Vertebral body estimation model learning device, vertebral body estimation device, fixed condition estimation device, vertebral body estimation model learning method, vertebral body estimation method, fixed condition estimation method, and program

Abstract

This vertebral body estimation model learning device comprises: a vertebral body estimation model executing unit for executing a vertebral body estimation model, which is a mathematical model for estimating, on the basis of image data of a vertebral body two-dimensional image being a two-dimensional image in which a vertebral body appears, positions of each vertex of the vertebral body and vectors at each vertex, the vectors being oriented from the vertices toward a center of the vertebral body, to thereby estimate the positions of the vertices of a vertebral body appearing in image data of a vertebral body two-dimensional image to be estimated, and the vectors at each vertex; and an updating unit for updating the vertebral body estimation model on the basis of the result of the estimation performed by the vertebral body estimation model executing unit.

Description

Technical Field

[0001] The present invention relates to a vertebral body estimation model learning device, a vertebral body estimation device, a fixation condition estimation device, a vertebral body estimation model learning method, a vertebral body estimation method, a fixation condition estimation method, and a program. This application claims priority based on Japanese Patent Application No. 2021-178270 filed in Japan on October 29, 2021, and incorporates the content thereof herein.

Background Art

[0002] There is a technique for treating abnormalities of the body related to the vertebral body. In such a technique, the vertebral body may be photographed, and a treatment policy may be determined based on the photographing result. Specifically, a person (such as a doctor) who estimates the state of the vertebral body from the photographing result (such as a radiological technologist) reads the state of the vertebral body from the photographing result, and a treatment policy is determined based on the read result. Therefore, the person who estimates the state of the vertebral body from the photographing result needs to read the state of the vertebral body from the photographing result.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, it is not always easy to read the state of the vertebral body from the photographing result, and there are cases where it depends on the skill of the person who reads the position of the vertebral body such as a doctor, or the burden is large. As a result, it is not easy to determine the treatment policy, and there are cases where the burden of determining the treatment policy is large.

[0005] In view of the above circumstances, the present invention aims to provide a technology that reduces the burden required to determine a treatment plan when treating physical abnormalities related to the vertebral body. [Means for solving the problem]

[0006] One aspect of the present invention is a vertebral body estimation model learning device comprising: a vertebral body estimation model execution unit that estimates the position of each vertex of a vertebral body and the vector for each vertex as seen in a vertebral body 2D image of a target vertebral body, by executing a vertebral body estimation model, which is a mathematical model that estimates the position of each vertex of a vertebral body, the position of the center of the vertebral body, and a vector for each vertex that points from the vertex to the center of the vertebral body, based on image data of a 2D image of a vertebral body, which is a 2D image of a vertebral body; and an update unit that updates the vertebral body estimation model based on the results of the estimation by the vertebral body estimation model execution unit and training data which is pairs of correct values ​​of the position of each vertex of a vertebral body, the position of the center, and the vector from the vertex to the center of the vertebral body.

[0007] One aspect of the present invention is a vertebral body estimation device comprising: a target data acquisition unit that acquires image data of a 2D image of a vertebral body, which is a two-dimensional image of a vertebral body; a vertebral body estimation model execution unit that estimates the position of each vertex of a vertebral body and the vector for each vertex as seen in a 2D image of a vertebral body to be estimated by executing a vertebral body estimation model, which is a mathematical model that estimates the position of each vertex of a vertebral body, the position of the center of a vertebral body, and a vector for each vertex that points from the vertex to the center of a vertebral body, based on the image data of a 2D image of a vertebral body, which is a two-dimensional image of a vertebral body; and an update unit that updates the vertebral body estimation model based on the results of the estimation by the vertebral body estimation model execution unit and training data which is pairs of correct values ​​of the position of each vertex of a vertebral body, the position of the center of a vertebral body, and the vector from the vertex to the center of a vertebral body. The vertebral body estimation device comprises: a target data acquisition unit that acquires image data of a 2D image of a vertebral body, which is a two-dimensional image of a vertebral body; a vertebral body estimation model execution unit that estimates the position of each vertex of a vertebral body and the vector for each vertex as seen in the image shown by the image data, based on the image data acquired by the target data acquisition unit.

[0008] One aspect of the present invention is a fixed-condition estimation model learning device comprising: a target data acquisition unit that acquires image data of a vertebral body, which is an image of a vertebral body; a fixed-condition estimation model execution unit that executes a fixed-condition estimation model, which is a mathematical model that estimates coordinate system information indicating each position of the image in the vertebral body image and the size and installation position of an internal spinal fixation device to be installed on the vertebral body, based on the image data of the vertebral body image; and an update unit that updates the fixed-condition estimation model based on the estimation results of the fixed-condition estimation model execution unit. The device uses the updated fixed-condition estimation model until a predetermined termination condition is met to estimate, based on the image data acquired by the target data acquisition unit, coordinate system information indicating each position of the image in the image data and the size and installation position of an internal spinal fixation device to be installed on the vertebral body. 。 A fixed-condition estimation device is a vertebral body learning device comprising: a fixed-condition estimation unit; the vertebral body image is a three-dimensional image showing a vertebral body; the image data of the vertebral body image is a two-dimensional image showing a vertebral body; the vertebral body estimation model execution unit estimates the position of each vertex of the vertebral body, the position of the center of the vertebral body, and a vector for each vertex that points from the vertex to the center of the vertebral body, based on the image data of the vertebral body image, which is a two-dimensional image showing a vertebral body; and an update unit updates the vertebral body estimation model based on the results of the vertebral body estimation model being executed on a plurality of two-dimensional images of vertebral bodies, until a predetermined termination condition is met.

[0009] One aspect of the present invention is a vertebral body estimation model learning method comprising: a vertebral body estimation model execution step, which estimates the position of each vertex of a vertebral body and the vector for each vertex, which is a vector from the vertex to the center of the vertebral body, based on image data of a 2D vertebral body image, which is a 2D image of a vertebral body; and an update step, which updates the vertebral body estimation model based on the estimation results from the vertebral body estimation model execution step.

[0010] One aspect of the present invention is a vertebral body estimation method comprising: a target data acquisition step of acquiring image data of a 2D vertebral body image, which is a two-dimensional image of a vertebral body; a vertebral body estimation model execution unit that estimates the position of each vertex of a vertebral body and the vector for each vertex as depicted in the image data of a 2D vertebral body image to be estimated by executing a vertebral body estimation model, which is a mathematical model that estimates the position of each vertex of a vertebral body and the vector for each vertex as depicted in the image data of a 2D vertebral body image to be estimated; and an update unit that updates the vertebral body estimation model based on the estimation results by the vertebral body estimation model execution unit until a predetermined termination condition is met, and using the vertebral body estimation model acquired in the target data acquisition step, the vertebral body estimation method is a vertebral body estimation model learning device comprising: a target data acquisition step of acquiring image data of a 2D vertebral body image, which is a two-dimensional image of a vertebral body image; a vertebral body estimation model execution unit that estimates the position of each vertex of a vertebral body and the vector for each vertex as depicted in the image data of a 2D vertebral body image to be estimated.

[0011] One aspect of the present invention is a fixed condition estimation method comprising: a target data acquisition step of acquiring image data of a vertebral body image, which is an image of a vertebral body; a fixed condition estimation model execution unit that estimates coordinate system information indicating each position of an image in the image data of a vertebral body image, and the size and installation position of an internal spinal fixation device to be installed on the vertebral body, by executing a fixed condition estimation model, which is a mathematical model that estimates coordinate system information indicating each position of an image in the image data of a vertebral body image based on the image data of the vertebral body image, and the size and installation position of an internal spinal fixation device to be installed on the vertebral body; and an update unit that updates the fixed condition estimation model based on the estimation results by the fixed condition estimation model execution unit, using the fixed condition estimation model which has been updated until a predetermined termination condition is met, based on the image data acquired in the target data acquisition step, to estimate coordinate system information indicating each position of an image in the image data, and the size and installation position of an internal spinal fixation device to be installed on the vertebral body image.

[0012] One aspect of the present invention is a program for causing a computer to function as the above-mentioned vertebral body estimation model learning device.

[0013] One aspect of the present invention is a program for causing a computer to function as the vertebral body estimation device described above. [Effects of the Invention]

[0014] This invention makes it possible to reduce the burden required to determine a treatment plan when treating physical abnormalities related to the vertebral body. [Brief explanation of the drawing]

[0015] [Figure 1] An explanatory diagram illustrating the outline of the vertebral body estimation model learning device of the first embodiment. [Figure 2] A diagram showing an example of the hardware configuration of the vertebral body estimation model learning device in the first embodiment. [Figure 3]A diagram showing an example of the configuration of a control unit included in a vertebral body estimation model learning device in the first embodiment. [Figure 4] A flowchart showing an example of the flow of processing executed by the vertebral body estimation model learning device in the first embodiment. [Figure 5] A diagram showing an example of the hardware configuration of a vertebral body estimation device in the first embodiment. [Figure 6] A diagram showing an example of the configuration of a control unit included in a vertebral body estimation device in the first embodiment. [Figure 7] A flowchart showing an example of the flow of processing executed by the vertebral body estimation device in the first embodiment. [Figure 8] A diagram showing an example of the configuration of a control unit included in a vertebral body estimation device in the fourth modification of the first embodiment. [Figure 9] A diagram showing an example of the configuration of a control unit included in a vertebral body estimation device in the second modification of the first embodiment. [Figure 10] A flowchart showing an example of the flow of processing executed by the control unit in the second modification of the first embodiment. [Figure 11] The first figure explaining an example of the coordinate system indicated by the correct coordinate system information in the second embodiment. [Figure 12] The second figure explaining an example of the coordinate system indicated by the correct coordinate system information in the second embodiment. [Figure 13] A diagram showing an example of the hardware configuration of a fixed condition estimation model learning device in the second embodiment. [Figure 14] A diagram showing an example of the configuration of a control unit included in a fixed condition estimation model learning device in the second embodiment. [Figure 15] A flowchart showing an example of the flow of processing executed by the fixed condition estimation model learning device in the second embodiment. [Figure 16] A diagram showing an example of the hardware configuration of a fixed condition estimation device in the second embodiment. [Figure 17] A diagram showing an example of the configuration of a control unit included in a fixed condition estimation device in the second embodiment. [Figure 18]A flowchart showing an example of the processing flow performed by the fixed-condition estimation device in the second embodiment. [Figure 19] This figure shows an example of the configuration of the control unit included in the fixed-condition estimation model learning device in the first modified example of the second embodiment. [Figure 20] This figure shows an example of the configuration of the control unit included in the fixed-condition estimation device in a second modified example of the second embodiment. [Modes for carrying out the invention]

[0016] (First Embodiment) Figure 1 is an explanatory diagram illustrating the outline of the vertebral body estimation model learning device 1 of the first embodiment. The vertebral body estimation model learning device 1 obtains a mathematical model for estimating the position of human or animal vertebral bodies using machine learning methods.

[0017] When acquiring a mathematical model for estimating the position of vertebral bodies in humans or animals, the vertebral body estimation model learning device 1 performs training on the vertebral body estimation model. The vertebral body estimation model is a mathematical model that estimates the position of each vertebral body vertex and a center-directing vector in the input 2D vertebral body image data. The 2D vertebral body image data is image data of a 2D image in which vertebral bodies are depicted. Vertebral body vertices are the individual vertices of a vertebra. The center-directing vector is a vector for each vertebral body vertex, and is a vector that points from each vertebral body vertex towards the center of the vertebra. The center is, for example, the centroid. The center may also be, for example, the intersection of the diagonals of a quadrilateral formed by the four vertebral body vertices. Hereinafter, the image shown by the 2D vertebral body image data will be referred to as the 2D vertebral body image.

[0018] Image G1 in Figure 1 is an example of a two-dimensional image of a vertebral body. Image G1 is a coronal plane image. Image G1 shows three vertebral bodies. Image G2 in Figure 1 shows the vertebral vertices shown in image G1. More specifically, points P1 to P12 in image G2 are examples of vertebral vertices. In image G2, vertebral vertices P1 to P4 are four vertices belonging to one of the three vertebral bodies. In image G2, vertebral vertices P5 to P8 are four vertices belonging to the other vertebral body of the three vertebral bodies. In image G2, vertebral vertices P9 to P12 are four vertices belonging to the last of the three vertebral bodies.

[0019] Image G2 in Figure 1 shows center-directed vectors for three of the vertebral vertices visible in Image G1, as an example of center-directed vectors. Vectors V1, V2, and V3 in Image G2 are all examples of center-directed vectors. Vector V1 is the center-directed vector at vertebral vertex P3. Vector V2 is the center-directed vector at vertebral vertex P7. Vector V3 is the center-directed vector at vertebral vertex P11. Thus, a center-directed vector is a vector that points from the vertex towards the center of the vertebra, corresponding to the position of each vertebral vertex and the position of the center of the vertebra.

[0020] A mathematical model is a set of one or more processes whose execution conditions and order (hereinafter referred to as "execution rules") are predetermined. Learning means updating the mathematical model using machine learning methods. Updating a mathematical model means adjusting the parameter values ​​in the mathematical model to suit the purpose. Execution of a mathematical model means executing each process included in the mathematical model according to the execution rules. The processes included in a mathematical model may also include processes that execute other mathematical models.

[0021] The updating of mathematical models through learning continues until a predetermined termination condition related to learning (hereinafter referred to as the "learning termination condition") is met. The learning termination condition is, for example, that a predetermined number of learning iterations have been performed.

[0022] The 2D vertebral body image data may be 2D image data obtained from 3D CT (Computed Tomography) images, 2D image data obtained using MRI (Magnetic Resonance Imaging), or X-ray image data obtained from radiographs. When obtaining 2D vertebral body image data from 3D CT images, for example, Digitally Reconstructed Radiograph (DRR) is used.

[0023] Two-dimensional vertebral body image data can be any image data that contains vertebral bodies; however, it is desirable to implement techniques that improve the accuracy of the vertebral body estimation model and reduce the burden required to acquire the two-dimensional vertebral body image data. Examples of such techniques will be explained in the modified examples.

[0024] <Details of the vertebral body estimation model training> In training a vertebral body estimation model, a set of vertebral body 2D image data, ground truth vertex information, and ground truth vector information is used as training data. Ground truth vertex information indicates the position of each vertebral body vertex in the image shown by the vertebral body 2D image data belonging to the same set. Ground truth vector information indicates the center-oriented vector for each vertebral body vertex in the image shown by the vertebral body 2D image data belonging to the same set. One training data set is used for each trial during training. The vertebral body estimation model is updated after each trial during training. Note that the vertebral body vertex positions indicated by the ground truth data may be, for example, the positions of vertebral body vertices manually identified by a doctor through diagnosis.

[0025] Let's explain the training of the vertebral body estimation model in more detail. In training the vertebral body estimation model, the model is run on the 2D vertebral body image data included in the training data. During training, the vertebral body estimation model is updated to minimize the difference between the estimation results obtained by running the vertebral body estimation model and the positions and center-direction vectors of the vertebral vertices indicated by the ground truth vertex information. In other words, the ground truth vertex information and ground truth vector information are the ground truth data in training the vertebral body estimation model. Note that the loss function representing the difference may be, for example, the squared error function.

[0026] Figure 2 shows an example of the hardware configuration of the vertebral body estimation model learning device 1 in the first embodiment. The vertebral body estimation model learning device 1 includes a control unit 11 which has a processor 91 such as a CPU (Central Processing Unit) and memory 92 connected by a bus, and executes a program. The vertebral body estimation model learning device 1 functions as a device comprising the control unit 11, input unit 12, communication unit 13, storage unit 14 and output unit 15 by executing the program.

[0027] More specifically, the processor 91 reads the program stored in the memory unit 14 and stores the read program in the memory 92. By executing the program stored in the memory 92, the vertebral body estimation model learning device 1 functions as a device comprising a control unit 11, an input unit 12, a communication unit 13, a memory unit 14, and an output unit 15.

[0028] The control unit 11 controls the operation of various functional units of the vertebral body estimation model learning device 1. The control unit 11 executes the vertebral body estimation model. The control unit 11 controls the operation of, for example, the output unit 15 and causes the output unit 15 to output the execution result of the vertebral body estimation model. The control unit 11 records various information generated by the execution of the vertebral body estimation model in the storage unit 14. The various information stored in the storage unit 14 includes, for example, the learning result of the vertebral body estimation model.

[0029] The input unit 12 includes input devices such as a mouse, keyboard, or touch panel. The input unit 12 may also be configured as an interface for connecting these input devices to the vertebral body estimation model learning device 1. The input unit 12 receives various types of information for input to the vertebral body estimation model learning device 1. For example, training data used for learning the vertebral body estimation model is input to the input unit 12.

[0030] The communication unit 13 includes a communication interface for connecting the vertebral body estimation model learning device 1 to an external device. The communication unit 13 communicates with the external device via wired or wireless connection. The external device is, for example, a device that transmits training data used for learning the vertebral body estimation model. The communication unit 13 receives training data used for learning the vertebral body estimation model from the source of training data used for learning the vertebral body estimation model through communication with the source of training data used for learning the vertebral body estimation model. The external device is, for example, a device that runs a trained vertebral body estimation model. The communication unit 13 transmits the trained vertebral body estimation model to the device that runs the trained vertebral body estimation model through communication with the device.

[0031] The memory unit 14 is configured using a computer-readable storage medium such as a magnetic hard disk drive or a semiconductor storage device. The memory unit 14 stores various information related to the vertebral body estimation model learning device 1. The memory unit 14 stores information input via, for example, the input unit 12 or the communication unit 13. The memory unit 14 pre-stores, for example, a vertebral body estimation model. The memory unit 14 stores, for example, various information generated by the execution of the vertebral body estimation model.

[0032] The output unit 15 outputs various types of information. The output unit 15 is comprised of a display device such as a CRT (Cathode Ray Tube) display, a liquid crystal display, or an organic EL (Electro-Luminescence) display. The output unit 15 may be configured as an interface for connecting these display devices to the vertebral body estimation model learning device 1. The output unit 15 outputs information input to, for example, the input unit 12 or the communication unit 13. The output unit 15 may display, for example, the execution result of the vertebral body estimation model.

[0033] Figure 3 shows an example of the configuration of the control unit 11 of the vertebral body estimation model learning device 1 in the first embodiment. The control unit 11 comprises a training data acquisition unit 111, a learning unit 112, a memory control unit 113, and an output control unit 114.

[0034] The training data acquisition unit 111 acquires training data. The training data acquisition unit 111 acquires training data input to, for example, the input unit 12 or the communication unit 13. The training data acquisition unit 111 may also acquire training data by reading training data that has been previously stored in the storage unit 14.

[0035] The learning unit 112 performs training of the vertebral body estimation model. The learning unit 112 includes a vertebral body estimation model execution unit 121 and an update unit 122. The vertebral body estimation model execution unit 121 performs the vertebral body estimation model on the 2D vertebral body image data included in the training data acquired by the training data acquisition unit 111. By performing the vertebral body estimation model, the vertebral body estimation model execution unit 121 estimates the position of each vertebral body vertex and the center-direction vector shown in the image represented by the 2D vertebral body image data of the target. Note that the target of execution means the object of execution of the processing or mathematical model. Therefore, the target of execution of the vertebral body estimation model is the estimation target of the estimation by the vertebral body estimation model execution unit 121.

[0036] The vertebral body estimation model execution unit 121 is specifically a neural network that represents the vertebral body estimation model. Therefore, the execution of the vertebral body estimation model by the vertebral body estimation model execution unit 121 means that the neural network representing the vertebral body estimation model is operating.

[0037] The update unit 122 updates the vertebral body estimation model based on the estimation results of the vertebral body estimation model execution unit 121 and the ground truth data included in the training data, in order to reduce the difference between the estimation results and the ground truth data.

[0038] The memory control unit 113 records various information in the memory unit 14. The output control unit 114 controls the operation of the output unit 15.

[0039] Figure 4 is a flowchart showing an example of the processing flow performed by the vertebral body estimation model learning device 1 in the first embodiment. The training data acquisition unit 111 acquires training data (step S101). Next, the vertebral body estimation model execution unit 121 executes the vertebral body estimation model on the 2D vertebral body image data included in the training data acquired in step S101 (step S102). By executing the vertebral body estimation model, the positions of each vertebral body vertex and the center-direction vectors shown in the 2D vertebral body image data to be executed are estimated.

[0040] Next, the update unit 122 updates the vertebral body estimation model based on the estimation results obtained from step S102 and the ground truth vertex information and ground truth vector information included in the training data acquired in step S101 (step S103). Next, the update unit 122 determines whether the learning termination condition has been met (step S104). If the learning termination condition is met (step S104: YES), the process ends. On the other hand, if the learning termination condition is not met (step S104: NO), the process returns to step S101.

[0041] The trained vertebral body estimation model obtained in this way is used in the process of estimating the position of each vertebral body vertex and each center-direction vector in the input image data, which is the image to be estimated. An example of a device that performs such processing is the vertebral body estimation device 2, which will be described below. The vertebral body estimation device 2 acquires the trained vertebral body estimation model in advance before execution, for example by obtaining it from the vertebral body estimation model learning device 1 via communication. The vertebral body estimation device 2 may also acquire the trained vertebral body estimation model in advance before execution, for example by equipping it with a neural network that represents the trained vertebral body estimation model.

[0042] The vertebral body estimation device 2 uses the trained vertebral body estimation model obtained by the vertebral body estimation model learning device 1 to estimate the position of each vertebral body vertex and each center-direction vector in the image to be estimated.

[0043] Figure 5 shows an example of the hardware configuration of the vertebral body estimation device 2 in the first embodiment. The vertebral body estimation device 2 includes a control unit 21 which has a processor 93 such as a CPU and a memory 94 connected by a bus, and executes a program. The vertebral body estimation device 2 functions as a device comprising the control unit 21, input unit 22, communication unit 23, storage unit 24 and output unit 25 by executing the program.

[0044] More specifically, the processor 93 reads the program stored in the storage unit 24 and stores the read program in the memory 94. By executing the program stored in the memory 94, the processor 93 functions as a device comprising a control unit 21, an input unit 22, a communication unit 23, a storage unit 24, and an output unit 25.

[0045] The control unit 21 controls the operation of various functional units of the vertebral body estimation device 2. The control unit 21 executes the trained vertebral body estimation model. The control unit 21 controls the operation of the output unit 25, for example, and causes the output unit 25 to output the execution result of the trained vertebral body estimation model. The control unit 21 records various information generated by the execution of the trained vertebral body estimation model in the storage unit 24. The various information stored in the storage unit 24 includes, for example, the execution result of the trained vertebral body estimation model.

[0046] The input unit 22 includes input devices such as a mouse, keyboard, or touch panel. The input unit 22 may also be configured as an interface for connecting these input devices to the vertebral body estimation device 2. The input unit 22 receives various types of information for input to the vertebral body estimation device 2. For example, image data of the image to be estimated is input to the input unit 22.

[0047] The communication unit 23 is configured to include a communication interface for connecting the vertebral body estimation device 2 to an external device. The communication unit 23 communicates with the external device via wired or wireless connection. The external device is, for example, a device that transmits image data of the image to be estimated. The communication unit 23 may receive image data of the image to be estimated from the device that transmits image data of the image to be estimated by communicating with the device. The external device is, for example, a vertebral body estimation model learning device 1. The communication unit 23 may receive a trained vertebral body estimation model from the vertebral body estimation model learning device 1 by communicating with the vertebral body estimation model learning device 1.

[0048] The memory unit 24 is configured using a computer-readable storage medium such as a magnetic hard disk drive or a semiconductor storage device. The memory unit 24 stores various information related to the vertebral body estimation device 2. The memory unit 24 stores information input via, for example, the input unit 22 or the communication unit 23. The memory unit 24 stores, for example, a trained vertebral body estimation model in advance before execution of the trained vertebral body estimation model. The memory unit 24 stores, for example, various information generated by the execution of the trained vertebral body estimation model.

[0049] The output unit 25 outputs various types of information. The output unit 25 is comprised of a display device such as a CRT display, liquid crystal display, or organic EL display. The output unit 25 may be configured as an interface for connecting these display devices to the vertebral body estimation device 2. The output unit 25 outputs information input to, for example, the input unit 22 or the communication unit 23. The output unit 25 may also output, for example, the execution result of a trained vertebral body estimation model.

[0050] Figure 6 shows an example of the configuration of the control unit 21 of the vertebral body estimation device 2 in the first embodiment. The control unit 21 includes a target data acquisition unit 211, a vertebral body estimation unit 212, a memory control unit 213, and an output control unit 214.

[0051] The target data acquisition unit 211 acquires 2D image data of vertebrae that are the target of the trained vertebral body estimation model. The 2D image data of vertebrae acquired by the target data acquisition unit 211 is the image data of the image to be estimated. The target data acquisition unit 211 acquires 2D image data of vertebrae input via, for example, the input unit 22 or the communication unit 23 as the 2D image data of vertebrae that are the target of the trained vertebral body estimation model.

[0052] The vertebral body estimation unit 212 executes a pre-trained vertebral body estimation model on the 2D vertebral body image data acquired by the target data acquisition unit 211. By executing the vertebral body estimation model, the vertebral body estimation unit 212 estimates the position of each vertebral body vertex and its center-direction vector as shown in the 2D vertebral body image data being executed.

[0053] The memory control unit 213 records various types of information in the memory unit 14. The output control unit 214 controls the operation of the output unit 15.

[0054] Figure 7 is a flowchart showing an example of the processing flow performed by the vertebral body estimation device 2 in the first embodiment. The target data acquisition unit 211 acquires image data of the image to be estimated (step S201). Next, the vertebral body estimation unit 212 estimates the position of each vertebral body vertex and the center-direction vector in the image to be estimated by running a trained vertebral body estimation model on the image data acquired by the target data acquisition unit 211 (step S202). Next, the output control unit 214 controls the operation of the output unit 25 to output the estimation result obtained in step S202 to the output unit 25 (step S203). The output method is, for example, display.

[0055] The vertebral body estimation model learning device 1 of the first embodiment, configured in this way, obtains a mathematical model that estimates the vertebral bodies in an image and the vectors pointing from the vertebral bodies to the center. When determining a treatment plan for physical abnormalities related to the vertebral bodies, the position of the vertebral bodies is read from an image, but this can be difficult or depend on the skill of the person reading the vertebral body position, such as a doctor. However, if the person reading the vertebral body position knows the vertebral body position and the vectors pointing from the vertebral body to the center, they can estimate the position of the vertebral body even if the endpoints of the vectors are close to each other and are the vertebral body positions, thus reducing the burden required to read the position of the vertebral body. Therefore, the burden on the person reading the vertebral body position when determining a treatment plan when treating physical abnormalities related to the vertebral bodies is reduced. Thus, the vertebral body estimation model learning device 1, which obtains a mathematical model that estimates the vertebral body positions and the vectors pointing from the vertebral body to the center through learning, can reduce the burden required to determine a treatment plan when treating physical abnormalities related to the vertebral bodies.

[0056] The vertebral body estimation device 2 of the first embodiment, configured in this way, estimates the vertebral body vertex and the vector from the vertex to the center using a trained vertebral body estimation model. Therefore, the vertebral body estimation device 2 can reduce the burden required to determine the treatment plan when treating physical abnormalities related to the vertebral body.

[0057] (First embodiment, first modified example) The position of the vertebral vertebral vertex estimated by the vertebral body estimation model may be indicated, for example, by information showing the distribution of the likelihood of a location being a vertebral vertex within an image (hereinafter referred to as "heatmap information"). Heatmap information may be shown, for example, by contour lines. If the position of the vertebral vertex estimated by the vertebral body estimation model is indicated by heatmap information, the trained vertebral body estimation model will also indicate the position of the vertebral vertex using heatmap information.

[0058] (Second modified example of the first embodiment) The vertebral body estimation model may be input with reduced resolution 2D images of vertebral bodies, such as 2D images of vertebral bodies with 1 / 4 resolution. Inputting such reduced-resolution images of vertebral bodies reduces the computational load. When reduced-resolution 2D images of vertebral bodies are input to the vertebral body estimation model, the trained vertebral body estimation model will also be input with reduced-resolution 2D images of vertebral bodies.

[0059] (Third modified example of the first embodiment) When a vertebral body estimation model is input with a reduced-resolution 2D image of vertebral bodies, the accuracy of the vertebral body estimation model's estimation of the vertebral body vertex position also decreases to the same degree as the reduction in resolution. Therefore, in training the vertebral body estimation model, it may be possible to train the model to estimate not only the information indicating the position of the vertebral body vertex on the reduced-resolution 2D image of vertebral bodies, but also the information indicating the position of the vertebral body vertex when the reduced resolution is restored. Hereinafter, the information indicating the position of the vertebral body vertex when the reduced resolution is restored will be referred to as the original position indication information. The original position indication information is, for example, represented by a position vector that starts with the information indicating the position of the vertebral body vertex on the reduced-resolution 2D image of vertebral bodies as the starting point and ends with the position of the vertebral body vertex when the reduced resolution is restored as the ending point.

[0060] In training a vertebral body estimation model that estimates original position information, the training data includes ground truth data, as well as information indicating the position of the vertebral vertex when the reduced resolution is restored.

[0061] When the vertebral body estimation device 2 uses a pre-trained vertebral body estimation model that has been trained to estimate original position indication information, the vertebral body estimation device 2 also estimates the original position indication information for the 2D image of the vertebral body to be estimated by executing the pre-trained vertebral body estimation model.

[0062] (Fourth modified example of the first embodiment) The vertebral body estimation device 2 distinguishes each vertebra in the target image based on the estimation results obtained by running the trained vertebral body estimation model. In other words, the vertebral body estimation device 2 determines that each vertebra in the target image is different from the other vertebrae based on the estimation results obtained by running the trained vertebral body estimation model. Hereinafter, the process of determining that each vertebra in the target image is different from the other vertebrae will be referred to as the vertebral body determination process.

[0063] The vertebral body identification process includes a vertebral body center estimation process and a grouping process. The vertebral body center estimation process estimates the center of each vertebra in the target image based on the estimation results obtained by running a trained vertebral body estimation model.

[0064] Since there is only one vertebral center per vertebra, it also serves as information for identifying a vertebra. Therefore, it is possible to classify each vertebral vertex and each center-directed vector in the image of the target of estimation using the vertebral center. Specifically, the process of determining which vertebra each vertebral vertex and each center-directed vector in the image of the target of estimation belongs to can be replaced by the process of determining which vertebral center each vertebral vertex and each center-directed vector in the image of the target of estimation belongs to.

[0065] The grouping process determines which vertebral center each vertex and center-direction vector projected onto the target vertebral body belongs to. Specifically, the classification in the grouping process is based on information indicating the center of each vertebral body estimated by the vertebral body center estimation process, and the positions of each vertebral body vertex and each center-direction vector obtained by running the trained vertebral body estimation model. By executing the grouping process, the vertebral body vertices, center-direction vectors, and vertebral body centers projected onto the target vertebral body are classified together according to which they belong to the same vertebral body. Since there is only one center for each vertebral body, the vertebral body center serves as an identifier for the vertebral body. Therefore, in other words, the grouping process is the process of grouping the positions of each vertebral body vertex and the center-direction vector for each vertex by vertebral body.

[0066] In the grouping process, if the distance between the endpoint of a center-directed vector and one of the centers estimated by the vertebral body center estimation process is shorter than a predetermined threshold, it is determined that the center, its center-directed vector, and the vertebral body vertex at the starting point of the center-directed vector belong to the same vertebral body. In this way, the grouping process determines which vertebral body center each vertebral body vertex and center-directed vector that appear in the estimation target belong to.

[0067] In a 2D image of a vertebra, it is expected that four vertebrae will be visible when the vertebra is photographed from the front. However, depending on the circumstances of the vertebra's image acquisition, such as when the vertebra is photographed in a curved position, there may be vertebrae in the 2D image of the vertebra to which the estimation target is located that only show three or fewer vertebral vertices. Therefore, in the vertebral body determination process, it is acceptable for vertebrae to be estimated that have three or fewer vertebral vertices and three or fewer center-directed vectors.

[0068] The output control unit 214 controls the operation of the output unit 25 to output the estimation result obtained by executing the vertebral body determination process. The output method is, for example, display.

[0069] Figure 8 shows an example of the configuration of the control unit 21a of the vertebral body estimation device 2 in the fourth modified example of the first embodiment. For the sake of simplicity in the following explanation, components having the same functions as the control unit 21 are denoted by the same reference numerals as in Figure 6, and their explanation is omitted. The control unit 21a differs from the control unit 21 of the first embodiment in that it includes a vertebral body determination unit 215. The vertebral body determination unit 215 performs vertebral body determination processing.

[0070] (Most modified example of the first embodiment) The vertebral body estimation device 2 may estimate the Cobb angle of the spine visible in the image to be estimated based on the estimation results obtained by executing the vertebral body determination process. The Cobb angle is a quantity that indicates the degree of scoliosis. The value of the Cobb angle is calculated by the Cobb method. Hereinafter, the process of calculating the Cobb angle of the spine visible in the image to be estimated based on the estimation results obtained by executing the vertebral body determination process will be referred to as the Cobb angle calculation process. Note that if the position of each vertebral body vertex estimated by the vertebral body estimation model is shown by heatmap information, each position of the vertebral body vertex in the Cobb angle calculation process is the position of the maximum value shown by the heatmap information for each vertebral body vertex position.

[0071] Figure 9 shows an example of the configuration of the control unit 21b of the vertebral body estimation device 2 in the second modified example of the first embodiment. For the sake of simplicity in the following explanation, components having the same functions as the control unit 21 are denoted by the same reference numerals as in Figure 6, and their explanation is omitted. The control unit 21b differs from the control unit 21a of the fourth modified example of the first embodiment in that it includes a Cobb angle calculation unit 216. The Cobb angle calculation unit 216 performs the Cobb angle calculation process, which will be explained using Figure 10.

[0072] Figure 10 is a flowchart showing an example of the processing flow performed by the control unit 21b in a second modified example of the first embodiment. The Cobb angle calculation process is performed after the vertebral body determination process is performed. The vertebral body determination process is performed, for example, after the process in step S202 is performed. The vertebral body determination unit 215 obtains the estimation result estimated in step S202. That is, the vertebral body determination unit 215 obtains the estimation result by the vertebral body estimation unit 212 (step S301). Next, the vertebral body determination unit 215 distinguishes each vertebra that appears in the image to be estimated by performing the vertebral body determination process (step S302).

[0073] Next, the Cobb angle calculation unit 216 determines the adjacent vertebral body pair (step S303). For the sake of simplicity, an example of the processing flow executed by the control unit 21b will be described below, assuming that the direction with an inclination of 0 is the direction parallel to the axis perpendicular to the line where the sagittal plane and the coronal plane intersect and parallel to the axis parallel to the coronal plane. An adjacent vertebral body pair is a pair of vertebral bodies that are adjacent to each other, in which the sign of the inclination of one vertebral body and the sign of the inclination of the other vertebral body are opposite (different) with respect to a predetermined direction. The predetermined direction is, for example, the horizontal direction in the result of photographing the vertebral bodies from the front of a person. Note that using the predetermined direction as a reference means that the predetermined direction is set to angle 0. For example, an adjacent vertebral body pair may be a pair in which the sign of the inclination between adjacent vertebral bodies reverses when the inclination changes from one vertebral body to the inclination of the other vertebral body. Therefore, an extreme point vertebral body pair is a pair of adjacent vertebral bodies where, for example, if we define the side of the spine closer to the neck as superior and the side of the spine closer to the lumbar spine as inferior, the sign of the vertebral angle formed by the upper edge of the superior vertebral body is different from the sign of the vertebral angle formed by the lower edge of the inferior vertebral body. The vertebral angle may also be the angle with respect to the line from the neck to the lumbar spine.

[0074] Specifically in step S303, the Cobb angle calculation unit 216 first calculates the inclination of the upper and lower edges of each vertebral body based on the position of each vertex of each vertebral body, for each vertebral body distinguished in step S302. Next, for each pair of adjacent vertebral bodies, the Cobb angle calculation unit 216 determines whether the signs of the inclination of the upper edge and the inclination of the lower edge of a pair of upper and lower edges that satisfy the condition that they are not adjacent to each other are reversed. Based on the determination result for each pair, the Cobb angle calculation unit 216 determines pairs in which the sign of the change in the inclination of the vertebral bodies is reversed. In this way, the Cobb angle calculation unit 216 determines adjacent vertebral body pairs.

[0075] In step S303, there are multiple pairs of adjacent vertebral bodies. Next, the memory control unit 213 records information indicating the upper vertebral body of the pair of vertebral bodies determined to be adjacent as the extremum vertebral body in a predetermined memory device such as the memory unit 24 (step S304). That is, the extremum vertebral body is the upper vertebral body of the pair of vertebral bodies determined to be adjacent. The Cobb angle calculation process will now be explained using the memory unit 24 as an example.

[0076] Next, the memory control unit 213 records the combination of two extreme point vertebrae that are closest to each other from among the multiple extreme point vertebrae (step S305). For example, if the detected extreme point vertebrae are vertebrae T1 and vertebrae L5, the memory control unit 213 records vertebrae T1 and vertebrae L5 in the memory unit 24. T1 represents the first thoracic vertebra, and L5 represents the fifth lumbar vertebra. Next, the Cobb angle calculation unit 216 calculates the Cobb angle as the absolute value of the difference in the slope between the vertebral angle formed by the upper edge of the upper vertebra and the vertebral angle formed by the lower edge of the lower vertebra for the combination of extreme point vertebrae stored in the memory unit 24 (step S306). For example, in the above example, the Cobb angle is calculated as the absolute value of the difference between the vertebral angle formed by the upper edge of vertebrae T1 and the vertebral angle formed by the lower edge of vertebrae L5. At this time, in order to remove noise, a certain slope condition may be set as a threshold, and values ​​below this threshold may not be included in the calculation of the Cobb angle. A certain slope condition is, for example, 5 degrees in absolute value.

[0077] The process from step S303 to step S307 is an example of the cob angle calculation process. The cob angle calculation unit 216 may, after step S307, determine the cob angles obtained in step S307 in order from largest to smallest as Major, Minor1, and Minor2.

[0078] (Sixth variation of the first embodiment) As mentioned above, the 2D vertebral body image data can be any image data in which vertebral bodies are visible within the image. However, from the perspective of improving the accuracy of the vertebral body estimation model and reducing the burden required to acquire the 2D vertebral body image data, it is desirable to implement specially designed techniques. Therefore, here we will describe an example of such a specially designed technique.

[0079] Two-dimensional vertebral body image data can be obtained, for example, by performing an ROI estimation process. The ROI estimation process is a process that targets two-dimensional image data. The ROI estimation process estimates the region of interest (ROI) within the image indicated by the two-dimensional image data being processed. In such a case, the two-dimensional vertebral body image data is image data obtained by removing the image of the region of interest estimated by the ROI estimation process from the image of the two-dimensional image data that was the target of the ROI estimation process.

[0080] More specifically, the ROI estimation process is the process of executing a mathematical model (hereinafter referred to as the "ROI model") that estimates the region of interest within an image represented by the target 2D image data, and is obtained by learning using pairs of 2D image data and ground truth data representing the region of interest.

[0081] Thus, the ROI model is a mathematical model obtained through learning. In learning the ROI model, information indicating the region of interest is used as ground truth data. During learning, the ROI model is updated to minimize the difference between the ROI model's execution result on the 2D image data included in the training data and the ground truth data. The 2D image data included in the training data used for learning is image data of 2D images showing vertebral bodies, and the ground truth data indicates the region showing the vertebral bodies as the region of interest. Note that the position of the vertebral vertex indicated by the ground truth data used may be, for example, the position of the vertebral vertex manually identified by a doctor through diagnosis. In this way, an ROI model is obtained that estimates the region of interest showing the vertebral bodies from the 2D image data to be executed.

[0082] Furthermore, some of the training data in learning an ROI model may include 2D image data that does not show images of vertebrae. In such cases, the ground truth data indicates that there is no region of interest. Alternatively, the 2D image data included in the training data for learning an ROI model may consist only of 2D image data that shows images of vertebrae. In such cases, the ground truth data may always indicate the region showing images of vertebrae as the region of interest. Also, the ground truth data included in the training data may indicate a randomly selected region within the 2D image data targeted by the ROI model that satisfies the condition of showing images of vertebrae. Therefore, for example, the ground truth data included in the training data may indicate each of the multiple regions within the 2D image data targeted by the ROI model that show images of at least one vertebra as the region of interest.

[0083] Spine imaging can be performed in various environments. Therefore, image variability occurs depending on the imaging conditions. If 2D vertebral body image data obtained through ROI estimation processing is used, the position of each vertebral vertex and the center-direction vector are estimated using image data with reduced imaging variability. This improves the efficiency of training the vertebral body estimation model and the accuracy of estimation by the trained model. When random selection of the region of interest, as described above, is performed during training, the accuracy of estimation by the trained vertebral body estimation model improves. The reason for this is explained below.

[0084] <Why random selection of the region of interest improves estimation accuracy> The datasets used for training sometimes consist mostly of images with large margins on both sides, and a small amount of images with almost no margins. If such datasets are used directly for training, the estimation accuracy may be high for images with large margins, but low for other image data, potentially leading to overfitting. When an overfitted mathematical model is incorporated into an application, the estimation accuracy may be low for image data obtained under different conditions than those used for training.

[0085] Therefore, it is desirable that training be performed in a way that reduces the bias of the training dataset. Random selection of the region of interest can reduce this bias in the training dataset because it randomly selects the data used for training. More specifically, by randomly selecting the region of interest, it is possible to generate data that is close to the minority data with less margin from the majority of data with a lot of margin, increasing the relative proportion of data with less margin. Therefore, random selection of the region of interest can reduce this bias in the training dataset because it randomly selects the data used for training. Consequently, random selection of the region of interest can prevent overfitting and improve the accuracy of estimation by the trained vertebral body estimation model. This concludes the explanation of why random selection of the region of interest improves estimation accuracy.

[0086] The ROI estimation process is performed, for example, by the training data acquisition unit 111. When pairs of 2D image data and ground truth data are input to the vertebral body estimation model learning device 1, the training data acquisition unit 111 generates 2D vertebral body image data from the input 2D image data by executing the ROI estimation process. The training data acquisition unit 111 acquires the pairs of the generated 2D vertebral body image data and the input ground truth data as training data.

[0087] The ROI estimation process is performed, for example, by the target data acquisition unit 211. When 2D image data is input to the vertebral body estimation device 2, the target data acquisition unit 211 generates 2D vertebral body image data from the input 2D image data by executing the ROI estimation process. The target data acquisition unit 211 acquires the generated 2D vertebral body image data as the target for execution of the trained vertebral body estimation model.

[0088] (Seventh modified example of the first embodiment) In training a vertebral body estimation model, multiple 2D images of vertebral bodies may be used. In such cases, some of the multiple 2D images of vertebral bodies may be horizontally flipped, or have their colors inverted (e.g., black and white inversion), or have their orientation inverted. In other words, the set of 2D image data targeted by the vertebral body estimation model may include a set of 2D image data that has been inverted in color, orientation, or left and right.

[0089] In training the vertebral body estimation model, the model may be run on four different sets of 2D vertebral body images: one with no color or left-right inversion, one with color inversion but no left-right inversion, one with left-right inversion but no color inversion, and one with both color and left-right inversion. In such cases, the vertebral body estimation model may be updated based on the estimation results of the vertebral body estimation model for each image, so as to minimize the difference between each estimation result.

[0090] When the vertebral body estimation device 2 uses a pre-trained vertebral body estimation model that has been trained using image data of these four types of 2D vertebral body images, the pre-trained vertebral body estimation model may be run on the image data of the four types of images mentioned above. In such a case, there may be vertebral bodies for which the vertex position and center-direction vector are estimated in only one to three of the four estimations. Thus, if the vertex position and center-direction vector are estimated in at least one of the four, even if not all four, the user may interpret the result as indicating that a vertebral body actually exists.

[0091] In one experiment, a vertebra that was not detected when using only one type of 2D vertebral body image data was detected consecutively in 17 trials when using these four types of 2D vertebral body image data.

[0092] Thus, training a vertebral body estimation model using image data of 2D vertebral body images that are color-inverted or horizontally inverted has the effect of improving estimation accuracy. Furthermore, estimation results obtained by running a trained vertebral body estimation model using image data of 2D vertebral body images that are color-inverted or horizontally inverted have the effect of higher estimation accuracy than estimation based on a single type of 2D vertebral body image data without color or horizontal inversion. Left / right and up / down are matters of definition, which side of the image is defined as right. Therefore, estimation results obtained by running a trained vertebral body estimation model using image data of 2D vertebral body images that are color-inverted, vertically inverted or horizontally inverted also have the effect of higher estimation accuracy than estimation based on a single type of 2D vertebral body image data without color, vertically inverted or horizontally inverted.

[0093] (Eighth variation of the first embodiment) The vertebral body estimation model learning device 1 may be implemented using multiple information processing devices connected to each other via a network. In this case, each functional unit of the vertebral body estimation model learning device 1 may be distributed and implemented across multiple information processing devices.

[0094] The vertebral body estimation device 2 may be implemented using multiple information processing devices connected to each other via a network. In this case, each functional unit of the vertebral body estimation device 2 may be distributed and implemented across multiple information processing devices.

[0095] Furthermore, the vertebral body estimation model learning device 1 and the vertebral body estimation device 2 do not necessarily need to be implemented as separate devices. For example, they may be implemented as a single device that combines the functions of both.

[0096] (Second Embodiment) The fixed-condition estimation model learning device 5 of the second embodiment will be described. The fixed-condition estimation model learning device 5 obtains a mathematical model for estimating the size and installation position of an internal spinal fixation device to be installed in the vertebral body of a person or animal, using machine learning methods.

[0097] The fixed-condition estimation model learning device 5 performs learning using three-dimensional image data of vertebral bodies (hereinafter referred to as "vertebral body three-dimensional image data"). The three-dimensional image data of vertebral bodies can be obtained in any way. For example, in learning, three-dimensional image data of virtual vertebral bodies obtained by simulation, etc., may be used. Alternatively, three-dimensional image data of vertebral bodies generated using image data in a format obtainable in existing medical settings may be used. Therefore, in explaining the fixed-condition estimation model learning device 5, we will first explain an example of the process for generating the three-dimensional image data of vertebral bodies used for learning (hereinafter referred to as "vertebral body three-dimensional image data generation process"), and then explain the learning process.

[0098] <Generation of 3D image data of vertebral bodies> In the vertebral body 3D image data generation process, a dimensional transformation process is performed. The dimensional transformation process converts the image data of a 3D image obtained by imaging such as a 3D CT scan into image data of multiple 2D images. Specifically, the multiple 2D images generated by the dimensional transformation process are images of cross-sections at multiple points on a predetermined axis (hereinafter referred to as the "cross-sectional perpendicular axis") of the image projected in the 3D image, and are images of cross-sections perpendicular to the cross-sectional perpendicular axis. In other words, the dimensional transformation process generates a set of images of cross-sections at multiple points on the cross-sectional perpendicular axis of the image projected in the 3D image, and are images of cross-sections perpendicular to the cross-sectional perpendicular axis, from the image data of the 3D image. An example of a dimensional transformation process is a Digitally Reconstructed Radiograph (DRR).

[0099] The 3D image subject to the dimensional transformation process is the 3D image of the vertebral body. Therefore, the 3D image subject to the dimensional transformation process contains an image of the vertebral body. Because the vertebral body is captured in the 3D image subject to the dimensional transformation process, the cross-sections of the vertebral body are captured in the multiple 2D images generated by the dimensional transformation process. Hereafter, the 2D image of the cross-section of the vertebral body will be referred to as a vertebral cross-sectional image.

[0100] In the vertebral body 3D image data generation process, a vertebral body detection process is performed on each obtained vertebral body cross-sectional image. The vertebral body detection process is the process of detecting the vertebral bodies that appear in the target vertebral body cross-sectional image. Specifically, detecting the vertebral bodies that appear in the vertebral body cross-sectional image means recording information indicating the pixels that represent the vertebral bodies among the pixels that make up the vertebral body cross-sectional image in a predetermined storage device such as the storage unit 54 or storage unit 64, which will be described later. Thus, the vertebral body detection process is the process of obtaining information about the position of the vertebral bodies in the vertebral body cross-sectional image.

[0101] The position of the vertebral body may be, for example, the position of the four vertebral vertebral bodies, the position of the center of the vertebral body, or the position of the contour of the vertebral body. The vertebral body detection process is a process that executes, for example, a trained vertebral body estimation model and a vertebral body determination process. In the vertebral body detection process, for example, a ROI estimation process may also be executed in addition to the trained vertebral body estimation model and the vertebral body determination process.

[0102] In the vertebral body 3D image data generation process, information indicating the positional relationship of each vertebral body cross-sectional image (hereinafter referred to as "cross-sectional positional relationship information") is used. The cross-sectional positional relationship information may be stored in advance in a predetermined storage device such as the storage unit 54 or storage unit 64, or it may be recorded in a predetermined storage device such as the storage unit 54 or storage unit 64 after the execution of the dimensional conversion process as one of the results of the execution of the dimensional conversion process.

[0103] Next, the vertebral body 3D image data generation process performs a 3D conversion process. The 3D conversion process generates 3D image data of the vertebral body based on the vertebral body cross-sectional image, cross-sectional positional relationship information, and the results of the vertebral body detection process. The cross-sectional positional relationship information is information that indicates the positional relationship of each vertebral body cross-sectional image. The cross-sectional positional relationship information may be stored in advance in a predetermined storage device such as the storage unit 54 or storage unit 64, or it may be obtained as one of the results of the execution of the dimensional conversion process.

[0104] The cross-sectional positional relationship information indicates the positional relationship of the vertebral cross-sectional images in three-dimensional space, and the position of each vertebra within the cross-sectional image is indicated as a result of the vertebral detection process. Therefore, the position of the vertebra in three-dimensional space is indicated by the cross-sectional positional relationship information and the results of the vertebral detection process.

[0105] Therefore, the 3D modeling process is a process that generates image data of a 3D image of a vertebra based on information indicating the position of the vertebra in 3D space, which is derived from the cross-sectional positional relationship information and the results of the vertebral body detection process (hereinafter referred to as "vertebral body position information in 3D space"). In the 3D modeling process, for example, interpolation may be performed based on the vertebral body position information in 3D space to generate image data of an even smoother 3D image.

[0106] Furthermore, in the vertebral body 3D image data generation process, various image processing steps, such as thresholding, may be performed between the vertebral body detection process and the 3D modeling process.

[0107] Up to this point, we have explained that vertebral cross-sectional images are obtained through dimensional transformation processing. However, the vertebral 3D image data generation process does not necessarily need to include dimensional transformation processing, as long as it can obtain 3D image data of the vertebrae. For example, if cross-sectional positional relationship information and multiple vertebral cross-sectional images are obtained in advance, the vertebral 3D image data generation process can be performed by executing a sample extraction process and a 3D transformation process, and 3D image data of the vertebrae can be obtained without performing dimensional transformation processing. In such cases, the vertebral cross-sectional images do not need to be obtained through dimensional transformation processing. Thus, the 3D image data of the vertebrae is image data generated using the results of a sample extraction process performed on multiple 2D images of vertebrae.

[0108] The vertebral body 3D image data generation process may be performed by the fixed-condition estimation model learning device 5 or by another device. Below, the vertebral body 3D image data generation process will be explained assuming that it is performed by a device other than the fixed-condition estimation model learning device 5. In other words, the fixed-condition estimation model learning device 5 will first be explained as a device that does not perform the vertebral body 3D image data generation process. Similarly, the device that runs the trained fixed-condition estimation model (i.e., the fixed-condition estimation device 6 described later) will also first be explained as a device that does not perform the vertebral body 3D image data generation process. The fixed-condition estimation model learning device 5 that does perform the vertebral body 3D image data generation process will be explained in a modified example. Similarly, the device that runs the trained fixed-condition estimation model will also be explained in a modified example when it performs the vertebral body 3D image data generation process.

[0109] <About training fixed-condition estimation models> In acquiring a mathematical model for estimating the size and placement of spinal internal fixation devices to be installed in the vertebrae of humans or animals, the fixation condition estimation model learning device 5 learns the fixation condition estimation model. The fixation condition estimation model is a mathematical model that estimates coordinate system information and spinal internal fixation device fixation conditions based on input 3D vertebral body image data. The coordinate system information is coordinate system information that indicates the position of each image in the image shown by the 3D vertebral body image data (hereinafter referred to as "3D vertebral body image"). The spinal internal fixation device fixation conditions are information that indicates the size and position of each spinal internal fixation device to be installed in each vertebra shown in the image of the 3D vertebral body image data.

[0110] Furthermore, 3D vertebral body image data may be obtained by performing a 3D vertebral body image data generation process on 3D CT (Computed Tomography) images, or by performing a 3D vertebral body image data generation process using 2D image data obtained using MRI (Magnetic Resonance Imaging) as vertebral body cross-sectional images. It may also be obtained by performing a 3D vertebral body image data generation process using X-ray image data obtained from radiographs as vertebral body cross-sectional images.

[0111] In training a fixed-condition estimation model, a set of data consisting of 3D vertebral body image data, ground truth coordinate system information, and ground truth spinal fixation device fixation conditions is used as training data. The ground truth coordinate system information is information about the coordinate system that shows the position of each image in the 3D vertebral body image data belonging to the same set. The ground truth spinal fixation device fixation conditions are information that shows the size and position of the spinal fixation device to be placed on each vertebra shown in the 3D vertebral body image data belonging to the same set. One training data set is used for each trial during training. The fixed-condition estimation model is updated after each trial during training.

[0112] Let's explain the training of the fixed-condition estimation model in more detail. In the training of the fixed-condition estimation model, the model is executed on 3D vertebral body image data included in the training data. During the training of the fixed-condition estimation model, the model is updated to minimize the coordinate system difference and the fixed condition difference. The coordinate system difference is the difference between the coordinate system information estimated by the execution of the fixed-condition estimation model and the coordinate system information indicated by the ground truth coordinate system information. The fixed condition difference is the difference between the spinal internal fixation device fixation conditions estimated by the execution of the fixed-condition estimation model and the ground truth spinal internal fixation device fixation conditions. In other words, the ground truth coordinate system information and the ground truth spinal internal fixation device fixation conditions are the ground truth data in the training of the fixed-condition estimation model. The loss function representing the difference may be, for example, cross-entropy or the squared error function.

[0113] <Regarding the coordinate system indicated by the correct coordinate system information> Here, an example of the coordinate system indicated by the ground truth coordinate system information will be explained using Figures 11 and 12. Figure 11 is the first diagram illustrating an example of the coordinate system indicated by the ground truth coordinate system information in the second embodiment. Figure 12 is the second diagram illustrating an example of the coordinate system indicated by the ground truth coordinate system information in the second embodiment. Image G3 in Figure 11 is an image of the contour of the cross-section on the axial plane of the vertebra. Image G5 in Figure 12 is an image with the background color of image G3 inverted.

[0114] The coordinate system indicated by the correct coordinate system information is a three-dimensional Cartesian coordinate system. The coordinate system indicated by the correct coordinate system information is a coordinate system in which each axis is defined by the following geometrical process.

[0115] <Process to define each axis of the coordinate system indicated by the correct coordinate system information> In the axial cross-section of a vertebra, the line whose endpoints are the center points of the narrowest parts of the left and right pedicles is defined as axis 1. The line that intersects axis 1 and connects the anterior edge of the vertebral body to the anterior edge of the vertebral foramen in the axial plane, and is approximately parallel to the superior edge of the vertebral body, is defined as axis 2. Axis 2 is the line segment J1 shown in image G4 of Figure 11 and image G6 of Figure 12. In the plane stretched by axes 1 and 2, the line segment that is perpendicular to axis 2 and includes one of the endpoints of axis 1 is defined as axis 3. The axis 2 thus defined is the Y-axis of the coordinate system shown in the correct coordinate system information, axis 3 is the X-axis of the coordinate system shown in the correct coordinate system information, and the axis perpendicular to axes 2 and 3 is the Z-axis. The origin of the coordinate system shown in the correct coordinate system information is the intersection of the X-axis and the Y-axis.

[0116] Figure 13 shows an example of the hardware configuration of the fixed-condition estimation model learning device 5 in the second embodiment. The fixed-condition estimation model learning device 5 includes a control unit 51 which has a processor 95 such as a CPU and a memory 96 connected by a bus, and executes a program. The fixed-condition estimation model learning device 5 functions as a device comprising the control unit 51, input unit 52, communication unit 53, storage unit 54 and output unit 55 by the execution of the program.

[0117] More specifically, the processor 95 reads the program stored in the memory unit 54 and stores the read program in the memory 96. By executing the program stored in the memory 96, the fixed-condition estimation model learning device 5 functions as a device comprising a control unit 51, an input unit 52, a communication unit 53, a memory unit 54, and an output unit 55.

[0118] The control unit 51 controls the operation of various functional units of the fixed-condition estimation model learning device 5. The control unit 51 executes the fixed-condition estimation model. The control unit 11 controls the operation of, for example, the output unit 55 and causes the output unit 55 to output the execution results of the fixed-condition estimation model. The control unit 51 records various information generated by the execution of the fixed-condition estimation model in the storage unit 54. The various information stored in the storage unit 54 includes, for example, the learning results of the fixed-condition estimation model.

[0119] The input unit 52 includes input devices such as a mouse, keyboard, or touch panel. The input unit 52 may also be configured as an interface connecting these input devices to the fixed-condition estimation model learning device 5. The input unit 52 receives various types of information for input to the fixed-condition estimation model learning device 5. For example, 3D image data of vertebral bodies may be input to the input unit 52.

[0120] The communication unit 53 is configured to include a communication interface for connecting the fixed-condition estimation model learning device 5 to an external device. The communication unit 53 communicates with the external device via wired or wireless connection. The external device is, for example, a device that transmits training data used for learning the fixed-condition estimation model. The communication unit 53 receives training data used for learning the fixed-condition estimation model by communicating with the device that transmits training data used for learning the fixed-condition estimation model. One of the devices that transmit training data used for learning the fixed-condition estimation model is a device that transmits 3D vertebral body image data. The external device is, for example, a device that runs a trained fixed-condition estimation model. The communication unit 53 transmits the trained fixed-condition estimation model to the device that runs the trained fixed-condition estimation model by communicating with the device.

[0121] The storage unit 54 is configured using a computer-readable storage medium such as a magnetic hard disk drive or a semiconductor storage device. The storage unit 54 stores various information related to the fixed-condition estimation model learning device 5. The storage unit 54 stores information input via, for example, the input unit 52 or the communication unit 53. The storage unit 54 pre-stores, for example, a fixed-condition estimation model. The storage unit 54 stores, for example, various information generated by the execution of the fixed-condition estimation model.

[0122] The output unit 55 outputs various types of information. The output unit 55 is comprised of a display device such as a CRT display, liquid crystal display, or organic EL display. The output unit 55 may be configured as an interface for connecting these display devices to the fixed-condition estimation model learning device 5. The output unit 55 outputs information input to, for example, the input unit 52 or the communication unit 53. The output unit 55 may display, for example, the execution results of the fixed-condition estimation model.

[0123] Figure 14 shows an example of the configuration of the control unit 51 of the fixed-condition estimation model learning device 5 in the second embodiment. The control unit 51 includes a training data acquisition unit 511, a learning unit 512, a memory control unit 513, and an output control unit 514.

[0124] The training data acquisition unit 511 acquires training data. The training data acquisition unit 511 acquires training data input to, for example, the input unit 52 or the communication unit 53. The training data acquisition unit 511 may also acquire training data by reading training data that has been previously stored in the storage unit 54. The training data acquired by the training data acquisition unit 511 is a set of 3D image data of the vertebral body, ground truth coordinate system information, and ground truth spinal internal fixation device fixation conditions.

[0125] The learning unit 512 performs training of a fixed-condition estimation model. The learning unit 512 includes a fixed-condition estimation model execution unit 521 and an update unit 522. The fixed-condition estimation model execution unit 521 performs the fixed-condition estimation model on the 3D vertebral body image data included in the training data acquired by the training data acquisition unit 511. By performing the fixed-condition estimation model, the fixed-condition estimation model execution unit 521 estimates the size and position of the spinal fixation device installed on each vertebra shown in the image of the 3D vertebral body image data being executed, as well as coordinate system information indicating the position of each image shown in the image.

[0126] The fixed-condition estimation model execution unit 521 is specifically a neural network that represents the fixed-condition estimation model. Therefore, the execution of the fixed-condition estimation model by the fixed-condition estimation model execution unit 521 means that the neural network representing the fixed-condition estimation model is operating.

[0127] The update unit 522 updates the fixed-condition estimation model based on the estimation results of the fixed-condition estimation model execution unit 521 and the ground truth data included in the training data, in order to reduce the difference between the estimation results and the ground truth data. The ground truth data is, for example, a set of image data of vertebrae in which spinal internal fixation devices are actually installed, the size and position of each installed spinal internal fixation device, and coordinate system information indicating the position of each image in the image.

[0128] The memory control unit 513 records various information in the memory unit 54. The output control unit 514 controls the operation of the output unit 55.

[0129] Figure 15 is a flowchart showing an example of the processing flow performed by the fixed-condition estimation model learning device 5 in the second embodiment. The training data acquisition unit 511 acquires training data (step S401). Next, the fixed-condition estimation model execution unit 521 executes the fixed-condition estimation model on the 3D vertebral body image data included in the training data acquired in step S401 (step S402). By executing the fixed-condition estimation model, the size and position of the spinal fixation device to be installed on each vertebra shown in the image of the 3D vertebral body image data to be executed, and coordinate system information indicating the position of each image shown in the image are estimated.

[0130] Next, the update unit 522 updates the fixed-condition estimation model based on the estimation results obtained from step S402 and the ground truth data included in the training data acquired in step S401 (step S403). Next, the update unit 522 determines whether or not the learning termination condition has been met (step S404). If the learning termination condition is met (step S404: YES), the process ends. On the other hand, if the learning termination condition is not met (step S404: NO), the process returns to step S401.

[0131] The trained fixed-condition estimation model obtained in this way is used to estimate the size and position of the spinal fixation devices to be placed on each vertebra in the input image data, and the coordinate system information indicating the position of each image in that image. An example of a device that performs such processing is the fixed-condition estimation device 6, which will be described below. The fixed-condition estimation device 6 acquires the trained fixed-condition estimation model in advance before execution, for example by obtaining it from the fixed-condition estimation model learning device 5 via communication. The fixed-condition estimation device 6 may also acquire the trained fixed-condition estimation model in advance before execution, for example by equipping it with a neural network that represents the trained fixed-condition estimation model.

[0132] The fixed-condition estimation device 6 uses the trained fixed-condition estimation model obtained by the fixed-condition estimation model learning device 5 to estimate the size and position of each spinal fixation device installed on each vertebra in the image data to be estimated, as well as coordinate system information indicating the position of each image in that image.

[0133] Figure 16 shows an example of the hardware configuration of the fixed-condition estimation device 6 in the second embodiment. The fixed-condition estimation device 6 includes a control unit 61 which has a processor 97 such as a CPU and a memory 98 connected by a bus, and executes a program. The fixed-condition estimation device 6 functions as a device comprising a control unit 61, an input unit 62, a communication unit 63, a storage unit 64, and an output unit 65 through the execution of the program.

[0134] More specifically, the processor 97 reads the program stored in the storage unit 64 and stores the read program in the memory 98. By executing the program stored in the memory 98, the fixed condition estimation device 6 functions as a device comprising a control unit 61, an input unit 62, a communication unit 63, a storage unit 64, and an output unit 65.

[0135] The control unit 61 controls the operation of various functional units of the fixed-condition estimation device 6. The control unit 61 executes the trained fixed-condition estimation model. The control unit 61 controls the operation of the output unit 65, for example, and causes the output unit 65 to output the execution result of the trained fixed-condition estimation model. The control unit 61 records various information generated by the execution of the trained fixed-condition estimation model in the storage unit 64. The various information stored in the storage unit 64 includes, for example, the execution result of the trained fixed-condition estimation model.

[0136] The input unit 62 includes input devices such as a mouse, keyboard, or touch panel. The input unit 62 may also be configured as an interface for connecting these input devices to the fixed-condition estimation device 6. The input unit 62 receives various types of information for input to the fixed-condition estimation device 6. For example, the input unit 62 receives 3D image data of the vertebra to be estimated.

[0137] The communication unit 63 is configured to include a communication interface for connecting the fixed-condition estimation device 6 to an external device. The communication unit 63 communicates with the external device via wired or wireless connection. The external device is, for example, the device that transmits the 3D image data of the vertebra to be estimated. The communication unit 63 may receive the 3D image data of the vertebra to be estimated from the device that transmits the 3D image data of the vertebra to be estimated by communicating with the device. The external device is, for example, the fixed-condition estimation model learning device 5. The communication unit 63 may receive the trained vertebra estimation model from the fixed-condition estimation model learning device 5 by communicating with the device.

[0138] The storage unit 64 is configured using a computer-readable storage medium such as a magnetic hard disk drive or a semiconductor storage device. The storage unit 64 stores various information related to the fixed condition estimation device 6. The storage unit 64 stores information input via, for example, the input unit 62 or the communication unit 63. The storage unit 64 stores, for example, a trained fixed condition estimation model in advance before execution of the trained fixed condition estimation model. The storage unit 64 stores, for example, various information generated by the execution of the trained fixed condition estimation model.

[0139] The output unit 65 outputs various types of information. The output unit 65 is comprised of a display device such as a CRT display, liquid crystal display, or organic EL display. The output unit 65 may be configured as an interface for connecting these display devices to the fixed-condition estimation device 6. The output unit 65 outputs information input to, for example, the input unit 62 or the communication unit 63. The output unit 65 may also output, for example, the execution results of a trained fixed-condition estimation model.

[0140] Figure 17 shows an example of the configuration of the control unit 61 of the fixed condition estimation device 6 in the second embodiment. The control unit 61 comprises a target data acquisition unit 611, a fixed condition estimation unit 612, a storage control unit 613, and an output control unit 614.

[0141] The target data acquisition unit 611 acquires 3D image data of vertebrae that are the target of the trained fixed-condition estimation model. The 3D image data of vertebrae acquired by the target data acquisition unit 611 is the image data of the 3D image to be estimated. The target data acquisition unit 611 acquires 3D image data of vertebrae input, for example via the input unit 62 or the communication unit 63, as the 3D image data of vertebrae that are the target of the trained fixed-condition estimation model.

[0142] The fixed condition estimation unit 612 executes a trained fixed condition estimation model on the 3D vertebral body image data acquired by the target data acquisition unit 611. By executing the fixed condition estimation model, the fixed condition estimation unit 612 estimates the size and position of the spinal internal fixation device installed on each vertebra in the image data of the target data, as well as coordinate system information indicating the position of each image in the image.

[0143] The memory control unit 613 records various types of information in the memory unit 64. The output control unit 614 controls the operation of the output unit 65.

[0144] Figure 18 is a flowchart showing an example of the processing flow performed by the fixed-condition estimation device 6 in the second embodiment. The target data acquisition unit 611 acquires image data of the image to be estimated (step S501). Next, the fixed-condition estimation unit 612 executes the trained fixed-condition estimation model on the image data acquired by the target data acquisition unit 611 (step S502). By executing the trained fixed-condition estimation model, the size and position of each spinal fixation device to be installed on each vertebra shown in the image of the target image data are estimated. In addition, by executing the trained fixed-condition estimation model, coordinate system information indicating the position of each image shown in the image of the target image data is also estimated.

[0145] Next, the output control unit 614 controls the operation of the output unit 65 to output the estimation result obtained in step S502 (step S503). The output method is, for example, display.

[0146] The fixed-condition estimation model learning device 5 of the second embodiment, configured in this way, obtains a mathematical model for estimating the size and placement of the spinal internal fixation device. Determining the size and placement of the spinal internal fixation device is one of the matters to be determined when deciding on a treatment plan for physical abnormalities related to the vertebrae. Therefore, the fixed-condition estimation model learning device 5, which obtains a mathematical model for estimating the size and placement of the spinal internal fixation device through learning, can reduce the burden required to determine a treatment plan when treating physical abnormalities related to the vertebrae.

[0147] The fixed condition estimation device 6 of the second embodiment, configured in this way, determines the size and placement of the spinal internal fixation device using the fixed condition estimation model learned by the fixed condition estimation model learning device 5. Therefore, the fixed condition estimation device 6 can reduce the burden required to determine the treatment plan when treating physical abnormalities related to the vertebral body.

[0148] (First modified example of the second embodiment) The fixed-condition estimation model learning device 5 may perform vertebral body 3D image data generation processing. When the fixed-condition estimation model learning device 5 performs vertebral body 3D image data generation processing, the input unit 52 or the communication unit 53 receives image data to be processed for vertebral body 3D image data generation processing instead of vertebral body 3D image data. The image data to be processed for vertebral body 3D image data generation processing depends on the content of the vertebral body 3D image data generation processing performed by the fixed-condition estimation model learning device 5.

[0149] If the vertebral body 3D image data generation process performed by the fixed-condition estimation model learning device 5 includes dimensional transformation processing, the image data targeted for the vertebral body 3D image data generation process is a 3D image obtained by imaging such as a 3D CT scan. If the vertebral body 3D image data generation process performed by the fixed-condition estimation model learning device 5 does not include dimensional transformation processing, the image data targeted for the vertebral body 3D image data generation process is multiple cross-sectional images of vertebral bodies.

[0150] Figure 19 shows an example of the configuration of the control unit 51a of the fixed-condition estimation model learning device 5 in the first modified example of the second embodiment. For the sake of simplicity in the following explanation, components having the same functions as the control unit 51 will be given the same reference numerals as in Figure 14 and their explanation will be omitted. The control unit 51a differs from the control unit 51 of the second embodiment in that it includes a training data acquisition unit 511a instead of a training data acquisition unit 511.

[0151] The training data acquisition unit 511a acquires the correct coordinate system information and the correct spinal internal fixation device fixation conditions input to the input unit 52 or the communication unit 53. The training data acquisition unit 511a performs vertebral body 3D image data generation processing on the image data to be processed, which is input to the input unit 52 or the communication unit 53. The training data acquisition unit 511a acquires vertebral body 3D image data by executing the vertebral body 3D image data generation processing. In this way, the training data acquisition unit 511a acquires training data.

[0152] (Second modified example of the second embodiment) The fixed-condition estimation device 6 may perform vertebral body 3D image data generation processing. When the fixed-condition estimation device 6 performs vertebral body 3D image data generation processing, the input unit 62 or the communication unit 63 receives image data to be processed for vertebral body 3D image data generation processing instead of vertebral body 3D image data. The image data to be processed for vertebral body 3D image data generation processing depends on the content of the vertebral body 3D image data generation processing performed by the fixed-condition estimation device 6.

[0153] If the vertebral body 3D image data generation process performed by the fixed-condition estimation device 6 includes a dimensional transformation process, the image data targeted for the vertebral body 3D image data generation process is a 3D image obtained by imaging such as a 3D CT scan. If the vertebral body 3D image data generation process performed by the fixed-condition estimation device 6 does not include a dimensional transformation process, the image data targeted for the vertebral body 3D image data generation process is multiple cross-sectional images of vertebral bodies.

[0154] Figure 20 shows an example of the configuration of the control unit 61a of the fixed condition estimation device 6 in a second modified example of the second embodiment. For the sake of simplicity in the following explanation, components having the same functions as the control unit 61 are given the same reference numerals as in Figure 17 and their explanation is omitted. The control unit 61a differs from the control unit 61 of the second embodiment in that it includes a target data acquisition unit 611a instead of a target data acquisition unit 611.

[0155] The target data acquisition unit 611a executes the vertebral body 3D image data generation process on the image data that is input to the input unit 62 or the communication unit 63 and is the target of the vertebral body 3D image data generation process. The target data acquisition unit 611a acquires vertebral body 3D image data by executing the vertebral body 3D image data generation process.

[0156] (Third modified example of the second embodiment) Furthermore, the fixed-condition estimation model learning device 5 and the fixed-condition estimation device 6 may perform processing on 2D image data. Therefore, the fixed-condition estimation model learning device 5 and the fixed-condition estimation device 6 may perform processing on N-dimensional image data (where N is 2 or 3). That is, a 3D vertebral body image may be replaced with an N-dimensional vertebral body image, and a 3D vertebral body image data may be replaced with N-dimensional vertebral body image data. In such cases, the ground truth coordinate system information used for training the fixed-condition estimation model is an orthogonal N-dimensional coordinate system.

[0157] (Fourth modified example of the second embodiment) The fixed-condition estimation model learning device 5 may be implemented using multiple information processing devices connected to each other via a network. In this case, each functional unit of the fixed-condition estimation model learning device 5 may be distributed and implemented across multiple information processing devices.

[0158] The fixed-condition estimation device 6 may be implemented using multiple information processing devices connected to each other via a network. In this case, each functional unit of the fixed-condition estimation device 6 may be distributed and implemented across multiple information processing devices.

[0159] Furthermore, the fixed-condition estimation model learning device 5 and the fixed-condition estimation device 6 do not necessarily need to be implemented as separate devices. For example, the fixed-condition estimation model learning device 5 and the fixed-condition estimation device 6 may be implemented as a single device that combines the functions of both.

[0160] (Other variations) Furthermore, all or part of the functions of the vertebral body estimation model learning device 1, the vertebral body estimation device 2, the fixed condition estimation model learning device 5, and the fixed condition estimation device 6 may be implemented using hardware such as ASICs (Application Specific Integrated Circuits), PLDs (Programmable Logic Devices), or FPGAs (Field Programmable Gate Arrays). The program may be recorded on a computer-readable recording medium. Computer-readable recording media include, for example, portable media such as flexible disks, magneto-optical disks, ROMs, and CD-ROMs, and storage devices such as hard disks built into computer systems. The program may also be transmitted via telecommunications lines.

[0161] Each embodiment may be combined.

[0162] Note that the 2D vertebral body image and the 3D vertebral body image are examples of vertebral body images, respectively.

[0163] Although embodiments of this invention have been described in detail above with reference to the drawings, the specific configuration is not limited to these embodiments and includes designs and the like that do not depart from the spirit of this invention.

[0164] (Note 1) A fixing condition estimation model execution unit executes a mathematical model that estimates coordinate system information indicating the position of each image in the vertebral body image, and the size and installation position of the spinal internal fixation device to be installed on the vertebral body, based on image data of a vertebral body image which is an image of the vertebral body. An update unit updates the fixed-condition estimation model based on the estimation results by the fixed-condition estimation model execution unit, A fixed-condition estimation model learning device equipped with the following features.

[0165] (Note 2) The aforementioned vertebral body image is a three-dimensional image showing the vertebral body. The image data of the vertebral body is image data obtained using the results obtained by running the updated vertebral body estimation model on a plurality of 2D vertebral body images until a predetermined termination condition is met. The vertebral body estimation model is a mathematical model that estimates the position of each vertex at the corner of the vertebral body and the vector for each vertex, which is a vector from the vertex to the center of the vertebral body, based on the image data of a 2D vertebral body image, which is a 2D image of the vertebral body. The vertebral body estimation model execution unit estimates the position of the vertex at the corner of the vertebral body and the vector for each vertex as seen in the image data of the 2D vertebral body image to be estimated. The update unit updates the vertebral body estimation model based on the estimation results from the vertebral body estimation model execution unit. A fixed-condition estimation model learning device as described in Appendix 1.

[0166] (Note 3) A target data acquisition unit acquires image data of vertebral bodies, which are images showing the vertebral bodies. A fixed condition estimation model learning device comprises: a fixed condition estimation model execution unit that estimates coordinate system information indicating the position of each image in the vertebral body image based on image data of the vertebral body image, and the size and installation position of the spinal internal fixation device to be installed on the vertebral body, by executing a fixed condition estimation model, which is a mathematical model that estimates coordinate system information indicating the position of each image in the image data of the vertebral body image to be estimated, and the size and installation position of the spinal internal fixation device to be installed on the vertebral body; and an update unit that updates the fixed condition estimation model based on the estimation results by the fixed condition estimation model execution unit. The device then uses the updated fixed condition estimation model until a predetermined termination condition is met, and based on the image data acquired by the target data acquisition unit, it estimates coordinate system information indicating the position of each image in the image data, and the size and installation position of the spinal internal fixation device to be installed on the vertebral body image. A fixed-condition estimation device equipped with the following features.

[0167] (Note 4) The fixation condition estimation model is a mathematical model that estimates coordinate system information indicating the position of each image in the vertebral body image, and the size and position of the spinal internal fixation device to be installed in the vertebral body, based on image data of the vertebral body image which is an image of the vertebral body. This is performed as a fixation condition estimation model execution step, which estimates coordinate system information indicating the position of each image in the vertebral body image to be estimated, and the size and position of the spinal internal fixation device to be installed in the vertebral body. An update step to update the fixed-condition estimation model based on the estimation results from the fixed-condition estimation model execution step, A fixed-condition estimation model learning method having the following characteristics.

[0168] (Note 5) A program to enable a computer to function as a learning device for the fixed-condition estimation model described above.

[0169] (Note 6) A program to make the computer function as the fixed-condition estimation device described above. [Explanation of Symbols]

[0170] 1…Vertebral body estimation model learning device, 11…Control unit, 12…Input unit, 13…Communication unit, 14…Storage unit, 15…Output unit, 111…Training data acquisition unit, 112…Learning unit, 113…Storage control unit, 114…Output control unit, 121…Vertebral body estimation model execution unit, 122…Update unit, 2…Vertebral body estimation device, 21, 21a, 21b…Control unit, 22…Input unit, 23…Communication unit, 24…Storage unit, 25…Output unit, 211…Target data acquisition unit, 212…Vertebral body estimation unit, 213…Storage control unit, 214…Output control unit, 215…Vertebral body determination unit, 216…Cobb angle calculation unit, 5…Fixed condition estimation model learning device, 51, 51a…Control unit, 52…Input unit, 53…Communication unit, 54…Storage unit, 55…Output unit, 511, 511a…Training data acquisition unit, 512…Learning unit, 513…Memory control unit, 514…Output control unit, 521…Fixed condition estimation model execution unit, 522…Update unit, 61, 61a…Control unit, 62…Input unit, 63…Communication unit, 64…Memory unit, 65…Output unit, 611, 611a…Target data acquisition unit, 612…Fixed condition estimation unit, 613…Memory control unit, 614…Output control unit, 91…Processor, 92…Memory, 93…Processor, 94…Memory, 95…Processor, 96…Memory, 97…Processor, 98…Memory

Claims

1. A vertebral body estimation model execution unit estimates the position of the vertebral body's vertebrae, the position of the vertebral body's center, and the vector for each vertex, which is a vector from the vertex to the center of the vertebral body, based on image data of a 2D image of a vertebral body, which is a 2D image of a vertebral body, by executing a vertebral body estimation model, which is a mathematical model that estimates the position of each vertex of the vertebral body, the position of the vertebral body's center, and the vector for each vertex, as seen in the 2D image of the vertebral body to be estimated. An update unit updates the vertebral body estimation model based on the estimation results by the vertebral body estimation model execution unit and training data which consists of pairs of correct values ​​for the position of each vertex of the vertebral body, the position of the center, and the vector from the vertex to the center of the vertebral body. A vertebral body estimation model learning device equipped with the following features.

2. The set of image data of the two-dimensional vertebral body to be estimated includes image data of the two-dimensional vertebral body in which one or more of the following are reversed: color, vertical orientation, and horizontal orientation. The vertebral body estimation model learning device according to claim 1.

3. The aforementioned two-dimensional vertebral body image data is obtained using an ROI model, which is a mathematical model that estimates the region of interest within the image shown by the two-dimensional image data being executed, and is obtained by learning using pairs of two-dimensional image data and ground truth data indicating the region of interest. The vertebral body estimation model learning device according to claim 1 or 2.

4. The region of interest in the training of the ROI model is a randomly selected region of the two-dimensional image data targeted by the ROI model, satisfying the condition that it captures an image of a vertebral body. The vertebral body estimation model learning device according to claim 3.

5. A data acquisition unit acquires image data of a 2D vertebral body image, which is a 2D image of the vertebral body, A vertebral body estimation model learning device comprising: a vertebral body estimation model execution unit that estimates the position of each vertex of a vertebra, the position of the center of a vertebra, and a vector for each vertex that points from the vertex to the center of the vertebra, based on image data of a 2D image of a vertebral body which is a two-dimensional image of a vertebral body which is a two-dimensional image of a vertebral body which is a two-dimensional image of a vertebral body which is the target of estimation; an update unit that updates the vertebral body estimation model based on the results of the estimation by the vertebral body estimation model execution unit and training data which is pairs of correct values ​​of the position of each vertex of a vertebra, the position of the center of a vertebra, and the vector for each vertex, as seen in the 2D image of a vertebral body which is the target of estimation; and a vertebral body estimation unit that uses the updated vertebral body estimation model until a predetermined termination condition is met, based on the image data acquired by the target data acquisition unit, to estimate the position of each vertex of a vertebra, the position of the center of a vertebra, and the vector for each vertex as seen in the image shown by the image data. A vertebral body estimation device equipped with the following features.

6. The vertebral body estimation unit groups the positions of each vertex of the vertebral body and the vectors for each vertex, which are the results of estimation using the vertebral body estimation model that has been updated by the vertebral body estimation model learning device until a predetermined termination condition is met, for each vertebral body. The vertebral body estimation device according to claim 5.

7. Based on the estimation results of the vertebral body estimation unit, a Cobb angle calculation unit calculates the Cobb angle. The vertebral body estimation device according to claim 6, comprising:

8. The image data of the vertebral body 2D image acquired by the target data acquisition unit is image data obtained using an ROI model, which is a mathematical model that estimates the region of interest within the image shown by the 2D image data of the target, and is obtained by learning using pairs of 2D image data and ground truth data indicating the region of interest. The vertebral body estimation device according to any one of claims 5 to 7.

9. The region of interest in the training of the ROI model is a randomly selected region of the two-dimensional image data targeted by the ROI model, satisfying the condition that it captures an image of a vertebral body. The vertebral body estimation device according to claim 8.

10. A target data acquisition unit acquires image data of vertebral bodies, which are images showing the vertebral bodies. A fixed-condition estimation model learning device comprises: a fixed-condition estimation model execution unit that estimates coordinate system information indicating the position of each image in the vertebral body image data and the size and installation position of the spinal internal fixation device to be installed on the vertebral body, by executing a fixed-condition estimation model, which is a mathematical model that estimates coordinate system information indicating the position of each image in the vertebral body image data and the size and installation position of the spinal internal fixation device to be installed on the vertebral body, based on image data of the vertebral body image data to be estimated; and an update unit that updates the fixed-condition estimation model based on the results of the estimation by the fixed-condition estimation model execution unit and training data which is pairs of correct values ​​for coordinate system information indicating the position of each image in the vertebral body image data and the size and installation position of the spinal internal fixation device to be installed on the vertebral body, based on the image data acquired by the target data acquisition unit, using the fixed-condition estimation model updated until a predetermined termination condition is met, and a fixed-condition estimation unit that estimates coordinate system information indicating the position of each image in the image data and the size and installation position of the spinal internal fixation device to be installed on the vertebral body as seen in the image data, based on the image data acquired by the target data acquisition unit. Equipped with, The aforementioned vertebral body image is a three-dimensional image showing the vertebral body. The image data of the vertebral body is image data obtained using the results obtained by running the vertebral body estimation model, which is a mathematical model that estimates the position of each vertex of the vertebral body, the position of the center of the vertebral body, and the vector for each vertex that points from the vertex to the center of the vertebral body, based on the image data of the 2D vertebral body image, which is a two-dimensional image of the vertebral body, and estimating the position of each vertex of the vertebral body, the position of the center of the vertebral body, and the vector for each vertex as seen in the 2D vertebral body image to be estimated; and updating the vertebral body estimation model based on the results of the estimation by the vertebral body estimation model execution unit and training data which is pairs of correct values ​​for the position of each vertex of the vertebral body, the position of the center of the vertebral body, and the vector pointing from the vertex to the center of the vertebral body, until a predetermined termination condition is met, on a plurality of 2D vertebral body images. Fixed condition estimator.

11. A vertebral body estimation model is executed, which is a mathematical model that estimates the position of each vertex of a vertebral body, the position of the center of a vertebral body, and the vector for each vertex that points from the vertex to the center of the vertebral body, based on image data of a 2D image of a vertebral body, which is a 2D image of a vertebral body, thereby estimating the position of the vertex of a vertebral body, the position of the center of a vertebral body, and the vector for each vertex as seen in the image data of the 2D image of a vertebral body to be estimated. An update step to update the vertebral body estimation model based on the results of the estimation performed in the vertebral body estimation model execution step and training data which consists of pairs of correct values ​​for the positions of each vertex of the vertebral body, the position of the center of the vertebral body, and the vectors pointing from the vertices to the center of the vertebral body. A vertebral body estimation model learning method having the following characteristics.

12. The target data acquisition step involves acquiring image data of a 2D image of a vertebral body, which is a 2D image showing the vertebral body, A vertebral body estimation model learning device comprises: a vertebral body estimation model execution unit that estimates the position of each vertex of a vertebra, the position of the center of a vertebra, and a vector for each vertex that points from the vertex to the center of the vertebra, based on image data of a 2D image of a vertebral body which is a two-dimensional image of a vertebral body which is a two-dimensional image of a vertebral body which is a vertebral body estimation model execution unit that estimates the position of each vertex of a vertebra, the position of the center of a vertebra, and the vector for each vertex as seen in the image data of a 2D image of a vertebral body to be estimated; and an update unit that updates the vertebral body estimation model based on the results of the estimation by the vertebral body estimation model execution unit and training data which is pairs of correct values ​​for the position of each vertex of a vertebra, the position of the center of a vertebra, and the vector pointing from the vertex to the center of the vertebra, using the vertebral body estimation model that has been updated until a predetermined termination condition is met, based on the image data acquired in the target data acquisition step, the vertebral body estimation step that estimates the position of each vertex of a vertebra as seen in the image shown by the image data, and the vector for each vertex, by executing a vertebral body estimation model, which is a mathematical model that estimates the position of each vertex of a vertebra, the position of the center of a vertebra, and a vector for each vertex that points from the vertex to the center of the vertebra, A method for estimating vertebral bodies.

13. The target data acquisition step involves acquiring image data of a vertebral body, which is a three-dimensional image of the vertebral body, and A fixed-condition estimation model learning device comprises: a fixed-condition estimation model execution unit that estimates coordinate system information indicating each position of an image in a vertebral body image based on image data of a vertebral body image, and the size and installation position of an internal spinal fixation device to be installed on the vertebral body, by executing a fixed-condition estimation model execution unit that estimates coordinate system information indicating each position of an image in image data of a vertebral body image to be estimated, and the size and installation position of an internal spinal fixation device to be installed on the vertebral body; and an update unit that updates the fixed-condition estimation model based on the results of the estimation by the fixed-condition estimation model execution unit and training data which is a pair of correct values ​​for coordinate system information indicating each position of an image in image data of a vertebral body image and the size and installation position of an internal spinal fixation device to be installed on the vertebral body; and a fixed-condition estimation step that estimates coordinate system information indicating each position of an image in image data, and the size and installation position of an internal spinal fixation device to be installed on the vertebral body, based on image data acquired in the target data acquisition step, using the fixed-condition estimation model that has been updated until a predetermined termination condition is met. It has, The image data acquired in the aforementioned target data acquisition step is image data obtained using the results obtained by running the vertebral body estimation model, which is a mathematical model that estimates the position of each vertex of a vertebra, the position of the center of a vertebra, and a vector for each vertex that points from the vertex to the center of the vertebra, based on the image data of a 2D image of a vertebra, which is a two-dimensional image of a vertebra, to estimate the position of each vertex of a vertebra, the position of the center of a vertebra, and the vector for each vertex as seen in the 2D image of a vertebra to be estimated; and an update unit that updates the vertebral body estimation model based on the results of the estimation by the vertebral body estimation model execution unit and training data which is pairs of correct values ​​for the position of each vertex of a vertebra, the position of the center of a vertebra, and the vector pointing from the vertex to the center of the vertebra, until a predetermined termination condition is met. Fixed condition estimation method.

14. A program for causing a computer to function as a vertebral body estimation model learning device according to claim 1 or 2.

15. A program for causing a computer to function as a vertebral body estimation device according to any one of claims 5 to 7.

Images