Image processing device, program, and image processing method

Abstract

An image processing device and the like that can efficiently determine the direction of positioning correction and the like is provided. [Solution] The radiography control device 2 includes a first acquisition unit that acquires a two-dimensional radiographic image, an estimation unit that performs image analysis of the two-dimensional radiographic image acquired by the first acquisition unit and estimates three-dimensional structural information of structures inside the subject, and an output unit that outputs the three-dimensional structural information estimated by the estimation unit. The estimation unit estimates information about the position in the radiation irradiation direction as the three-dimensional structural information inside the subject.

Description

[Technical Field]

[0001] The present invention relates to an image processing device, a program, and an image processing method. [Background technology]

[0002] In radiography using radiation such as X-rays performed in hospitals and other medical facilities, if the patient's posture is not appropriate for imaging, the positioning will be off. In this case, the radiological technologist, etc., will look at the previously captured radiological image to grasp the three-dimensional positional relationship of the internal structures of the subject and guide the patient to the appropriate positioning.

[0003] The following documents are cited as examples of technologies for capturing radiographic images using three-dimensional information. Patent Document 1 describes a method for detecting a body region based on an image of a patient captured by a 3D scanner or the like, and automatically displaying on a screen a scanning range that selectively covers the detected body region. Patent Document 2 describes an X-ray diagnostic device that calculates the arc-shaped trajectory of a blood vessel of interest based on standard three-dimensional model data related to blood vessels in any location, and supports optimization of the imaging angle. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent No. 4484462 [Patent Document 2] Patent No. 4709600 Summary of the Invention [Problem to be solved by the invention]

[0005] Conventional techniques can assist in positioning the scanning range and optimizing the imaging angle relative to blood vessels. However, conventional techniques cannot present the radiologist with the three-dimensional positional relationship of structures inside the subject in the radiographic image. Therefore, there is no information that can serve as a basis for correcting the positioning, which makes it difficult to efficiently determine the direction in which to correct the positioning.

[0006] SUMMARY OF THE INVENTION It is therefore an object of the present invention to provide an image processing device, a program, and an image processing method that can efficiently determine the direction of positioning correction, etc. [Means for solving the problem]

[0007] The image processing device according to the present invention comprises: a first acquisition unit that acquires a two-dimensional radiation image; an estimation unit that performs image analysis on the acquired two-dimensional radiographic image to estimate three-dimensional structural information of a structure inside the subject; an output unit that outputs the second shooting support information; Equipped with.

[0008] The program according to the present invention comprises: Computer, a first acquisition unit for acquiring a two-dimensional radiation image; an estimation unit that performs image analysis on the acquired two-dimensional radiographic image to estimate three-dimensional structural information of the structure inside the subject; Function as.

[0009] The image processing method according to the present invention comprises: an acquisition step of acquiring a two-dimensional radiographic image; an estimation step of estimating three-dimensional structural information of a structure inside the subject by image analysis of the acquired two-dimensional radiographic image; It has. [Effects of the Invention]

[0010] According to the present invention, three-dimensional structural information of structures inside a subject in a radiological image is estimated, so that the direction of positioning correction and the like can be efficiently determined. [Brief explanation of the drawings]

[0011] [Figure 1] 1 is a diagram showing a schematic configuration of a photography support system according to an embodiment of the present invention. [Figure 2] FIG. 1 is a block diagram of an imaging control device according to the present embodiment. [Figure 3] 10 is a diagram showing an example of the configuration of a shooting support information output table stored in a storage unit according to the present embodiment. FIG. [Figure 4A] FIG. 10 is a diagram for explaining the movements of external rotation and internal rotation. [Figure 4B] FIG. 10 is a diagram illustrating the movements of abduction and adduction. [Figure 5] FIG. 10 is a diagram showing an example of a conventional analysis result of positioning deviation in a radiological image of the "lateral surface of the knee (right knee) joint." [Figure 6A] FIG. 1 is a diagram illustrating an example of input data used when training a machine learning model according to an embodiment of the present invention. [Figure 6B] FIG. 1 is a diagram illustrating an example of correct answer data used when training a machine learning model according to this embodiment. [Figure 7] FIG. 10 is a diagram showing an example of other correct answer data used when training the machine learning model according to the present embodiment. [Figure 8] 6 is a flowchart showing an example of the operation of the imaging control device when capturing a radiographic image of a subject according to the present embodiment. [Figure 9] FIG. 4 is a diagram showing an example of the configuration of a shooting screen displayed on a display unit according to the present embodiment. [Figure 10] 10 is a flowchart showing an example of the operation of the imaging control device during re-photographing determination processing according to the present embodiment. [Figure 11A] FIG. 10 is a diagram showing an example of three-dimensional structural information of a radiological image estimated by the machine learning model according to the present embodiment. [Figure 11B]FIG. 10 is a diagram showing an example of three-dimensional structural information of a radiological image estimated by the machine learning model according to the present embodiment. [Figure 12] 6 is a flowchart showing an example of a first determination process according to the present embodiment. [Figure 13] 10 is a flowchart showing an example of a second determination process according to the present embodiment. [Figure 14] FIG. 10 is a diagram showing an example of shooting support information displayed on a shooting screen according to the present embodiment. [Figure 15] FIG. 10 is a diagram showing an example of a photographing screen in the case where character information is displayed in addition to each line information as three-dimensional structure information. [Figure 16] FIG. 10 is a diagram showing an example of an imaging screen when only medial condyle line information is displayed as three-dimensional structural information. [Figure 17] FIG. 10 is a diagram showing a schematic configuration of a photography support system according to another embodiment 1. DETAILED DESCRIPTION OF THE INVENTION

[0012] Preferred embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings.

[0013] [Configuration example of the photography support system 10A] FIG. 1 is a diagram showing a schematic configuration of an imaging support system 10A according to this embodiment. The imaging support system 10A includes a radiographic imaging device 1, an imaging control device 2, a radiation generating device 3, an image management device 4, and an HIS / RIS 5. Hereinafter, the radiographic imaging device may be referred to as an imaging device 1, and the radiation generating device may be referred to as a generator 3. HIS is an abbreviation for Hospital Information System. RIS is an abbreviation for Radiology Information System, and is a radiology information system. A radiographic image is an example of a medical image. The imaging control device 2 is an example of an image processing device.

[0014] The photographing device 1, photographing control device 2, generating device 3, image management device 4, and HIS / RIS 5 are connected to each other so as to be able to communicate with each other via a network N. Examples of the network N include a LAN, a WAN, and the Internet. LAN is an abbreviation for Local Area Network. WAN is an abbreviation for Wide Area Network. The communication method of the network N may be wired communication or wireless communication. Examples of wireless communication include Wi-Fi (registered trademark).

[0015] The generating device 3 includes a generator 31, a switch 32, and a radiation source 33. The generator 31 applies a voltage according to preset imaging conditions to the radiation source 33, which includes, for example, a tube, based on the operation of the switch 32. The generator 31 may have an operation unit that accepts input of irradiation conditions, etc.

[0016] When a voltage is applied from the generator 31, the radiation source 33 generates radiation R at a dose corresponding to the applied voltage. The radiation R is, for example, an X-ray.

[0017] The generator 3 generates radiation R in a manner corresponding to the type of radiological image, for example, a still image or a dynamic image. Specifically, in the case of a still image, the generator 3 irradiates radiation R only once per pressing of the switch 32. In the case of a dynamic image, the generator 3 irradiates pulsed radiation R repeatedly multiple times per predetermined time per pressing of the switch 32, for example.

[0018] The imaging device 1 generates digital image data showing the imaging region of the subject S. For example, a portable FPD is used as the imaging device 1. FPD is an abbreviation for Flat Panel Detector. The imaging device 1 may be configured integrally with the generating device 3.

[0019] Although not shown, the imaging device 1 includes, for example, an imaging element, a sensor substrate, a scanning unit, a readout unit, a control unit, and a communication unit. The imaging element generates an electric charge according to the radiation dose when exposed to radiation R. The sensor substrate has switch elements arranged two-dimensionally (in a matrix) that accumulate and release electric charge. The scanning unit switches each switch element on / off. The readout unit reads out the amount of electric charge released from each pixel as a signal value. The control unit generates image data of the radiation image from the multiple signal values ​​read out by the readout unit. The image data includes still image data or dynamic image data. The communication unit transmits the generated image data and various signals to other devices such as the imaging control device 2, and receives various information and signals from other devices.

[0020] The imaging control device 2 sets imaging conditions for the imaging device 1 and the generator 3, etc., and controls the reading operation of the radiographic image captured by the imaging device 1. The imaging control device 2 is also called a console and is configured, for example, by a personal computer. The imaging control device 2 determines whether re-imaging is necessary based on a positioning error in the radiographic image obtained by imaging. Here, positioning refers, for example, to the way the patient is positioned during imaging. If the imaging control device 2 determines that re-imaging is necessary, it displays imaging support information I and three-dimensional structure information T on the screen of the display unit 22, which will be described later. The imaging support information I assists in correcting the positioning by presenting, for example, the direction in which the positioning should be corrected using words or sentences. The three-dimensional structure information T is information indicating the three-dimensional positional relationship of structures inside the subject in the radiographic image, and provides information that serves as a basis for correcting the positioning.

[0021] The imaging conditions include, for example, patient conditions related to the subject S, irradiation conditions related to the irradiation of radiation R, and image reading conditions related to the image reading of the imaging device 1. The patient conditions include, for example, the imaging region, imaging direction, and physique. The irradiation conditions include, for example, tube voltage (kV), tube current (mA), irradiation time (ms), and current-time product (mAs value). The image reading conditions include, for example, pixel size, image size, and frame rate. The imaging control device 2 may automatically set the imaging conditions based on order information acquired from the HIS / RIS 5 or the like. Alternatively, the imaging control device 2 may allow a user, such as a radiologist, to manually set the imaging conditions by operating the operation unit 21, which will be described later.

[0022] The image management device 4 manages the image data generated by the imaging device 1. The image management device 4 is an image archiving and communication system, an image diagnosis workstation, etc. The image archiving and communication system is sometimes called PACS. PACS is an abbreviation for Picture Archiving and Communication System.

[0023] The HIS / RIS 5 receives order information regarding radiography of a patient from, for example, a doctor, and transmits the received order information to the radiography control device 2. The order information includes various information such as the patient's ID, radiography region, radiography direction, and physique.

[0024] [Example of block diagram of imaging control device 2] 2 is a block diagram of the imaging control device 2. The imaging control device 2 includes a control unit 20, an operation unit 21, a display unit 22, a storage unit 23, and a communication unit 24. The control unit 20, the operation unit 21, the display unit 22, the storage unit 23, and the communication unit 24 are connected to each other so as to be able to communicate with each other via a bus 25, for example.

[0025] The control unit 20 includes, for example, a processor such as a CPU. CPU is an abbreviation for Central Processing Unit. The processor performs various processes including imaging control and re-imaging determination by executing programs stored in a memory such as RAM (which may be the storage unit 23). The control unit 20 may also include electronic circuits such as ASIC and FPGA. ASIC is an abbreviation for Application Specific Integrated Circuit. FPGA is an abbreviation for Field Programmable Gate Array.

[0026] The operation unit 21 receives instructions in response to various input operations from the user, converts the received instructions into operation signals, and outputs the operation signals to the control unit 20. The operation unit 21 includes, for example, a mouse, a keyboard, switches, buttons, etc. The operation unit 21 may be, for example, a touch panel integrally combined with a display. The operation unit 21 may be, for example, a user interface that receives audio input from a microphone or the like.

[0027] The display unit 22 displays a radiographic image based on image data received from the radiographing device 1 and a GUI for accepting various input operations from the user. GUI is an abbreviation for Graphical User Interface. The display unit 22 is, for example, a display such as a liquid crystal display or an organic EL display. EL is an abbreviation for Electro Luminescence. Specifically, the display unit 22 displays a radiographic image obtained by radiographing with the radiographing device 1, as well as radiographing support information I and three-dimensional structure information T according to the result of the re-photographing determination process.

[0028] The storage unit 23 stores, for example, system programs, application programs, and various data. The storage unit 23 includes any storage module, for example, an HDD, an SSD, a ROM, and a RAM. HDD is an abbreviation for Hard Disk Drive. SSD is an abbreviation for Solid State Drive. ROM is an abbreviation for Read Only Memory. Specifically, the storage unit 23 stores a shooting support information output table 23b and a machine learning model (trained model) 23c. The machine learning model 23c and the like may be stored in an external storage device or the like. Details of the shooting support information output table 23b and the machine learning model 23c will be described later.

[0029] The communication unit 24 includes, for example, a communication module including a NIC, a receiver, and a transmitter. NIC is an abbreviation for Network Interface Card. The communication unit 24 communicates various data such as image data with the photographing device 1 and the image management device 4 via the network N.

[0030] In this embodiment, the control unit 20 functions as a first acquisition unit (acquisition step), an extraction unit (extraction step), an estimation unit, and an output unit. The functions of the first acquisition unit, extraction unit, estimation unit, and output unit are realized by the processor of the control unit 20 executing a program stored in the storage unit 23 or the like.

[0031] The first acquisition unit acquires two-dimensional radiographic images captured by the imaging device 1. The control unit 20 may also function as a second acquisition unit that acquires imaging body part information from order information transmitted from the HIS / RIS 5 or the like. The imaging body part information can be used when changing parameters and algorithms of a machine learning model, which will be described later. The estimation unit extracts internal structures of the subject from the two-dimensional radiographic images based on the imaging body part information acquired from the second acquisition unit.

[0032] The estimation unit performs image analysis on the two-dimensional radiographic image acquired by the first acquisition unit and estimates three-dimensional structural information T of structures inside the subject of the radiographic image. In this embodiment, the structures inside the subject include at least a first structure located on the front side and a second structure located on the back side. The front side refers to the side of the generating device 3, such as the radiation source 33, and the back side refers to the side of the imaging device 1. The estimation unit may estimate the three-dimensional structural information T of structures inside the subject of the radiographic image using a pre-trained machine learning model 23c. In this case, the estimation unit inputs the acquired two-dimensional radiographic image into the machine learning model 23c, thereby being able to distinguish between two structures inside the subject of the radiographic image, namely, the first structure located on the front side and the second structure located on the back side. Details of the three-dimensional structural information T of structures using the machine learning model 23c will be described later. It should be noted that the estimation of the three-dimensional structural information T of the structure inside the subject can also be performed without using the machine learning model 23c. In this case, the control unit 20 functions as an extraction unit and an estimation unit. The functions of the extraction unit and the estimation unit are realized by the processor of the control unit 20 executing a program stored in the storage unit 23 or the like. Specifically, the extraction unit and the estimation unit estimate the three-dimensional structural information T of the structure inside the subject by executing techniques such as recognition of the structure by edge detection, which is an image processing method, estimation of the structure by histogram analysis, and recognition of the structure by comparing it with a correct image by pattern matching.

[0033] The output unit outputs three-dimensional structural information T of the structure inside the subject of the radiographic image estimated by the estimation unit. For example, the output unit controls output so that the estimated three-dimensional structural information T of the structure is superimposed on the radiographic image displayed on the display unit. The output unit also functions as a re-photographing support information output unit, and outputs re-photographing support information based on the three-dimensional structural information T estimated by the estimation unit. The re-photographing support information includes imaging support information I for changing the position of the subject S or the imaging device 1 when re-photographing.

[0034] The imaging control device 2 may be configured without the operation unit 21 and the display unit 22. In this case, the imaging control device 2 may receive a control signal from an operation unit provided in an external device connected via the communication unit 24. The imaging control device 2 may also output an image signal to a display unit provided in the external device to display a radiographic image, etc. The external device may be the image management device 4, etc., or another device.

[0035] [Configuration example of shooting support information output table 23b] Next, an example of the configuration of the shooting support information output table 23b stored in the storage unit 23 will be described.

[0036] 3 shows an example of the configuration of the photography support information output table 23b. The photography support information output table 23b stores a photography region, a correction direction when correcting a positioning deviation, and photography support information I for presenting information on the basis of the correction direction, in association with each other. Examples of photography regions include "lateral surface of the knee joint," "lateral surface of the leg," and "lateral surface of the elbow."

[0037] Examples of correction directions when correcting positioning deviations include "external rotation," "internal rotation," "abduction," and "adduction." FIG. 4A is a diagram for explaining the movements of external rotation and internal rotation. External rotation is a movement in which the femur is rotated outward with respect to the long axis of the bone, as shown on the left side of FIG. 4A. Internal rotation is a movement in which the femur is rotated inward with respect to the long axis of the bone, as shown on the right side of FIG. 4A. FIG. 4B is a diagram for explaining the movements of abduction and adduction. Abduction is a movement in which the femur is moved away from the central axis of the body. Adduction is a movement in which the femur is moved closer to the central axis of the body.

[0038] Specifically, when the imaging region is "lateral side of knee joint" and the positioning correction direction is "external rotation", imaging support information I such as "Please externally rotate the knee" is associated. When the imaging region is "lateral side of knee joint" and the positioning correction direction is "internal rotation", imaging support information I such as "Please internally rotate the knee" is associated. When the imaging region is "lateral side of knee joint" and the positioning correction direction is "abduction", imaging support information I of "abduction the knee" is associated, for example. When the imaging region is "lateral side of knee joint" and the positioning correction direction is "adduction", imaging support information I such as "adduct the knee" is associated.

[0039] [About machine learning] FIG. 5 shows an example of the results of a conventional analysis of positioning deviations in a radiological image Ga of the "lateral side of the right knee joint." In FIG. 5, the horizontal direction (left-right direction) of the radiological image Ga is defined as the X direction, and the vertical direction (up-down direction) of the radiological image Ga is defined as the Y direction. The direction perpendicular to the X and Y directions of the radiological image Ga and the direction in which radiation R is applied is defined as the Z direction. The following radiological images and the imaging screen 80 on which the radiological images are displayed are also based on the Cartesian coordinate system including the X, Y, and Z directions.

[0040] If there is a positioning error in the radiographic image Ga of the "lateral side of the right knee joint," the medial and lateral condyles, which are located in the Z direction of the epiphysis, which are structures inside the subject, will be misaligned. In this case, because the radiographic image Ga is constructed in two dimensions, the lines representing the medial and lateral condyles of the "femoral condyle" are displayed on the same plane. Therefore, as shown in Figure 5, the medial and lateral condyles are displayed as two lines in the "femoral condyle" of the radiographic image Ga. These two lines are called the first epiphysis Tx and the second epiphysis Ty. Conventional image analysis has not been able to determine which of the first epiphysis Tx and second epiphysis Ty is the line representing the medial condyle located closer to the subject and which is the line representing the lateral condyle located further back. In other words, it has not been possible to determine the three-dimensional positional relationship in the Z direction of the two first epiphysis Tx and second epiphysis Ty.

[0041] In this embodiment, the machine learning model 23c is trained by machine learning using machine learning data by a learning device. The learning device is configured, for example, by a computer and has processors such as a CPU and a GPU. GPU is an abbreviation for Graphics Processing Unit. The processor realizes a predetermined machine learning function by executing a program stored in a memory such as a RAM. The learning device may be a client device or a server device.

[0042] The machine learning model 23c outputs, as estimated data, a line on the medial condyle side and a line on the lateral condyle side of the "femoral condyle" in the radiographic image Ga shown in FIG. 5. The machine learning model 23c may be, for example, a regression model such as polynomial regression, multiple regression, support vector regression, or random forest regression. The machine learning model 23c may also be another model such as a neural network.

[0043] Fig. 6A shows an example of input data Gb used when training the machine learning model 23c, and Fig. 6B shows an example of correct answer data Gc used when training the machine learning model 23c.

[0044] The machine learning data includes input data Gb input to the machine learning model 23c and correct answer data Gc output from the machine learning model 23c. For example, as shown in FIG. 6A, the input data Gb is radiation image data of the "lateral surface of the knee joint" when the lateral side of the patient's right knee is imaged with the imaging device 1 in contact with the knee. The input data Gb has a positioning deviation, for example, in the "femoral condyle." The input data Gb may be a past radiation image of the patient obtained by actual imaging.

[0045] As shown in FIG. 6B, the correct answer data Gc is radiographic image data of the "lateral surface of the knee joint" when the lateral side of the patient's right knee is photographed with the imaging device 1 in contact with the input data Gb. The correct answer data Gc includes medial condyle side line information T1 indicating the medial condyle region and lateral condyle side line information T2 indicating the lateral condyle region, which are correct for the input data Gb. Here, the medial condyle side line is the boundary between the medial condyle side of the femoral condyle and the knee soft tissue. The lateral condyle side line is the boundary between the lateral condyle side of the femoral condyle and the knee soft tissue. Hereinafter, the medial condyle side line information T1 will be referred to as medial condyle side line information T1, and the lateral condyle side line information T2 will be referred to as lateral condyle side line information T2. ​​The correct answer data Gc may be created by a user, such as a radiologist. The user may specify the medial condyle side line information T1 and the lateral condyle side line information T2 from the radiographic image and tag them with line information. This allows the attenuated correct answer data Gc to be created. The medial condyle side line information T1 and the lateral condyle side line information T2 according to this embodiment correspond to the three-dimensional structure information T.

[0046] The learning device performs machine learning using a dataset including the above-described input data Gb and correct answer data Gc to create a trained machine learning model 23c. When the input data Gb of the "lateral surface of the right knee joint" is input, the machine learning model 23c outputs correct medial condyle side line information T1 and lateral condyle side line information T2 if there is a positioning misalignment of the "femoral condyle." In other words, the machine learning model 23c distinguishes the medial and lateral condyles of the "femoral condyle," which are the epiphysis, into medial condyle side line information T1 located on the near side and lateral condyle side line information T2 located on the far side. The trained machine learning model 23c is stored, for example, in the memory unit 23 of the imaging control device 2. The imaging control device 2 can identify the type of positioning misalignment based on the medial condyle side line information T1 and lateral condyle side line information T2 output from the machine learning model 23c. In this case, the imaging control device 2 can identify whether the "femoral condyle" is internally rotated or externally rotated as the type of positioning misalignment.

[0047] Although the positional relationship between the medial and lateral condyles of the femoral condyle in the Z direction was learned using the medial condyle side line information T1 actually specified by the user, the learning method is not limited to this. For example, as another learning method, the coordinate points of the medial and lateral condyles of the femoral condyle may be extracted, and the positional relationship between the medial and lateral condyles in the Z axis direction may be estimated by, for example, regressing the extracted consecutive coordinate points.

[0048] Next, we will explain the case where the type of positioning misalignment is determined to be whether the "femoral condyle" is adduced or abducted. In this case, the machine learning model 23c is trained by machine learning using the correct answer data Gd, which is different from the correct answer data Gc shown in Figure 6B.

[0049] 7 shows an example of other correct answer data Gd used when training the machine learning model 23c. Note that the input data Gb can be the radiographic image shown in FIG. 6A, and therefore a detailed description thereof will be omitted.

[0050] The machine learning data includes input data Gb input to the machine learning model 23c and correct answer data Gd indicating the correct answer output from the machine learning model 23c. As shown in FIG. 7, the correct answer data Gd includes information C1 indicating the femoral condyle center and information C2 indicating the tibia condyle center, which are correct answers for the input data Gb. Hereinafter, the information C1 indicating the femoral condyle center will be referred to as femoral condyle center information C1, and the information C2 indicating the tibia condyle center will be referred to as tibia condyle center information C2. The correct answer data Gd also includes joint information for identifying whether the radiographic image of the input data Gb is of a right knee joint or a left knee joint. The correct answer data Gd may be created by a user such as a radiologist. For example, the user visually determines the coordinates of the femoral condyle center information C1 and the tibia condyle center information C2, which are correct answers for the "femoral condyle." Next, two heat maps are created according to the accuracy of the determined coordinates of the femoral condyle center information C1 and the tibia condyle center information C2. The created heat maps are used as correct answer data Gd.

[0051] When the input data Gb of the "lateral side of the knee joint" is input, the machine learning model 23c estimates the correct femoral condyle center information C1, tibial condyle center information C2, and joint information indicating whether the knee joint is left or right. The trained machine learning model 23c is stored, for example, in the memory unit 23 of the imaging control device 2. The imaging control device 2 identifies the type of positioning deviation based on the femoral condyle center information C1, etc. output from the machine learning model 23c. In this case, the imaging control device 2 can identify whether the "femoral condyle" is adduced or abducted as the type of positioning deviation.

[0052] Although the femoral condyle center information C1 of the femoral condyle and the like are learned using a heat map, other learning methods may be used. As another learning method, learning may be performed using the center coordinates of the femoral condyle themselves. Furthermore, while the knee joint is identified by choosing between the right knee and the left knee, the knee joint may also be identified using other information, such as the positional relationship between the femur and the patella. Furthermore, all of the above-mentioned medial condyle side line information T1, femoral condyle center information C1, etc. may be estimated using one machine learning model 23c. Alternatively, multiple machine learning models 23c may be used. In this case, the medial condyle side line information T1, etc. may be estimated using one machine learning model 23c, and the femoral condyle center information C1, etc. may be estimated using another machine learning model 23c.

[0053] In the above example, the 3D structure of the femoral condyle at the "lateral side of the right knee joint" was learned when the lateral side of the patient's right knee was imaged with the imaging device 1 in contact with the image, but the present invention is not limited to this. For example, the 3D structure of the femoral condyle at the "lateral side of the right knee joint" may be learned using a radiographic image of the "lateral side of the right knee joint" when the medial side of the patient's right knee is imaged with the imaging device 1 in contact with the image. In this case, the medial condyle side line information T1 is located at the far side in the Z direction, and the lateral condyle side line information T2 is located at the near side in the Z direction. Furthermore, the imaging site to be subjected to machine learning may be other than the "lateral side of the right knee joint." For example, the imaging site to be subjected to machine learning may be the lateral side of the left knee joint, or other sites such as the ankle joint or elbow joint.

[0054] [Example of operation of the imaging control device 2] Next, a flow will be described for capturing a radiographic image of the subject S. Fig. 8 is a flowchart showing an example of the operation of the imaging control device 2 when capturing a radiographic image of the subject S. In the following, a case will be described in which the imaging region and imaging direction for capturing a radiographic image are "the lateral surface of the knee (right knee) joint."

[0055] The communication unit 24 of the imaging control device 2 receives the order information transmitted from the HIS / RIS 5 or the like. For example, the user selects predetermined order information from the examination list displayed on the screen of the display unit 22 of the imaging control device 2. The control unit 20 acquires the order information selected by the user (step S10).

[0056] When the control unit 20 acquires predetermined order information, it causes the display unit 22 to display the photographing screen 80 (step S11).

[0057] FIG. 9 shows an example of the configuration of an imaging screen 80 displayed on the display unit 22 of the imaging control device 2. The imaging screen 80 includes an imaging selection section 81, a condition setting section 82, an image display area 83, a patient information display section 84, and an examination end button 85. The imaging selection section 81 displays order information selected from the examination list, for example. The order information includes imaging details such as the imaging region and imaging direction, and thumbnail images showing the captured radiographic images. The condition setting section 82 includes buttons for setting imaging conditions for the radiographic images and buttons for adjusting the captured radiographic images, for example. The image display area 83 displays radiographic images captured by the imaging device 1 based on the set imaging conditions, etc. Note that in FIG. 9, no radiographic images are displayed in the image display area 83 because the radiographic images have not yet been captured.

[0058] When predetermined order information is selected by the photography selection unit 81 or the like, the control unit 20 sets photography conditions for each of the photography device 1 and the generation device 3 (step S12). The photography conditions include image reading conditions to be set for the photography device 1 and irradiation conditions to be set for the generation device 3. For example, the control unit 20 sets image reading conditions for the photography device 1 based on the photography part, photography direction, etc. of the selected order information. The control unit 20 also sets irradiation conditions for the generation device 3 based on the photography part, photography direction, etc. of the selected order information.

[0059] The imaging conditions may be set manually by the user. Specifically, the control unit 20 may set the image reading conditions received through an input operation by the user on the condition setting unit 82 to the imaging device 1. The control unit 20 may set the radiation irradiation conditions received through an input operation by the user on the operation panel of the radiation generator 3 to the radiation generator 3.

[0060] Next, when the user turns on the switch 32, the control unit 20 controls the imaging device 1, the generator 3, etc. to capture a radiographic image of the subject S (step S13). The generator 3 irradiates the imaging region of the subject S with radiation R. The imaging device 1 detects the radiation R that has passed through the subject S from the generator 3, and generates image data showing the imaging region based on the detected radiation R. The imaging device 1 transmits the generated image data to the imaging control device 2. The control unit 20 acquires a radiographic image based on the image data transmitted from the imaging device 1 (step S13).

[0061] When step S13 is completed, the process branches to steps S14 and S15. In this embodiment, an example in which the process of step S14 and the process of step S15 are performed in parallel will be described, but the present invention is not limited to this. For example, the process of step S14 and the process of step S15 may be performed serially in order. In this case, step S15 may be performed first, and step S14 may be performed together with step S16, which will be described later.

[0062] First, the processing of step S14 will be described. The control unit 20 displays the acquired radiographic image in the image display area 83 of the imaging screen 80 (step S14). In this embodiment, a radiographic image of the "lateral side of the right knee joint" is displayed in the image display area 83 as the imaging site, etc. A thumbnail image showing the radiographic image that has been captured is displayed in the order information 81a of the imaging selection unit 81. When step S14 is completed, the process proceeds to step S16.

[0063] Next, the process of step S15 branched from step S13 will be described. The control unit 20 executes a re-photographing determination process to determine whether or not re-photographing is necessary using the acquired radiographic image. The control unit 20 proceeds to a subroutine shown in FIG.

[0064] Fig. 10 is a flowchart showing an example of the operation of the imaging control device 2 during the re-photographing determination process. Fig. 11A and Fig. 11B show an example of three-dimensional structural information T of the femoral condyle of the radiographic image G estimated by the machine learning model 23c.

[0065] The control unit 20 uses the machine learning model 23c to estimate medial condyle-side line information T1 and lateral condyle-side line information T2 of the femoral condyle from the radiographic image G obtained by imaging (step S20). Specifically, the control unit 20 inputs the radiographic image G of the "lateral side of the knee joint" into the machine learning model 23c. Based on the input radiographic image G, the machine learning model 23c outputs the medial condyle-side line information T1 and lateral condyle-side line information T2 of the femoral condyle, which are the correct answers, as estimated data. In FIG. 11A, the medial condyle-side line information T1 is indicated by a thin line, and the lateral condyle-side line information T2 is indicated by a thick line. In this way, the control unit 20 acquires three-dimensional structural information T that is distinguished into the medial condyle-side line information T1 located on the near side and the lateral condyle-side line information T2 located on the far side.

[0066] Next, the control unit 20 uses the machine learning model 23c to estimate femoral condyle center information C1, tibia condyle center information C2, and joint information from the radiographic image G obtained by imaging (step S21). Specifically, the control unit 20 inputs a radiographic image of the "lateral side of the knee joint" to the machine learning model 23c. Based on the input radiographic image G, the machine learning model 23c outputs femoral condyle center information C1 and tibia condyle center information C2 of the femoral condyle as estimated data. Furthermore, based on the input radiographic image G, the machine learning model 23c outputs joint information indicating that the radiographic image G is a right knee joint. In this way, the control unit 20 respectively acquires the femoral condyle center information C1, tibia condyle center information C2, and joint information. Note that the joint information may be acquired from the examination order information transmitted from the HIS / RIS 5.

[0067] In this embodiment, steps S20 and S21 have been described separately, but steps S20 and S21 may be combined into one step. Specifically, the machine learning model 23c may output all of the medial condyle side line information T1, lateral condyle side line information T2, femoral condyle center information C1, tibia condyle center information C2, and joint information based on the input radiographic image G. After step S21 is completed, the process proceeds to step S22.

[0068] The control unit 20 determines whether the positioning deviation is internal rotation or external rotation based on the intersection pattern between the lines radially extending from the femoral condyle center information C1 and the medial condyle side line information T1, etc. (Step S22). In this embodiment, the process of determining whether the positioning deviation is internal rotation or external rotation is referred to as the first determination process.

[0069] The control unit 20 proceeds to a subroutine shown in Fig. 12 and functions as a first determination unit to execute the first determination process. Fig. 12 is a flowchart showing an example of the first determination process.

[0070] The control unit 20 extends multiple lines L radially from the estimated femoral condyle center information C1 (step S30). Specifically, as shown in FIG. 11A, six lines L are extended radially at equal intervals from the femoral condyle center information C1 in the circumferential direction. A line L that does not intersect with either the medial condyle side line information T1 or the lateral condyle side line information T2 is referred to as a first line L1. In FIG. 11A, the first line L1 is indicated by a dashed line. A line L that intersects with the medial condyle side line information T1 and the lateral condyle side line information T2 in this order is referred to as a second line L2. In FIG. 11A, the second line L2 is indicated by a dashed line. A line L that intersects with the lateral condyle side line information T2 and the medial condyle side line information T1 in this order is referred to as a third line L3. In FIG. 11A, the third line L3 is indicated by a dashed line. Note that the first line L1, the second line L2, and the third line L3 may be collectively referred to as lines L.

[0071] The control unit 20 determines whether to view the multiple lines L in a clockwise or counterclockwise direction based on the estimated joint information. For example, the "right knee joint" in the joint information is associated with a counterclockwise direction, and the "left knee joint" in the joint information is associated with a clockwise direction. In this embodiment, since the joint information is a "right knee joint," the multiple lines L are viewed in a counterclockwise direction. The control unit 20 determines whether the order of the multiple lines L is the first line L1, the second line L2, and the third line L3 when viewed counterclockwise based on the first line L1 in the center of FIG. 11A (step S31). If the control unit 20 determines that the condition of step S31 is satisfied, the process proceeds to step S32.

[0072] If the order is first line L1, second line L2, and third line L3, the control unit 20 determines that the femoral condyle is internally rotated (step S32). That is, the control unit 20 determines that the femoral condyle should be externally rotated to correct the positioning in the correct direction. Note that FIG. 11A shows an example in which the femoral condyle in radiographic image G is externally rotated, and step S32 does not apply to FIG. 11A.

[0073] On the other hand, if the control unit 20 determines that the condition of step S31 is not satisfied, the control unit 20 proceeds to step S33. The control unit 20 determines whether the order of the first line L1, the third line L3, and the second line L2 is the order when viewed counterclockwise from the first line L1 in the center of Fig. 11A (step S33). If the control unit 20 determines that the condition of step S33 is satisfied, the control unit 20 proceeds to step S34.

[0074] If the order is first line L1, third line L3, and second line L2, the control unit 20 determines that the femoral condyle is externally rotated (step S34). That is, the control unit 20 determines that the femoral condyle should be internally rotated to correct the positioning in the correct direction. Note that FIG. 11A shows an example in which the femoral condyle in radiographic image G is externally rotated, and step S34 corresponds to FIG. 11A.

[0075] On the other hand, if the control unit 20 determines that the condition of step S33 is not satisfied, the control unit 20 proceeds to step S35. In this case, the control unit 20 determines that the medial condyle side line information T1 and the lateral condyle side line information T2 of the femoral condyle overlap, and that no deviation of the femoral condyle has occurred (step S35). Note that the determination that no deviation of the femoral condyle has occurred is not limited to when the medial condyle side line information T1 and the lateral condyle side line information T2 completely overlap. If the amount of deviation between the medial condyle side line information T1 and the lateral condyle side line information T2 is within the allowable range, it may be determined that no deviation of the femoral condyle has occurred. After step S35 is completed, the control unit 20 ends the subroutine of the first determination process and proceeds to step S23 of FIG. 10.

[0076] Although the first determination process has been described for the case of a right knee joint, the type of positioning deviation can also be determined for the case of a left knee joint, etc., using similar processing. In the case of a left knee joint, the intersections of multiple lines L extending radially from femoral condyle center information C1 and medial condyle side line information T1, etc. are viewed clockwise on the radiographic image G. The control unit 20 determines that the femoral condyle is internally rotated when the order is first line L1, second line L2, and third line L3. The control unit 20 determines that the femoral condyle is externally rotated when the order is first line L1, third line L3, and second line L2.

[0077] Next, the control unit 20 determines whether the positioning deviation is adduction or abduction based on the intersection pattern of the line connecting the femoral condyle center information C1 and the tibia condyle center information C2 with the medial condyle side line information T1, etc. (Step S23). In this embodiment, the process of determining whether the positioning deviation is adduction or abduction is referred to as the second determination process.

[0078] The control unit 20 proceeds to a subroutine shown in Fig. 13 and functions as a second determination unit to execute the second determination process. Fig. 13 is a flowchart showing an example of the second determination process.

[0079] As shown in FIG. 11B, the control unit 20 connects the estimated femoral condyle center information C1 and the tibia condyle center information C2 with a fourth line L4 (step S40).

[0080] The control unit 20 determines whether the fourth line L4 intersects with the medial condyle side line information T1 and the lateral condyle side line information T2 in this order when viewing the fourth line L4 from the femoral condyle center information C1 toward the tibia condyle center information C2 (step S41). If the control unit 20 determines that the condition of step S41 is satisfied, the process proceeds to step S42.

[0081] If the fourth line L4 intersects with the medial condyle side line information T1 and then with the lateral condyle side line information T2, the control unit 20 determines that the femoral condyle is adducted (step S42). That is, the control unit 20 determines that the femoral condyle should be abducted to correct the positioning in the correct direction. Note that FIG. 11B shows an example in which the femoral condyle in the radiographic image G is abducted, and step S42 does not apply to FIG. 11B.

[0082] On the other hand, if the control unit 20 determines that the condition of step S41 is not satisfied, the control unit 20 proceeds to step S43. When the fourth line L4 is viewed from the femoral condyle center information C1 toward the tibia condyle center information C2, the control unit 20 determines whether the fourth line L4 intersects with the lateral condyle side line information T2 and the medial condyle side line information T1 in this order (step S43). If the control unit 20 determines that the condition of step S43 is satisfied, the control unit 20 proceeds to step S44.

[0083] If the fourth line L4 intersects with the lateral condyle side line information T2 and then with the medial condyle side line information T1, the control unit 20 determines that the femoral condyle is abducted (step S44). That is, the control unit 20 determines that the femoral condyle should be adducted in order to correct the positioning in the correct direction. Note that FIG. 11B shows an example in which the femoral condyle is abducted in the radiographic image G, and step S44 corresponds to FIG. 11B.

[0084] On the other hand, if the control unit 20 determines that the condition of step S43 is not satisfied, the control unit 20 proceeds to step S45. In this case, the control unit 20 determines that the medial condyle side line information T1 and the lateral condyle side line information T2 of the femoral condyle overlap, and that no misalignment has occurred (step S45). The determination that no misalignment has occurred also includes a case where the amount of misalignment between the medial condyle side line information T1 and the lateral condyle side line information T2 is within the allowable range. Note that step S45 is a process common to step S35 in FIG. 12 and may be omitted. After step S45 is completed, the control unit 20 terminates the subroutine of the second determination process and proceeds to step S24 in FIG. 10.

[0085] While the second determination process has been described for the case where the imaging site is the lateral surface of the right knee joint, the type of positioning deviation can be determined using a similar process for the case of the lateral surface of the left knee joint, etc. The specific determination method is the same as for the case of the lateral surface of the right knee joint, and therefore a detailed description will be omitted. Furthermore, whether the femoral condyle is adducted, etc., is determined using the fourth line L4 connecting the femoral condyle center information C1 and the tibia condyle center information C2 shown in FIG. 11B, but this is not limiting. The following determination method may also be used. For example, a triangular virtual area V, indicated by the dashed line in FIG. 11B, is assumed between the femoral condyle center information C1 and the tibia condyle center information C2. Next, multiple lines are drawn within the virtual area V in a direction from the femoral condyle center information C1 toward the tibia condyle center information C2. The number of lines that intersect the medial condyle side line information T1 and the lateral condyle side line information T2 in this order is compared with the number of lines that intersect the lateral condyle side line information T2 and the medial condyle side line information T1 in this order. Finally, whether the femoral condyles are adducted or abducted may be determined based on which number is greater.

[0086] Next, the control unit 20 derives from the storage unit 23 photographing support information I associated with the positioning determination results in steps S22 and S23 (step S24). Specifically, for example, if the result of the first determination process is that the knee is externally rotated as shown in Fig. 11A, the control unit 20 refers to the photographing support information output table 23b to acquire photographing support information I such as "Please internally rotate your knee." Furthermore, for example, if the result of the second determination process is that the knee is abducted as shown in Fig. 11B, the control unit 20 refers to the photographing support information output table 23b to acquire photographing support information I such as "Please internally rotate your knee."

[0087] If the determination results in steps S22 and S23 indicate that there is no deviation in positioning, the control unit 20 may acquire instruction information indicating that the positioning is normal. After step S24 is completed, the control unit 20 ends the subroutine and returns to step S16 shown in FIG. 8.

[0088] The control unit 20 executes display control to display the obtained three-dimensional structure information T and shooting support information I, which are the determination results of the re-shooting determination process, on the shooting screen 80 of the display unit 22 (step S16).

[0089] FIG. 14 shows an example of three-dimensional structural information T and imaging support information I displayed on the imaging screen 80. In addition to a radiographic image G obtained by imaging, the three-dimensional structural information T is displayed in the image display area 83 of the imaging screen 80. The three-dimensional structural information T includes medial condyle side line information T1 indicating the medial condyle region and lateral condyle side line information T2 indicating the lateral condyle region of the femoral condyle in the radiographic image G. In the image display area 83 of the imaging screen 80, the medial condyle side line information T1 and lateral condyle side line information T2, which are correct for the femoral condyle, are displayed superimposed on the radiographic image G. The medial condyle side line information T1 and lateral condyle side line information T2 may be displayed in different colors. Specifically, the medial condyle side line information T1 may be displayed in red (thin line in FIG. 14), and the lateral condyle side line information T2 may be displayed in blue (thick line in FIG. 14). Alternatively, the medial condyle line information T1 and the lateral condyle line information T2 may be displayed with different line thicknesses, or may be displayed as solid or dashed lines. To achieve this, a correspondence relationship between the "medial condyle line information T1" and the "lateral condyle line information T2" and the "color, thickness, and line pattern (solid line, dashed line)" is set in advance. The correspondence relationship may be a default setting, or may be set by the user after shipment. The user can quickly identify the three-dimensional positional relationship between the medial condyle line information T1 and the lateral condyle line information T2 by memorizing the correspondence relationship in advance or by checking a correspondence table or the like showing the correspondence relationship. The correspondence table or the like showing the correspondence relationship may be displayed on the imaging screen 80.

[0090] Furthermore, in addition to the radiographic image G obtained by imaging, imaging support information I based on the three-dimensional structure information T is displayed in the image display area 83 of the imaging screen 80. The imaging support information I is composed of, for example, words or sentences including technical terms. Specifically, when the positioning correction direction is internal rotation, the image display area 83 displays the sentence "Please internally rotate your knee" as the imaging support information I. For example, when the positioning correction direction is adduction, the image display area 83 displays the sentence "Please internally rotate your knee" as the imaging support information I.

[0091] Rank information Ic and deviation amount information Id are displayed above the imaging support information I. The rank information Ic and deviation amount information Id are information for alerting the user that positioning needs to be corrected and for presenting detailed correction content. In this embodiment, the imaging support information I and the like are displayed in the empty space other than the radiographic image G in the image display area 83, but the present invention is not limited to this.

[0092] The rank information Ic is information that indicates the degree of deviation in positioning by rank. For example, if there is no deviation in positioning and re-imaging is not required, "Positioning: A" is displayed as the rank information Ic. If the deviation in positioning is within the allowable range and re-imaging is not required, "Positioning: B" is displayed as the rank information Ic. If the deviation in positioning exceeds the allowable range and re-imaging is required, "Positioning: C" is displayed as the rank information Ic. FIG. 14 illustrates an example in which the rank information Ic is "Positioning: C."

[0093] The deviation amount information Id is information indicating the distance (deviation width) in a predetermined direction between the estimated medial condyle side line information T1 and lateral condyle side line information T2 of the femoral condyle. For example, if the distance D in the X direction between the medial condyle side line information T1 and lateral condyle side line information T2 is "4 mm," "deviation amount: 4.0 mm" is displayed in the image display area 83 as the deviation amount information Id. Note that, although FIG. 14 shows the deviation amount in the X direction as the deviation amount information Id, this is not limiting. For example, the deviation amount in the Y direction may be displayed, or the deviation amounts in both the X and Y directions may be displayed.

[0094] The user guides the patient and corrects any positioning deviations by checking the three-dimensional structural information T and imaging support information I displayed on the imaging screen 80. Once the positioning deviations have been corrected, the radiographic image is re-acquired.

[0095] On the other hand, if the control unit 20 determines that there is no positioning deviation, it may display only the radiographic image G in the image display area 83 of the imaging screen 80. Furthermore, if the control unit 20 determines that there is no positioning deviation, it may display "Positioning: A" as the rank information Ic and "Deviation: 0 mm" as the deviation amount information Id in the image display area 83. Furthermore, the control unit 20 may display text or the like indicating that there is no positioning deviation as imaging support information I in the image display area 83.

[0096] Furthermore, the display method of the medial condyle side line information T1 and the lateral condyle side line information T2 is not limited to the display method shown in FIG. 14. FIG. 15 shows an example of an imaging screen 80 in which text information is displayed in addition to each line information as the three-dimensional structural information T. In the image display area 83 of the imaging screen 80, the medial condyle side line information T1 and the lateral condyle side line information T2 indicating the medial condyle region of the femoral condyle are respectively displayed as the three-dimensional structural information T. The medial condyle side line information T1 is displayed in association with text information "medial condyle (near side)" indicating that the line information is the medial condyle and is located on the near side. The lateral condyle side line information T2 is displayed in association with text information "lateral condyle (far side)" indicating that the line information is the lateral condyle and is located on the far side. Note that the display method of the text information is not limited to that shown in FIG. 14. For example, only the text information "medial condyle" and "lateral condyle" may be displayed.

[0097] 14 and the like, for example, only one of the medial condyle side line information T1 and the lateral condyle side line information T2 may be displayed superimposed on the "femoral condyle" of the radiographic image G. This is because, depending on the user, if they can recognize one of the line information of the medial condyle side line information T1 and the lateral condyle side line information T2, they can estimate the line information of the other line information.

[0098] FIG. 16 shows an example of an imaging screen 80 in which only medial condyle line information T1 is displayed as the three-dimensional structural information T. In the image display area 83 of the imaging screen 80, the medial condyle line information T1 indicating the medial condyle region of the femoral condyle is displayed superimposed on the radiographic image G as the three-dimensional structural information T. As in FIG. 14, the medial condyle line information T1 may be displayed in a color such as red. When two lines indicating epiphysis can be recognized in the femoral condyle of the radiographic image G, displaying the medial condyle line information T1 on the imaging screen 80 allows identification of the line indicating one of the epiphysis as the medial condyle line. This also allows identification of the line indicating the remaining epiphysis as the lateral condyle line. Note that while FIG. 16 shows an example in which the medial condyle line information T1 is displayed as the three-dimensional structural information T, lateral condyle line information T2 may also be displayed.

[0099] As described above, according to this embodiment, three-dimensional structural information T of the internal structure of the subject in a radiographic image is estimated. Specifically, by inputting a radiographic image of the "lateral side of the right knee joint" into the machine learning model 23c, it is possible to distinguish and estimate the medial condyle-side line information T1 located on the near side of the "femoral condyle" and the lateral condyle-side line information T2 located on the far side. A user such as a radiologist can identify the three-dimensional positional relationship of the "femoral condyle" by checking the medial condyle-side line information T1 and the lateral condyle-side line information T2 superimposed on the radiographic image on the radiographic screen 80. This allows the user to efficiently determine the direction of positioning correction. Furthermore, according to this embodiment, radiographic support information I is displayed on the radiographic screen 80, allowing the user to quickly determine positioning by checking the radiographic support information I. As a result, radiographic imaging can be performed more quickly and the burden on the patient during positioning can be reduced.

[0100] [Another embodiment 1] In another embodiment 1, positioning assistance is performed using an optical camera image captured by an optical camera. Note that the following description will focus on differences from the above embodiment, and a description of commonalities with the above embodiment will be omitted. In addition, in the description of another embodiment 1, parts common to the above embodiment will be described with the same reference numerals.

[0101] 17 is a diagram showing a schematic configuration of an imaging support system 10B according to another embodiment 1. The imaging support system 10B includes a radiographic image capturing device 1, an imaging control device 2, a radiation generating device 3, an image management device 4, and an HIS / RIS 5. The generating device 3 includes a generator 31, a switch 32, a radiation source 33, and an optical camera 34. The optical camera 34 and the like are connected to the imaging control device 2 via a network N.

[0102] The optical camera 34 is disposed, for example, adjacent to the radiation source 33. The radiation source 33 and the optical camera 34 may be integrally mounted in a single housing. The optical camera 34 optically photographs the subject S at a timing before capturing a radiographic image of the subject S. The optical camera 34 transmits the photographed optical camera image according to the positioning of the patient to the imaging control device 2. The optical camera image includes a still image or continuously captured dynamic images.

[0103] The imaging control device 2 determines the presence or absence, type, etc. of positioning deviation based on the optical camera image acquired from the optical camera 34. Specifically, the imaging control device 2 estimates three-dimensional structural information T, such as medial condyle line information T1, using the machine learning model 23c. The imaging control device 2 identifies the type of positioning deviation based on the three-dimensional structural information T, etc., and acquires imaging support information I corresponding to the type of deviation. The imaging control device 2 displays the imaging support information I and the three-dimensional structural information T, etc., on, for example, the display device 26 and the display unit 22. A user, such as a radiologist, can recognize the three-dimensional positional relationship of the deviation area based on the three-dimensional structural information T, and can easily understand the positioning correction direction based on the imaging support information I. If the positioning deviation is resolved, the user proceeds to the next step, capturing a radiographic image of the subject S.

[0104] According to another embodiment 1, similar to the above-described embodiment, a user such as a radiologist can identify the three-dimensional positional relationship of the "femoral condyle" by checking the medial condyle line information T1 and the lateral condyle line information T2 superimposed on the radiographic image on the radiographic screen 80. This allows the user to efficiently determine the direction of positioning correction. Furthermore, according to this embodiment, radiographic support information I is displayed on the radiographic screen 80, so the user can quickly determine positioning by checking the radiographic support information I. As a result, radiographic imaging can be performed more quickly and the burden on the patient during positioning can be reduced. Furthermore, by using the optical camera 34, the positioning can be corrected before radiographic imaging. This reduces the number of re-imaging operations and the patient's total radiation exposure, thereby reducing the burden on the patient.

[0105] While the preferred embodiments of the present disclosure have been described in detail above with reference to the accompanying drawings, the technical scope of the present disclosure is not limited to such examples. Furthermore, various modifications and improvements will naturally fall within the technical scope of the present disclosure, provided that they are within the scope of the technical ideas described in the claims of those skilled in the art.

[0106] In the above-described embodiment, the three-dimensional structural information T of the "femoral condyle" is estimated when the radiographic image is of the lateral side of the knee joint, but the imaging site to be estimated is not limited to the lateral side of the knee joint. For example, when the radiographic image is of the lateral side of the ankle joint, the lateral side of the elbow joint, etc., the three-dimensional structural information T of the internal structure of each subject can be estimated by applying the re-photographing determination process described above. [Explanation of symbols]

[0107] 1. Radiation imaging device (imaging device) 10A, 10B Photography Support System 2. Shooting control device (image processing device) 20 Control Unit 23 Memory section 23b Shooting support information output table I. Shooting support information T 3D structure information

Claims

1. a first acquisition unit that acquires a two-dimensional radiation image; an estimation unit that performs image analysis on the acquired two-dimensional radiographic image to estimate three-dimensional structural information of a structure inside the subject; An image processing device comprising:

2. an output unit that outputs the estimated three-dimensional structure information; The image processing device according to claim 1 .

3. the estimation unit estimates information about a position in a radiation irradiation direction as the three-dimensional structure information inside the subject; The image processing device according to claim 1 .

4. the estimation unit estimates, as the three-dimensional structure information inside the subject, information distinguishing between a structure on a front side and a structure on a back side in a radiation irradiation direction; The image processing device according to claim 1 .

5. an extraction unit that extracts the structure inside the subject from the two-dimensional radiation image; the estimation unit estimates, as the three-dimensional structure information, three-dimensional structure information of the extracted structure; The image processing device according to claim 1 .

6. a second acquisition unit that acquires imaging region information; the extraction unit extracts a structure of the subject from the two-dimensional radiographic image based on the acquired imaging region information. The image processing device according to claim 5 .

7. the three-dimensional structure information estimated by the estimation unit includes information on a medial condyle line and a lateral condyle line of a femoral condyle; The image processing device according to claim 1 .

8. the output unit displays the estimated three-dimensional structural information superimposed on the two-dimensional radiographic image. The image processing device according to claim 2 .

9. a re-photographing support information output unit that outputs re-photographing support information based on the three-dimensional structure information estimated by the estimation unit; The image processing device according to claim 1 .

10. The re-photographing support information is information for changing the position of the subject or the photographing device when re-photographing. The image processing device according to claim 9 .

11. The estimation unit a machine learning model that uses a radiographic image including an internal structure of a subject as input data and three-dimensional structural information of the internal structure of the subject as correct output data, and inputs the radiographic image acquired by the first acquisition unit into the machine learning model that has been trained using the radiographic image including an internal structure of the subject as input data, thereby estimating the three-dimensional structural information of the internal structure of the radiographic image; The image processing device according to claim 1 .

12. Computer, a first acquisition unit that acquires a two-dimensional radiation image; an estimation unit that performs image analysis on the acquired two-dimensional radiation image to estimate three-dimensional structural information of a structure inside the subject; A program to function as a

13. an output unit that outputs the estimated three-dimensional structure information; The program according to claim 12, for causing the program to function as

14. the estimation unit estimates information about a position in a radiation irradiation direction as the three-dimensional structure information inside the subject; The program according to claim 12.

15. the estimation unit estimates, as the three-dimensional structure information inside the subject, information distinguishing between a structure on a front side and a structure on a back side in a radiation irradiation direction; The program according to claim 12.

16. causing the computer to function as an extraction unit that extracts the structure inside the subject from the two-dimensional radiation image; the estimation unit estimates, as the three-dimensional structure information, three-dimensional structure information of the extracted structure; The program according to claim 12.

17. causing the computer to function as a second acquisition unit that acquires imaging region information; the extraction unit extracts a structure of the subject from the two-dimensional radiographic image based on the acquired imaging region information. The program according to claim 16.

18. the three-dimensional structure information estimated by the estimation unit includes information on a medial condyle line and a lateral condyle line of a femoral condyle; The program according to claim 12.

19. the output unit displays the estimated three-dimensional structural information superimposed on the two-dimensional radiographic image. The program according to claim 13.

20. a re-photographing support information output unit that outputs re-photographing support information based on the three-dimensional structure information estimated by the estimation unit; The program according to claim 12.

21. The re-photographing support information is information for changing the position of the subject or the photographing device when re-photographing. The program according to claim 20.

22. an acquisition step of acquiring a two-dimensional radiation image; an estimation step of estimating three-dimensional structural information of a structure inside the subject by image analysis of the acquired two-dimensional radiographic image; An image processing method comprising:

Images