Image processing device, program, and image processing method

Abstract

To provide an image processing device, etc. capable of efficiently determining a positioning correction direction, etc.SOLUTION: An imaging control device 2 includes a first acquisition unit for acquiring a two-dimensional radiation image, an estimation unit for performing image analysis for the two-dimensional radiation image acquired by the first acquisition unit and estimating three-dimensional structure information on a structure inside a subject, and an output unit for outputting the three-dimensional structure information estimated by the estimation unit. The estimation unit estimates information on the position in a radiation irradiation direction as the three-dimensional structure information inside the subject.SELECTED DRAWING: Figure 14

Description

[Technical field]

[0001] The present invention relates to an image processing device, a program, and an image processing method. [Background technology]

[0002] In radiography using radiation such as X-rays performed in medical facilities such as hospitals, if the patient's posture during imaging is not appropriate, the positioning will be off. In this case, the radiologist, etc., will grasp the three-dimensional positional relationship of the internal structures of the subject while looking at the previously captured radiographic image, and guide the patient to the appropriate positioning.

[0003] The following documents are cited as examples of technology for capturing radiographic images using three-dimensional information. Patent Document 1 describes a method for detecting a body area based on an image of a patient taken by a 3D scanner or the like, and automatically presenting on a screen a scanning range that selectively covers the detected body area. Patent Document 2 describes an X-ray diagnostic device that calculates the arc-shaped trajectory of a blood vessel of interest based on standard three-dimensional model data relating to blood vessels at any location, and assists in optimizing the imaging angle. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent No. 4484462 [Patent Document 2] Patent No. 4709600 Summary of the Invention [Problem to be solved by the invention]

[0005] According to the conventional technology, it is possible to support the positioning of the scanning range and the optimization of the imaging angle with respect to the blood vessel. However, the conventional technology cannot present the three-dimensional positional relationship of the internal structure of the subject in the radiation image to the radiologist. Therefore, there is no information that can be used as a basis for correcting the positioning, and therefore there is a problem that the direction of correction of the positioning cannot be efficiently determined.

[0006] SUMMARY OF THE PRESENT EMBODIMENTS Accordingly, an object of the present invention is to provide an image processing apparatus, a program, and an image processing method that are capable of efficiently determining the direction of positioning correction, etc. [Means for solving the problem]

[0007] The image processing device according to the present invention comprises: A first acquisition unit that acquires a two-dimensional radiation image; an estimation unit that performs image analysis on the acquired two-dimensional radiation image and estimates three-dimensional structural information of a structure inside the subject; an output unit that outputs the second photographing support information; Equipped with.

[0008] The program according to the present invention comprises: Computer, A first acquisition unit for acquiring a two-dimensional radiation image; an estimation unit that performs image analysis on the acquired two-dimensional radiation image and estimates three-dimensional structural information of a structure inside the subject; Function as.

[0009] The image processing method according to the present invention comprises: An acquisition step of acquiring a two-dimensional radiation image; an estimation step of estimating three-dimensional structural information of a structure inside a subject by image analysis of the acquired two-dimensional radiation image; has. Effect of the Invention

[0010] According to the present invention, since three-dimensional structural information of a structure inside a subject of a radiological image is estimated, it is possible to efficiently determine a direction for correcting positioning, etc. [Brief description of the drawings]

[0011] [Figure 1] 1 is a diagram showing a schematic configuration of a photography support system according to an embodiment of the present invention. [Diagram 2] FIG. 2 is a block diagram of an imaging control device according to the present embodiment. [Diagram 3] 11 is a diagram showing an example of a configuration of a shooting support information output table stored in a storage unit according to the embodiment. FIG. [Figure 4A] FIG. 1 is a diagram for explaining the movements of external rotation and internal rotation. [Figure 4B] FIG. 1 is a diagram for explaining abduction and adduction movements. [Diagram 5] FIG. 13 is a diagram showing an example of a conventional analysis result of positioning deviation in a radiological image of the “lateral surface of the knee (right knee) joint.” [Figure 6A] FIG. 2 is a diagram illustrating an example of input data used when training a machine learning model according to the present embodiment. [Figure 6B] FIG. 1 is a diagram showing an example of correct answer data used when training a machine learning model according to an embodiment of the present invention. [Figure 7] FIG. 13 is a diagram showing an example of other correct answer data used when training the machine learning model according to the present embodiment. [Figure 8] 5 is a flowchart showing an example of the operation of the imaging control device when capturing a radiation image of a subject according to the present embodiment. [Figure 9] FIG. 4 is a diagram showing an example of the configuration of a shooting screen displayed on a display unit according to the embodiment. [Figure 10] 10 is a flowchart showing an example of an operation of the shooting control device during re-shooting determination processing according to the present embodiment. [Figure 11A] 1 is a diagram showing an example of three-dimensional structural information of a radiological image estimated by a machine learning model according to the present embodiment. FIG. [Figure 11B]1 is a diagram showing an example of three-dimensional structural information of a radiological image estimated by a machine learning model according to the present embodiment. FIG. [Figure 12] 5 is a flowchart showing an example of a first determination process according to the embodiment. [Figure 13] 10 is a flowchart showing an example of a second determination process according to the embodiment. [Figure 14] 11 is a diagram showing an example of shooting support information displayed on a shooting screen according to the embodiment. FIG. [Figure 15] FIG. 13 is a diagram showing an example of a photographed screen in the case where character information is displayed in addition to each line information as three-dimensional structure information. [Figure 16] FIG. 13 is a diagram showing an example of an imaging screen when only medial condyle side line information is displayed as three-dimensional structural information. [Figure 17] FIG. 13 is a diagram showing a schematic configuration of a photography support system according to another embodiment 1. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Preferred embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings.

[0013] [Example of configuration of the imaging support system 10A] FIG. 1 is a diagram showing a schematic configuration of a photography support system 10A according to this embodiment. The imaging support system 10A includes a radiographic imaging device 1, an imaging control device 2, a radiation generating device 3, an image management device 4, and an HIS / RIS 5. In the following, the radiographic imaging device may be referred to as an imaging device 1, and the radiation generating device may be referred to as a generating device 3. HIS is an abbreviation for Hospital Information System, which is a hospital information system. RIS is an abbreviation for Radiology Information System, which is a radiology department information system. A radiographic image is an example of a medical image. The imaging control device 2 is an example of an image processing device.

[0014] The photographing device 1, the photographing control device 2, the generating device 3, the image management device 4, and the HIS / RIS 5 are connected to each other so as to be able to communicate with each other via a network N. Examples of the network N include a LAN, a WAN, and the Internet. LAN is an abbreviation for Local Area Network. WAN is an abbreviation for Wide Area Network. The communication method of the network N may be wired communication or wireless communication. Examples of wireless communication include Wi-Fi (registered trademark).

[0015] The generating device 3 includes a generator 31 , a switch 32 and a radiation source 33 . The generator 31 applies a voltage according to preset imaging conditions to a radiation source 33 including, for example, a tube, based on the operation of the switch 32. The generator 31 may have an operation unit that accepts input of irradiation conditions and the like.

[0016] When a voltage is applied from the generator 31 to the radiation source 33, the radiation source 33 generates radiation R with a dose corresponding to the applied voltage. The radiation R is, for example, an X-ray.

[0017] The generating device 3 generates radiation R in a manner corresponding to the type of radiation image, for example, a still image or a dynamic image. Specifically, in the case of a still image, the generating device 3 irradiates radiation R only once per pressing of the switch 32. In the case of a dynamic image, the generating device 3 irradiates pulsed radiation R repeatedly multiple times per predetermined time per pressing of the switch 32, for example.

[0018] The imaging device 1 generates digital image data showing the imaging region of the subject S. For example, a portable FPD is used as the imaging device 1. FPD is an abbreviation for Flat Panel Detector. The imaging device 1 may be configured integrally with the generating device 3.

[0019] Although not shown, the imaging device 1 includes, for example, an imaging element, a sensor board, a scanning unit, a readout unit, a control unit, and a communication unit. The imaging element generates an electric charge according to the dose when exposed to radiation R. The sensor board has switch elements arranged two-dimensionally (in a matrix) and stores and releases electric charge. The scanning unit switches each switch element on / off. The readout unit reads out the amount of electric charge released from each pixel as a signal value. The control unit generates image data of the radiation image from the multiple signal values ​​read out by the readout unit. The image data includes still image data or dynamic image data. The communication unit transmits the generated image data and various signals to other devices such as the imaging control device 2, and receives various information and various signals from other devices.

[0020] The imaging control device 2 sets imaging conditions for the imaging device 1 and the generator 3, and controls the reading operation of the radiation image captured by the imaging device 1. The imaging control device 2 is also called a console, and is composed of, for example, a personal computer. The imaging control device 2 determines whether reimaging is necessary or not depending on the positioning deviation of the radiation image obtained by imaging. Here, positioning refers to, for example, the positioning of the patient during imaging. When the imaging control device 2 determines that reimaging is necessary, it displays imaging support information I and three-dimensional structure information T on the screen of the display unit 22 described later. The imaging support information I assists in correcting the positioning by presenting the direction of correction of the positioning by words or sentences. The three-dimensional structure information T is information indicating the three-dimensional positional relationship of the structure inside the subject in the radiation image, and presents information that serves as a basis for correcting the positioning.

[0021] The imaging conditions include, for example, patient conditions related to the subject S, irradiation conditions related to the irradiation of radiation R, and image reading conditions related to the image reading of the imaging device 1. The patient conditions include, for example, the imaging part, imaging direction, physique, etc. The irradiation conditions include, for example, tube voltage (kV), tube current (mA), irradiation time (ms), current-time product (mAs value), etc. The image reading conditions include, for example, pixel size, image size, frame rate, etc. The imaging control device 2 may automatically set the imaging conditions based on order information acquired from the HIS / RIS 5, etc. Alternatively, the imaging control device 2 may allow a user, such as a radiologist, to manually set the imaging conditions by operating an operation unit 21, which will be described later.

[0022] The image management device 4 manages the image data generated by the imaging device 1. The image management device 4 is an image archiving and communication system, an image diagnosis workstation, etc. The image archiving and communication system is sometimes called PACS. PACS is an abbreviation for Picture Archiving and Communication System.

[0023] The HIS / RIS 5 receives order information regarding radiography of a patient from, for example, a doctor, and transmits the received order information to the radiography control device 2. The order information includes various information such as the patient's ID, the body part to be radiographed, the radiography direction, and the physique.

[0024] [Example of block diagram of imaging control device 2] FIG. 2 is a block diagram of the imaging control device 2. The imaging control device 2 includes a control unit 20, an operation unit 21, a display unit 22, a storage unit 23, and a communication unit 24. The control unit 20, the operation unit 21, the display unit 22, the storage unit 23, and the communication unit 24 are connected to each other so as to be able to communicate with each other via a bus 25, for example.

[0025] The control unit 20 includes a processor such as a CPU. CPU is an abbreviation for Central Processing Unit. The processor executes programs stored in a memory such as a RAM (which may be the storage unit 23) to realize various processes including shooting control and re-shooting determination. The control unit 20 may include electronic circuits such as ASIC and FPGA. ASIC is an abbreviation for Application Specific Integrated Circuit. FPGA is an abbreviation for Field Programmable Gate Array.

[0026] The operation unit 21 receives instructions according to various input operations from the user, converts the received instructions into operation signals, and outputs the operation signals to the control unit 20. The operation unit 21 includes, for example, a mouse, a keyboard, a switch, a button, and the like. The operation unit 21 may be, for example, a touch panel integrally combined with a display. The operation unit 21 may be, for example, a user interface that receives voice input from a microphone or the like.

[0027] The display unit 22 displays a radiographic image based on image data received from the imaging device 1, a GUI for receiving various input operations from the user, and the like. GUI is an abbreviation for Graphical User Interface. The display unit 22 is, for example, a display such as a liquid crystal display or an organic EL display. EL is an abbreviation for Electro Luminescence. Specifically, the display unit 22 displays a radiographic image obtained by imaging with the imaging device 1, and also displays imaging support information I and three-dimensional structure information T according to the result of the re-imaging determination process.

[0028] The storage unit 23 stores, for example, a system program, an application program, and various data. The storage unit 23 includes any storage module, for example, an HDD, an SSD, a ROM, and a RAM. HDD is an abbreviation for Hard Disk Drive. SSD is an abbreviation for Solid State Drive. ROM is an abbreviation for Read Only Memory. Specifically, the storage unit 23 stores a shooting support information output table 23b and a machine learning model (trained model) 23c. The machine learning model 23c and the like may be stored in an external storage device or the like. Details of the shooting support information output table 23b and the machine learning model 23c will be described later.

[0029] The communication unit 24 includes, for example, a communication module including a NIC, a receiver, and a transmitter. NIC is an abbreviation for Network Interface Card. The communication unit 24 communicates various data such as image data between the photographing device 1 and the image management device 4 via the network N.

[0030] In this embodiment, the control unit 20 functions as a first acquisition unit (acquisition step), an extraction unit (extraction step), an estimation unit, and an output unit. The functions of the first acquisition unit, the extraction unit, the estimation unit, and the output unit are realized by the processor of the control unit 20 executing a program stored in the storage unit 23 or the like.

[0031] The first acquisition unit acquires a two-dimensional radiographic image captured by the imaging device 1. The control unit 20 may also function as a second acquisition unit that acquires imaging body part information from order information or the like transmitted from the HIS / RIS 5 or the like. The imaging body part information can be used when changing parameters or algorithms of a machine learning model described later. The estimation unit extracts structures inside the subject from the two-dimensional radiographic image based on the imaging body part information acquired from the second acquisition unit.

[0032] The estimation unit performs image analysis of the two-dimensional radiographic image acquired by the first acquisition unit, and estimates three-dimensional structural information T of the structure inside the subject of the radiographic image. In this embodiment, the structure inside the subject includes at least a first structure located on the front side and a second structure located on the back side. The front side is the side of the generating device 3 such as the radiation source 33, and the back side is the side of the imaging device 1. The estimation unit may estimate the three-dimensional structural information T of the structure inside the subject of the radiographic image using a machine learning model 23c that has been trained in advance. In this case, the estimation unit inputs the acquired two-dimensional radiographic image into the machine learning model 23c, so that the two structures inside the subject in the radiographic image can be distinguished into a first structure located on the front side and a second structure located on the back side. Details of the three-dimensional structural information T of the structure using the machine learning model 23c will be described later. The estimation of the three-dimensional structural information T of the structure inside the subject can also be performed without using the machine learning model 23c. In this case, the control unit 20 functions as an extraction unit and an estimation unit. The functions of the extraction unit and the estimation unit are realized by the processor of the control unit 20 executing a program stored in the storage unit 23 or the like. Specifically, the extraction unit and the estimation unit estimate the three-dimensional structural information T of the structure inside the subject by executing techniques such as recognition of the structure by edge detection, which is image processing, estimation of the structure by histogram analysis, and recognition of the structure by matching with a correct image by pattern matching.

[0033] The output unit outputs three-dimensional structure information T of a structure inside the subject of the radiographic image estimated by the estimation unit. For example, the output unit controls output so that the estimated three-dimensional structure information T of the structure is superimposed on the radiographic image displayed on the display unit. The output unit also functions as a re-photographing support information output unit, and outputs re-photographing support information based on the three-dimensional structure information T estimated by the estimation unit. The re-photographing support information includes photographing support information I for changing the position of the subject S or the photographing device 1 when re-photographing.

[0034] The imaging control device 2 may be configured not to include the operation unit 21 and the display unit 22. In this case, the imaging control device 2 may receive a control signal from an operation unit provided in an external device connected via the communication unit 24. The imaging control device 2 may also output an image signal to a display unit provided in the external device to display a radiation image or the like. The external device may be an image management device 4 or the like, or may be another device.

[0035] [Example of configuration of shooting support information output table 23b] Next, an example of the configuration of the shooting support information output table 23b stored in the storage unit 23 will be described.

[0036] FIG. 3 shows an example of the configuration of the shooting support information output table 23b. The photographing support information output table 23b stores a photographing part, a correction direction when correcting a positioning deviation, and photographing support information I for presenting information on the basis of the correction direction in association with each other. Examples of the photographing part include a "side surface of the knee joint," a "side surface of the leg," and a "side surface of the elbow."

[0037] Examples of the types of correction directions when correcting positioning deviation include "external rotation", "internal rotation", "abduction", and "adduction". FIG. 4A is a diagram for explaining the movements of external rotation and internal rotation. External rotation is a movement in which the femur is rotated outward with the long axis of the bone as a reference, as shown on the left side of FIG. 4A. Internal rotation is a movement in which the femur is rotated inward with the long axis of the bone as a reference, as shown on the right side of FIG. 4A. FIG. 4B is a diagram for explaining the movements of abduction and adduction. Abduction is a movement in which the femur is moved away from the central axis of the body. Adduction is a movement in which the femur is moved closer to the central axis of the body.

[0038] Specifically, when the imaging part is the "lateral side of the knee joint" and the positioning correction direction is "external rotation", imaging support information I, for example, "Please externally rotate the knee" is associated with the imaging part. When the imaging part is the "lateral side of the knee joint" and the positioning correction direction is "internal rotation", imaging support information I, for example, "Please internally rotate the knee" is associated with the imaging part. When the imaging part is the "lateral side of the knee joint" and the positioning correction direction is "abduction", imaging support information I of "please abduct the knee", for example, is associated with the part. When the imaging part is the "lateral side of the knee joint" and the positioning correction direction is "adduction", imaging support information I, for example, "please adduct the knee" is associated with the imaging part.

[0039] [About machine learning] Fig. 5 shows an example of a conventional analysis result of positioning deviation in a radiological image Ga of a "side of a right knee joint". In Fig. 5, the horizontal direction (left-right direction) of the radiological image Ga is the X direction, and the vertical direction (up-down direction) of the radiological image Ga is the Y direction. The direction perpendicular to the X direction and Y direction of the radiological image Ga and the direction of irradiation of radiation R is the Z direction. The orthogonal coordinate system including the above-mentioned X direction, Y direction, and Z direction is also applied to the radiological images and the imaging screen 80 that displays the radiological images described below.

[0040] When there is a positioning deviation in the radiographic image Ga of the "side of the right knee joint", a deviation occurs in the medial condyle and the lateral condyle, which are located in the Z direction of the epiphysis, which is a structure inside the subject. In this case, since the radiographic image Ga is constructed in two dimensions, the line indicating the medial condyle and the line indicating the lateral condyle of the "femoral condyle" are displayed on the same plane. Therefore, as shown in FIG. 5, the medial condyle and the lateral condyle are displayed by two lines in the "femoral condyle" of the radiographic image Ga. These two lines are called the first epiphysis Tx and the second epiphysis Ty. In the conventional image analysis, it was not possible to determine which of the first epiphysis Tx and the second epiphysis Ty is the line indicating the medial condyle side located on the near side and which is the line indicating the lateral condyle side located on the far side. In other words, it was not possible to determine the three-dimensional positional relationship in the Z direction of the two first epiphysis Tx and the second epiphysis Ty.

[0041] In this embodiment, the machine learning model 23c is trained by machine learning using machine learning data by a learning device. The learning device is, for example, configured by a computer and has a processor such as a CPU and a GPU. GPU is an abbreviation for Graphics Processing Unit. The processor realizes a predetermined machine learning function by executing a program stored in a memory such as a RAM. The learning device may be a client device or a server device.

[0042] The machine learning model 23c outputs, as estimated data, each of the lines on the medial condyle side and the lateral condyle side of the "femoral condyle" in the radiographic image Ga shown in Fig. 5. The machine learning model 23c may be, for example, a regression model such as polynomial regression, multiple regression, support vector regression, or random forest regression. The machine learning model 23c may also be another model such as a neural network.

[0043] Fig. 6A shows an example of input data Gb used when training the machine learning model 23c. Fig. 6B shows an example of correct answer data Gc used when training the machine learning model 23c.

[0044] The machine learning data includes input data Gb input to the machine learning model 23c and answer data Gc output from the machine learning model 23c. For example, as shown in FIG. 6A, the input data Gb is radiation image data of the "lateral side of the knee joint" when the outer side of the patient's right knee is photographed in a state where the outer side is in contact with the imaging device 1. The input data Gb has a positioning deviation in, for example, the "femoral condyle." The input data Gb may be a past radiation image of the patient obtained by actual imaging.

[0045] The correct answer data Gc is radiation image data of the "lateral side of the knee joint" when the outside of the patient's right knee is photographed in contact with the photographing device 1 as shown in FIG. 6B. The correct answer data Gc has information T1 of the medial condyle side line indicating the medial condyle region and information T2 of the lateral condyle side line indicating the lateral condyle region, which are correct for the input data Gb. Here, the medial condyle side line is the boundary between the medial condyle side of the femoral condyle and the knee soft tissue. Also, the lateral condyle side line is the boundary between the lateral condyle side of the femoral condyle and the knee soft tissue. Hereinafter, the information T1 of the medial condyle side line is referred to as the medial condyle side line information T1, and the information T2 of the lateral condyle side line is referred to as the lateral condyle side line information T2. ​​The correct answer data Gc may be created by a user such as a radiologist, for example. The user may specify the medial condyle side line information T1 and the lateral condyle side line information T2 from the radiation image and tag the line information thereto. This allows the attenuated correct answer data Gc to be created. The medial condyle side line information T1 and the lateral condyle side line information T2 according to this embodiment correspond to the three-dimensional structure information T.

[0046] The learning device performs machine learning using a data set including the above-mentioned input data Gb and correct answer data Gc, and creates a trained machine learning model 23c. When the input data Gb of the "right knee joint side" is input, the machine learning model 23c outputs the medial condyle side line information T1 and the lateral condyle side line information T2, which are correct answers, if there is a positioning deviation in the "femoral condyle". In other words, the medial condyle and the lateral condyle of the "femoral condyle", which is the epiphysis, are distinguished into the medial condyle side line information T1 located on the near side and the lateral condyle side line information T2 located on the far side. The trained machine learning model 23c is stored in, for example, the storage unit 23 of the imaging control device 2. The imaging control device 2 can identify the type of positioning deviation based on the medial condyle side line information T1 and the lateral condyle side line information T2 output from the machine learning model 23c. In this case, the imaging control device 2 can identify whether the "femoral condyle" is internally rotated or externally rotated as the type of positioning deviation.

[0047] Although the positional relationship in the Z direction between the medial condyle and the medial condyle of the femoral condyle is learned using the medial condyle side line information T1 and the like actually specified by the user, the learning method is not limited to this. For example, as another learning method, the coordinate points of the medial condyle and the lateral condyle of the femoral condyle may be extracted, and the positional relationship in the Z axis direction between the medial condyle and the medial condyle may be estimated by, for example, regressing the extracted consecutive coordinate points.

[0048] Next, a case will be described in which the "femoral condyle" is determined to be inverted or abducted as a type of positioning deviation. In this case, the machine learning model 23c is trained by machine learning using the correct answer data Gd different from the correct answer data Gc shown in FIG.

[0049] Fig. 7 shows an example of another correct answer data Gd used when training the machine learning model 23c. Note that, since the input data Gb can use the radiation image shown in Fig. 6A, a detailed description thereof will be omitted.

[0050] The machine learning data includes input data Gb input to the machine learning model 23c and correct answer data Gd indicating the correct answer of the output of the machine learning model 23c. As shown in FIG. 7, the correct answer data Gd includes information C1 indicating the femoral condyle center and information C2 indicating the crus condyle center, which are correct answers for the input data Gb. Hereinafter, the information C1 indicating the femoral condyle center is referred to as femoral condyle center information C1, and the information C2 indicating the crus condyle center is referred to as crus condyle center information C2. The correct answer data Gd also includes joint information for identifying whether the radiographic image of the input data Gb is a right knee joint or a left knee joint. The correct answer data Gd may be created by a user such as a radiologist. For example, the coordinates of the femoral condyle center information C1 and the coordinates of the crus condyle center information C2, which are correct answers for the "femoral condyle", are determined by the user by visual inspection or the like. Next, two heat maps are created according to the accuracy of the determined coordinates of the femoral condyle center information C1 and the femoral condyle center information C2. The created heat maps are used as the correct answer data Gd.

[0051] When the input data Gb of the "side of the knee joint" is input, the machine learning model 23c estimates the femoral condyle center information C1, the tibial condyle center information C2, and joint information indicating whether the knee joint is left or right, which are correct answers. The trained machine learning model 23c is stored, for example, in the memory unit 23 of the imaging control device 2. The imaging control device 2 identifies the type of positioning deviation based on the femoral condyle center information C1, etc. output from the machine learning model 23c. In this case, the imaging control device 2 can identify whether the "femoral condyle" is adduced or abducted as the type of positioning deviation.

[0052] Although the femoral condyle center information C1 of the femoral condyle and the like are learned using a heat map, other learning methods may be used. As another learning method, learning may be performed using the center coordinates of the femoral condyle themselves. In addition, although the knee joint is specified by choosing between the right knee and the left knee, the knee joint may be specified using other information such as the positional relationship between the femur and the patella. Moreover, all of the above-mentioned medial condyle side line information T1, femoral condyle center information C1, etc. may be estimated using one machine learning model 23c. Moreover, a plurality of machine learning models 23c may be used. In this case, the medial condyle side line information T1, etc. may be estimated using one machine learning model 23c, and the femoral condyle center information C1, etc. may be estimated using another machine learning model 23c.

[0053] In the above example, the 3D structure of the femoral condyle in the "side of the right knee joint" when the outside of the patient's right knee is photographed in a state where it is in contact with the imaging device 1 is learned, but the present invention is not limited to this. For example, the 3D structure of the femoral condyle in the "side of the right knee joint" may be learned using a radiographic image of the "side of the right knee joint" when the inside of the patient's right knee is photographed in a state where it is in contact with the imaging device 1. In this case, the medial condyle side line information T1 is located at the back side in the Z direction, and the lateral condyle side line information T2 is located at the front side in the Z direction. In addition, the imaging site to be the subject of machine learning may be other than the "side of the right knee joint". For example, the imaging site to be the subject of machine learning may be the side of the left knee joint, or may be another site such as an ankle joint or an elbow joint.

[0054] [Example of operation of the imaging control device 2] Next, a flow of operations for capturing a radiation image of the subject S will be described. 8 is a flowchart showing an example of the operation of the imaging control device 2 when capturing a radiographic image of the subject S. In the following, a case will be described in which the imaging part and imaging direction of the radiographic image are to be captured of the "lateral surface of the knee (right knee) joint."

[0055] The communication unit 24 of the imaging control device 2 receives the order information transmitted from the HIS / RIS 5 or the like. For example, the user selects predetermined order information from the examination list displayed on the screen of the display unit 22 of the imaging control device 2. The control unit 20 acquires the order information selected by the user (step S10).

[0056] When the control unit 20 acquires predetermined order information, it causes the display unit 22 to display the photographing screen 80 (step S11).

[0057] FIG. 9 shows an example of the configuration of a shooting screen 80 displayed on the display unit 22 of the shooting control device 2. As shown in FIG. The imaging screen 80 is provided with an imaging selection section 81, a condition setting section 82, an image display area 83, a patient information display section 84, and an examination end button 85. The imaging selection section 81 displays, for example, order information selected in the examination list. The order information includes, for example, imaging details such as the imaging part and imaging direction, and a thumbnail image showing the captured radiation image. The condition setting section 82 is provided with, for example, a button for setting imaging conditions for the radiation image, a button for adjusting the image of the captured radiation image, and the like. The image display area 83 displays a radiation image captured by the imaging device 1 based on the set imaging conditions, etc. In FIG. 9, since the radiation image has not yet been captured, no radiation image is displayed in the image display area 83.

[0058] When predetermined order information is selected by the photography selection unit 81 or the like, the control unit 20 sets photography conditions for each of the photography device 1 and the generation device 3 (step S12). The photography conditions include image reading conditions to be set for the photography device 1 and irradiation conditions to be set for the generation device 3. For example, the control unit 20 sets image reading conditions for the photography device 1 based on the body part to be photographed, the photography direction, etc. of the selected order information. The control unit 20 also sets irradiation conditions for the generation device 3 based on the body part to be photographed, the photography direction, etc. of the selected order information.

[0059] The photographing conditions may be manually set by the user. Specifically, the control unit 20 may set the image reading conditions received by the user's input operation on the condition setting unit 82 to the photographing device 1. The control unit 20 may set the radiation irradiation conditions received by the user's input operation on the operation panel of the generator 3 to the generator 3.

[0060] Next, when the user turns on the switch 32, the control unit 20 controls the imaging device 1, the generator 3, etc. to capture a radiographic image of the subject S (step S13). The generator 3 irradiates radiation R to the imaging region of the subject S. The imaging device 1 detects the radiation R that has passed through the subject S from the generator 3, and generates image data showing the imaging region based on the detected radiation R. The imaging device 1 transmits the generated image data to the imaging control device 2. The control unit 20 acquires a radiographic image based on the image data transmitted from the imaging device 1 (step S13).

[0061] When step S13 ends, the process branches to step S14 and step S15. In this embodiment, an example in which the process of step S14 and the process of step S15 are performed in parallel will be described, but the present invention is not limited to this. For example, the process of step S14 and the process of step S15 may be performed in series in order. In this case, step S15 may be the first process, and step S14 may be processed together with step S16, which will be described later.

[0062] First, the process of step S14 will be described. The control unit 20 displays the acquired radiographic image in the image display area 83 of the imaging screen 80 (step S14). In this embodiment, a radiographic image of the "lateral side of the right knee joint" is displayed in the image display area 83 as the imaging site, etc. A thumbnail image showing the radiographic image that has been captured is displayed in the order information 81a of the imaging selection unit 81. When step S14 is completed, the process proceeds to step S16.

[0063] Next, the process of step S15 branched from step S13 will be described. The control unit 20 executes a re-photographing determination process for determining whether or not re-photographing is required using the acquired radiographic image. The control unit 20 proceeds to a subroutine shown in FIG.

[0064] Fig. 10 is a flowchart showing an example of the operation of the imaging control device 2 during the re-imaging determination process. Fig. 11A and Fig. 11B show an example of three-dimensional structure information T of the femoral condyle of the radiographic image G estimated by the machine learning model 23c.

[0065] The control unit 20 uses the machine learning model 23c to estimate the medial condyle side line information T1 and the lateral condyle side line information T2 of the femoral condyle from the radiographic image G obtained by imaging (step S20). Specifically, the control unit 20 inputs the radiographic image G of the "knee joint side" to the machine learning model 23c. The machine learning model 23c outputs the medial condyle side line information T1 and the lateral condyle side line information T2 of the femoral condyle, which are the correct answers, as estimated data based on the input radiographic image G. In FIG. 11A, the medial condyle side line information T1 is shown by a thin line, and the lateral condyle side line information T2 is shown by a thick line. In this way, the control unit 20 acquires the three-dimensional structure information T that is distinguished into the medial condyle side line information T1 located on the near side and the lateral condyle side line information T2 located on the far side.

[0066] Next, the control unit 20 uses the machine learning model 23c to estimate femoral condyle center information C1, tibia condyle center information C2, and joint information from the radiographic image G obtained by imaging (step S21). Specifically, the control unit 20 inputs a radiographic image of the "knee joint lateral surface" to the machine learning model 23c. Based on the input radiographic image G, the machine learning model 23c outputs femoral condyle center information C1 and tibia condyle center information C2 of the femoral condyle as estimated data. Also, based on the input radiographic image G, the machine learning model 23c outputs joint information indicating that the radiographic image G is a right knee joint. In this way, the control unit 20 acquires femoral condyle center information C1, tibia condyle center information C2, and joint information. The joint information may be acquired from the examination order information transmitted from the HIS / RIS5.

[0067] In this embodiment, steps S20 and S21 are described separately, but steps S20 and S21 may be combined into one step. Specifically, the machine learning model 23c may output all of the medial condyle side line information T1, the lateral condyle side line information T2, the femoral condyle center information C1, the tibial condyle center information C2, and the joint information based on the input radiographic image G. After step S21 is completed, the process proceeds to step S22.

[0068] The control unit 20 judges whether the positioning deviation is internal rotation or external rotation based on the intersection pattern between the lines radially extending from the femoral condyle center information C1 and the medial condyle side line information T1, etc. (Step S22). In this embodiment, the process of judging whether the positioning deviation is internal rotation or external rotation is referred to as a first judgment process.

[0069] The control unit 20 proceeds to a subroutine shown in Fig. 12 and executes the first determination process by functioning as a first determination unit. Fig. 12 is a flowchart showing an example of the first determination process.

[0070] The control unit 20 extends a plurality of lines L radially from the estimated femoral condyle center information C1 (step S30). Specifically, as shown in FIG. 11A, six lines L are extended radially from the femoral condyle center information C1 at equal intervals in the circumferential direction. The line L that does not intersect with either the medial condyle side line information T1 or the lateral condyle side line information T2 is called the first line L1. In FIG. 11A, the first line L1 is shown by a dashed line. The line L that intersects with the medial condyle side line information T1 and the lateral condyle side line information T2 in this order is called the second line L2. In FIG. 11A, the second line L2 is shown by a dashed line. The line L that intersects with the lateral condyle side line information T2 and the medial condyle side line information T1 in this order is called the third line L3. In FIG. 11A, the third line L3 is shown by a two-dot chain line. Note that the first line L1, the second line L2, and the third line L3 may be collectively called the line L.

[0071] The control unit 20 judges whether the order of the multiple lines L is to be viewed clockwise or counterclockwise based on the estimated joint information. For example, the "right knee joint" in the joint information is associated with the counterclockwise direction, and the "left knee joint" in the joint information is associated with the clockwise direction. In this embodiment, since the joint information is the "right knee joint", the order of the multiple lines L is to be viewed counterclockwise. The control unit 20 judges whether the order is the first line L1, the second line L2, and the third line L3 when viewed counterclockwise based on the first line L1 in the center of FIG. 11A (step S31). If the control unit 20 judges that the condition of step S31 is satisfied, the process proceeds to step S32.

[0072] When the order is the first line L1, the second line L2, and the third line L3, the control unit 20 determines that the femoral condyle is internally rotated (step S32). That is, the control unit 20 determines that the femoral condyle should be externally rotated in order to correct the positioning in the correct direction. Note that FIG. 11A shows an example in which the femoral condyle in the radiographic image G is externally rotated, and step S32 does not apply to FIG. 11A.

[0073] On the other hand, if the control unit 20 determines that the condition of step S31 is not satisfied, the process proceeds to step S33. The control unit 20 determines whether the order is the first line L1, the third line L3, and the second line L2 when viewed counterclockwise from the first line L1 in the center of FIG. 11A (step S33). If the control unit 20 determines that the condition of step S33 is satisfied, the process proceeds to step S34.

[0074] When the order is the first line L1, the third line L3, and the second line L2, the control unit 20 determines that the femoral condyle is externally rotated (step S34). That is, the control unit 20 determines that the femoral condyle should be internally rotated in order to correct the positioning in the correct direction. Note that Fig. 11A shows an example in which the femoral condyle in the radiographic image G is externally rotated, and step S34 corresponds to Fig. 11A.

[0075] On the other hand, when the control unit 20 determines that the condition of step S33 is not satisfied, the control unit 20 proceeds to step S35. In this case, the control unit 20 determines that the medial condyle side line information T1 and the lateral condyle side line information T2 of the femoral condyle overlap, and that no deviation of the femoral condyle has occurred (step S35). Note that the determination that no deviation of the femoral condyle has occurred is not limited to the case where the medial condyle side line information T1 and the lateral condyle side line information T2 completely overlap. When the amount of deviation of the overlap between the medial condyle side line information T1 and the lateral condyle side line information T2 is within the allowable range, it may be determined that no deviation of the femoral condyle has occurred. When step S35 ends, the control unit 20 ends the subroutine of the first determination process, and proceeds to step S23 in FIG.

[0076] In the first determination process, the case of the right knee joint has been described, but the type of positioning deviation can be determined in the case of the left knee joint, etc., by the same process. In the case of the left knee joint, the intersections of the multiple lines L radially extending from the femoral condyle center information C1 and the medial condyle side line information T1, etc. are viewed clockwise in the radiographic image G. If the order is the first line L1, the second line L2, and the third line L3, the control unit 20 determines that the femoral condyle is internally rotated. If the order is the first line L1, the third line L3, and the second line L2, the control unit 20 determines that the femoral condyle is externally rotated.

[0077] Next, the control unit 20 judges whether the positioning deviation is adduction or abduction based on the intersection pattern of the line connecting the femoral condyle center information C1 and the tibia condyle center information C2 with the medial condyle side line information T1, etc. (Step S23). In this embodiment, the process of judging whether the positioning deviation is adduction or abduction is called the second judgment process.

[0078] The control unit 20 proceeds to a subroutine shown in Fig. 13 and functions as a second determination unit to execute the second determination process. Fig. 13 is a flowchart showing an example of the second determination process.

[0079] As shown in FIG. 11B, the control unit 20 connects the estimated femoral condyle center information C1 and the tibia condyle center information C2 with a fourth line L4 (step S40).

[0080] When the fourth line L4 is viewed from the femoral condyle center information C1 toward the tibia condyle center information C2, the control unit 20 determines whether the fourth line L4 intersects with the medial condyle side line information T1 and the lateral condyle side line information T2 in that order (step S41). If the control unit 20 determines that the condition of step S41 is satisfied, the process proceeds to step S42.

[0081] When the fourth line L4 intersects with the medial condyle side line information T1 and the lateral condyle side line information T2 in this order, the control unit 20 determines that the femoral condyle is adducted (step S42). That is, the control unit 20 determines that the femoral condyle should be abducted in order to correct the positioning to the correct direction. Note that FIG. 11B shows an example in which the femoral condyle in the radiographic image G is abducted, and step S42 does not apply to FIG. 11B.

[0082] On the other hand, when the control unit 20 determines that the condition of step S41 is not satisfied, the control unit 20 proceeds to step S43. When the fourth line L4 is viewed from the femoral condyle center information C1 toward the tibia condyle center information C2, the control unit 20 determines whether the fourth line L4 intersects with the lateral condyle side line information T2 and the medial condyle side line information T1 in this order (step S43). When the control unit 20 determines that the condition of step S43 is satisfied, the control unit 20 proceeds to step S44.

[0083] When the fourth line L4 intersects with the lateral condyle side line information T2 and the medial condyle side line information T1 in this order, the control unit 20 determines that the femoral condyle is abducted (step S44). That is, the control unit 20 determines that the femoral condyle should be adducted in order to correct the positioning to the correct direction. Note that FIG. 11B shows an example in which the femoral condyle in the radiographic image G is abducted, and step S44 corresponds to FIG. 11B.

[0084] On the other hand, when the control unit 20 determines that the condition of step S43 is not satisfied, the control unit 20 proceeds to step S45. In this case, the control unit 20 determines that the medial condyle side line information T1 and the lateral condyle side line information T2 of the femoral condyle overlap, and that no misalignment has occurred (step S45). The determination that no misalignment has occurred also includes the case where the amount of misalignment of the medial condyle side line information T1 and the lateral condyle side line information T2 is within the allowable range. Note that step S45 is a process common to step S35 in FIG. 12, and may be omitted. When step S45 ends, the control unit 20 ends the subroutine of the second determination process, and proceeds to step S24 in FIG.

[0085] In the second determination process, the case where the imaging part is the lateral side of the right knee joint has been described, but the type of positioning deviation can be determined by the same process even in the case of the lateral side of the left knee joint, etc. The specific determination method is the same as in the case of the lateral side of the right knee joint, so a detailed description will be omitted. In addition, whether the femoral condyle is adducted or not is determined using the fourth line L4 connecting the femoral condyle center information C1 and the tibia condyle center information C2 shown in FIG. 11B, but is not limited to this. The following determination method may be used. For example, a triangular virtual area V shown by a broken line in FIG. 11B is assumed between the femoral condyle center information C1 and the tibia condyle center information C2. Next, multiple lines are drawn in the virtual area V in the direction from the femoral condyle center information C1 to the tibia condyle center information C2. At this time, the number of lines that cross the medial condyle side line information T1 and the lateral condyle side line information T2 in this order is compared with the number of lines that cross the lateral condyle side line information T2 and the medial condyle side line information T1 in this order. Finally, whether the femoral condyle is adducted or abducted may be determined based on which number is larger.

[0086] Next, the control unit 20 derives from the storage unit 23 the photographing support information I associated with the positioning determination results in steps S22 and S23 (step S24). Specifically, for example, when the result of the first determination process is that the knee is externally rotated as shown in Fig. 11A, the control unit 20 refers to the photographing support information output table 23b to acquire the photographing support information I of "Please internally rotate your knee." Also, for example, when the result of the second determination process is that the knee is abducted as shown in Fig. 11B, the control unit 20 refers to the photographing support information output table 23b to acquire the photographing support information I of "Please internally rotate your knee."

[0087] If the determination results in steps S22 and S23 indicate that there is no deviation in positioning, the control unit 20 may obtain instruction information indicating that the positioning is normal. After step S24 is completed, the control unit 20 ends the subroutine and returns to step S16 shown in FIG.

[0088] The control unit 20 executes display control to display the three-dimensional structure information T and the shooting support information I, which are the obtained determination results of the re-shooting determination process, on the shooting screen 80 of the display unit 22 (step S16).

[0089] FIG. 14 shows an example of the three-dimensional structure information T and the shooting support information I displayed on the shooting screen 80. In the image display area 83 of the photographing screen 80, three-dimensional structure information T is displayed in addition to the radiographic image G obtained by photographing. The three-dimensional structure information T includes medial condyle side line information T1 indicating the medial condyle region and lateral condyle side line information T2 indicating the lateral condyle region in the femoral condyle of the radiographic image G. In the image display area 83 of the photographing screen 80, the medial condyle side line information T1 and lateral condyle side line information T2, which are correct in the femoral condyle, are displayed superimposed on the radiographic image G. The medial condyle side line information T1 and lateral condyle side line information T2 may be displayed in different colors. Specifically, the medial condyle side line information T1 may be displayed in red (thin line in FIG. 14), and the lateral condyle side line information T2 may be displayed in blue (thick line in FIG. 14). In addition, the medial condyle side line information T1 and lateral condyle side line information T2 may be displayed with different line thicknesses, or may be displayed with a solid line and a dashed line. In order to achieve this, the correspondence between the "medial condyle side line information T1" and the "lateral condyle side line information T2" and the "color, thickness, and line pattern (solid line, broken line)" is set in advance. The correspondence may be a default setting, or may be set by the user as appropriate after shipment. The user can quickly identify the three-dimensional positional relationship between the medial condyle side line information T1 and the lateral condyle side line information T2 by memorizing the correspondence in advance or by checking a correspondence table or the like showing the correspondence. The correspondence table or the like showing the correspondence may be displayed on the shooting screen 80.

[0090] Furthermore, in addition to the radiographic image G obtained by imaging, imaging support information I based on the three-dimensional structure information T is displayed in the image display area 83 of the imaging screen 80. The imaging support information I is composed of, for example, words or sentences including technical terms. Specifically, when the positioning correction direction is internal rotation, the image display area 83 displays the sentence "Please internally rotate your knee" as the imaging support information I. For example, when the positioning correction direction is adduction, the image display area 83 displays the sentence "Please internally adduce your knee" as the imaging support information I.

[0091] Rank information Ic and deviation amount information Id are displayed above the imaging support information I. The rank information Ic and deviation amount information Id are information for alerting the user that correction of positioning is necessary and presenting detailed correction contents. In this embodiment, the imaging support information I and the like are displayed in an empty space other than the radiation image G in the image display area 83, but the present invention is not limited to this.

[0092] The rank information Ic is information that indicates the degree of the positioning deviation by rank. For example, when there is no positioning deviation and re-imaging is not required, "positioning: A" is displayed as the rank information Ic. When the positioning deviation is within the allowable range and re-imaging is not required, "positioning: B" is displayed as the rank information Ic. When the positioning deviation exceeds the allowable range and re-imaging is required, "positioning: C" is displayed as the rank information Ic. FIG. 14 illustrates an example of the rank information Ic being "positioning: C."

[0093] The deviation amount information Id is information indicating the distance (deviation width) in a predetermined direction between the estimated medial condyle side line information T1 and lateral condyle side line information T2 of the femoral condyle. For example, when the distance D in the X direction between the medial condyle side line information T1 and lateral condyle side line information T2 is "4 mm", "deviation amount: 4.0 mm" is displayed in the image display area 83 as the deviation amount information Id. Note that, although the deviation amount in the X direction is displayed as the deviation amount information Id in FIG. 14, this is not limiting. For example, the deviation amount in the Y direction may be displayed, or the deviation amounts in both the X direction and the Y direction may be displayed.

[0094] The user guides the patient and corrects any deviation in positioning by checking the three-dimensional structure information T and the imaging support information I displayed on the imaging screen 80. Once the deviation in positioning has been corrected, a radiographic image is captured again.

[0095] On the other hand, when the control unit 20 determines that there is no deviation in positioning, it may display only the radiation image G in the image display area 83 of the shooting screen 80. Furthermore, when the control unit 20 determines that there is no deviation in positioning, it may display "Positioning: A" as the rank information Ic and "Deviation: 0 mm" as the deviation amount information Id in the image display area 83. Furthermore, the control unit 20 may display text or the like indicating that there is no deviation in positioning as shooting support information I in the image display area 83.

[0096] Moreover, the display method of the medial condyle side line information T1 and the lateral condyle side line information T2 is not limited to the display method shown in FIG. FIG. 15 shows an example of a photographed screen 80 in which character information is displayed as the three-dimensional structure information T in addition to each line information. In the image display area 83 of the photographing screen 80, medial condyle side line information T1 and lateral condyle side line information T2 indicating the medial condyle region of the femoral condyle are displayed as three-dimensional structure information T. The medial condyle side line information T1 is displayed in association with text information of "medial condyle (near side)" indicating that the line information is the medial condyle and is located on the near side. The lateral condyle side line information T2 is displayed in association with text information of "lateral condyle (far side)" indicating that the line information is the lateral condyle and is located on the far side. The method of displaying the text information is not limited to FIG. 14, and for example, only the text information of "medial condyle" and "lateral condyle" may be displayed.

[0097] 14 and the like, for example, only one of the medial condyle side line information T1 and the lateral condyle side line information T2 may be displayed by superimposing it on the "femoral condyle" of the radiological image G. This is because, depending on the user, if one of the medial condyle side line information T1 and the lateral condyle side line information T2 can be recognized, the other line information can be estimated.

[0098] FIG. 16 shows an example of the photographing screen 80 when only the medial condyle side line information T1 is displayed as the three-dimensional structure information T. In the image display area 83 of the photographing screen 80, the medial condyle side line information T1 indicating the medial condyle region of the femoral condyle is displayed superimposed on the radiographic image G as the three-dimensional structural information T. The medial condyle side line information T1 may be displayed in a color such as red, as in FIG. 14. When two lines indicating epiphysis can be recognized in the femoral condyle of the radiographic image G, the line indicating one epiphysis can be specified as the medial condyle side line by displaying the medial condyle side line information T1 on the photographing screen 80. This also allows the line indicating the remaining other epiphysis to be specified as the epicondyle side line. Note that, although FIG. 16 shows an example in which the medial condyle side line information T1 is displayed as the three-dimensional structural information T, the epicondyle side line information T2 may be displayed.

[0099] As described above, according to this embodiment, the three-dimensional structural information T of the structure inside the subject of the radiographic image is estimated. Specifically, by inputting the radiographic image of the "side of the right knee joint" into the machine learning model 23c, the medial condyle side line information T1 located on the near side of the "femoral condyle" and the lateral condyle side line information T2 located on the far side can be distinguished and estimated. A user such as a radiologist can specify the three-dimensional positional relationship in the "femoral condyle" by checking the medial condyle side line information T1 and the lateral condyle side line information T2 superimposed on the radiographic image on the imaging screen 80. This allows the user to efficiently determine the correction direction of the positioning. Furthermore, according to this embodiment, the imaging support information I is displayed on the imaging screen 80, so the user can quickly determine the positioning by checking the imaging support information I. As a result, the radiographic imaging can be performed quickly and the burden on the patient during positioning can be reduced.

[0100] [Another embodiment 1] In another embodiment 1, positioning support is performed using an optical camera image captured by an optical camera. In the following, differences from the above embodiment will be mainly described, and explanations of points in common with the above embodiment will be omitted. In addition, in the explanation of the other embodiment 1, parts in common with the above embodiment will be described with the same reference numerals.

[0101] FIG. 17 is a diagram showing a schematic configuration of a photographing support system 10B according to another embodiment 1. As shown in FIG. The radiography support system 10B includes a radiographic image radiography device 1, an radiography control device 2, a radiation generation device 3, an image management device 4, and an HIS / RIS 5. The generation device 3 includes a generator 31, a switch 32, a radiation source 33, and an optical camera 34. The optical camera 34 and the like are connected to the radiography control device 2 via a network N.

[0102] The optical camera 34 is disposed, for example, next to the radiation source 33. The radiation source 33 and the optical camera 34 may be integrally mounted in one housing. The optical camera 34 optically captures an image of the subject S at a timing before capturing a radiation image of the subject S. The optical camera 34 transmits the captured optical camera image according to the positioning of the patient to the imaging control device 2. The optical camera image includes a still image or a dynamic image captured continuously.

[0103] The imaging control device 2 judges the presence or absence, type, etc. of positioning deviation based on the optical camera image acquired from the optical camera 34. Specifically, the imaging control device 2 estimates three-dimensional structure information T such as medial condyle side line information T1 using the machine learning model 23c. The imaging control device 2 identifies the type of positioning deviation based on the three-dimensional structure information T, etc., and acquires imaging support information I corresponding to the type of deviation. The imaging control device 2 displays the imaging support information I and the three-dimensional structure information T, etc., on, for example, the display device 26 and the display unit 22. A user such as a radiologist can recognize the three-dimensional positional relationship of the deviation area from the three-dimensional structure information T, and can easily understand the correction direction of the positioning from the imaging support information I. If the positioning deviation is resolved, the user proceeds to the next step of capturing a radiographic image of the subject S.

[0104] According to another embodiment 1, as in the above-mentioned embodiment, a user such as a radiologist can specify the three-dimensional positional relationship in the "femoral condyle" by checking the medial condyle side line information T1 and the lateral condyle side line information T2 displayed superimposed on the radiation image on the imaging screen 80. This allows the user to efficiently determine the correction direction of the positioning. In addition, according to this embodiment, since the imaging support information I is displayed on the imaging screen 80, the user can quickly determine the positioning by checking the imaging support information I. As a result, the radiation imaging can be performed quickly and the burden on the patient during positioning can be reduced. Furthermore, by using the optical camera 34, the positioning can be corrected before the radiation imaging. This reduces the number of re-imagings, and reduces the total radiation exposure of the patient, thereby reducing the burden on the patient.

[0105] Although the preferred embodiments of the present disclosure have been described in detail above with reference to the accompanying drawings, the technical scope of the present disclosure is not limited to such examples. In addition, various modifications and improvements will naturally belong to the technical scope of the present disclosure within the scope of the technical ideas described in the claims of those skilled in the art.

[0106] In the above embodiment, the three-dimensional structural information T of the "femoral condyle" is estimated when the radiographic image is of the side of the knee joint, but the imaging site to be estimated is not limited to the side of the knee joint. For example, when the radiographic image is of the side of the ankle joint, the side of the elbow joint, etc., the three-dimensional structural information T of the structure inside each subject can be estimated by applying the re-photographing determination process described above. [Explanation of symbols]

[0107] 1. Radiation imaging device (imaging device) 10A, 10B Photography Support System 2. Shooting control device (image processing device) 20 Control section 23 Memory section 23b Shooting support information output table I. Shooting support information T 3D structure information

Claims

1. A first acquisition unit that acquires a two-dimensional radiation image; an estimation unit that performs image analysis on the acquired two-dimensional radiation image and estimates three-dimensional structural information of a structure inside a subject; An image processing device comprising:

2. An output unit that outputs the estimated three-dimensional structure information. The image processing device according to claim 1 .

3. the estimation unit estimates information about a position in a radiation irradiation direction as the three-dimensional structure information inside the subject; The image processing device according to claim 1 .

4. the estimation unit estimates, as the three-dimensional structure information inside the subject, information distinguishing between a structure on a front side and a structure on a back side in a radiation irradiation direction; The image processing device according to claim 1 .

5. an extraction unit that extracts the structure inside the subject from the two-dimensional radiation image, The estimation unit estimates three-dimensional structure information of the extracted structure as the three-dimensional structure information. The image processing device according to claim 1 .

6. A second acquisition unit that acquires imaging site information, The extraction unit extracts a structure of a subject from the two-dimensional radiation image based on the acquired imaging region information. The image processing device according to claim 5 .

7. The three-dimensional structure information estimated by the estimation unit includes information on the medial condyle line and the lateral condyle line of the femoral condyle. The image processing device according to claim 1 .

8. the output unit displays the estimated three-dimensional structural information by superimposing it on the two-dimensional radiation image. The image processing device according to claim 2 .

9. a re-photographing support information output unit that outputs re-photographing support information based on the three-dimensional structure information estimated by the estimation unit; The image processing device according to claim 1 .

10. The re-photographing support information is information for changing the position of a subject or a photographing device when re-photographing. The image processing device according to claim 9 .

11. The estimation unit is a machine learning model that is trained using a radiographic image including an internal structure of a subject as input data and three-dimensional structural information of the internal structure of the subject as correct output data, by inputting the radiographic image acquired by the first acquisition unit into the machine learning model; and estimating the three-dimensional structural information of the internal structure of the radiographic image. The image processing device according to claim 1 .

12. Computer, A first acquisition unit that acquires a two-dimensional radiation image; an estimation unit that performs image analysis on the acquired two-dimensional radiation image and estimates three-dimensional structural information of a structure inside the subject; A program to function as a

13. an output unit for outputting the estimated three-dimensional structure information; The program according to claim 12, for causing the program to function as a

14. the estimation unit estimates information about a position in a radiation irradiation direction as the three-dimensional structure information inside the subject; The program according to claim 12.

15. the estimation unit estimates, as the three-dimensional structure information inside the subject, information distinguishing between a structure on a front side and a structure on a back side in a radiation irradiation direction; The program according to claim 12.

16. causing the computer to function as an extraction unit that extracts the structure inside the subject from the two-dimensional radiation image; The estimation unit estimates three-dimensional structure information of the extracted structure as the three-dimensional structure information. The program according to claim 12.

17. The computer is caused to function as a second acquisition unit that acquires imaging site information; The extraction unit extracts a structure of a subject from the two-dimensional radiation image based on the acquired imaging region information. The program according to claim 16.

18. The three-dimensional structure information estimated by the estimation unit includes information on the medial condyle line and the lateral condyle line of the femoral condyle. The program according to claim 12.

19. the output unit displays the estimated three-dimensional structural information by superimposing it on the two-dimensional radiation image. The program according to claim 13.

20. a re-photographing support information output unit that outputs re-photographing support information based on the three-dimensional structure information estimated by the estimation unit; The program according to claim 12.

21. The re-photographing support information is information for changing the position of a subject or a photographing device when re-photographing. The program according to claim 20.

22. An acquisition step of acquiring a two-dimensional radiation image; an estimation step of estimating three-dimensional structural information of a structure inside a subject by image analysis of the acquired two-dimensional radiation image; An image processing method comprising the steps of:

Images