Photography auxiliary device, working method thereof and storage medium

Abstract

A photography assisting apparatus for a radiographic apparatus having a radiation source and a radiation image detector that detects a radiation image of an object from radiation that is irradiated from the radiation source and transmitted through the object, the photography assisting apparatus includes an optical camera that optically photographs an area including an irradiation field of the radiation that is irradiated from the radiation source to the object and outputs an optical image, and at least one processor that performs a determination process that determines, before radiography is started, whether there is a possibility of re-photography when the radiography is performed based on the optical image acquired by the optical camera using a learned model that learns a relationship between an optical image taken when the radiography is performed and whether re-photography of a radiation image taken when the radiography is performed is needed.

Description

Technical Field

[0001] The present invention relates to a photographic aid device, its operating method, and a storage medium. Background Technology

[0002] In radiographic systems used in the medical field, as preparation for radiography, after a radiographer or physician (hereinafter referred to as technicians, etc.) positions the radiographic site of the subject, radiographic imaging is performed according to the technicians' instructions. However, after the radiographic site is positioned relative to the radiation field but before radiographic imaging is performed, the position of the radiographic site may shift due to movement of the subject. Furthermore, for sites that are difficult to position, such as the sides of joints, even if the technicians believe that the positioning has been accurate, a slight positional shift may still occur.

[0003] Therefore, if there is a positional shift at the imaging site, the desired radiographic image may not be obtained. The situation where the desired radiographic image cannot be obtained through radiography, i.e., radiographic failure, is called "radiographic loss." When radiographic loss occurs, re-enhancing the image is performed. Re-enhancing is both time-consuming and labor-intensive, so it is preferable to minimize re-enhancing.

[0004] It is known that, in order to reduce the need for re-photographing of radiation-based subjects, an optical camera is used to capture optical images of the subject. During the preparation stage of photography, the positional offset of the subject is detected based on the optical image (see Japanese Patent Application Publication No. 2019-033830). In the radiation photography system described in Japanese Patent Application Publication No. 2019-033830, a positioning index image representing a set position as the ideal position of the subject relative to the radiation detector is pre-registered. The positional offset of the subject is detected based on the positioning index image and the optical image. Then, in the radiation photography system described in Japanese Patent Application Publication No. 2019-033830, if the positional offset is greater than a threshold, and radiation photography is performed directly in this state, there is a possibility of photographic loss, requiring re-photographing. Therefore, a warning is issued urging the subject to be repositioned. Summary of the Invention

[0005] The technical problem to be solved by the invention

[0006] According to the technology described in Japanese Patent Application Publication No. 2019-033830, during the preparation for radiographic imaging, it is possible to determine whether a re-enhancing imaging is necessary. However, in the technology described in Japanese Patent Application Publication No. 2019-033830, various positioning index images need to be pre-registered based on the characteristics of the subject to accommodate different subjects. For example, when the imaging site is the knee, the width of the knee, etc., varies depending on the physique of the patient being imaged, therefore, various positioning index images corresponding to different physiques need to be pre-registered.

[0007] Therefore, in the technology described in Japanese Patent Application Publication No. 2019-033830, in addition to pre-registering various positioning indicator images, it is also necessary to select positioning indicator images suitable for the subject, which is quite laborious. Thus, in the technology described in Japanese Patent Application Publication No. 2019-033830, it is not easy to grasp the possibility of re-photographing during the preparation for photography.

[0008] The technical objective of this invention is to provide a photographic aid device, its working method, and its working procedure that allows for easy assessment of the possibility of re-photographing during the pre-photographing preparation stage before radiography.

[0009] means for solving technical problems

[0010] To achieve the above objectives, the present invention provides a photographic aid device for use in a radiographic device, the radiographic device having a radiation source and a radiation image detector for detecting a radiation image of a subject based on radiation irradiated from and transmitted through the radiation source. The photographic aid device includes an optical camera for outputting an optical image by optically photographing a region containing radiation from the radiation source irradiating the subject, and at least one processor. The processor performs the following determination process: using a learned model, before radiographic photography begins, based on the optical image acquired by the optical camera, it is determined whether there is a possibility of re-photographing if radiographic photography has been performed. The learned model learns the relationship between the optical image captured during radiographic photography and whether re-photographing of the radiation image captured during radiographic photography is necessary.

[0011] The processor preferably performs the following warning notification process: when the determination process determines that there is a possibility of re-photographing, a warning is issued.

[0012] The processor preferably performs the following prohibition process: when the determination process determines that there is a possibility of re-photography, irradiation from the radiation source is prohibited.

[0013] The processor preferably performs the following prompting process: prompting corrective measures for correcting the position or orientation of the subject.

[0014] In the prompting process, the processor preferably provides, in addition to corrective measures, reasons for determining that there is a possibility of re-photographing.

[0015] The processor preferably displays at least one of the corrective measures and reasons on the display unit during the prompting process.

[0016] The processor preferably performs the following association establishment process: establishing an association between the optical image captured during radiography and the result information indicating whether the radiographic image captured during radiography has been re-photographed.

[0017] The present invention discloses a method for operating a photography assist device, the photography assist device being used in a radiography apparatus, the radiography apparatus having a radiation source and a radiation image detector for detecting a radiation image of a subject based on radiation irradiated from and transmitted through the radiation source, the photography assist device comprising an optical camera for outputting an optical image by performing optical photography on an area containing radiation from the radiation source irradiating the subject, the method of operating the photography assist device comprising the following steps: using a learned model, before radiography begins, determining, based on the optical image acquired by the optical camera, whether there is a possibility of re-photography when radiography has been performed, the learned model having learned the relationship between the optical image captured during radiography and whether re-photography of the radiation image captured during radiography is necessary.

[0018] The operating procedure of the present invention is a procedure for operating a photographic aid device for a radiographic device. The radiographic device has a radiation source and a radiation image detector for detecting a radiation image of a subject based on radiation irradiated from and transmitted through the radiation source. The photographic aid device includes an optical camera for optically photographing an area containing radiation from the radiation source irradiating the subject and at least one processor. The operating procedure causes the processor to perform the following determination process: using a learned model, before radiographic photography begins, based on the optical image acquired by the optical camera, it is determined whether there is a possibility of re-photographing if radiographic photography has been performed. The learned model learns the relationship between the optical image captured during radiographic photography and whether re-photographing of the radiation image captured during radiographic photography is necessary.

[0019] Invention Effects

[0020] According to the present invention, a photographic aid device, its working method, and its working procedure can be provided, which makes it easy to determine whether there is a possibility of re-photographing during the photographic preparation before radiography. Attached Figure Description

[0021] Figure 1This is a diagram showing the structure of an X-ray imaging system.

[0022] Figure 2 This is a 3D view of the electronic dark box.

[0023] Figure 3 It is a block diagram representing the structure of the console.

[0024] Figure 4 This is an example of a photography order.

[0025] Figure 5 This is a diagram illustrating a condition table.

[0026] Figure 6 This is a diagram representing an example of an image file.

[0027] Figure 7 It is a block diagram representing the various functional units that make up the CPU.

[0028] Figure 8 This is an example of a control panel display during photography preparation.

[0029] Figure 9 This is an example of a control panel display after X-ray imaging.

[0030] Figure 10 This diagram illustrates the generation process of the learned model.

[0031] Figure 11 This is a flowchart illustrating the first half of the CPU's processing sequence.

[0032] Figure 12 This is a flowchart illustrating the latter half of the CPU's processing sequence. Detailed Implementation

[0033] Figure 1 This diagram illustrates the structure of an X-ray imaging system 10 that uses X-rays as radiation. The X-ray imaging system 10 includes an X-ray source 11, a radiation source control device 12, an electronic cassette 13, a control console 14, and an optical camera 15. In this embodiment, the control console 14 and the optical camera 15 constitute a photographic aid device. The X-ray source 11 is an example of a radiation source. The electronic cassette 13 is an example of a radiation image detector.

[0034] In the X-ray imaging system 10, the electronic cassette 13 is positioned opposite the X-ray source 11. By positioning the subject H between the X-ray source 11 and the electronic cassette 13, the imaging portion of the subject H can be visualized. Figure 1X-ray imaging is performed on the knee (in the middle). The X-ray source 11 and the electron cassette 13 constitute an X-ray imaging apparatus. This X-ray imaging apparatus is an example of a radiographic imaging apparatus related to the technology of this invention.

[0035] The electronic cassette 13 can be configured on a standing or supine radiography table. In this embodiment, after the subject H is positioned by a radiology technician (hereinafter referred to as technician) RG, the technician RG performs the X-ray imaging operation.

[0036] X-ray source 11 includes an X-ray tube 11A that generates X-rays and a collimator 11B that defines the area to be irradiated by X-rays, namely the irradiation field RF. X-ray source 11 may have an irradiation field display light source (not shown) that emits irradiation field display light showing the irradiation field RF on the X-ray incident surface 13A of the electronic cassette 13.

[0037] X-ray tube 11A has a filament that releases thermionic electrons and a target from which the thermionic electrons released from the filament collide and emit X-rays. Collimator 11B, for example, is formed by arranging four lead plates that block X-rays on each side of a quadrilateral, creating a quadrilateral irradiation opening in the center through which X-rays are transmitted. In this case, collimator 11B sets the irradiation field RF by changing the size of the irradiation opening by moving the position of the lead plates.

[0038] The X-ray source control device 12 includes a touch panel 12A, a voltage generating unit 12B, and a control unit 12C. The touch panel 12A is operated by a technician RG when setting the X-ray irradiation conditions and the size of the irradiation opening of the collimator 11B. The X-ray irradiation conditions include the tube voltage, tube current, and X-ray irradiation time applied to the X-ray source 11.

[0039] The voltage generating unit 12B generates a tube voltage applied to the X-ray tube 11A. The control unit 12C controls the operation of the voltage generating unit 12B to set the tube voltage, tube current, and X-ray irradiation time to values ​​set via the touch panel 12A. The control unit 12C has a timer that starts counting when X-rays are generated from the X-ray tube 11A. For example, when the timer reaches the irradiation time specified in the irradiation conditions, the control unit 12C stops the operation of the X-ray tube 11A. Furthermore, the control unit 12C activates the collimator 11B to set the size of its irradiation opening to the size set via the touch panel 12A.

[0040] Furthermore, an irradiation switch 16 is connected to the control unit 12C via a cable or the like. The irradiation switch 16 is operated by a technician RG when X-ray irradiation begins. If the irradiation switch 16 is operated, the X-ray source control device 12 causes the X-ray tube 11A to generate X-rays. As a result, X-rays are irradiated toward the irradiation field RF.

[0041] The electronic cassette 13 detects an X-ray image XP based on X-rays emitted from the X-ray source 11 and transmitted through a radiographic part of the subject H. The electronic cassette 13 has a wireless communication unit and a battery, and operates wirelessly. The electronic cassette 13 wirelessly transmits the detected X-ray image XP to the control console 14. The X-ray image XP is an example of a radiographic image.

[0042] Furthermore, the X-ray source 11 is suspended vertically downwards from the ceiling 2 of the radiography studio. The X-ray source 11 is held by a suspension holding mechanism 17. The suspension holding mechanism 17 is mounted on the ceiling 2 via a horizontal movement mechanism 18. The suspension holding mechanism 17 holds the X-ray source 11 so that it can be freely raised and lowered in the vertical direction (±Z direction). The horizontal movement mechanism 18 holds the suspension holding mechanism 17 so that it can be freely moved along the X-ray irradiation axis direction (±X direction) and in a direction orthogonal to the X-ray irradiation axis direction (±Y direction).

[0043] Motors (not shown) are respectively provided in the suspension holding mechanism 17 and the horizontal moving mechanism 18, enabling the X-ray source 11 to be moved in various directions manually or electrically. The operation of the suspension holding mechanism 17 and the horizontal moving mechanism 18 is controlled by the control unit 12C. The X-ray source 11 can be moved manually or electrically via the touch panel 12A. By moving the X-ray source 11, the position of the irradiation field (RF) can be adjusted.

[0044] The optical camera 15 is an optical digital camera comprising a CMOS (Complementary Metal Oxide Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and, as an example, performs visible light-based photography. The optical camera 15 is capable of both still image photography and video recording. The optical camera 15 is an example of a video recording device according to the technology of this invention.

[0045] The optical axis of the optical camera 15 is parallel to the illumination axis of the X-rays passing through the center of the illumination field RF. The optical camera 15 generates an optical image by optically photographing the region containing the illumination field RF. The optical image is an image showing the photographic portion of the subject H located within the illumination field RF. The optical image can be, for example, a color still image or a moving image.

[0046] The optical camera 15 is mounted on the outer periphery of the X-ray source 11. Alternatively, the optical camera 15 may not be mounted on the outer periphery of the X-ray source 11, but may be built into the X-ray source 11. Furthermore, the objective lens and the imaging element in the optical camera 15 may be separate components. In this case, the objective lens may be disposed on the outer periphery of the X-ray source 11, and the imaging element may be built into a part other than the X-ray source 11 (e.g., the arm supporting the X-ray source 11).

[0047] The optical camera 15 is connected to the control console 14 via wired or wireless means. The control console 14 functions as a photography control device to control the photographic actions of the optical camera 15. In addition to enabling the optical camera 15 to perform still image photography in conjunction with X-ray photography, the control console 14 also enables it to perform motion image photography during the photography preparation period before the start of X-ray photography. For example, the control console 14 is located in an operating room adjacent to the photography room where the X-ray source 11 is located.

[0048] When the illumination switch 16 is activated, the control console 14 sends a still image photography instruction signal to the optical camera 15. Based on the still image photography instruction signal input from the control console 14, the optical camera 15 performs still image photography on the area containing the illumination field RF. The optical image obtained through this still image photography (hereinafter referred to as the still image SP) is sent to the control console 14.

[0049] When technician RG performs the operation to prepare for the start of photography, console 14 sends a motion image photography start signal to optical camera 15. Optical camera 15 begins motion image photography of the area containing the illumination field RF based on the motion image photography start signal input from console 14. The optical image obtained through this motion image photography (hereinafter referred to as the motion image MP) is sent to console 14 in real time as a so-called instant preview image during motion image photography.

[0050] The console 14 is connected via network N to the RIS (Radiology Information System) and PACS (Picture Archiving and Communication System) installed in the X-ray imaging system 10. Sometimes, the console 14 is also connected via network N to a radiographic loss management system. This system collects X-ray images XP marked as radiographic loss and analyzes the radiographic loss rate and causes.

[0051] The console 14 has the function of performing X-ray imaging through the operation of technician RG based on imaging orders and various information obtained from the RIS. Furthermore, the console 14 has the function of outputting the X-ray image XP received from the electronic cassette 13 to the PACS after X-ray imaging. The console 14 also associates and outputs the motion image MP acquired simultaneously with the X-ray image XP during X-ray imaging with the X-ray image XP.

[0052] The control console 14 is, for example, located in an operating room adjacent to the radiography room where the X-ray source 11 is located. The X-ray image XP received by the control console 14 from the electronic cassette 13 is displayed on a monitor 30 located on the control console 14 (see reference). Figure 3 The technician RG can determine whether the X-ray image XP displayed on the monitor 30 is suitable for diagnosis based on the X-ray image XP.

[0053] Furthermore, the console 14 is connected to a learned model providing server 40 via network N. The learned model providing server 40 stores a learned model LM, which learns the relationship between still images SP captured during X-ray imaging and whether or not re-enhancing X-ray images XP captured during X-ray imaging is necessary. Using the learned model LM provided by the learned model providing server 40, the console 14 determines, based on each frame acquired during motion imaging, whether there is a possibility of needing to re-enhance the X-ray image when performing X-ray imaging in the state of the subject H represented by each frame. That is, during motion imaging in the preparation period before X-ray imaging begins, the console 14 determines, each time a frame is acquired, whether there is a possibility of needing to re-enhance the X-ray image when that frame is acquired.

[0054] The learned model provider server 40 stores multiple learned learned models (LMs). For example, each learned model LM is associated with a photographic technique. A photographic technique is information related to the photographic part of the subject H, the pose of that part, and its orientation. For example, a learned model LM is generated by machine learning using a still image SP stored in the PACS and associated with an X-ray image XP taken using the same photographic technique. The machine learning used to generate the learned model LM is performed, for example, by the learned model provider server 40.

[0055] exist Figure 2In this embodiment, the electronic cassette 13 comprises a sensor panel 20, a circuit section 21, and a portable, rectangular frame 22 that houses these components. The frame 22 is, for example, of a size similar to that of a film cassette, an IP (Imaging Plate) cassette, or a CR (Computed Radiography) cassette, conforming to the international standard ISO (International Organization for Standardization) 4090:2001.

[0056] The electronic cassette 13 is positioned with its X-ray incident surface 13A, which serves as the upper surface of the frame 22, facing the X-ray source 11, and X-rays are irradiated onto the X-ray incident surface 13A. In addition, although not shown in the diagram, the frame 22 is equipped with a switch for switching the main power supply on / off, an indicator for the remaining battery life, and a status indicator for the completion of photographic preparation, among other operational status indicators for the electronic cassette 13.

[0057] The sensor panel 20 comprises a scintillator 20A and a photodetector substrate 20B. The scintillator 20A and photodetector substrate 20B are stacked in the order of scintillator 20A and photodetector substrate 20B when viewed from the X-ray incident surface 13A. The scintillator 20A has a phosphor such as CsI:Tl (thallium-activated cesium iodide) or GOS (Gd₂O₂S:Tb, terbium-activated gadolinium oxysulfide) to convert X-rays incident through the X-ray incident surface 13A into visible light for emission. Alternatively, a sensor panel in which the photodetector substrate 20B and scintillator 20A are stacked in the order of photodetector substrate 20B when viewed from the X-ray incident surface 13A can also be used. Furthermore, a direct conversion type sensor panel that directly converts X-rays into signal charges using a photoconductive film such as amorphous selenium can also be used.

[0058] The photodetector substrate 20B detects the visible light emitted from the scintillator 20A and converts it into electrical charge. The circuit section 21 controls the driving of the photodetector substrate 20B and generates an X-ray image XP based on the charge output from the photodetector substrate 20B.

[0059] Multiple pixels are arranged in a two-dimensional matrix on the photodetector substrate 20B. Each pixel generates and stores electrical charge by photoelectric conversion of visible light emitted by the scintillator 20A. The charge stored in each pixel is converted into a digital signal by the circuit section 21 to generate an X-ray image XP.

[0060] Furthermore, the electronic cassette 13, for example, has a function to detect the start of X-ray irradiation. This irradiation start detection function is, for example, constituted by an irradiation start detection sensor disposed on the photodetector substrate 20B. The irradiation start detection sensor is, for example, composed of a portion of a plurality of pixels arranged in a two-dimensional matrix. When the amount of X-ray signal periodically output from the irradiation start detection sensor exceeds a threshold, it is determined that X-ray irradiation has started.

[0061] Furthermore, the electronic cassette 13, like the X-ray source control device 12, has a timer that starts counting when X-ray irradiation begins. The electronic cassette 13 determines that X-ray irradiation has ended when the time counted by the timer becomes the irradiation time included in the irradiation conditions set by the control console 14. Thus, by performing X-ray detection only during the period corresponding to the irradiation time included in the irradiation conditions, the electronic cassette 13 can detect an X-ray image XP based on the irradiated X-rays.

[0062] Furthermore, the electronic cassette 13 has an image memory and a wireless communication circuit. The electronic cassette 13 stores the X-ray image XP generated by the circuit section 21 in the image memory, and transmits the X-ray image XP stored in the image memory to the control console 14 through the wireless communication circuit.

[0063] exist Figure 3 In this system, the console 14 includes a display 30, an input device 31, a CPU (Central Processing Unit) 32, a storage device 34, a memory 33, and a communication unit 35. These components are interconnected via a data bus 36.

[0064] The display 30 is a display unit that displays various operation screens, X-ray images (XP), and optical images (still images (SP) and moving images (MP)) based on GUI (Graphical User Interface) operation functions. The input device 31 is an input operation unit that includes a touch panel or keyboard, etc.

[0065] Storage device 34 is, for example, an HDD (Hard Disk Drive) array, either built into console 14 or externally connected to console 14. External connection is via cable or network. Storage device 34 stores operating system and other control programs, various applications, and various data associated with these programs.

[0066] Furthermore, the storage device 34 stores the operating program P for enabling the control console 14 and optical camera 15 to function as photographic aids, and multiple learned models LM provided from the learned model providing server 40. Additionally, the storage device 34 stores the condition table 38 (described later) and image files 39 containing X-ray images XP and still images SP received from the electronic cassette 13 (see reference). Figure 6 ).

[0067] Memory 33 is the working memory used by CPU 32 for processing. CPU 32 loads the program stored in storage device 34 into memory 33 and executes the processing according to the program, thereby centrally controlling the various parts of console 14. Communication unit 35 transmits and receives various data such as X-ray images XP and optical images (still images SP and moving images MP) with electronic cassette 13 and optical camera 15. In addition, communication unit 35 communicates with control unit 12C of X-ray source control device 12. Moreover, communication unit 35 communicates with RIS, PACS and learning completed model providing server 40 via network N.

[0068] Console 14 Receive Figure 4 The input for imaging order 37 is shown. Imaging order 37 is, for example, information from a client requesting imaging in a clinic to instruct technician RG on X-ray imaging. Imaging order 37 is, for example, transmitted from RIS to console 14.

[0069] Photography order 37 includes items such as Order ID (Identification Data), Subject ID, and Photography Technique. The Order ID is a marker or number that identifies each photography order 37 and is automatically assigned by RIS. The Subject ID field records the Subject ID of the subject H, which is the subject of the photograph. The Subject ID is a marker or number that identifies each subject H.

[0070] Photographic technique refers to information related to the photographic part of the subject H, the pose of that part, and its orientation. The photographic part, besides... Figure 1 Besides the knees, which are shown in the example, other areas include the head, cervical spine, chest, abdomen, hands, fingers, or elbows. Posture refers to the subject H's position, such as standing, lying down, or sitting. Orientation refers to the subject H's orientation relative to the X-ray source 11, such as front, side, or back. In addition to these items, photography order 37 also includes subject information such as the subject H's name, gender, age, height, and weight.

[0071] The storage device 34 of the console 14 stores Figure 5 The conditions are shown in Table 38. In Table 38, corresponding lighting conditions are registered for each photographic technique.

[0072] The console 14, operated by technician RG, displays the following on monitor 30: Figure 4 The photography order list shown is a list of the contents of photography order 37. Technician RG can browse the photography order list to confirm the contents of photography order 37. Furthermore, console 14 will... Figure 5 The contents of the conditions table 38 are displayed on the monitor 30. The technician RG can select and set the illumination conditions to match the photographic technique specified in the photographic order 37.

[0073] The console 14 wirelessly transmits a condition setting signal containing various information such as the irradiation conditions set by the technician RG, the order ID, and the console ID as identification information of the console to the electronic cassette 13.

[0074] The console 14 associates the X-ray image XP received from the electronic cassette 13 with a still image SP captured simultaneously with the X-ray image XP, for example, setting it as an image file conforming to the DICOM (Digital Imaging and Communication in Medicine) standard, and saves it in the storage device 34, which serves as the storage unit. The image file contains accompanying information. The accompanying information includes the order ID, subject ID, photographic technique, irradiation conditions, photographic loss indicator, and reason for photographic loss, etc.

[0075] Figure 6 This represents an example of an image file stored in storage device 34. For example... Figure 6 As shown, image file 39 contains an X-ray image XP, a still image SP, and accompanying information SI. The still image SP is associated with an image ID (optical image ID) that corresponds to the image ID (X-ray image ID) of the simultaneously captured X-ray image XP. The X-ray image XP and the still image SP obtained in a single X-ray imaging session are associated with the X-ray image ID and the optical image ID and stored in one image file 39.

[0076] The accompanying information SI includes a photography loss flag indicating whether the technician RG determines the X-ray image XP displayed on the monitor 30 to be a photography loss image. The technician RG can input the determination result using the input device 31. For example, a photography loss flag of "1" indicates that the technician RG determines the X-ray image XP to be a photography loss image. On the other hand, a photography loss flag of "0" indicates that the technician RG determines the X-ray image XP to be a normal image.

[0077] When the imaging loss flag is "1", X-ray re-enhancing is performed; when the imaging loss flag is "0", X-ray re-enhancing is not performed. Therefore, the imaging loss flag indicates whether X-ray image re-enhancing has been performed. The imaging loss flag is an example of the result information involved in the technology of this invention.

[0078] The reason for photographic loss is the reason determined by technician RG for photographic loss, and is input by technician RG using input device 31.

[0079] When the imaging technique is "knee / bent posture / side view," the physician diagnoses the joint cavity (JC) of the knee based on the X-ray image XP. Therefore, the joint cavity JC needs to be clearly depicted in the X-ray image XP. For example, the technician RG determines whether there is radiographic loss based on whether the joint cavity JC is clearly depicted and infers the reason for the radiographic loss (reason for radiographic loss).

[0080] Figure 7 This represents the various functions configured within the CPU 32. The operating program P is stored in the storage device 34. Furthermore, although the illustration is omitted, Figure 5 The condition table 38 shown is also stored in the storage device 34. The CPU 32 is configured with multiple functional units by executing the working program P.

[0081] Operating program P causes CPU 32 to function as a still image photography instruction unit 51, a still image acquisition unit 52, a correlation establishment unit 53, a moving image photography instruction unit 54, a moving image acquisition unit 55, and a display control unit 56. Furthermore, operating program P causes CPU 32 to function as a model selection unit 60, a judgment unit 61, a warning unit 62, a prompt unit 63, and an illumination prohibition indication unit 64.

[0082] The still image imaging instruction unit 51 receives an X-ray irradiation start signal ES, which is generated by the control unit 12C of the X-ray source control device 12 and supplied to the voltage generation unit 12B when the irradiation switch 16 is pressed. If the still image imaging instruction unit 51 receives the X-ray irradiation start signal ES, it instructs the optical camera 15 to perform still image imaging.

[0083] The still image acquisition unit 52 acquires a still image SP generated by still image photography performed by the optical camera 15. The still image SP acquired by the still image acquisition unit 52 is input to the association establishment unit 53. Furthermore, in the association establishment unit 53, the communication unit 35 (see reference) Figure 3 The input is an X-ray image XP detected by the electronic cassette 13 based on the X-rays irradiated from the X-ray source 11 as the irradiation switch 16 is pressed.

[0084] The association establishment unit 53 stores the image file 39, created by establishing an association between the still image SP input to the still image acquisition unit 52 and the X-ray image XP input from the electronic cassette 13, in the storage device 34. Furthermore, the image file 39 contains the aforementioned incidental information SI.

[0085] The motion image photography instruction unit 54 sends a motion image photography start signal to instruct the optical camera 15 to begin motion image photography upon receiving a photography preparation start signal RS from the input device 31. When the technician RG positions the subject H as a preparation stage before performing X-ray photography of the subject H, he can switch the photography aid to the X-ray photography preparation mode by operating the input device 31.

[0086] The motion image acquisition unit 55 acquires motion images MP generated by motion image photography by the optical camera 15 frame by frame in real time. The motion image acquisition unit 55 acquires optical images frame by frame and outputs motion images MP consisting of multiple acquired frames. The motion images MP output from the motion image acquisition unit 55 are input frame by frame to the display control unit 56 and the determination unit 61.

[0087] The display control unit 56 displays the dynamic image MP input from the dynamic image acquisition unit 55 frame by frame on the display 30 during photography preparation (i.e., real-time display). Furthermore, the display control unit 56 displays the X-ray image XP acquired through X-ray photography on the display 30.

[0088] The dynamic image photography based on the optical camera 15 ends upon receiving an execution instruction for still image photography from the aforementioned still image photography instruction unit 51. The still image acquisition unit 52 can acquire one frame of the dynamic image MP as a still image SP.

[0089] When the technician RG confirms that the X-ray image XP displayed on the display 30 is a photographic loss, he can operate the input device 31 to mark the X-ray image XP as a photographic loss (i.e., set the photographic loss flag to "1").

[0090] The model selection unit 60 selects from the plurality of learned models LM stored in the storage device 34 the learned model LM that corresponds to the photographic technique included in the photographic order 37 selected by the technician RG using the input device 31. The model selection unit 60 supplies the selected learned model LM to the determination unit 61.

[0091] The determination unit 61 uses the learned model LM supplied from the model selection unit 60 to perform determination processing on each frame of the dynamic image MP input from the dynamic image acquisition unit 55. The determination unit 61 determines whether there is a possibility of re-photographing when X-ray photography is performed on the subject H in the state represented by each frame (i.e., whether there is a possibility of re-photographing).

[0092] The learned model (LM) is constructed using neural networks. For example, the learned model (LM) can be constructed using a multi-layered neural network, known as a deep neural network (DNN), which is the object of deep learning. As a DNN, for example, a convolutional neural network (CNN) can be used, treating images as objects.

[0093] After performing the determination process, the determination unit 61 provides a "determination result" indicating whether a re-photograph is necessary. Furthermore, when the determination unit 61 determines that a re-photograph is necessary, it provides the determined "reason" to the prompt unit 63. Hereinafter, the reason for determining that a re-photograph is necessary will be referred to as the "reason for re-photographing." As described above, when the knee is the photographic site, "external rotation" or "internal rotation" becomes a reason for re-photographing. The determination unit 61 may also generate more detailed reasons for re-photographing.

[0094] Based on the determination result supplied by the determination unit 61, the warning unit 62 performs a warning notification process by supplying a warning message to the display control unit 56 and displaying it on the display 30 when there is a possibility that re-photography is required. That is, the warning unit 62 notifies the user that there is a possibility that re-photography is required due to photographic loss when X-ray photography is performed in the current positioning state of the subject H.

[0095] Furthermore, the warning unit 62 can display warnings on devices such as mobile terminals connected to the X-ray imaging system 10, and is not limited to the display 30 of the control console 14. Moreover, the warning unit 62 is not limited to displaying warnings on the display 30, etc.; warnings can also be delivered via sound or other means. The warning notification can be delivered in any way that stimulates the perception of technicians such as RG.

[0096] The prompting unit 63 performs prompting processing by providing information indicating the reason for re-photographing, supplied from the determination unit 61, to the display control unit 56 for display on the monitor 30. Furthermore, the prompting unit 63 derives correction measures for correcting the position or orientation of the subject H based on the reason for re-photographing, and provides information indicating the derived correction measures to the display control unit 56, thereby displaying it on the monitor 30. For example, when the reason for re-photographing is "inward rotation," the prompting unit 63 derives the correction measure "preferably outward rotation," and conversely, when the reason for re-photographing is "outward rotation," it derives the correction measure "preferably inward rotation." The prompting unit 63 can also derive more detailed correction measures.

[0097] In addition, the prompting unit 63 may display at least one of the corrective measures and reasons for re-photographing on the display 30.

[0098] For example, the display control unit 56 overlays a message indicating a warning, the reason for re-enactment, and corrective measures onto the moving image MP displayed in real time on the display 30. The technician RG corrects the position or orientation of the subject H based on the message displayed on the display 30, thereby preventing photographic loss.

[0099] Furthermore, the determination unit 61 supplies the determination result to the irradiation prohibition instruction unit 64. When, based on the determination result supplied by the determination unit 61, there is a possibility that re-irradiation is required, the irradiation prohibition instruction unit 64 performs a prohibition process by supplying an irradiation prohibition signal RP to the control unit 12C of the radiation source control device 12 to prohibit X-ray irradiation from the X-ray source 11. Thus, when there is a possibility that re-irradiation is required, interlocking control is performed to prohibit X-ray irradiation of the subject H.

[0100] Figure 8 This represents an example of a console screen displayed on the monitor 30 by the display control unit 56. For example... Figure 8 As shown, the console screen 70 has an image display area 70A for displaying dynamic images (MP) or X-ray images (XP).

[0101] Furthermore, the console screen 70 displays a first operation button 71 for starting photography preparation, a second operation button 72 for marking photographic loss, and a third operation button 73 for outputting image file 39 to PACS. The first operation button 71, the second operation button 72, and the third operation button 73 are operated via a touch panel formed on the screen of the display 30.

[0102] Figure 8 This is an example of a control panel screen 70 displaying the preparation for X-ray imaging. The preparation process begins, for example, by technician RG pressing the first operation button 71. Figure 8In the example shown, the moving image MP acquired by the optical camera 15 is displayed in real time in the image display area 70A. Furthermore, a message box 74 indicating the determination result based on the determination unit 61 is displayed in the image display area 70A.

[0103] exist Figure 8 In the example shown, a warning mark 75 is displayed in message box 74, indicating that the judgment result based on the judgment unit 61 is not good and there is a possibility that re-photographing is required. Warning mark 75 is displayed based on the warning information supplied from the warning unit 62 to the display control unit 56. Furthermore, message box 74 displays the reason 76 for re-photographing and corrective measures 77 supplied from the prompt unit 63 to the display control unit 56.

[0104] The content displayed in message box 74 is updated sequentially based on the determination results of each frame of the moving image MP performed by the determination unit 61. The technician RG can correct the position or orientation of the subject H based on the content displayed in message box 74. By properly positioning the subject H, message box 74 may not be displayed, for example, when the determination result based on the determination unit 61 becomes favorable.

[0105] Figure 9 This shows an example of the console screen 70 after X-ray imaging. Figure 9 In the example shown, an X-ray image XP obtained through X-ray imaging is displayed in image display area 70A. When the technician RG determines that the X-ray image XP is not suitable for diagnosis and requires re-imaging (i.e., imaging loss), they can mark the X-ray image XP as an imaging loss image by pressing the second operation button 72. As a result, the imaging loss flag included in the accompanying information SI of the X-ray image XP is set to "1". Furthermore, at this time, the technician RG can input the reason for the imaging loss via a keyboard (not shown).

[0106] By having a technician press the third operation button 73, the aforementioned image file 39, containing the X-ray image XP displayed in the image display area 70A, is output to the PACS. Additionally, the image file 39, containing the X-ray image XP with the photography loss flag set to "1", is output via network N to the PACS or a photography loss management system (not shown).

[0107] Figure 10 The generation process of the learned model (LM) is explained. The generation of the learned model (LM) is performed, for example, on the learned model providing server 40. The learned model providing server 40 uses image files 39 output to PACS and photogrammetry loss management systems, etc., for machine learning.

[0108] like Figure 10As shown, the learned model server 40 stores the unlearned learned model M. Still images SP contained in image file 39 are input to the learned model M. The learned model M outputs a judgment result and a reason for re-photographing the input still image SP. The judgment result and reason for re-photographing output from the learned model M are compared with the photographic loss flag and reason for photographic loss, which are included as positive solution data, in the accompanying information SI. Then, the parameters of the learned model M are corrected in a way that minimizes the difference between the two. The learned model LM is obtained by repeatedly correcting the parameters of the learned model M (i.e., learning) using multiple image files 39.

[0109] For example, using image files 39 obtained through the same photographic technique, a learned model (LM) can be generated for each photographic technique. Furthermore, the generation of the learned model (LM) is not limited to an external server such as the learned model provider server 40; it can also be performed within the console 14. Moreover, the learned model (LM) can be updated by using the generated image file 39 for learning each time an image file 39 is generated. Additionally, to reduce the load on the console 14, the generated learned model (LM) can be performed on a dedicated computer outside of the console 14.

[0110] Next, refer to Figure 11 and Figure 12 The flowchart shown illustrates the function of the photographic aid device described above. First, before taking photographs, the technician RG confirms the contents of the photographic order 37 via the display 30 and sets the illumination conditions using the input device 31 and the touch panel 12A.

[0111] Next, technician RG positions the X-ray source 11, electron cassette 13, and subject H according to the photographic technique included in photographic order 37. Here, the photographic technique is set to "knee / bent posture / side view". Technician RG bends one leg of subject H, positioning subject H so that the side of the knee faces the X-ray incident surface 13A of electron cassette 13 and the knee is located in the center of the irradiation field RF (see reference). Figure 1 When the technician RG is positioning the subject H, he activates the photography aid by pressing the first operation button 71 on the control panel screen 70. If the first operation button 71 is pressed, the photography preparation start signal RS is input from the input device 31 to the moving image photography instruction unit 54.

[0112] The motion image photography instruction unit 54 determines whether it has received the photography preparation start signal RS output from the input device 31 by the technician RG pressing the first operation button 71 (step S10). If the motion image photography instruction unit 54 determines that the photography preparation start signal RS has been received (step S10: Yes), it sends a motion image photography start signal to the optical camera 15 (step S11). The motion image acquisition unit 55 acquires the motion image MP generated by the motion image photography performed by the optical camera 15 frame by frame (step S12). The display control unit 56 displays the motion image MP acquired by the motion image acquisition unit 55 frame by frame on the image display area 70A of the console screen 70 (step S13).

[0113] Furthermore, each frame of the motion image MP acquired by the motion image acquisition unit 55 is supplied to the determination unit 61. The determination unit 61 uses the learned model LM selected by the model selection unit 60 to perform determination processing on each frame of the motion image MP (step S14). The model selection unit 60 selects the learned model LM corresponding to the photography technique included in the photography order 37.

[0114] The determination unit 61 determines whether there is a possibility that re-photographing is required when X-ray photography is performed on the subject H in the state represented by each frame (step S15). When the determination unit 61 determines that there is a possibility that re-photographing is required (step S15: Yes), the process proceeds to step S16.

[0115] In step S16, the warning unit 62, as... Figure 8 As shown, a warning mark 75 is displayed in the image display area 70A, thereby warning of the possibility that re-enhancing is required. In the next step S17, the prompting unit 63 generates a reason for re-enhancing 76 and a corrective measure 77 based on the judgment result and judgment reason supplied from the judgment unit 61, and displays them in the image display area 70A. In the next step S18, the irradiation prohibition indication unit 64 supplies an irradiation prohibition signal RP to the control unit 12C of the radiation source control device 12, thereby prohibiting X-ray irradiation from the X-ray source 11. After step S18 ends, the process returns to step S12. In addition, the processes in steps S16 to S18 can be executed in parallel.

[0116] On the other hand, when the determination unit 61 determines that there is no possibility of needing to perform a new imaging (step S15: No), the process proceeds to step S19. That is, when there is no possibility of needing to perform a re-imaging, no warning notification, no prompt of the reason for re-imaging and corrective measures, and no prohibition of X-ray irradiation are given. In step S19, the still image imaging instruction unit 51 determines whether it has received the X-ray irradiation start signal ES emitted from the control unit 12C when the irradiation switch 16 is pressed by the technician RG.

[0117] When the still image imaging instruction unit 51 determines that the X-ray irradiation start signal ES has been received (step S19: Yes), the process proceeds to step S20. On the other hand, when the still image imaging instruction unit 51 determines that the X-ray irradiation start signal ES has not been received (step S19: No), the process returns to step S12.

[0118] In step S20, the still image photography instruction unit 51 instructs the optical camera 15 to perform still image photography. The still image acquisition unit 52 acquires the still image SP generated by the still image photography performed by the optical camera 15 (step S21). Meanwhile, the control console 14 acquires the X-ray image XP detected by the electronic cassette 13 (step S22).

[0119] Next, the association establishment unit 53 establishes an association between the acquired X-ray image XP and the still image SP, and saves them together with the accompanying information SI as an image file 39 in the storage device 34 (step S23). Then, the display control unit 56 displays the X-ray image XP stored in the image file 39 in the storage device 34 on the image display area 70A of the console screen 70 as shown in Figure 9 (step S24).

[0120] Next, the CPU 32 determines whether there is an instruction to output an image by the technician RG pressing the third operation button 73 (step S25). When the technician RG confirms the X-ray image XP displayed in the image display area 70A and determines that re-photographing is unnecessary, he / she instructs to output the image by pressing the third operation button 73. When the CPU 32 determines that the third operation button 73 has been pressed (step S25: Yes), it outputs the image file 39 stored in the storage device 34 to the PACS (step S29) and ends the process.

[0121] On the other hand, when the CPU32 determines that no instruction was given to output an image by pressing the third operation button 73 by the technician RG (step S25: No), it determines whether the X-ray image XP was marked as photographic loss by pressing the second operation button 72 by the technician RG (step S26). This marking includes inputs such as the reason for the photographic loss.

[0122] When the CPU32 determines that the X-ray image XP is marked as a photographic loss (step S26: Yes), it outputs the image file 39 containing the X-ray image XP marked as a photographic loss to the PACS or the photographic loss management system (step S27). On the other hand, when the CPU32 determines that the X-ray image XP is not marked as a photographic loss (step S26: No), the process returns to step S25.

[0123] In step S28, similar to step S10, the moving image photography instruction unit 54 determines whether it has received the photography preparation start signal RS output from the input device 31 by the technician RG pressing the first operation button 71. If the moving image photography instruction unit 54 determines that the photography preparation start signal RS has been received (step S28: Yes), the process proceeds to step S11. On the other hand, if the moving image photography instruction unit 54 determines that the photography preparation start signal RS has not been received (step S28: No), the process returns to step S25. Then, when the CPU 32 determines that the third operation button 73 has been pressed (step S25: Yes), the image file 39 is output to the PACS (step S29), and the process ends.

[0124] As described above, in the photographic aid device with the above structure, during the preparation for X-ray imaging, the learned model is used to determine whether there is a possibility of re-imaging. Therefore, with the photographic aid device according to the above structure, it is not necessary to pre-register various positioning index images as in the past, nor is it necessary to select positioning index images suitable for the subject, and technicians can easily determine whether there is a possibility of re-imaging.

[0125] In addition, the determination unit 61 determines the dynamic image MP input from the dynamic image acquisition unit 55 frame by frame, but in order to reduce the processing load, the determination can be performed on multiple frames.

[0126] The above embodiments are illustrated by taking an X-ray imaging system set up in a radiography studio as an example, but the X-ray imaging system can also be a system that utilizes a so-called mobile medical vehicle.

[0127] Furthermore, the technology of the present invention is not limited to X-rays, but can also be applied to systems that use other types of radiation, such as gamma rays, to photograph a subject.

[0128] In the above embodiments, the hardware structure of the processing units that perform various processes, such as the still image photography instruction unit 51, the still image acquisition unit 52, the association establishment unit 53, the moving image photography instruction unit 54, the moving image acquisition unit 55, the display control unit 56, the model selection unit 60, the determination unit 61, the warning unit 62, the prompt unit 63, and the illumination prohibition instruction unit 64, is the various processors shown below.

[0129] Processors include CPUs, programmable logic devices (PLDs), and special-purpose circuits. As is well known, a CPU is a general-purpose processor that executes software (programs) to function as various processing units. A PLD is a processor such as a field-programmable gate array (FPGA) whose circuit structure can be modified after manufacturing. Special-purpose circuits are processors with circuit structures specifically designed to perform specific processes, such as application-specific integrated circuits (ASICs).

[0130] A processing unit can consist of one of these various processors, or it can consist of a combination of two or more processors of the same or different types (e.g., multiple FPGAs or a combination of a CPU and an FPGA). Furthermore, multiple processing units can also be composed of a single processor.

[0131] As examples of a single processor comprising multiple processing units, firstly, there are processors that combine one or more CPUs and software to form a single processor, which functions as multiple processing units. Secondly, there are systems-on-chips (SoCs), which use a single IC chip to implement the functions of the entire system containing multiple processing units. Thus, various processing units are constructed as hardware structures using one or more of the aforementioned processors.

[0132] Moreover, the hardware structure of these various processors, more specifically, is capable of utilizing circuits that combine circuit elements such as semiconductor elements.

[0133] This invention is not limited to the embodiments described above. Various structures can be employed as long as they do not depart from the spirit of the invention. Furthermore, in addition to programs, this invention also relates to a computer-readable storage medium that non-temporarily stores programs.

Claims

1. A photographic aid for use in a radiographic apparatus, the radiographic apparatus having a radiation source and a radiation image detector for detecting a radiation image of a subject based on radiation irradiated from and transmitted through the radiation source, the photographic aid comprising: An optical camera outputs an optical image by optically photographing a region including an irradiation field of radiation from the radiation source that is incident on the subject; and At least one processor, The processor performs the following processing: The determination process uses a learned model to determine, before radiography begins, whether there is a possibility of re-photographing based on the optical images acquired by the optical camera. The learned model has learned the relationship between the optical images captured during radiography and whether re-photographing of the radiographic images is necessary. The association establishment process establishes an association between the optical image captured during radiography and the result information indicating whether the radiographic image captured during radiography was re-enhanced.

2. The photographic auxiliary device according to claim 1, wherein, The processor performs the following warning notification processing: If the determination process indicates that there is a possibility of re-photographing, a warning is issued.

3. The photographic auxiliary device according to claim 1, wherein, The processor performs the following disable process: If the determination process determines that there is a possibility of re-photographing, irradiation from the radiation source is prohibited.

4. The photographic auxiliary device according to claim 1, wherein, The processor performs the following prompt processing: Tips are provided for corrective measures to correct the position or orientation of the subject.

5. The photographic auxiliary device according to claim 4, wherein, In the prompting process, in addition to the corrective measures, the processor also provides reasons for determining that there is a possibility of re-photographing.

6. The photographic aid device according to claim 5, wherein, In the prompting process, the processor displays at least one of the corrective measures and the reasons on the display unit.

7. A method of operating a photographic aid device, the photographic aid device being used in a radiographic device, the radiographic device having a radiation source and a radiation image detector for detecting a radiation image of a subject based on radiation irradiated from and transmitted through the radiation source, the photographic aid device comprising an optical camera for outputting an optical image by optically photographing an area including an irradiation field of radiation irradiated from the radiation source onto the subject, the method of operating the photographic aid device comprising the following steps: Using a learned model, before radiography begins, based on the optical images acquired by the optical camera, it is determined whether re-photography is necessary. The learned model has learned the relationship between the optical images captured during radiography and whether re-photography of the radiographic images captured during radiography is required. The optical image captured during radiography is correlated with information indicating whether a re-photograph of the radiographic image captured during radiography was performed.

8. A storage medium, readable by a computer and storing an operating program that causes a photographic aid device to operate, the photographic aid device being used in a radiographic apparatus, the radiographic apparatus having a radiation source and a radiation image detector for detecting a radiation image of a subject based on radiation irradiated from and transmitted through the radiation source, the photographic aid device comprising an optical camera for outputting an optical image by optically photographing an area including an irradiation field of radiation irradiated from the radiation source onto the subject, and at least one processor, the operating program causing the processor to perform the following processing: The determination process uses a learned model to determine, before radiography begins, whether there is a possibility of re-photographing based on the optical images acquired by the optical camera. The learned model has learned the relationship between the optical images captured during radiography and whether re-photographing of the radiographic images is necessary. The association establishment process establishes an association between the optical image captured during radiography and the result information indicating whether the radiographic image captured during radiography was re-enhanced.

Images