Information processing device, radiography system, information processing method, and program

Abstract

If a change in the subject's condition that exceeds a predetermined amount of movement is detected, the optical image will be displayed to notify the system of the change in the subject's condition. [Solution] The information processing device for processing radiographic images acquired by radiography of a subject comprises: an analysis unit that analyzes the movement of the subject in the optical image acquired from a first optical imaging device; a determination unit that determines whether to display the optical image on a display unit based on the analysis; and a display control unit that controls the display on the display unit based on the determination.

Description

Technical Field

[0001] The present invention relates to an information processing apparatus, a radiation imaging system, an information processing method, and a program.

Background Art

[0002] As an imaging device used for medical image diagnosis and non-destructive inspection by radiation, a radiation imaging device using a flat panel detector (FPD) formed of a semiconductor material has become widespread. Such a radiation imaging device is used, for example, in medical image diagnosis as a digital imaging device for still image imaging such as general imaging or moving image imaging such as fluoroscopy.

[0003] In recent years, in radiation imaging, the imaging site is photographed with an optical camera, and imaging support using the optical camera has come to be performed. For example, in Patent Document 1, a technique for performing radiation imaging while confirming the imaging situation by an optical video has been proposed.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] In the radiation imaging system as disclosed in Patent Document 1, although the optical video is displayed on the display, there are scenes where the optical video becomes unnecessary at timings when it is not necessary to confirm the imaging situation, such as during confirmation of the captured radiation image, and the optical video may be made non-displayed. In addition, since the subject is reflected in the optical video and there is a high possibility of being seen by eyes other than the examiner, considering the privacy of the subject, it is required not to display the optical video on the display as much as possible.

[0006] However, if the optical image is not displayed, it becomes impossible to monitor the subject using the optical image, which could lead to delays in noticing changes in the subject's condition or other emergencies.

[0007] The present invention has been made in view of the above problems, and one of its objectives is to provide an information processing technology that can notify the detected change in the subject's condition by displaying the optical image when a change in the subject's condition that exceeds a predetermined amount of movement is detected, even when the optical image is hidden for the sake of an efficient shooting flow. [Means for solving the problem]

[0008] An information processing device according to one aspect of the present invention is an information processing device that processes radiographic images acquired by radiography of a subject, An analysis unit that analyzes the movement of the subject in the optical image acquired from the first optical imaging device, A determination unit that determines whether to display the optical image on the display unit based on the above analysis, The system includes a display control unit that controls the display of the display unit based on the above determination. [Effects of the Invention]

[0009] According to the present invention, if a change in the condition of a subject that exceeds a predetermined amount of movement is detected, an optical image can be displayed to notify the user of the change in the subject's condition. [Brief explanation of the drawing]

[0010] [Figure 1] A diagram showing an example of the basic configuration of a radiography system according to the first embodiment. [Figure 2] A diagram showing the detailed configuration of the optical image analysis unit 105 according to the first embodiment. [Figure 3] A flowchart illustrating the display determination process for optical images using the optical imaging device 140 according to the first embodiment. [Figure 4]A flowchart illustrating the analysis process of optical images acquired from the optical imaging device 140 according to the first embodiment. [Figure 5] A diagram showing an example configuration of a radiography system according to the second embodiment. [Figure 6] A flowchart illustrating the display determination process of optical images using the surveillance camera 170 according to the first embodiment. [Figure 7] A flowchart illustrating the analysis process of sensor data acquired from the surveillance camera 170 according to the first embodiment. [Modes for carrying out the invention]

[0011] The embodiments will be described in detail below with reference to the attached drawings. Note that the following embodiments do not limit the invention as defined in the claims. While the embodiments describe multiple features, not all of these features are essential to the invention, and the features may be combined in any way. Furthermore, in the attached drawings, identical or similar configurations are given the same reference numerals, and redundant descriptions are omitted.

[0012] (First embodiment) Figure 1 shows an example of the basic configuration of a radiography system according to this embodiment. The radiography system comprises an information processing device 100, a radiation generator 120, a radiation detection device 130, an optical imaging device 140, a display unit 150, and an operation unit 160. The information processing device 100 controls radiography using the radiation detection device 130 and the radiation generator 120. The information processing device 100 is communicated with the radiation detection device 130, the radiation generator 120, and the optical imaging device 140.

[0013] The radiation generator 120 is equipped with a radiation tube that generates radiation and irradiates a subject such as a patient with radiation. The radiation detection device 130 generates image data (radiation image) based on the radiation irradiated from the radiation generator 120. The radiation detection device 130 may be composed of, for example, a flat panel detector (FPD). The radiation detection device 130 has a sensor panel in which multiple pixels are arranged to form multiple rows and multiple columns. The multiple pixels of the sensor panel function as detection elements that detect radiation. The radiation detection device 130 detects radiation irradiated from the radiation generator 120 that has passed through a subject (not shown), generates image data corresponding to the detected radiation, and outputs it. The image data generated by the radiation detection device 130 can also be referred to as a medical image or a radiation image.

[0014] Specifically, the radiation detection device 130 detects radiation that has passed through the subject as an electric charge equivalent to the amount of transmitted radiation. For example, the radiation detection device 130 may use a direct conversion sensor that directly converts radiation into electric charge, such as a-Se, or an indirect sensor that uses a scintillator such as CsI and a photoelectric conversion element such as a-Si to convert radiation into visible light. Furthermore, the radiation detection device 130 generates image data by performing A / D conversion on the detected charge and outputs it to the information processing device 100.

[0015] (Information processing device 100) The information processing apparatus 100 processes a radiation image acquired by radiographic imaging of a subject. The information processing apparatus 100 is connected to a radiation generator 120, a radiation detector 130, and an optical imaging apparatus 140 via, for example, a wired or wireless network or a dedicated line. The information processing apparatus 100 can be configured using a computer including a processor, a memory, etc., but may also be configured as a dedicated control device for the radiographic imaging system. The processor of the information processing apparatus 100 may be a CPU (Central Processing Unit). Also, the processor may be, for example, an MPU (Micro Processing Unit), a GPU (Graphical Processing Unit), an FPGA (Field-Programmable Gate Array), or the like.

[0016] Each function of the information processing apparatus 100 may be realized by a processor such as a CPU or an MPU executing a software module stored in the internal storage unit of the information processing apparatus 100. Note that the processor may be, for example, a GPU, an FPGA, or the like. Also, each function of the information processing apparatus 100 may be configured by a circuit or the like that performs a specific function such as an ASIC. For example, each functional configuration of the information processing apparatus 100 may be realized by dedicated hardware such as an ASIC.

[0017] The operation unit 160 includes, for example, input devices such as a keyboard, a pointing device (e.g., a mouse, etc.), and a touch panel, and receives various inputs from an operator. The information processing apparatus 100 sets radiographic imaging conditions based on the inspection information received by the operation unit 160, and controls the timing at which the radiation generator 120 generates radiation based on the set imaging conditions.

[0018] (Radiation Image Acquisition Unit 101) The radiation image acquisition unit 101 controls the timing at which the radiation detector 130 captures image data and the timing at which it outputs the data, and receives the image data generated by the radiation detector 130.

[0019] (Image processing unit 102) The image processing unit 102 performs image processing on the image data received by the radiographic image acquisition unit 101. The image processing unit 102 applies diagnostic image processing such as gradation processing, enhancement processing, and noise reduction processing to the image data to create an image suitable for diagnosis.

[0020] (Display control unit 104) The display unit 150 is implemented by, for example, a liquid crystal display or a CRT (Cathode Ray Tube) display. The display unit 150 displays various information under the control of the display control unit 104. The display control unit 104 can control the display of the display unit 150 and can display radiation images and subject information before and after image processing by the image processing unit 102 on the display unit 150. The display control unit 104 performs display control to display the image processed by the image processing unit 102 on the display unit 150.

[0021] The display control unit 104 provides a graphical user interface (GUI) using the display unit 150 and receives instructions from the operator via the operation unit 160.

[0022] (Inspection Information Management Department 106) The inspection information management unit 106 manages multiple inspection information items for multiple radiography operations. The inspection information management unit 106 displays the multiple inspection information items in a list format on the display unit 150 via the display control unit 104. The inspection information management unit 106 allows the operator to select one of the multiple inspection information items displayed in the list format based on input from the operation unit 160, and sets the selected inspection information as the inspection information to be used for radiography. The inspection information management unit 106 outputs the set inspection information to the optical image analysis unit 105 and the display control unit 104. Alternatively, the operator may directly input the inspection information from the operation unit 160 to the optical image analysis unit 105 and the display control unit 104. The information processing device 100 can control the timing of acquiring radiographic images and optical images based on the set inspection information. For example, based on the inspection information received by the operation unit 160, the information processing device 100 sets the shooting conditions for controlling radiography, and controls the timing of radiation generation by the radiation generator 120 and the generation of image data (radiographic images) by the radiation detection device 130. The radiation detection device 130 detects the radiation generated by the radiation generator 120 and outputs image data (radiation image) corresponding to the detected radiation to the information processing device 100.

[0023] Here, the examination information managed by the examination information management unit 106 may include the name of the department that requested the examination, the examination ID, the examination items, the imaging conditions (imaging protocol) in radiography, and patient information (subject information). In addition, the examination information may include information that identifies the imaging environment (e.g., standing stand, supine table, radiation detector 130 (FPD) unit only), the imaging site (e.g., frontal chest or lateral abdomen), and the imaging posture (e.g., standing or supine).

[0024] (Optical imaging device 140) The radiography system is equipped with an optical imaging device 140 (e.g., an optical camera) for capturing optical images of the subject. The optical imaging device 140 captures video as an optical image of the subject. The optical imaging device 140 may be positioned near the radiation generator 120, for example, so that the imaging area of ​​the subject in radiography can be photographed. Furthermore, the optical imaging device 140 is not limited to a single optical imaging device 140, but multiple optical imaging devices 140 may be used. For example, the optical imaging devices 140 may be positioned in different directions to photograph the subject so that optical images can be acquired from the front and lateral directions.

[0025] The information processing device 100 controls the conditions, timing, frame rate, etc., for the optical imaging device 140 to acquire optical images.

[0026] (Optical image acquisition unit 103) The optical image acquisition unit 103 acquires optical images from the optical imaging device 140. If multiple optical imaging devices 140 are provided, the optical image acquisition unit 103 acquires optical images from multiple optical imaging devices 140.

[0027] The optical image acquisition unit 103 outputs the optical image acquired from the optical imaging device 140 to the display control unit 104.

[0028] The display control unit 104 performs display control to display the optical image output from the optical image acquisition unit 103 on the display unit 150. If multiple optical images are output, the display control unit 104 performs display control to display multiple optical images on the display unit 150. The display control unit 104 adds information to be displayed in addition to the optical image (additional information) and controls the content to be displayed using the display unit 150. Additional information may include the imaging conditions (imaging protocol) in radiography. When displaying an optical image, the display control unit 104 may also display additional information on the display unit 150 to notify the movement of the subject. The additional information displayed on the display unit 150 may include a message notifying the movement of the subject, or an icon (notification mark, symbol, etc.), or a combination of a message and an icon. When the movement of the subject exceeds a predetermined amount of movement, the display control unit 104 may change the display color of the additional information and display it on the display unit 150. Here, the predetermined amount of motion is a threshold that can be changed by setting, and the optical image analysis unit 105, which will be described later, determines that the subject is a moving object if the movement of the subject detected by the motion detection process exceeds the threshold.

[0029] The additional information is not limited to the information displayed on the display unit 150, but may also include sound output from the sound output unit 180, which is composed of a speaker or the like.

[0030] The determination unit 107 may output a notification sound from the sound output unit 180 to notify the subject's movement when the display control unit 104 displays an optical image. Furthermore, the determination unit 107 may change the type of notification sound or increase the volume of the notification sound and output it from the sound output unit 180 when the subject's movement exceeds a predetermined amount. Alternatively, a combination of changing the type of notification sound and changing the volume of the notification sound may be used.

[0031] (Optical image analysis unit 105) The optical image analysis unit 105 analyzes the movement of the subject in the optical image acquired from the optical imaging device 140 (first optical imaging device). If multiple different human bodies are detected within the frame of the optical image by human body detection processing based on image processing or inference processing using a trained model, as described later, the optical image analysis unit 105 may acquire feature information of the multiple detected human bodies and, based on the acquired feature information, detect one of the multiple human bodies as the subject.

[0032] In this case, an example of a situation where multiple different human bodies are detected within a frame is when the operator and the subject are captured within the optical image frame. In such cases, the operator's characteristic information may include, for example, the type and color of a white coat or hospital-specific clothing, or a name tag, which may be known operator characteristics. Similarly, the subject's characteristics may include, for example, subject information obtained from examination data (e.g., adult, child, gender, height, etc.) or the examination gown worn by the subject, which may be known subject characteristics.

[0033] The optical image analysis unit 105, when detecting multiple different human bodies within the optical image frame through human body detection processing, acquires characteristic information of multiple human bodies and, based on the acquired characteristic information, detects one of the multiple human bodies as the subject. For example, if characteristic information of both the operator and the subject can be acquired, or if characteristic information of only the subject can be acquired, the optical image analysis unit 105 can detect the subject from among the multiple human bodies based on the acquired characteristic information of the subject.

[0034] Alternatively, if characteristic information of only the operator can be obtained, the optical image analysis unit 105 can detect the remaining human bodies as subjects based on the obtained characteristic information of the operator, excluding the operator from the multiple human bodies.

[0035] Furthermore, the optical image analysis unit 105 uses the optical image acquired from the optical image acquisition unit 103 and the inspection information acquired from the inspection information management unit 106 to analyze the subject's position (positioning position), the area of ​​the subject being photographed, and the photographing posture in the shooting environment, and determines whether they match the settings of the inspection information. Here, the inspection information includes information that identifies the shooting environment (e.g., standing stand, supine table, radiation detection device 130 (FPD) alone, etc.), the area being photographed (e.g., front of the chest, side of the abdomen, etc.), and the photographing posture (e.g., standing, supine, etc.). The optical image analysis unit 105 compares the analysis information obtained by analyzing the optical image with various information contained in the inspection information to determine whether they match the settings of the inspection information.

[0036] The optical image analysis unit 105 determines whether the subject is located in a predetermined position in the shooting environment set in the examination information by performing a human body detection process based on image processing or inference processing using a trained model, as described later. The optical image analysis unit 105 also determines whether the shooting area and shooting posture of the subject obtained by analyzing the optical image match the shooting area and shooting posture of the subject set in the examination information.

[0037] (Judgment unit 107) The determination unit 107 determines whether to display the optical image on the display unit 150 based on the analysis results of the optical image analysis unit 105 and the analysis results of the sensor data analysis unit 109, which will be described later in the second embodiment. The display control unit 104 controls the display of the content on the display unit 150 based on the determination result of the determination unit 107.

[0038] (Detailed configuration of the optical image analysis unit 105) Figure 2 shows a detailed configuration of the optical image analysis unit 105 in this embodiment. The optical image analysis unit 105 consists of a human body detection processing unit 200, a human body detection inference processing unit 201, a motion detection processing unit 210, and a motion detection inference processing unit 211. The human body detection processing unit 200 has a human body detection inference processing unit 201, and the motion detection processing unit 210 has a motion detection inference processing unit 211.

[0039] The human body detection processing unit 200 performs human body detection processing to determine whether a subject (human body) is captured within the frame of the optical image acquired from the optical image acquisition unit 103. In the following explanation, the detection of a subject may also be referred to as the detection of a human body.

[0040] The human body detection processing unit 200 may, for example, determine whether a subject is captured in the optical image by processing the optical image. As part of the image processing of the optical image, the human body detection processing unit 200 may, for example, extract the outline of a human body by contour extraction, or divide the image (video frame) into multiple regions and identify candidate regions of a person from the feature quantities of each region. Alternatively, a human body captured in the image (video frame) may be detected by performing pattern matching using a human body matching pattern.

[0041] Furthermore, the human body detection processing performed by the human body detection processing unit 200 is not limited to image processing. For example, the human body detection inference processing unit 201 may perform human body detection by using a neural network and a trained model obtained through deep learning to perform inference processing.

[0042] The optical image analysis unit 105 may, when a human body is detected at multiple locations within the optical image frame by human body detection processing based on image processing or inference processing using a trained model, compare the positions of the human body at multiple locations and detect the human body that is photographed in a position close to the center of the frame as the subject.

[0043] When the human body detection processing unit 200 detects a subject (human body) in the optical image, it notifies the optical image analysis unit 105. At this time, the optical image analysis unit 105 uses the motion detection processing unit 210 to perform motion detection processing on the optical image in which the subject (human body) was detected. The motion detection processing unit 210 can monitor the movement (presence or absence of movement) of the detected subject by performing motion detection processing on the human body in the optical image, for example.

[0044] The motion detection processing unit 210 performs motion detection processing to determine whether there is a moving object in the optical image acquired from the optical image acquisition unit 103. The motion detection processing unit 210 determines whether the subject detected by the human body detection processing is a moving object exceeding a predetermined amount of movement by performing image processing using multiple frames of the optical image or inference processing using a trained model. As motion detection processing using image processing using multiple frames of the optical image, the motion detection processing unit 210 may, for example, calculate the difference between frames in the optical image and determine that there was an object (moving object) moving between frames of the optical image if there was a difference between frames.

[0045] Furthermore, motion detection processing is not limited to image processing (inter-frame difference). For example, the motion detection inference processing unit 211 may implement motion detection processing by using a neural network and a trained model obtained through deep learning to perform inference processing. In motion detection processing using image processing (inter-frame difference) or motion detection using a trained model, the criterion (threshold) for determining the presence or absence of motion can be changed by settings input from the operation unit 160. For example, the setting of the criterion (threshold) can be changed to exclude slight movements that may occur after the positioning of the subject in radiography, and to detect changes in body position that may occur due to changes in the subject's condition. This makes it possible to efficiently detect the subject's movement as desired by the operator (user).

[0046] When the motion detection processing unit 210 detects a moving object in the optical image, it notifies the optical image analysis unit 105. The optical image analysis unit 105 notifies the determination unit 107 of the information processing device 100 of the determination results of the human body detection processing unit 200 and the motion detection processing unit 210. Based on the notification from the optical image analysis unit 105, the determination unit 107 of the information processing device 100 determines whether to display the optical image on the display unit. Here, the determination results of each processing unit notified by the optical image analysis unit 105 may be either the determination results of the human body detection processing unit 200 or the motion detection processing unit 210, or a combination of determination results other than those described above.

[0047] For example, if the human body detection processing unit 200 does not detect a human body, the optical image analysis unit 105 may notify the determination unit 107 of the detection result of the human body detection processing unit 200 (no human body detected). Alternatively, if the human body detection processing unit 200 detects a human body and the motion detection processing determines whether or not there is a moving body, the optical image analysis unit 105 may notify the determination unit 107 of the detection result of the human body detection processing unit 200 (human body detected) and the determination result of the motion detection processing unit 210 (whether or not the detected human body is moving).

[0048] Furthermore, as a determination result other than the processing described above, in the second embodiment, the sensor data analysis unit 109 may analyze the surveillance optical image from the surveillance camera 170 (described later) and sensor data acquired from sensors (e.g., infrared sensor, ultrasonic sensor, thermal sensor, acceleration sensor, etc.), and the determination unit 107 may be notified of a determination result that combines the analysis result of the sensor data analysis unit 109 and the analysis result of the optical image analysis unit 105. By combining the analysis results of the surveillance optical image and the analysis results of various sensor data in addition to the analysis results of the optical image, it is possible to suppress misjudgments in the human body detection processing unit 200 and the motion detection processing unit 210 and improve the determination accuracy.

[0049] (Processing flow) Next, following the flowchart in Figure 3, the optical image display determination process using the optical imaging device 140 in this embodiment will be explained. In the optical image display determination process shown in Figure 3, an example is described in which the processes S301-S304 are performed when radiography and radiographic images are displayed (S300), but the optical image display determination process in this embodiment is not limited to this example. For example, if a change in the subject's condition occurs after the subject's positioning is complete and before the operator presses the exposure switch (before radiography begins), the optical image may be displayed to quickly notify the operator of the change in the subject's condition. Alternatively, after radiography is completed, the captured radiographic image may be displayed on the display unit 150, and if a change in the subject's condition occurs while the operator is checking the radiographic image, the optical image may be displayed to quickly notify the operator of the change in the subject's condition.

[0050] In S300, radiography is performed and the radiographic image is displayed on the display unit 150. In this step, the radiographic image acquisition unit 101 receives image data generated by the radiation detection device 130 and performs image processing on the received image data. The display control unit 104 displays the radiographic image on the display unit 150 before and after image processing by the image processing unit 102. The radiographic image acquired by radiography may be a still image or a moving image from fluoroscopy.

[0051] In S301, the determination unit 107 of the information processing device 100 determines whether an optical image is displayed on the display unit 150. If no optical image is displayed on the display unit 150 (S301-No), the process proceeds to S302. On the other hand, if the determination in S301 indicates that an optical image is displayed on the display unit 150 (S301-Yes), the process ends.

[0052] In S302, the optical image analysis unit 105 performs optical image analysis processing. Details of the optical image analysis processing will be described later with reference to Figure 4.

[0053] In S303, the determination unit 107 of the information processing device 100 determines whether to display the optical image on the display unit 150 based on the results of the optical image analysis processing in S302 by the optical image analysis unit 105. The determination unit 107 of the information processing device 100 determines whether to display the optical image on the display unit 150 based on the human body detection processing and motion detection processing in the optical image analysis unit 105. In the motion detection processing by image processing (inter-frame difference) and motion detection by inference processing using a trained model performed by the optical image analysis unit 105, the degree of change in the subject's movement can be changed by settings input from the operation unit 160. For example, the determination unit 107 of the information processing device 100 determines to display the optical image on the display unit 150 when there is a large movement exceeding a predetermined amount in the subject detected within the frame of the optical image.

[0054] At this time, the amount of movement of the subject (degree of change in movement) used as the criterion for determining whether to display the optical image can be changed by a setting input from the operation unit 160. This allows the display of the optical image to be changed according to the degree of movement of the subject. For example, the system may be set to display the optical image and notify if the subject moves after positioning is complete and repositioning is required (when the degree of movement is relatively small). Alternatively, the system may be set to display the optical image and notify only if there is a change in the subject's condition, such as the subject collapsing (when the degree of movement is relatively large). If it is determined that the optical image should be displayed (S303-Yes), the process proceeds to S304. On the other hand, if the determination process in S303 determines not to display the optical image (S303-No), the process ends. The determination unit 107 determines that if the movement of the subject analyzed by the optical image analysis unit 105 is smaller than a predetermined amount of movement, it should display the radiation image on the display unit 150 and hide the optical image. Based on this determination, the display control unit 104 displays the radiation image on the display unit 150.

[0055] In S304, the display control unit 104 performs display control to display the optical image acquired by the optical image acquisition unit 103 on the display unit 150. The determination unit 107 determines that the optical image should be displayed on the display unit 150 if the movement of the subject analyzed by the optical image analysis unit 105 is greater than a predetermined amount of movement, and the display control unit 104 displays the optical image on the display unit 150 based on the result of the determination. At this time, the display control unit 104 may change the display method of the optical image according to the result of the optical image analysis process in S302.

[0056] The display control unit 104 may display the radiographic image acquired by radiography on the display unit 150, and when displaying the optical image, it may hide the radiographic image and display the optical image on the display unit 150.

[0057] The display control unit 104 may display both the radiation image and the optical image on the display unit 150 when displaying an optical image.

[0058] When displaying optical images, the display control unit 104 may further display additional information on the display unit 150 to notify the movement of the subject. Here, the additional information may include a message notifying the movement of the subject, an icon, or a combination of a message and an icon.

[0059] The display control unit 104 may change the display color of the additional information and display it on the display unit 150 when the subject's movement exceeds a predetermined amount. The determination unit 107 may output a notification sound from the sound output unit 180 to notify the subject's movement when the display control unit 104 displays an optical image.

[0060] The determination unit 107 may change the type of notification sound or increase the volume of the notification sound and output it from the sound output unit 180 if the movement of the subject exceeds a predetermined amount of movement.

[0061] As an example of changing the display method, if the analysis results of the optical image analysis unit 105 show that the subject's movement is within a predetermined range that exceeds a predetermined amount of movement set as a threshold, the display control unit 104 may display the optical image of the first size (reference size) on the display unit 150. If the analysis results show that the subject's movement exceeds the upper limit of the predetermined range and a large movement is detected, the display control unit 104 may display the optical image enlarged to a size larger than the first size (reference size) (second size) on the display unit 150. Furthermore, the above-mentioned additional information may be displayed along with the enlarged optical image, or an alert sound may be output.

[0062] When multiple optical images are acquired from multiple optical imaging devices 140, the display control unit 104 may display the multiple optical images acquired from the multiple optical imaging devices 140 on the display unit 150. When the movement of the subject is within a predetermined range and multiple optical images or monitoring optical images are to be displayed on the display unit 150, the display control unit 104 may display the optical images on the display unit 150 that have been reduced to a size smaller than the first size (reference size) (third size).

[0063] Depending on the degree of the subject's movement, the method of displaying the optical image may be changed. If the subject's movement exceeds the upper limit of a predetermined range and a large movement is detected, the display control may be performed to enlarge the size of the optical image from the first size to the second size, in addition to displaying the additional information and outputting sound as a warning.

[0064] (Detailed flow of optical image analysis processing) Next, following the flowchart in Figure 4, the analysis process of the optical image acquired from the optical imaging device 140, which is performed in S302 in this embodiment, will be described.

[0065] In S401, the optical image acquisition unit 103 acquires an optical image from the connected optical imaging device 140.

[0066] In S402, the human body detection processing unit 200 performs human body detection processing on the optical image acquired from the optical imaging device 140 in S401. The human body detection processing may be implemented by image processing. The human body detection inference processing unit 201 may implement the human body detection processing by using a neural network and an inference processing using a trained model obtained by deep learning.

[0067] In S403, the human body detection processing unit 200 determines from the processing results in S402 whether a human body (subject) has been detected in the optical image. If a human body is detected (S403-Yes), the process proceeds to S404; if no human body is detected (S403-No), the analysis process is terminated.

[0068] In S404, the motion detection processing unit 210 performs motion detection processing on the optical image acquired from the optical imaging device 140 in S401. The motion detection processing may be implemented by image processing that calculates the inter-frame difference of the optical image and, if there is a difference between frames, identifies the location of the difference as an object (motion) moving between frames of the optical image. Alternatively, the motion detection inference processing unit 211 may implement the motion detection processing by using a neural network and an inference process using a trained model obtained by deep learning.

[0069] In S405, the motion detection processing unit 210 determines whether the subject is moving based on the processing result in S404. If the motion detection processing unit 210 determines that the subject is moving (S405-Yes), it proceeds to S406. On the other hand, if the motion detection processing unit 210 determines in S405 that the subject is not moving (S405-No), it terminates the process.

[0070] In S406, the optical image analysis unit 105 notifies the determination unit 107 of the information processing device 100 of the processing results analyzed by the optical image analysis unit 105.

[0071] According to this embodiment, the optical imaging device 140 can analyze the optical image to determine whether the subject is being photographed in the optical image and whether the subject is moving, thereby determining whether to display the optical image on the display unit 150. This makes it possible to quickly detect and notify the system when the subject moves significantly beyond a predetermined amount of motion or when there is a change in the subject's condition, even when the optical image is not being displayed on the display unit 150.

[0072] Even when the optical image is hidden for an efficient imaging workflow, if an emergency such as a change in the subject's condition is detected, the optical image can be displayed to notify the system of the detected emergency.

[0073] (Second embodiment) In the first embodiment, a configuration was described in which a human body detection process is performed using an optical image captured by an optical imaging device 140 to detect the presence or absence of a subject, and a motion detection process is performed to detect whether or not the subject is moving. When the subject moves significantly, the optical image is displayed to notify the user of a change in the subject's condition.

[0074] In the second embodiment, sensor data acquired from various sensors is used to perform human body detection processing, subject position analysis, and motion detection processing of the subject at a predetermined examination position. When the subject moves significantly, an optical image is displayed to notify the system of a change in the subject's condition.

[0075] In the following description, to avoid duplication with the first embodiment, we will explain the differences. In this embodiment, a surveillance camera 170 is used as one type of sensor, and a configuration example is described in which human body detection processing, whether the person is at the inspection location, and motion detection processing are performed using the surveillance optical image (hereinafter also referred to as surveillance camera image) captured by the surveillance camera 170 as sensor data.

[0076] Figure 5 shows an example of the configuration of a radiography system in this embodiment. The radiography system comprises an information processing device 100, a radiation generator 120, a radiation detection device 130, an optical imaging device 140, a display unit 150, an operation unit 160, and a surveillance camera 170. The information processing device 100 controls radiography using the radiation detection device 130 and the radiation generator 120. The information processing device 100 is communicated with the radiation detection device 130, the radiation generator 120, the optical imaging device 140, and the surveillance camera 170.

[0077] The surveillance camera 170 is a second optical imaging device having a wider field of view (second field of view) compared to the field of view (first field of view) of the optical imaging device 140 (first optical imaging device). The surveillance camera 170 is placed in the examination room where radiography is performed and is capable of continuously photographing the entire examination room, including the radiography system, such as the subject, the radiation generator 120, and the radiation detection device 130 that generates radiographic images. The surveillance camera 170 (second optical imaging device) photographs the radiation detection device 130 and the subject as surveillance camera footage. The surveillance camera 170 is connected to the information processing device 100, for example, by a wired or wireless network or a dedicated line. The information processing device 100 controls the conditions, timing, frame rate, etc., under which the surveillance camera 170 acquires surveillance camera footage.

[0078] The sensor data acquisition unit 108 of this embodiment acquires surveillance camera images from the surveillance camera 170 as sensor data.

[0079] The sensor data analysis unit 109 analyzes sensor data acquired from the sensor that detects the subject. The sensor data analysis unit 109 analyzes the surveillance camera video acquired from the sensor data acquisition unit 108 and determines the presence and movement of the subject in the surveillance camera video. The sensor data analysis unit 109 may also determine the presence and movement of the subject in the surveillance camera video by image processing of the surveillance camera video or by inference processing using a trained model such as machine learning or deep learning.

[0080] The sensor data analysis unit 109 performs human body detection processing to detect whether the subject is captured in the frames of surveillance camera footage acquired from the radiation detection device 130 that generates radiation images and the surveillance camera 170 (second optical imaging device) that photographs the subject, as sensor data. The sensor data analysis unit 109 also analyzes the movement of the subject by performing a determination process to determine whether the subject detected by the human body detection process is in a predetermined position relative to the radiation detection device 130, and a motion detection process to determine whether the subject, determined to be in a predetermined position, is a moving object exceeding a predetermined amount of movement.

[0081] The determination unit 107 of the information processing device 100 determines whether to display the optical image acquired from the optical imaging device 140 on the display unit 150, based on the analysis results of the sensor data analysis unit 109. For example, the determination unit 107 of the information processing device 100 analyzes the movement of the subject from the surveillance camera image acquired by the surveillance camera 170, and if it determines that the subject is a moving object exceeding a predetermined amount of movement (i.e., if it determines that the subject's condition has changed), it determines to display the optical image on the display unit 150. The display control unit 104 displays the optical image on the display unit 150 based on the determination result of the determination unit 107.

[0082] (Processing flow) Next, the optical image display determination process using the surveillance camera 170 in this embodiment will be explained according to the flowchart in Figure 6. In the optical image display determination process shown in Figure 6, an example is described in which the processes S601-S604 are performed when radiography and radiographic images are displayed (S600), but the optical image display determination process in this embodiment is not limited to this example. For example, if a change in the subject's condition occurs after the subject's positioning is complete and before the operator presses the exposure switch (before radiography begins), the optical image may be displayed to quickly notify the operator of the change in the subject's condition. Alternatively, after radiography is completed, the captured radiographic image may be displayed on the display unit 150, and if a change in the subject's condition occurs while the operator is checking the radiographic image, the optical image may be displayed to quickly notify the operator of the change in the subject's condition.

[0083] In step S600, radiography is performed and the radiographic image is displayed on the display unit 150. In this step, the radiographic image acquisition unit 101 receives image data generated by the radiation detection device 130 and performs image processing on the received image data. The display control unit 104 displays the radiographic image on the display unit 150 before and after image processing by the image processing unit 102. The radiographic image acquired by radiography may be a still image or a moving image from fluoroscopy.

[0084] In S601, the determination unit 107 of the information processing device 100 determines whether an optical image is displayed on the display unit 150. If no optical image is displayed on the display unit 150 (S601-No), the process proceeds to S602. On the other hand, if the determination in S601 indicates that an optical image is displayed on the display unit 150 (S601-Yes), the process terminates.

[0085] In S602, the sensor data analysis unit 109 performs sensor data analysis processing. Details of the sensor data analysis processing will be described later with reference to Figure 7.

[0086] In S603, the determination unit 107 of the information processing device 100 determines whether to display an optical image on the display unit 150 based on the result of the sensor data analysis processing in S602 by the sensor data analysis unit 109.

[0087] For example, the determination unit 107 of the information processing device 100 determines to display an optical image on the display unit 150 when a large change in movement exceeding a predetermined amount of movement is detected in the subject by the sensor data analysis process. At this time, the amount of movement of the subject (degree of change in movement) can be changed by a setting input from the operation unit 160 as the criterion for determining whether to display the optical image. If it is determined to display the optical image (S603-Yes), the process proceeds to S604. On the other hand, if the determination process in S603 determines not to display the optical image (S603-No), the process ends. The determination unit 107 determines to display a radiation image on the display unit 150 and hide the optical image when the movement of the subject analyzed by the sensor data analysis unit 109 is less than a predetermined amount of movement, and the display control unit 104 displays the radiation image on the display unit 150 based on the determination.

[0088] In S604, the display control unit 104 performs display control to display the optical image acquired by the optical image acquisition unit 103 on the display unit 150. The determination unit 107 determines that the optical image should be displayed on the display unit 150 if the movement of the subject analyzed by the sensor data analysis unit 109 is greater than a predetermined amount of movement, and the display control unit 104 displays the optical image on the display unit 150 based on the result of the determination. The optical imaging device 140 (first optical imaging device) is an optical imaging device with a narrower field of view than the surveillance camera 170 (second optical imaging device) described in the second embodiment, and the display control unit 104 displays the optical image captured by the optical imaging device 140 (first optical imaging device), which is an optical image of the subject being photographed in radiography, on the display unit 150. At this time, the display control unit 104 may change the method of displaying the optical image according to the result of the sensor data analysis process in S602.

[0089] An example of display control for changing the display method of optical images may be the same as the process described earlier in S304. For example, the display method of optical images may be changed depending on the degree of movement of the subject, and if the subject's movement exceeds the upper limit of a predetermined range and a large movement is detected, the display control may be performed to enlarge the size of the optical image from the first size to the second size in conjunction with the display of the additional information and the output of sound warnings as described above. When the subject's movement is within a predetermined range and multiple optical images or monitoring optical images are to be displayed on the display unit 150, the display control unit 104 may display an optical image on the display unit 150 that has been reduced to a size smaller than the first size (reference size) (third size).

[0090] Furthermore, the display control unit 104 may, when displaying the radiographic image acquired by radiography on the display unit 150 and displaying the optical image, hide the radiographic image and display the optical image and the surveillance camera image (surveillance optical image) on the display unit.

[0091] Furthermore, when displaying optical images, the display control unit 104 may display the radiation image, the optical image, and the monitoring optical image on the display unit 150.

[0092] (Detailed flow of sensor data analysis process) Next, following the flowchart in Figure 7, the analysis process of surveillance camera images acquired from the surveillance camera 170, which is performed in S602 in this embodiment, will be described.

[0093] In S701, the sensor data acquisition unit 108 acquires surveillance camera images from the connected surveillance camera 170.

[0094] In S702, the sensor data analysis unit 109 performs human body detection processing on the surveillance camera video acquired from the surveillance camera 170 in S701. Human body detection processing may be implemented by image processing, or it may be implemented by using a neural network and inference processing using a trained model obtained by deep learning.

[0095] In S703, the sensor data analysis unit 109 determines from the processing results in S702 whether a human body (subject) has been detected in the surveillance camera video. If a human body is detected (S703-Yes), the process proceeds to S704; if no human body is detected (S703-No), the analysis process ends.

[0096] In S704, the sensor data analysis unit 109 uses the surveillance camera video acquired from the surveillance camera 170 in S701 to analyze the position of the subject within the surveillance camera video (frame) detected in S702.

[0097] In S705, the sensor data analysis unit 109 determines from the analysis results in S704 whether the subject is in the predetermined position for radiography. The inspection information managed by the inspection information management unit 106 includes settings for the imaging area, imaging direction, and imaging environment (imaging stand, supine table, cassette imaging, etc.) of the subject. The sensor data analysis unit 109 determines whether the position of the subject within the surveillance camera image (frame) detected in S702 is in the predetermined position relative to the position of the radiation detection device 130 installed in the imaging environment.

[0098] If the sensor data analysis unit 109 determines that the subject is in the designated position for radiography (S705-Yes), it proceeds to S706. If the subject is not in the designated position for radiography (S705-No), it returns to S701 and repeats the same process.

[0099] In S706, the sensor data analysis unit 109 performs motion detection processing on the surveillance camera video acquired from the surveillance camera 170 in S701. Motion detection processing may be implemented by image processing that calculates the inter-frame difference of the surveillance camera video and, if there is a difference between frames, identifies the location of the difference as an object (motion) moving between frames of the optical image. Alternatively, motion detection processing may be implemented using a neural network and inference processing with a trained model obtained by deep learning.

[0100] In S707, the sensor data analysis unit 109 determines whether the subject is moving based on the processing results in S706. If the sensor data analysis unit 109 determines that the subject is moving (S707-Yes), it proceeds to S708. On the other hand, if the sensor data analysis unit 109 determines in S707 that the subject is not moving (S707-No), it terminates the process.

[0101] In S708, the sensor data analysis unit 109 notifies the determination unit 107 of the information processing device 100 of the processing results analyzed by the sensor data analysis unit 109.

[0102] According to this embodiment, it is possible to determine whether to display the optical image captured by the optical imaging device 140 on the display unit 150 based on the presence, position, and movement of the subject obtained from the video analysis processing of the surveillance camera image from the surveillance camera 170 that is capturing the entire examination room. As a result, even when the optical image is not displayed on the display unit 150 due to the analysis processing of the optical image analysis unit 105, the analysis processing of the sensor data analysis unit 109 can detect when the subject, who is in a predetermined position where radiography is to be performed, moves significantly beyond a predetermined amount of movement, or when there is a change in the subject's condition, and the optical image acquired from the optical imaging device 140 by the optical image acquisition unit 103 can be displayed on the display unit 150. This makes it possible to detect when the subject moves significantly beyond a predetermined amount of movement or when there is a change in the subject's condition and to quickly notify the system.

[0103] In this embodiment, as one aspect of the sensor data acquired from the sensor, a configuration was described in which human body detection processing and motion detection processing are performed using surveillance camera images acquired from a surveillance camera 170. However, as long as it is possible to detect a subject and whether or not the subject is moving, various sensors can be used, not just the surveillance camera 170. The sensor includes a first sensor that detects the subject as a human body and detects whether it is in a predetermined position relative to the radiation detection device 130, and a second sensor that detects whether the subject detected as being in a predetermined position is a moving object exceeding a predetermined amount of movement. The first sensor may include, for example, an infrared sensor, an ultrasonic sensor, or a temperature sensor, and the second sensor may include, for example, an acceleration sensor. The first sensor (infrared sensor, ultrasonic sensor, or temperature sensor, etc.) should be placed near the radiation generator 120 so that, for example, the radiation detection device 130 and the subject are within the detection range of the sensor. The second sensor (accelerometer, etc.) should be attached to the subject so that, for example, the subject's movement can be detected. The sensor data acquisition unit 108 can acquire sensor data from the first sensor via wired or wireless communication, and acquire sensor data from the second sensor via wireless communication.

[0104] For example, as a process corresponding to the processes S701 to S705 in Figure 7, infrared sensors or the like may be used to detect and acquire infrared radiation emitted from the subject as sensor data. Based on the acquired infrared sensor data, the sensor data analysis unit 109 may acquire the distance (distance information) between the radiation detection device 130 and the subject and analyze whether the subject is at a predetermined position for radiography. Alternatively, heat sensors or the like may be used to detect and acquire heat emitted from the subject as sensor data. Based on the acquired heat sensor data, the sensor data analysis unit 109 may acquire the distance (distance information) between the radiation detection device 130 and the subject and analyze whether the subject is at a predetermined position for radiography. Alternatively, ultrasonic sensors or the like may be used to measure the distance to the subject using the reflection of ultrasound. The sensor data analysis unit 109 may use the reflection of ultrasound to acquire the distance (distance information) between the radiation detection device 130 and the subject and analyze whether the subject is at a predetermined position for radiography. Furthermore, the sensor used for human body detection may be a single sensor or a combination of multiple types of sensors.

[0105] For example, motion detection processing may be performed using an acceleration sensor as a process corresponding to the processes S706 to S708 in Figure 7. The sensor data analysis unit 109 may perform motion detection processing based on the detection results of the acceleration sensor. The sensor data analysis unit 109 determines that the subject is moving if the movement of the subject exceeds a threshold, based on the motion detection processing using the acceleration data of the acceleration sensor. Human body detection processing and motion detection processing may also be realized by combining sensors for each process and performing a comprehensive analysis.

[0106] According to this embodiment, even in situations where the optical image is hidden for an efficient shooting flow, if an emergency such as a change in the subject's condition is detected, the optical image can be displayed to notify the detected emergency state.

[0107] (Summary of the embodiments) The above embodiments disclose the following information processing device, radiography system, information processing method, and program. (Item 1) An information processing device for processing radiographic images acquired by radiography of a subject, An analysis unit that analyzes the movement of the subject in the optical image acquired from the first optical imaging device, A determination unit that determines whether to display the optical image on the display unit based on the above analysis, Based on the above determination, a display control unit controls the display of the display unit, An information processing device characterized by comprising: (Item 2) The analysis unit performs a human body detection process to detect the subject captured in the frame of the optical image by performing image processing or inference processing using a trained model, A motion detection process that determines whether the subject detected by the human body detection process is a moving body exceeding a predetermined amount of movement by performing image processing using multiple frames of the optical image or inference processing using a trained model, The information processing device according to item 1, characterized by analyzing the movement of the subject by performing the following. (Item 3) The information processing device according to item 2, characterized in that, when a human body is detected at multiple locations within the frame by the human body detection process, the analysis unit compares the positions of the human bodies at the multiple locations and detects the human body photographed at a position close to the center of the frame as the subject. (Item 4) The information processing apparatus according to item 2, characterized in that, when the analysis unit detects multiple different human bodies within the frame by the human body detection process, it acquires characteristic information of the multiple human bodies and, based on the characteristic information, detects one of the multiple human bodies as the subject. (Item 5) The aforementioned predetermined amount of movement is a threshold that can be changed by setting, The information processing device according to item 2, characterized in that the analysis unit determines that the subject is a moving object when the movement of the subject detected by the motion detection process exceeds the threshold. (Item 6) The determination unit determines that if the movement of the subject analyzed by the analysis unit is smaller than a predetermined amount of movement, it will display the radiation image on the display unit and hide the optical image. The information processing apparatus according to item 2, characterized in that the display control unit causes the radiation image to be displayed on the display unit based on the determination. (Item 7) The determination unit determines that the optical image should be displayed on the display unit if the movement of the subject analyzed by the analysis unit is greater than a predetermined amount of movement. The information processing apparatus according to item 2, characterized in that the display control unit causes the optical image to be displayed on the display unit based on the determination. (Item 8) The display control unit, The radiographic image acquired by the radiographic imaging is displayed on the display unit. The information processing apparatus according to item 7, characterized in that, when displaying the optical image, the radiation image is hidden and the optical image is displayed on the display unit. (Item 9) The information processing apparatus according to item 7, characterized in that the display control unit causes the display unit to display the radiation image and the optical image when displaying the optical image. (Item 10) The information processing apparatus according to item 8 or 9, characterized in that when the display control unit displays the optical image, it further displays additional information that notifies the movement of the subject on the display unit. (Item 11) The information processing device according to item 10, characterized in that the additional information includes one of the following: a message or an icon that notifies the movement of the subject, or a combination of the message and the icon. (Item 12) The information processing apparatus according to item 10 or 11, characterized in that the display control unit changes the display color of the additional information and displays it on the display unit when the movement of the subject exceeds a predetermined amount of movement. (Item 13) The information processing apparatus according to any one of items 7 to 12, characterized in that the determination unit causes the display control unit to output a notification sound from the sound output unit to notify the movement of the subject when the display control unit displays the optical image. (Item 14) The information processing apparatus according to item 13, characterized in that when the movement of the subject exceeds a predetermined amount of movement, the determination unit changes the type of notification sound or increases the volume of the notification sound and outputs it from the sound output unit. (Item 15) In the analysis results of the analysis unit, if the movement of the subject falls within a predetermined range that exceeds the predetermined amount of movement set as a threshold, the display control unit causes the display unit to display a first-size optical image. The information processing apparatus according to any one of items 7 to 14, characterized in that, in the analysis results, if the movement of the subject exceeds the upper limit of the predetermined range and a large movement is detected, the display control unit displays an optical image enlarged to a second size, which is larger than the first size, on the display unit. (Item 16) The information processing apparatus according to any one of items 7 to 15, characterized in that when the display control unit acquires a plurality of optical images acquired from a plurality of first optical imaging devices, it causes the acquired plurality of optical images to be displayed on the display unit. (Item 17) The system further includes a data analysis unit that analyzes sensor data acquired from the sensor that detects the subject, The aforementioned data analysis unit, The sensor data includes a radiation detection device that generates the radiation image and a human body detection process that detects whether the subject is captured in the frame of the surveillance optical image acquired from a second optical imaging device that photographs the subject, A determination process to determine whether the subject detected by the human body detection process is in a predetermined position relative to the radiation detection device, The information processing apparatus according to item 1, characterized by performing motion detection processing to determine whether the subject, which has been determined to be in the predetermined position, is a moving body exceeding a predetermined amount of movement, thereby analyzing the movement of the subject. (Item 18) The determination unit determines that the optical image should be displayed on the display unit if the movement of the subject analyzed by the data analysis unit is greater than a predetermined amount of movement. The information processing apparatus according to item 17, characterized in that the display control unit causes the optical image to be displayed on the display unit based on the determination. (Item 19) The display control unit, The radiographic image acquired by the radiographic imaging is displayed on the display unit. The information processing apparatus according to item 17 or 18, characterized in that when displaying the optical image, the radiation image is hidden and the optical image and the monitoring optical image are displayed on the display unit. (Item 20) The information processing apparatus according to item 17 or 18, characterized in that when the display control unit displays the optical image, it causes the radiation image, the optical image, and the monitoring optical image to be displayed on the display unit. (Item 21) The first optical imaging device has a narrower field of view compared to the second optical imaging device. The information processing apparatus according to item 17, characterized in that the display control unit causes the display unit to display an optical image of the imaging area of ​​the subject in the radiography as the optical image captured by the first optical imaging device. (Item 22) The sensor includes a first sensor that detects the subject as a human body and detects whether it is in a predetermined position relative to the radiation detection device, The information processing apparatus according to item 17 or 18, characterized in that it includes a second sensor for detecting whether the subject detected as being in the predetermined position is a moving body exceeding a predetermined amount of movement. (Item 23) The information processing device according to item 22, characterized in that the first sensor includes an infrared sensor, an ultrasonic sensor, or a temperature sensor, and the second sensor includes an acceleration sensor. (Item 24) The data analysis unit analyzes the sensor data acquired from the second optical imaging device and the second sensor. The determination unit determines, by analyzing the sensor data, that if the movement of the subject is greater than a predetermined amount of movement, the optical image will be displayed on the display unit. The information processing apparatus according to item 22 or 23, characterized in that the display control unit causes the optical image to be displayed on the display unit based on the determination. (Item 25) Radiation detection device, An information processing device described in any one of items 1 to 24, which is communicably connected to the aforementioned radiation detection device, A radiography system characterized by having the following features. (Item 26) An information processing method for an information processing device that processes radiographic images obtained from radiography of a subject, An analysis step of analyzing the movement of the subject in the optical image acquired from the first optical imaging device, Based on the above analysis, a determination step is made to determine whether to display the optical image on the display unit, A display control step that controls the display of the display unit based on the determination, An information processing method characterized by having the following features. (Item 27) A program that causes a computer to perform the information processing methods described in item 26. [Explanation of symbols]

[0108] 100: Information Processing Device 101: Radiation Image Acquisition Unit 102: Image Processing Unit 103: Optical Image Acquisition Unit 104: Display Control Unit 105: Optical Image Analysis Department (Analysis Department) 106: Inspection Information Management Department 107: Judgment section 108: Sensor data acquisition unit 109: Sensor Data Analysis Department (Data Analysis Department) 110: Inference Processing Unit 120: Radiation Generator 130: Radiation detection device 140: Optical imaging device (first optical imaging device) 150: Display section 160:Operation unit 170: Surveillance camera (second optical imaging device)

Claims

1. An information processing device for processing radiographic images acquired by radiography of a subject, An analysis unit that analyzes the movement of the subject in the optical image acquired from the first optical imaging device, A determination unit that determines whether to display the optical image on the display unit based on the above analysis, Based on the above determination, a display control unit controls the display of the display unit, An information processing device characterized by comprising:

2. The analysis unit performs a human body detection process to detect the subject captured in the frame of the optical image by performing image processing or inference processing using a trained model, A motion detection process that determines whether the subject detected by the human body detection process is a moving body exceeding a predetermined amount of movement by performing image processing using multiple frames of the optical image or inference processing using a trained model, The information processing apparatus according to claim 1, characterized in that it analyzes the movement of the subject by performing the following.

3. The information processing apparatus according to claim 2, characterized in that, when a human body is detected at multiple locations within the frame by the human body detection process, the analysis unit compares the positions of the human bodies at the multiple locations and detects the human body photographed at a position close to the center of the frame as the subject.

4. The information processing apparatus according to claim 2, characterized in that, when the analysis unit detects multiple different human bodies within the frame by the human body detection process, it acquires characteristic information of the multiple human bodies and, based on the characteristic information, detects one of the multiple human bodies as the subject.

5. The aforementioned predetermined amount of movement is a threshold that can be changed by setting, The information processing apparatus according to claim 2, characterized in that the analysis unit determines that the subject is a moving object when the movement of the subject detected by the motion detection process exceeds the threshold.

6. The determination unit determines that if the movement of the subject analyzed by the analysis unit is smaller than a predetermined amount of movement, it will display the radiation image on the display unit and hide the optical image. The information processing apparatus according to claim 2, characterized in that the display control unit causes the radiation image to be displayed on the display unit based on the determination.

7. The determination unit determines that the optical image should be displayed on the display unit if the movement of the subject analyzed by the analysis unit is greater than a predetermined amount of movement. The information processing apparatus according to claim 2, characterized in that the display control unit causes the optical image to be displayed on the display unit based on the determination.

8. The display control unit, The radiographic image acquired by the radiographic imaging is displayed on the display unit. The information processing apparatus according to claim 7, characterized in that, when displaying the optical image, the radiation image is hidden and the optical image is displayed on the display unit.

9. The information processing apparatus according to claim 7, characterized in that the display control unit causes the display unit to display the radiation image and the optical image when displaying the optical image.

10. The information processing apparatus according to claim 8, characterized in that when the display control unit displays the optical image, it further displays additional information that notifies the movement of the subject on the display unit.

11. The information processing apparatus according to claim 10, characterized in that the additional information includes one of the following: a message that notifies the movement of the subject, an icon, or a combination of the message and the icon.

12. The information processing apparatus according to claim 10, characterized in that the display control unit changes the display color of the additional information and displays it on the display unit when the movement of the subject exceeds a predetermined amount of movement.

13. The information processing apparatus according to claim 7, characterized in that the determination unit causes the display control unit to output a notification sound from the sound output unit to notify the movement of the subject when the display control unit displays the optical image.

14. The information processing apparatus according to claim 13, characterized in that when the movement of the subject exceeds a predetermined amount of movement, the determination unit changes the type of notification sound or increases the volume of the notification sound and outputs it from the sound output unit.

15. In the analysis results of the analysis unit, if the movement of the subject falls within a predetermined range that exceeds the predetermined amount of movement set as a threshold, the display control unit causes the display unit to display a first-size optical image. The information processing apparatus according to claim 7, characterized in that, in the analysis results, if the movement of the subject exceeds the upper limit of the predetermined range and a large movement is detected, the display control unit causes the display unit to display an optical image enlarged to a second size, which is larger than the first size.

16. The information processing apparatus according to claim 7, characterized in that when the display control unit acquires a plurality of optical images acquired from a plurality of first optical imaging devices, it causes the acquired plurality of optical images to be displayed on the display unit.

17. The system further includes a data analysis unit that analyzes sensor data acquired from the sensor that detects the subject, The aforementioned data analysis unit, The sensor data includes a radiation detection device that generates the radiation image and a human body detection process that detects whether the subject is captured in the frame of the surveillance optical image acquired from a second optical imaging device that photographs the subject, A determination process to determine whether the subject detected by the human body detection process is in a predetermined position relative to the radiation detection device, The information processing apparatus according to claim 1, characterized in that it analyzes the movement of the subject by performing a motion detection process to determine whether the subject, which has been determined to be in the predetermined position, is a moving body exceeding a predetermined amount of movement.

18. The determination unit determines that the optical image should be displayed on the display unit if the movement of the subject analyzed by the data analysis unit is greater than a predetermined amount of movement. The information processing apparatus according to claim 17, characterized in that the display control unit causes the optical image to be displayed on the display unit based on the determination.

19. The display control unit, The radiographic image acquired by the radiographic imaging is displayed on the display unit. The information processing apparatus according to claim 17, characterized in that when displaying the optical image, the radiation image is hidden and the optical image and the monitoring optical image are displayed on the display unit.

20. The information processing apparatus according to claim 17, characterized in that when the display control unit displays the optical image, it causes the radiation image, the optical image, and the monitoring optical image to be displayed on the display unit.

21. The first optical imaging device has a narrower field of view compared to the second optical imaging device. The information processing apparatus according to claim 17, characterized in that the display control unit causes the display unit to display an optical image of the area of ​​the subject being photographed in the radiography as the optical image captured by the first optical imaging device.

22. The sensor includes a first sensor that detects the subject as a human body and detects whether it is in a predetermined position relative to the radiation detection device, The information processing apparatus according to claim 17, further comprising a second sensor for detecting whether the subject detected as being in the predetermined position is a moving body exceeding a predetermined amount of movement.

23. The information processing apparatus according to claim 22, characterized in that the first sensor includes an infrared sensor, an ultrasonic sensor, or a temperature sensor, and the second sensor includes an acceleration sensor.

24. The data analysis unit analyzes the sensor data acquired from the second optical imaging device and the second sensor. The determination unit determines, by analyzing the sensor data, that if the movement of the subject is greater than a predetermined amount of movement, the optical image will be displayed on the display unit. The information processing apparatus according to claim 22, characterized in that the display control unit causes the optical image to be displayed on the display unit based on the determination.

25. Radiation detection device, An information processing device according to any one of claims 1 to 24, which is communicably connected to the radiation detection device, A radiography system characterized by having the following features.

26. An information processing method for an information processing device that processes radiographic images obtained from radiography of a subject, An analysis step of analyzing the movement of the subject in the optical image acquired from the first optical imaging device, Based on the above analysis, a determination step is made to determine whether to display the optical image on the display unit, A display control step that controls the display of the display unit based on the determination, An information processing method characterized by having the following features.

27. A program that causes a computer to execute the information processing method described in claim 26.

Images