Information processing apparatus, radiation imaging system, information processing method, and storage medium

Abstract

To appropriately determine the propriety of a photographing posture of a subject even when a plurality of subjects are reflected in an optical image.SOLUTION: An information processing apparatus according to an embodiment of the present disclosure includes an acquisition unit configured to acquire imaging order information related to radiography and an optical image, a prediction unit configured to predict an irradiation region of radiation using the imaging order information and the optical image, a detection unit configured to detect a human body part in the optical image, and an analysis unit configured to analyze a human body part closest to the irradiation region using the optical image when a plurality of human body parts are detected in the optical image.SELECTED DRAWING: Figure 16

Description

[Technical Field]

[0001] The present disclosure relates to an information processing device, a radiation imaging system, an information processing method, and a program. [Background technology]

[0002] In recent years, radiography support using optical images has become common in radiography for medical examinations. Optical images are acquired by capturing images of the radiography site using an optical camera, and additional information obtained by analyzing the optical images is provided to the operator along with live images. For example, Patent Document 1 proposes a technology that determines the radiography position of the subject from an optical image and outputs information regarding the appropriateness of the radiography position, thereby enabling efficient radiography that is not dependent on the skill or experience of the radiologist. Patent Document 2 also proposes a technology that stably displays a detection frame indicating the region of the subject detected in a live image. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2020-199163 [Patent Document 2] Japanese Patent Publication No. 2022-110441 Summary of the Invention [Problem to be solved by the invention]

[0004] Here, at the imaging site, if the subject is an elderly person or a child, the radiologist may adjust the imaging position of the subject while providing support to the subject. In this case, a person other than the subject (in this case, the radiologist) may appear in the optical image. Furthermore, the contrast between the background and the subject may be reduced depending on the lighting environment in the imaging room. If multiple people appear in the optical image, a person other than the subject may be recognized as the subject. Furthermore, reduced contrast between the background and the subject may make the recognition of the subject itself unstable. As a result, it may not be possible to properly determine whether the subject's imaging position is appropriate.

[0005] Therefore, one object of one embodiment of the present disclosure is to provide an information processing device that can appropriately determine whether the subject's shooting position is appropriate even when multiple subjects are reflected in the optical image. [Means for solving the problem]

[0006] An information processing device according to an embodiment of the present disclosure includes: an acquisition unit that acquires radiography order information and optical images related to radiography; a prediction unit that predicts an irradiation area of ​​radiation using the radiography order information and the optical image; a detection unit that detects a human body part in the optical image; When a plurality of human body parts are detected in the optical image, an analysis unit that uses the optical image to analyze a human body part that is closest to the irradiation area; Equipped with. [Effects of the Invention]

[0007] According to an embodiment of the present disclosure, even when a plurality of subjects are captured in an optical image, it is possible to appropriately determine whether the subject's imaging position is appropriate. [Brief explanation of the drawings]

[0008] [Figure 1] 1 shows a schematic configuration of a radiation imaging system according to an embodiment. [Figure 2] 1 shows a schematic configuration of an information processing apparatus according to an embodiment. [Figure 3] 10 is a flowchart showing a processing procedure according to Prior Art Example 1. [Figure 4] 10A and 10B are diagrams illustrating examples of optical images according to Prior Example 1. FIG. [Figure 5] 10A and 10B are diagrams illustrating an example of an output from a skeleton estimation unit according to Prior Art Example 1. FIG. [Figure 6] 10 is a flowchart illustrating an example of object information determination processing according to the first prior art example. [Figure 7] FIG. 2 is a diagram illustrating the arrangement of a radiation generating device in a camera coordinate system. [Figure 8] 10 is a flowchart of an example of processing using object information according to Prior Example 1. [Figure 9] 10 is a flowchart showing a processing procedure according to Prior Art Example 2. [Figure 10] 10A and 10B are diagrams illustrating examples of optical images according to Prior Art Example 2. FIG. [Figure 11] 10A and 10B are diagrams illustrating an example of an output from a skeleton estimation unit according to Prior Art Example 2. FIG. [Figure 12] 10 is a flowchart illustrating an example of object information determination processing according to the second prior art example. [Figure 13] 10A and 10B are diagrams illustrating other examples of optical images according to Prior Art Example 2. FIG. [Figure 14] 10 is a flowchart showing another example of the object information determination process according to the second prior art example. [Figure 15] 10 is a flowchart showing a processing procedure according to a third prior art example. [Figure 16] 1 is a flowchart illustrating a processing procedure according to the first embodiment. [Figure 17] 3A to 3C are diagrams illustrating examples of optical images according to the first embodiment. [Figure 18] 5A and 5B are diagrams illustrating an example of an output from a subject information determination unit according to the first embodiment. [Figure 19] 10 is a flowchart showing a processing procedure according to the second embodiment. [Figure 20]10 is a diagram illustrating an example of an output from a subject information determination unit according to the second embodiment. FIG. [Figure 21] 10 is a flowchart showing a processing procedure according to a third embodiment. [Figure 22] 10A and 10B are diagrams illustrating examples of optical images according to the third embodiment. [Figure 23] 11 is a diagram illustrating an example of an output from a subject information determination unit according to the third embodiment. FIG. [Figure 24] 10 is a flowchart showing a processing procedure according to a fourth embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, exemplary embodiments and examples for carrying out the present disclosure will be described in detail with reference to the drawings. However, the dimensions, materials, shapes, relative positions of components, etc. described in the following embodiments and examples are arbitrary and can be changed depending on the configuration of the device to which the present invention is applied or various conditions. In addition, the same reference numerals are used between drawings to indicate identical or functionally similar elements.

[0010] In the following, the term radiation can include, for example, electromagnetic radiation such as X-rays and gamma rays, as well as particle radiation such as alpha rays, beta rays, particle rays, proton rays, heavy ion rays, and meson rays.

[0011] Furthermore, a machine learning model refers to a learning model based on a machine learning algorithm. Specific examples of machine learning algorithms include nearest neighbor algorithms, naive Bayes algorithms, decision trees, and support vector machines. Another example is deep learning, which uses a neural network to generate features and connection weighting coefficients for learning. Algorithms using decision trees include gradient boosting techniques such as LightGBM and XGBoost. Any of the above algorithms can be used as appropriate and applied to the following embodiments and examples. Furthermore, training data refers to training data and is composed of pairs of input data and output data. Furthermore, output data from training data is also referred to as ground truth data.

[0012] Furthermore, a trained model refers to a machine learning model that follows any machine learning algorithm, such as deep learning, and that has been trained (learned) in advance using appropriate training data (learning data). However, although a trained model is obtained in advance using appropriate training data, it does not mean that it does not undergo further learning, and additional learning can also be performed. Additional learning can also be performed after the device is installed at the site of use.

[0013] (Embodiment) (Schematic configuration) First, a radiation imaging system, an information processing device, and an information processing method according to an embodiment of the present disclosure will be described with reference to FIGS. 1 and 2. The embodiment of the present disclosure is applied to, for example, a radiation imaging system 100 and an information processing device 200 as shown in FIGS. 1 and 2. FIG. 1 shows a schematic configuration of the radiation imaging system 100 according to an embodiment of the present disclosure, and FIG. 2 shows a schematic configuration of the information processing device 200 according to an embodiment of the present disclosure. Note that while FIG. 1 shows a state in which the subject O is in a supine position, the subject O may be, for example, in an upright position or a sitting position. Furthermore, the imaging table used to support the subject O may be a table according to the position of the subject O.

[0014] The radiation imaging system 100 is provided with an information processing device 200, a radiation generation device 120, a radiation detector 130, and a camera 140. The information processing device 200 is connected to the radiation generation device 120, the radiation detector 130, and the camera 140 and can control them. The information processing device 200 can also perform image processing and analysis of various images obtained using the radiation detector 130 and the camera 140. The information processing device 200 is also connected to an external storage device 160 such as a server via an arbitrary network 150 such as the Internet or an intranet, and can exchange data with the external storage device 160. The external storage device 160 may also be directly connected to the information processing device 200.

[0015] The radiation generating device 120 includes, for example, a radiation generator such as a radiation tube, a collimator, a collimator lamp, etc., and can irradiate a radiation beam under the control of the information processing device 200. The radiation beam irradiated from the radiation generating device 120 passes through the object O while being attenuated, and enters the radiation detector 130.

[0016] The radiation detector 130 can detect an incident radiation beam and transmit a signal corresponding to the detected radiation beam to the information processing device 200. The radiation detector 130 may be any radiation detector that detects radiation and outputs a corresponding signal, and may be configured using, for example, an FPD (Flat Panel Detector). The radiation detector 130 may be an indirect conversion type detector that converts radiation into visible light using a scintillator or the like and then converts the visible light into an electrical signal using a photosensor or the like, or may be a direct conversion type detector that directly converts incident radiation into an electrical signal.

[0017] The camera 140 is an example of an optical device that captures an optical image of the subject O under the control of the information processing device 200 and acquires the optical image. The camera 140 transmits the captured optical image to the information processing device 200. The camera 140 may have any known configuration, and may be configured as a camera capable of capturing moving images such as a video camera, or may be configured as a camera that only captures still images. The camera 140 may be configured to capture images using visible light, or may be configured to capture images using invisible light other than radiation, such as infrared light.

[0018] The information processing device 200 is provided with an optical image acquisition unit 201, a skeleton estimation unit 202, a subject information determination unit 203, a consistency determination unit 204, a radiographic image acquisition unit 205, an annotation unit 206, a display control unit 207, a person detection unit 208, and an image analysis unit 209. The information processing device 200 also is provided with a CPU 231, a storage unit 232, a main memory 233, an operation unit 234, and a display unit 235. The various units of the information processing device 200 are connected via a CPU bus 230, and are capable of exchanging data with one another.

[0019] The optical image acquiring unit 201 can control the camera 140 and acquire an optical image of the subject O captured by the camera 140. The optical image acquiring unit 201 may also acquire an optical image of the subject O from the external storage device 160 or an optical device (not shown) connected to the information processing device 200 via an arbitrary network. The optical image acquiring unit 201 may also acquire an optical image stored in the storage unit 232.

[0020] The skeleton estimation unit 202 performs skeleton estimation using the optical image as input data for a trained model, and can estimate skeletal information of the human body in the optical image. The trained model used by the skeleton estimation unit 202 according to this embodiment can be a trained model generated by additionally training desired data on a general-purpose trained model for estimating skeletal information obtained using a large amount of training data. Here, the desired data may include skeletal information desired in a medical institution, a medical setting, or the like where the radiation imaging system 100 is used. Detailed processing by the skeleton estimation unit 202 will be described later.

[0021] The subject information determination unit 203 analyzes the estimated skeletal information and can determine and recognize subject information including information indicating the part of the subject O that is the subject of radiographic imaging in the optical image, information indicating the laterality, information indicating the direction, etc. Detailed processing by the subject information determination unit 203 will be described later.

[0022] The consistency determination unit 204 can determine whether there is consistency between the subject information determined by the subject information determination unit 203 and information about the subject O included in the radiographic image capturing order. The information processing device 200 can provide the operator with the determination result to assist in determining whether the subject information obtained using the optical image matches the radiographic imaging instructions.

[0023] The radiation image acquisition unit 205 controls the radiation generation device 120 and the radiation detector 130 to perform radiography of the subject O and acquire a radiation image of the subject O from the radiation detector 130. The radiation image acquisition unit 205 may also acquire a radiation image of the subject O from the external storage device 160 or a radiation detector (not shown) connected to the information processing device 200 via an arbitrary network. The radiation image acquisition unit 205 may also acquire a radiation image stored in the storage unit 232.

[0024] The annotation unit 206 annotates the object information determined by the object information determination unit 203 onto the radiation image. Here, annotation refers to a process of embedding object information indicating the part, laterality, direction, etc. of the object O into the radiation image. The annotation unit 206 may be configured to be able to annotate the object information onto the optical image.

[0025] The display control unit 207 can control the display of the display unit 235. The display control unit 207 can display, for example, patient information about the patient who is the subject O, imaging conditions, parameters set by the operator, generated optical images and radiation images, determined subject information, analysis information, and the like on the display unit 235. Here, the analysis information may include, for example, segmentation information, and the like. Furthermore, the display control unit 207 can display, on the display unit 235, any display or GUI, such as a button or slider for receiving an operation by the operator, according to a desired configuration.

[0026] The CPU (Central Processing Unit) 231 is an example of a processor that controls the operation of the information processing device 200. The CPU 231 uses the main memory 233 to control the operation of the entire device in accordance with operations from the operation unit 234 and parameters stored in the storage unit 232. Note that the processor in the information processing device 200 is not limited to a CPU, and may include, for example, a microprocessing unit (MPU) and a graphics processing unit (GPU). The processor may also include a digital signal processor (DSP), a data flow processor (DFP), and a neural processing unit (NPU).

[0027] The storage unit 232 can store various images, data, etc. processed by the information processing device 200. The storage unit 232 can also store patient information, imaging conditions, parameters set by the operator, etc. The storage unit 232 can also store information on a rule-based algorithm for analyzing skeletal information performed by the subject information determination unit 203. The storage unit 232 may be configured with any storage medium, such as an optical disk or memory. The main memory 233 is configured with memory, etc., and can be used for temporary data storage, etc.

[0028] Note that a GPU can perform efficient calculations by processing a larger amount of data in parallel. Therefore, when performing learning multiple times using a machine learning algorithm such as deep learning, it is effective to use a GPU for processing. Therefore, in this embodiment, a GPU may be used in addition to a CPU for processing by the information processing device 200, which functions as an example of a learning unit. Specifically, when executing a learning program including a learning model, learning can be performed by the CPU and GPU working together to perform calculations. Note that calculations may be performed only by the CPU or the GPU in the processing of the learning unit. Furthermore, the estimation process according to this embodiment may also be realized using a GPU, as with the learning unit. Note that if the trained model is provided in an external device, the information processing device 200 does not need to function as a learning unit.

[0029] The learning unit may also include an error detection unit and an update unit (not shown). The error detection unit obtains the error between correct data and output data output from the output layer of the neural network in response to input data input to the input layer. The error detection unit may use a loss function to calculate the error between the output data from the neural network and correct data. The update unit updates the connection weighting coefficients between the nodes of the neural network based on the error obtained by the error detection unit so as to reduce the error. This update unit updates the connection weighting coefficients using, for example, an error backpropagation method. The error backpropagation method is a technique for adjusting the connection weighting coefficients between the nodes of each neural network so as to reduce the error.

[0030] As the machine learning model according to this embodiment, for example, FCN (Fully Convolutional Network) or SegNet can be used. As the machine learning model for object recognition, for example, RCNN (Region CNN), fastRCNN, or fasterRCNN can be used. Furthermore, as the machine learning model for object recognition in units of regions, YOLO (You Only Look Once), SSD (Single Shot Detector, or Single Shot MultiBox Detector) can be used.

[0031] The operation unit 234 includes input devices, such as a keyboard and a mouse, for operating the information processing device 200. The operator can also input parameters and the like related to a rule-based algorithm for analysis processing using skeletal information performed by the subject information determination unit 203 via the operation unit 234.

[0032] The display unit 235 includes, for example, any display, and displays various information such as subject information and various images under the control of the display control unit 207. The display unit 235 may be, for example, a monitor of a console for operating a radiation imaging apparatus including the radiation generation device 120 and the radiation detector 130. The display unit 235 may also be a sub-monitor installed at a position where the operator can observe while assisting in positioning the subject O, or a console monitor of a radiation irradiator. The display unit 235 may also be a device that allows the operator to reliably check the display with minimal eye movement, such as a head-mounted display that the operator can wear while working. The display unit 235 may also be configured as a touch panel display, in which case the display unit 235 can also serve as the operation unit 234.

[0033] The information processing device 200 can be configured by a computer provided with a processor and a memory. The information processing device 200 may be configured by a general computer or a computer dedicated to a radiation imaging system. The information processing device 200 may be, for example, a personal computer (PC), or a desktop PC, a notebook PC, or a tablet PC (portable information terminal). Furthermore, the information processing device 200 may be configured as a cloud-based computer in which some of the components are located in an external device.

[0034] Furthermore, the optical image acquisition unit 201, the skeleton estimation unit 202, the subject information determination unit 203, the consistency determination unit 204, the radiographic image acquisition unit 205, the annotation unit 206, the display control unit 207, the human detection unit 208, and the image analysis unit 209 may be configured as software modules executed by the CPU 231. Furthermore, each of these components may be configured as a circuit that performs a specific function, such as an ASIC, or an independent device.

[0035] Next, the operation of the information processing device 200 under the control of the CPU 231 will be described. First, based on radiography order information transmitted from an information management device (not shown), the information processing device 200 starts preparation for radiography, and the optical image acquisition unit 201 starts acquiring optical images under the control of the CPU 231. Here, the radiography order information is information corresponding to the unit of examination ordered by a doctor, and includes, for example, patient information, the (scheduled) radiography date and time, and the part, direction, and posture of the subject to be radiographed based on the doctor's findings. The radiography order information includes information necessary for radiography, such as the use of the radiation detection device to be used (standing, supine, portable, etc.), the patient's posture (radiography part, direction, etc.), and radiography conditions (tube voltage, tube current, presence or absence of a grid, etc.).

[0036] The optical image acquisition unit 201 controls the camera 140 to capture an optical image of the subject O and acquires the optical image from the camera 140. The optical image acquired by the optical image acquisition unit 201 is transferred sequentially via the CPU bus 230 to the main memory 233, the skeleton estimation unit 202, the human detection unit 208, and the subject information determination unit 203.

[0037] The skeleton estimation unit 202 estimates skeletal information of the subject O using the transferred optical images as input data for the trained model. Subsequently, the subject information determination unit 203 obtains subject information from the estimated skeletal information. The subject information is transferred to the consistency determination unit 204 via the CPU bus 230. The consistency determination unit 204 compares the imaging order information with the subject information and outputs a consistency determination result. The optical images, skeletal information, subject information, and consistency determination result are transferred to the storage unit 232 and display control unit 207 via the CPU bus 230. The storage unit 232 stores the transferred various pieces of information. The display control unit 207 displays the transferred various pieces of information on the display unit 235.

[0038] The operator checks the various types of displayed information and issues operation instructions as necessary via the operation unit 234. For example, if the consistency determination result is correct, the operator issues an instruction to capture a radiographic image via the operation unit 234. This instruction to capture a radiographic image is transmitted to the radiographic image acquisition unit 205 by the CPU 231.

[0039] Upon receiving an imaging instruction, the radiation image acquisition unit 205 controls the radiation generation device 120 and the radiation detector 130 to perform radiation imaging. In radiation imaging, first, a radiation beam is irradiated from the radiation generation device 120 toward the subject O, and the radiation beam that passes through the subject O while attenuating is detected by the radiation detector 130. The radiation image acquisition unit 205 acquires a signal corresponding to the intensity of the radiation beam detected by the radiation detector 130 as a radiation image. Data of this radiation image is transferred sequentially to the main memory 233 and the annotation unit 206 via the CPU bus 230.

[0040] The annotation unit 206 annotates the transferred radiographic image with the subject information stored in the storage unit 232. The annotated radiographic image is transferred to the storage unit 232 and the display control unit 207 via the CPU bus 230. The storage unit 232 stores the transferred annotated radiographic image. The display control unit 207 displays the transferred annotated radiographic image on the display unit 235. The operator can check the displayed radiographic image and issue operation instructions as necessary via the operation unit 234. The human detection unit 208 uses the optical image transferred from the optical image acquisition unit 201 as input data for the trained model to detect a human body reflected in the optical image. Here, the optical image may reflect only the subject O, or may reflect multiple human bodies (people). If multiple people are reflected in the optical image, the human detection unit 208 detects the multiple people. An example of a case where multiple people are reflected is when the subject O and a radiologist are both reflected in the optical image. If the subject O cannot assume the imaging posture (the posture included in the imaging order information) by himself / herself (for example, if the subject O is elderly or a child), the radiological technologist may adjust the imaging posture while supporting the subject O. Therefore, during the support, the subject O and the radiological technologist may appear in the optical image. Note that the human body includes not only the entire person but also body parts such as the hands, feet, and waist. Note that an existing neural network model can be used to detect the human body.

[0041] The detected information about the human body is transferred to the consistency determination unit 204 via the CPU bus 230. The consistency determination unit 204 compares the imaging order information with the subject information (imaging region, laterality, direction, etc.) and outputs a consistency determination result.

[0042] The optical image, skeletal information, subject information, and consistency determination results are transferred to the storage unit 232 and display control unit 207 via the CPU bus 230.

[0043] The storage unit 232 stores the transferred information. The display control unit 207 causes the display unit 235 to display the transferred information.

[0044] The image analysis unit 209 performs image analysis on the human body detected in the optical image by the human detection unit 208. Since the target of image analysis is one human body, if multiple human bodies are detected, the image analysis is performed on the human body that is closest to the irradiation area among the multiple human bodies.

[0045] (Image analysis and consistency determination) In general chest and abdominal radiography, radiography is often performed in the order of radiography of the chest followed by the abdomen. Therefore, if the radiography region in the radiography order information is only for the abdomen, there is a risk that the chest will be radiographed by mistake when only the abdomen should be radiographed. Therefore, it is possible to analyze whether the radiography region in the radiography order information is being radiographed correctly, and if there is a risk of radiography being performed incorrectly, to display an alert such as "Confirm: Radiograph Region" on the display unit 235.

[0046] There is also a risk that the operator may misread the imaging region. For example, misreading of the lumbar vertebrae (LSPIEN) and the thoracic vertebrae (TSPIEN) may occur, particularly depending on the size of the characters displayed on the operation screen of the display unit 235. Therefore, it is possible to analyze whether the target region is correctly located in the center of the irradiation region for the lumbar vertebrae and the thoracic vertebrae, and if there is a risk of imaging the wrong region, to display an alert such as "Confirm: Imaging Region" on the display unit 235.

[0047] Depending on the imaging region, mistakes in the imaging direction (direction) may occur. For example, in upright chest imaging, PA imaging is often performed, in which the heart is imaged so as to be closer to the radiation detector 130, but AP imaging may also be performed in rare cases. Similarly, RL imaging is often performed in lateral imaging, but LL imaging may also be performed in rare cases. In this case, there is a risk that imaging will be performed in the wrong direction due to the radiologist's assumption. Therefore, the direction may be determined, and if there is a risk of incorrect imaging, an alert such as "Confirm: Field of View Position" may be displayed on the display unit 235. In addition, by analyzing images for RLO / LLO imaging from an oblique angle and RLD / LLD imaging in decubitus imaging, an alert may be displayed on the display unit 235 if there is a risk of incorrect imaging.

[0048] Furthermore, when imaging a limb joint that has laterality, there is a risk of mistaking the imaging site and also a risk of mistaking the left and right. For example, there is a risk of imaging the right hand when the left hand should be imaged. When imaging a limb joint that has laterality, in addition to the imaging site and laterality, the direction may also be specified. For these sites, it may be determined whether the imaging site, laterality, and direction are correct, and if there is a risk of any of them being incorrectly imaged, an alert such as "Confirm: Laterality" may be displayed on the display unit 235. For example, if the laterality of the imaging site, laterality, and direction is incorrect, an alert such as "Confirm: Laterality / Direction" may be displayed on the display unit 235. Furthermore, for example, if the laterality and direction of the imaging site, laterality, and direction are incorrect, an alert such as "Confirm: Laterality / Direction" may be displayed on the display unit 235.

[0049] There may be cases where it is not possible to determine the consistency between the imaging order information and the subject information (imaging region, laterality, direction, etc.). For example, when imaging a hand, if a hand that is not the target of imaging is captured in the image, an alert such as "Unable to determine" may be displayed on the display unit 235.

[0050] Furthermore, if no human body is detected, an alert such as "no patient detected" may be displayed on the display unit 235.

[0051] (Details of the image analysis and consistency determination sections) Next, the image analysis unit 209 and the consistency determination unit 204 will be described in detail.

[0052] The image analysis unit 209 analyzes the image acquired by the optical image acquisition unit 201 and transferred from the camera 140 .

[0053] Image analysis is performed sequentially on images from the camera 140. When images are acquired as a moving image, frames may be thinned out based on the processing load of the image analysis. Furthermore, the analysis processing may be performed using a GPU.

[0054] The image analysis unit 209 analyzes the posture of the subject by performing skeleton estimation on the human body to be analyzed. Note that the skeleton estimation can be performed using an existing neural network model.

[0055] When the camera 140 is provided in the radiation generating device 120, moving the radiation generating device 120 (or moving a collimator provided in the radiation generating device 120) also moves the camera 140. For example, when the radiation generating device is rotated by 90 degrees, the camera 140 also rotates by 90 degrees. As a result, the human body reflected in the optical image also rotates by 90 degrees (the body axis rotates by 90 degrees).

[0056] The accuracy of human detection and skeleton estimation may change when the body axis direction changes. Therefore, for example, human detection is performed with the optical image in its original state (rotated 0°), then human detection is performed with the optical image rotated 90°. Next, human detection is performed with the optical image rotated 180°, and then human detection is performed with the optical image rotated 270°. Finally, the result with the highest detection accuracy may be adopted from among these detection results.

[0057] In radiography, the radiation generating device is often rotated during adjustment before imaging, but is rarely rotated during imaging. Therefore, after adopting the result (rotation angle) with the highest detection accuracy using the above-mentioned method, the load of the analysis process can be reduced by analyzing only that rotation angle until moving on to the next imaging. Whether or not the next imaging has been performed may be determined based on whether new imaging order information has been acquired, or whether or not a human body is no longer present in the optical image.

[0058] Furthermore, depending on the rotation angle of the radiation generating device, a human body may not be detected. In such cases, a human body may be detected by rotating the optical image at each rotation angle of 0°, 90°, 180°, and 270° using the method described above and performing analysis on each image. If a human body is detected, analysis processing is performed using the optical image at the detected rotation angle.

[0059] Of course, if the radiation generating device is configured to be able to acquire the rotation angle, the rotation angle of the optical image to be analyzed may be adjusted based on the acquired rotation angle.

[0060] When determining the rotation angle of the optical image to be analyzed, the rotation angle with the highest detection accuracy among multiple optical images with different rotation angles can be stored as a history, and once the history has stabilized (when it becomes possible to predict the rotation angle with the highest detection accuracy), the analysis can be limited to that rotation angle.

[0061] Although the image rotation has been described as being of four degrees, 0°, 90°, 180°, and 270°, it is not limited to these and may be rotated in finer increments (for example, in 45° increments).

[0062] When a human body is not detected using the above-described method, or when skeleton estimation is not possible, an alert such as "not detected" may be displayed on the display unit 235.

[0063] The consistency determination unit 204 acquires the optical center position in the optical image, which will be described later, and uses this to determine consistency.

[0064] When the skeleton is estimated, first, consistency is determined for the imaging region.

[0065] In determining the imaging region, it is determined whether the imaging region included in the imaging order information is located near the center position of the optical image.

[0066] The direction is determined based on the estimated positional relationship of the skeleton.

[0067] In determining laterality, consistency is determined by whether or not a laterality region included in the radiography order information is present near the center position of the optical image. For example, when imaging a subject's hand, a hand other than the subject's (the radiologist's hand) may assist. In this case, if the subject's hand and the radiologist's hand overlap, making it difficult to determine the laterality, an alert such as "determination impossible" may be displayed on the display unit 235.

[0068] The order of determining the imaging region, direction, and laterality may be determined arbitrarily. Depending on the imaging region, the determination may start from the direction or from the laterality.

[0069] In addition, there are cases where the direction or laterality is not set in the imaging order information, in which case consistency determination regarding the direction or laterality is not performed.

[0070] Note that even if the imaging order information does not include a direction, the direction may be determined based on other information. For example, if the imaging technique name included in the imaging order information is the "Rosenberg technique" for imaging the knee, the direction may be determined.

[0071] As described above, if there is a risk of an error in the imaging region, direction, or laterality of the actual subject in relation to the imaging order information, an alert is displayed on the display unit 235. This allows the radiologist to check whether the imaging position of the subject is appropriate before imaging.

[0072] When an alert is displayed, the radiation generating device may be controlled so that radiation irradiation cannot be performed, or so that radiation irradiation can be performed. Even when the information processing device displays an alert, it may be possible to perform imaging, so it is useful to control the radiation generating device so that radiation irradiation can be performed.

[0073] (Prior example 1) 3 to 7, a radiography system, an information processing device, and an information processing method according to Prior Example 1 will be described. Prior Example 1 describes a process of recognizing the laterality of a subject to be imaged from optical images acquired at a predetermined frame rate using a video camera as camera 140, and outputting the image as subject information. Laterality is information indicating whether the body part to be imaged is on the left or right side.

[0074] (Processing flow) A series of processing procedures according to Prior Example 1 will be described below with reference to Fig. 3. Fig. 3 is a flowchart showing the processing procedures according to Prior Example 1. When the processing procedures according to Prior Example 1 start, the process proceeds to step S301.

[0075] (Step S301) In step S301, the optical image acquisition unit 201 controls the camera 140 to acquire an optical image of the radiation imaging site, including the subject being radiographed. In the preceding example 1, the camera 140 is a video camera attached to a radiation generator, and captures an image of the subject taking an imaging posture on the radiation detector 130 placed on a supine table, and outputs an optical image at a predetermined frame rate.

[0076] Here, with reference to FIG. 4, an example will be described in which a radiation detector 130, a right hand 402, and a left hand 403 are captured in an optical image 400. FIG. 4 shows an example of an optical image according to Prior Art Example 1. Here, a radiation detector region 401 in the optical image 400 is a region representing the radiation detector 130. Furthermore, in the example shown in FIG. 4, light from a collimator lamp is irradiated onto the subject's right hand, and a collimator lamp irradiation region 404 is depicted above the right hand 402 in the optical image 400. Here, the collimator lamp is a device that irradiates visible light and is attached to a collimator in order to confirm a radiation irradiation region, which is a region to be irradiated with radiation, before radiation irradiation. The collimator lamp irradiation region 404 generated by the collimator lamp coincides with the radiation irradiation region during radiography.

[0077] (Step S302) In step S302, the skeleton estimation unit 202 estimates skeletal information of the subject by using the optical image acquired by the optical image acquisition unit 201 as input data for a trained model. Here, the skeletal information is information indicating a plurality of predefined feature points on the subject, and includes, for example, coordinates indicating body parts such as the face, shoulders, hands, waist, and feet, and can be used to recognize the posture of the human body. In prior example 1, the skeleton estimation unit 202 uses a trained model constituted by a machine learning model typified by a neural network. Here, the trained model is a model generated in advance by supervised machine learning using a variety of image data not limited to the radiography site, and is additionally trained with data corresponding to an output desired in the radiography site.

[0078] First, in the case of the head, the feature points can include at least one of the eyes, nose, and mouth, for example, to distinguish between the front and back of the head. Furthermore, in the case of the torso, the feature points can include, for example, feature points for distinguishing between the neck, chest (thoracic vertebrae), abdomen, and lumbar region (lumbar vertebrae). Furthermore, in the case of the limbs, the feature points can include, for example, shoulders, elbows, wrists, hip joints, knees, and ankles. The skeleton estimation unit 202 estimates skeleton information indicating these feature points, allowing the information processing device 200 to recognize a wide range of postures. Furthermore, parts with laterality, such as the right hand and the left hand, can be defined as different feature points at the time of skeleton estimation. In this case, the information processing device 200 can estimate laterality based on the feature points.

[0079] The skeleton estimation process may be performed on the optical image using a single neural network model, or may include a subject region extraction process for extracting a subject region from the optical image and performing skeleton estimation from the extracted subject region. In this case, the subject region in the optical image may be used as input data for the trained model. Alternatively, the subject region in the optical image may be used as training data. Furthermore, the skeleton estimation process may include a classification process for classifying whether the optical image shows the subject's entire body or only parts such as the head, hands, or feet. In this case, the skeleton estimation unit 202 may select and apply a trained model for skeleton estimation specialized for each class, such as the whole body, head, arms, or feet, based on the results of the classification process.

[0080] Typically, neural network models used for skeletal estimation are trained models for general-purpose use, which are machine-learned using large amounts of data including optical images and skeletal information of the subject in the optical images. However, when such trained models are used as skeletal estimation units required in the medical settings targeted by Prior Example 1, there is a possibility that the number of feature points that can be output may be insufficient. For example, while the trained model is trained to output the shoulder joints, elbow joints, wrists, hip joints, knee joints, and ankles for the limbs, it is assumed that feature points representing the fingers, toes, and even the front and back of the fingers and toes may not be output. Therefore, Prior Example 1 uses a trained model that has been additionally trained on such a general-purpose trained model so that it can output feature points required for each medical institution or medical setting. Additional training of a neural network model to output additional feature points for some human body parts can be achieved by machine learning using a dataset that is relatively small compared to the dataset used to train a general-purpose trained model. For example, additional training can be performed using a dataset necessary and sufficient to output feature points required for each medical institution or medical setting where the radiography system 100 is used.

[0081] The training data used for additional training may include a dataset in which an optical image is used as input data and, for feature points desired for additional training, labels indicating the feature points are attached to the positions of the feature points in the optical image, or an image is used as output data (correct answer data). Note that the correct answer data may be generated by a doctor or the like based on the optical image. When an optical image is input, a trained model trained using such training data can output the coordinates of each feature point in the input optical image, including the additionally trained feature points, and a probability indicating the likelihood (likelihood) of the feature point. Note that if the probability is 0.0, the trained model does not need to output the position of the feature point. Furthermore, training data of a general-purpose trained model before additional training may also include datasets in a similar format, but the training data used for additional training includes skeletal information regarding a skeleton different from the skeleton indicated by the skeletal information trained by the general-purpose trained model. Note that the trained model used by the skeleton estimation unit 202 may have already undergone additional training, and training does not need to be performed each time processing is performed.

[0082] In Prior Example 1, an example will be described in which the skeleton estimation unit 202 applies arm skeleton estimation, which estimates the left wrist, left elbow joint, left shoulder joint, right wrist, right elbow joint, and right shoulder joint as skeleton information, to the optical image 400. In this example, the skeleton estimation unit 202 inputs the optical image 400 to the trained model. In this case, as shown in FIG. 5, the trained model detects the left wrist 501 and the right wrist 502 and outputs their respective coordinates (x501, y501) and (x502, y502) and probabilities P501 and P502 representing their likelihood as feature points. In contrast, the trained model estimates that the probability that the left shoulder joint, left elbow joint, and right shoulder joint are present in the optical image 400 is 0.0 and does not output the coordinates corresponding to each. On the other hand, the trained model estimates that the right elbow joint 503 is present at the coordinates (x503, y503) with a probability of P503. Here, it is assumed that the estimation regarding the right elbow joint is incorrect, and in this case, probability P503 will be smaller than probabilities P501 and P502.

[0083] The skeleton estimation unit 202 outputs the coordinates and probability of each feature point as skeleton information in a format like table 500. Note that the skeleton estimation unit 202 may output the skeleton information in a format according to the output of the trained model. For example, the trained model may output matrix data in a format like table 500, and the skeleton estimation unit 202 may output the matrix data or may output it in a format like table 500. Furthermore, for example, the trained model may output a map (heat map) that visualizes feature amounts extracted from an optical image. In this case, the skeleton estimation unit 202 may output the map, or may output the position and probability of the most probable feature point in the map in a format like table 500.

[0084] (Step S303) In step S303, the subject information determination unit 203 determines the laterality of the subject, who is the subject of radiographic imaging, from the skeletal information estimated by the skeleton estimation unit 202, and outputs the result as subject information. Specifically, the subject information determination unit 203 applies a rule-based algorithm such as that shown in Fig. 6 to the skeletal information estimated by the skeleton estimation unit 202. In this rule-based algorithm, the laterality of the subject is determined based on the probability and coordinates of the skeletal information output by the skeleton estimation unit 202.

[0085] Here, an example will be described in which the rule-based algorithm of Fig. 6 is applied to the output of the skeleton estimation shown in Fig. 5. The output of the skeleton estimation shown in Fig. 5 includes a left wrist 501, a right wrist 502, and a right elbow joint 503, and the probability P503 of the right elbow joint 503 is assumed to be small.

[0086] First, in step S601, the object information determination unit 203 deletes feature points whose probability of being a feature point is less than a threshold value from among the feature points included in the skeleton information output by the skeleton estimation unit 202. In the above example, the object information determination unit 203 determines that the probability P503 of the right elbow joint 503 is less than the threshold value and deletes the right elbow joint 503 as a feature point.

[0087] Next, in step S602, the object information determination unit 203 determines whether the number of remaining feature points is one or more. If it is determined in step S602 that the number of feature points is more than one, the process proceeds to step S603. On the other hand, if it is determined in step S602 that the number of feature points is one, the process proceeds to step S604. In the above example, the object information determination unit 203 determines that the number of remaining feature points is two, that is, the left wrist 501 and the right wrist 502, and therefore the process proceeds to step S603.

[0088] In step S603, the object information determination unit 203 determines the position of the planned radiation irradiation area (irradiation area) in the optical image, and deletes the remaining feature points except for the feature point closest to the position of the planned radiation irradiation area. In the above example, the object information determination unit 203 calculates the distance between the planned radiation irradiation area and the coordinates (x501, y501) of the left wrist, and the distance between the planned radiation irradiation area and the coordinates (x502, y502) of the right wrist.

[0089] Here, the position of the region to be irradiated with radiation in the optical image 400 may be the position of the collimator lamp irradiation region 404. Therefore, the object information determination unit 203 may extract the collimator lamp irradiation region 404 by, for example, threshold processing based on the brightness of the optical image, and calculate the distance from the feature point by regarding the collimator lamp irradiation region 404 as the region to be irradiated with radiation. Note that the method for extracting the collimator lamp irradiation region 404 in the optical image is not limited to this, and any known method may be used. For example, the object information determination unit 203 may extract the collimator lamp irradiation region 404 by using edge detection, corner detection, or the like.

[0090] The distance dn between the coordinates (xn, yn) of the feature point n and the representative point (for example, the center position (xc, yc)) of the region to be irradiated with radiation can be calculated according to the following formula 1.

[0091]

number

[0092] The subject information determination unit 203 compares the distance between each feature point and the planned radiation irradiation area, calculated using Equation 1, to find the feature point closest to the planned radiation irradiation area, and can delete feature points other than this feature point. In the above example, the right wrist 502 is closest to the planned radiation irradiation area among the skeleton estimation outputs shown in Figure 5, so the subject information determination unit 203 deletes the left wrist 501, which is a feature point.

[0093] In step S604, the subject information determination unit 203 outputs the laterality of the remaining feature points as subject information. In the above example, the subject information determination unit 203 outputs the laterality "right" of the right wrist 502, which is the remaining feature point. When the laterality of the feature points is output in step S604, the laterality determination process, which is the subject information determination process according to Prior Example 1, ends.

[0094] In the subject information determination process, the correspondence between the remaining feature points and the feature points to be output may be determined in advance by a rule. For example, if the right elbow joint or the right wrist remains as a feature point, the output laterality may be set to "right," and if the left elbow joint or the left wrist remains as a feature point, the output laterality may be set to "left."

[0095] Although the above description has been given of the case where the feature points are the right wrist and the left wrist, other feature points such as shoulder joints, elbow joints, hip joints, knee joints, or ankles may also be used. In this case, too, the object information determination unit 203 can output, as object information, the laterality of the feature points included in the skeleton information output by the skeleton estimation unit 202.

[0096] Furthermore, the rules for the process of determining object information are not limited to the above rules. For example, in the process of determining object information, if the remaining feature point is a "right shoulder joint" or a "left shoulder joint," and the laterality to be determined is a laterality related to a "hand," some kind of error may be considered to have occurred, and therefore a rule may be considered in which an error is output.

[0097] Furthermore, the planned radiation irradiation region is not limited to the collimator lamp irradiation region, and may be determined using other known methods. For example, more simply, the center of the optical image may be determined as the planned radiation irradiation region. In this case, the subject information determination unit 203 may output the laterality of a feature point close to the center of the optical image as the subject information. In this case, the laterality of the subject can be determined even if the collimator lamp irradiation region is not captured in the optical image. Furthermore, for example, the planned radiation irradiation region may be determined by providing a marker or the like on the imaging table to indicate the planned radiation irradiation region, and extracting the marker from the optical image using the subject information determination unit 203.

[0098] Furthermore, if higher accuracy is required in determining the object information, the radiation irradiation target region in the image may be calculated taking into account the spatial positions of the radiation generation device 120 and the camera 140. For example, as shown in Fig. 7, consider a camera coordinate system in which the optical center of the camera 140 is the origin (0,0,0), the optical axis direction of the camera 140 is the Z axis direction, and the horizontal and vertical directions of the image are the X axis direction and the Y axis direction, respectively. Here, consider a case in which the spatial coordinates (X120, Y120, Z120) of the radiation generation device 120 and the radiation irradiation direction (vx, vy, vz) are known.

[0099] This information can be obtained, for example, by installing the camera 140 relative to the radiation generation device 120 while measuring the position with a tape measure or the like. Alternatively, the information may be obtained based on the amount of drive from the installation position of the radiation generation device 120 using a configuration capable of mechanically determining the amount of drive, such as a stepping motor. Furthermore, the information may be obtained by attaching a gyro mechanism, an acceleration sensor, or the like to the radiation generation device 120 to acquire the amount of displacement of the position or angle from the initial position and inputting the acquired information to the information processing device 200. However, when the camera 140 is installed relative to the radiation generation device 120, the spatial coordinates (X120, Y120, Z120) and the radiation irradiation direction (vx, vy, vz) of the radiation generation device 120 in the camera coordinate system after installation can generally be considered to be fixed values. In this case, the coordinates (xp, yp) of the area to be irradiated with radiation in the acquired optical image can be expressed by the following equation 2, where D is the distance between the radiation generating device 120 and the radiation detector 130, (fx, fy) is the focal length of the camera 140, and (cx, cy) is the optical center.

[0100]

number

[0101] By determining the planned radiation irradiation region (xp, yp) as described above, the subject information determination unit 203 can more accurately determine the laterality of the subject by outputting the laterality of the feature point closest to the planned radiation irradiation region (xp, yp) as subject information.

[0102] (Example of use of determined specimen information) An example of processing using the subject information obtained above will be described with reference to Fig. 8. In this example, when processing using the subject information is started, the subject information obtained in the subject information determination processing is transferred to the consistency determination unit 204 via, for example, the CPU bus 230, and the processing proceeds to step S801. In step S801, the consistency determination unit 204 determines the consistency between the imaging order information and the subject information. For example, when the imaging target part of the imaging order is "hand" and the laterality is "right," the consistency determination unit 204 outputs "consistency" if the laterality included in the obtained subject information is "right," and outputs "inconsistency" otherwise.

[0103] In step S802, the display control unit 207 causes the display unit 235 to display the optical image acquired by the optical image acquisition unit 201, the object information output by the object information determination unit 203, the imaging order, and the consistency determination result. The display control unit 207 can also cause the display unit 235 to display the object region and skeletal information acquired by the skeleton estimation unit 202, and analysis information obtained during the object information determination process.

[0104] In step S803, the operator checks the information displayed on the display unit 235, and then inputs a radiographic image capturing instruction to the information processing device 200 via the operation unit 234. The radiographic image acquiring unit 205 acquires a radiographic image in response to the radiographic image capturing instruction from the operator. Specifically, the radiographic image acquiring unit 205 causes the radiation generator 120 to irradiate a radiation beam under imaging conditions according to the imaging order, and acquires a radiographic image from the radiation detector 130 that detects the radiation that has transmitted through the subject O. The radiographic image acquiring unit 205 may also acquire a radiographic image in response to the completion of the consistency determination process. In this case, the radiographic image acquiring unit 205 may acquire a radiographic image corresponding to the optical image used in the subject information determination process from the external storage device 160, the storage unit 232, etc.

[0105] In step S804, the subject information is transferred to the annotation unit 206 via, for example, the CPU bus 230, and the annotation unit 206 annotates the acquired radiographic image with the subject information. For example, because radiographic images are fluoroscopic images, it is difficult to determine whether a radiographic image of a right hand is a right hand or a left hand. However, by the annotation unit 206 annotating the radiographic image with information such as "right" as laterality information, the radiographic image can be provided with information that the subject depicted in the radiographic image has a right hand. The radiographic image annotated with the subject information can be transferred to the display control unit 207 or the storage unit 232 via the CPU bus 230. The annotation unit 206 may also annotate the subject information on the optical image. In this case, the optical image annotated with the subject information can be transferred to the display control unit 207 or the storage unit 232 via the CPU bus 230.

[0106] In step S805, the display control unit 207 can display the radiographic image annotated with the subject information on the display unit 235. The display control unit 207 can display the optical image and the radiographic image annotated with the subject information side by side or by switching between them on the display unit 235. When the display process of the radiographic image is completed, the process using the subject information is completed. The display unit 235 can also display the optical image annotated with the subject information as the optical image.

[0107] As described above, the radiography system 100 according to Prior Example 1 includes a radiation generator 120 and a radiation detector 130 that perform radiography of a subject, a camera 140 that functions as an example of an optical device that captures an optical image of the subject, and an information processing device 200. The information processing device 200 includes an optical image acquisition unit 201, a skeleton estimation unit 202, and a subject information determination unit 203. The optical image acquisition unit 201 functions as an example of an acquisition unit that acquires an optical image obtained by capturing an image of the subject at a radiographic image capture site. The skeleton estimation unit 202 functions as an example of an estimation unit that estimates skeletal information of the subject in the acquired optical image by using the acquired optical image as input data for a second trained model that is obtained by additionally training a general-purpose trained model (first trained model) that estimates skeletal information about the subject's skeleton using skeletal information about a skeleton different from the skeletal information learned by the first trained model. The object information determination unit 203 functions as an example of a determination unit that determines object information including information on the laterality of the object in the optical image, using skeletal information of the object in the optical image.

[0108] With the above configuration, the information processing device 200 according to Prior Example 1 can determine the laterality of a subject (a subject to be radiographed) as subject information using skeletal information estimated from an optical image. Therefore, the information processing device 200 can more appropriately assist an operator in determining whether the subject's laterality is appropriate using the determined subject information. Furthermore, the determined subject information can be used as additional information provided to an operator along with live images during radiography for medical examinations. Furthermore, the trained model used to estimate skeletal information according to Prior Example 1 is obtained by performing additional training on a general-purpose trained model according to the purpose of each medical institution or medical site. Therefore, the information processing device 200 can appropriately estimate skeletal information required by a medical institution or medical site without collecting a large amount of training data.

[0109] The subject information determination unit 203 can determine the subject information by rule-based processing. Therefore, the information processing device 200 can determine the subject information by rule-based processing, which is relatively easy to adjust, without adjusting a trained model, the configuration of which is difficult to change. Therefore, the information processing device 200 can determine the subject information by rules according to the operation and knowledge of a medical institution or medical site.

[0110] Furthermore, the skeleton estimation unit 202 can estimate, as skeleton information of the subject in the optical image, the coordinates of a plurality of feature points of the subject in the optical image and the probability that the plurality of feature points correspond to predefined feature points of the subject. In this case, the subject information determination unit 203 can select feature points of the subject in the optical image using a threshold value for the probability estimated by the skeleton estimation unit 202, and determine subject information using the selected feature points. By selecting feature points in this way, the information processing device 200 can determine subject information based on more appropriate feature points.

[0111] Furthermore, the object information determination unit 203 can determine a radiation irradiation target region (irradiation region) to be irradiated with radiation in the optical image. The object information determination unit 203 can select feature points of the object in the optical image based on the positional relationship between the coordinates estimated by the skeleton estimation unit 202 and the irradiation region, and determine object information using the selected feature points. By selecting feature points in this way, the information processing device 200 can determine object information based on more appropriate feature points.

[0112] The object information determination unit 203 can determine the irradiation area based on the irradiation area of ​​the collimator lamp in the optical image or the center of the optical image. The object information determination unit 203 can also determine the irradiation area based on the arrangement of the radiation generation device 120 that irradiates radiation and the camera 140 that functions as an example of an optical device that generates an optical image. In this case, by taking into account the arrangement of the radiation generation device 120 in the camera coordinate system, appropriate analysis processing can be performed according to the position of the device in each medical setting, and the analysis accuracy of the object information determination unit 203 can be improved.

[0113] The skeleton estimation unit 202 can also extract a subject region in the optical image and estimate skeletal information of the subject in the optical image by using the subject region in the optical image as input data for the second trained model. The skeleton estimation unit 202 can also classify the optical image into classes indicating whether the image shows the subject's entire body, head, hands, or feet, and select and use a second trained model corresponding to the class. In these cases, the skeleton estimation unit 202 can be expected to estimate more appropriate skeletal information.

[0114] The second trained model may be a trained model obtained by additional training using a smaller number of training data than the number of training data of the first trained model. For example, the second trained model may be a trained model obtained by additional training using skeletal information on at least one feature point of the left and right elbow joints, wrists, fingers, the front and back of the hands, knee joints, ankles, toes, and the front and back of the feet.

[0115] The information processing device 200 may further include a consistency determination unit 204 that functions as an example of a determination unit that determines whether or not there is consistency between the subject information and information included in the radiographic image capturing order. In this case, the information processing device 200 can more appropriately support the operator in determining whether or not the imaging conditions, such as the subject's posture, correspond to the radiographic image capturing order by presenting the determination result by the consistency determination unit 204.

[0116] In this regard, the information processing device 200 may further include a display control unit 207 that causes the display unit 235 to display at least one of the optical image, the subject skeletal information in the optical image, the subject region in the optical image, the subject information, and the determination result by the consistency determination unit 204. Note that, when the consistency determination unit 204 outputs a determination result indicating inconsistency, the display control unit 207 can cause the display unit 235 to display a warning. In this case, the information processing device 200 can notify the operator that inconsistency is not achieved and prompt the operator to adjust the imaging conditions, such as the subject's posture, in accordance with the imaging order, thereby more appropriately supporting radiography.

[0117] The information processing device 200 may further include an annotation unit 206 that arranges subject information on an optical image or a radiological image as annotation information. In this case, the information processing device 200 can more appropriately support the operator in determining the imaging conditions, such as the subject's posture, by presenting the optical image or the radiological image on which the annotation information is arranged. The display control unit 207 can display the annotation information and the optical image or the radiological image on which the annotation information is arranged on the display unit 235.

[0118] (Prior example 2) A radiography system, an information processing device, and an information processing method according to Prior Example 2 will be described below with reference to FIGS. 9 to 14. Prior Example 2 describes a process for recognizing a region of a subject to be imaged from optical images acquired at a predetermined frame rate using a video camera as the camera 140 and outputting the region as subject information. Here, the region of the subject includes, for example, the head, chest, and limbs of a human body. Furthermore, the granularity of the region to be recognized may vary depending on the purpose, such that, for example, the limbs can be further divided into regions such as the upper arm, elbow, wrist, hand, and finger. Here, a case where the region to be imaged in an imaging order is determined to be either the "chest" or the "abdomen" will be described as an example. The configurations of the radiography system and the information processing device according to Prior Example 2 are similar to those of the radiography system and the information processing device according to Prior Example 1, and therefore, the same reference numerals will be used and a description thereof will be omitted. Furthermore, a description of the same processes as those described in detail in Example 1 will be omitted.

[0119] (Processing flow) A series of processing procedures according to Prior Example 2 will be described with reference to Fig. 9. Fig. 9 is a flowchart showing the processing procedures according to Prior Example 2. When the processing procedures according to Prior Example 2 start, the process proceeds to step S901.

[0120] (Step S901) In step S901, the optical image acquisition unit 201 controls the camera 140 to acquire an optical image of the radiation imaging site including the subject of radiation imaging. In the preceding example 2, the camera 140 is a video camera attached to a radiation generator, and captures an image of the subject taking an imaging posture on the radiation detector 130 placed on a supine table, and outputs an optical image at a predetermined frame rate.

[0121] Here, an example will be described with reference to Fig. 10 in which a radiation detector 130, a head 1002, a chest 1003, and an abdomen 1004 are captured in an optical image 1000. Fig. 10 shows an example of an optical image according to Prior Art Example 2. Here, a radiation detector region 1001 in the optical image 1000 is a region representing the radiation detector 130. Furthermore, in the example shown in Fig. 10, it is assumed that light from a collimator lamp is irradiated onto the chest of the subject, and a collimator lamp irradiation region 1005 is depicted above the chest 1003 in the optical image 1000. The collimator lamp irradiation region 1005 coincides with the region to be irradiated with radiation during radiography.

[0122] (Step S902) In step S902, the skeleton estimation unit 202 estimates skeletal information of the subject by using the optical image acquired by the optical image acquisition unit 201 as input data for the trained model. In the preceding example 2, a case will be described in which the skeleton estimation unit 202 applies whole-body skeleton estimation to the optical image 1000, estimating left and right eyes, left and right shoulder joints, left and right hip joints, left and right elbows, left and right wrists, left and right knees, and left and right ankles as skeletal information. As shown in Fig. 11 , the skeleton estimation unit 202 detects left and right eyes 1101, 1102, left and right shoulder joints 1103, 1104, and left and right hip joints 1105, 1106 in the optical image 1000 as feature points.

[0123] The skeleton estimation unit 202 outputs the coordinates and probabilities of each feature point as skeleton information in a format such as Table 1100. Here, it is assumed that the skeleton estimation unit 202 outputs the coordinates (x1101, y1101) to (x1106, y1106) of each feature point and the probabilities P1101 to P1106 representing the likelihood of each feature point. However, it is assumed that the probability that other skeleton information is included in the optical image 1000 is 0.0, and the coordinates corresponding to each feature point are not output. Also, for simplicity of explanation, unlike the example in Prior Example 1, it is assumed that no erroneous estimation was made here. Note that the skeleton estimation unit 202 may output skeleton information in a format corresponding to the output of the trained model, similar to the skeleton estimation unit 202 in Prior Example 1.

[0124] Furthermore, the trained model used in Prior Example 2 may be a trained model that has undergone additional training on a trained model used for general-purpose skeletal estimation so as to be able to output feature points required for each medical institution or medical site, similar to the trained model related to Prior Example 1. Furthermore, the training data may be prepared in the same manner as the training data related to Prior Example 1.

[0125] (Step S903) In step S903, the subject information determination unit 203 determines the part of the subject that is the subject of radiographic imaging from the skeletal information estimated by the skeleton estimation unit 202, and outputs the determined part as subject information. Specifically, the subject information determination unit 203 applies a rule-based algorithm as shown in Fig. 12 to the skeletal information estimated by the skeleton estimation unit 202. In this rule-based algorithm, the part of the subject is determined based on the probability and coordinates of the skeletal information that is output from the skeleton estimation unit 202.

[0126] Here, an example will be described in which the rule-based algorithm of Fig. 12 is applied to the output of the skeleton estimation shown in Fig. 11 to determine whether the subject's body part is the chest or abdomen. The output of the skeleton estimation shown in Fig. 11 includes left and right eyes 1101, 1102, left and right shoulder joints 1103, 1104, and left and right hip joints 1105, 1106, and each of the probabilities P1101 to P1106 is assumed to be equal to or greater than a predetermined threshold.

[0127] First, in step S1201, the object information determination unit 203 deletes feature points whose probability of being a feature point is less than a threshold value from among the feature points included in the skeleton information output by the skeleton estimation unit 202. In the above example, since all of the probabilities P1101 to P1106 are equal to or greater than a predetermined threshold value, there are no feature points that are deleted as feature points below the threshold value.

[0128] Next, in step S1202, the object information determination unit 203 determines whether the number of remaining feature points is one or more. If it is determined in step S1202 that the number of feature points is more than one, the process proceeds to step S1203. On the other hand, if it is determined in step S1202 that the number of feature points is one, the process proceeds to step S1204. In the above example, the object information determination unit 203 determines that the number of remaining feature points is more than one, and therefore the process proceeds to step S1203.

[0129] In step S1203, the object information determination unit 203 obtains the position of the region to be irradiated with radiation in the optical image, and deletes, from the remaining feature points, those feature points other than the feature point closest to the position of the region to be irradiated with radiation. In the above example, the object information determination unit 203 calculates the distance between the position of the region to be irradiated with radiation and the coordinates (x1101, y1101) to (x1106, y1106) of the feature points.

[0130] Here, the position of the region to be irradiated with radiation in the optical image 1000 may be the collimator lamp irradiation region 1005. Therefore, similar to the prior art 1, the object information determination unit 203 may extract the collimator lamp irradiation region 1005 by threshold processing based on the brightness or the like of the optical image, and calculate the distance from the feature point by regarding the collimator lamp irradiation region 1005 as the region to be irradiated with radiation. Note that the region to be irradiated with radiation may be calculated taking into consideration the center of the optical image and the spatial positions of the radiation generation device 120 and the camera 140, as described in the prior art 1.

[0131] The subject information determination unit 203 compares the distance between each feature point and the region to be irradiated with radiation, finds the feature point that is closest to the region to which radiation sickness is present, and can delete feature points other than that feature point. In the above example, of the skeleton estimation outputs shown in Figure 11, either the left or right shoulder joint 1103 or 1104 is closest to the region to be irradiated with radiation, so the subject information determination unit 203 deletes the other feature points.

[0132] In step S1204, the object information determination unit 203 outputs the location of the remaining feature point as object information. In the above example, the object information determination unit 203 outputs the location corresponding to one of the remaining feature points, the left or right shoulder joint 1103, 1104. Since the rule-based algorithm according to Prior Example 2 is an algorithm for determining whether the location is the chest or abdomen, the determination result is output as "chest," which is close to the shoulder joint. In the algorithm for determining whether the location is the chest or abdomen, "chest" is output even if the feature point closest to the planned radiation irradiation region is either the left or right eye. On the other hand, "abdomen" is output when the feature point closest to the planned radiation irradiation region is the left or right hip joint 1105, 1106, or, although not shown here, the left or right wrist or knee joint. When the object location is output in step S1204, the laterality determination process, which is the process for determining object information according to Prior Example 2, is completed.

[0133] The above describes a case where feature points are the left and right eyes, shoulder joints, and hip joints, and the subject's region is determined to be the chest or abdomen. However, the subject information determination unit 203, which outputs the region as subject information, may use other feature points or determine other regions, such as the head or limbs. The rule-based algorithm for determining subject information may vary depending on the imaging posture adopted by the medical institution or medical site, the feature points captured within the angle of view of the camera 140 used, and other factors. For example, a rule-based algorithm intended to determine the head and limbs may output "head" if the feature point closest to the radiation irradiation target region is either the left or right eye. Furthermore, a rule-based algorithm intended to determine the left and right wrists and knee joints may output regions with granularity such as "left wrist," "right wrist," "left knee joint," and "right knee joint."

[0134] Other rule-based algorithms for determining the body part of a subject are also possible. For example, as shown in Fig. 13, it is assumed that a certain medical institution installs the camera 140 so that the optical image obtained when imaging the chest is optical image 1300 and the optical image obtained when imaging the abdomen is optical image 1301. In this case, the subject information determination unit 203 may apply a rule-based algorithm such as that shown in Fig. 14 to the output of the bone structure estimation.

[0135] In the rule-based algorithm shown in FIG. 14, first, in step S1401, the object information determination unit 203 simply determines whether there are any feature points belonging to the head. If it is determined in step S1401 that there are any feature points belonging to the head, the process proceeds to step S1402. In step S1402, the object information determination unit 203 outputs "chest" as the object information. On the other hand, if it is determined in step S1401 that there are no feature points belonging to the head, the process proceeds to step S1403. In step S1403, the object information determination unit 203 outputs "abdomen" as the object information. When the object's part is output in step S1402 or step S1403, the laterality determination process, which is the object information determination process according to Prior Example 2, is completed.

[0136] In this way, the rule-based algorithm for determining subject information may be adapted to the operations at the medical institution or medical site where the radiation imaging system 100 or the information processing device 200 is used. Here, the operations at the medical institution or medical site may include, for example, agreements regarding the posture of the subject during radiation imaging and the placement of the camera 140 depending on the imaging region.

[0137] (Example of use of determined specimen information) The subject information including the subject's body part obtained as described above may be used for processing such as annotation, as in the preceding example 1. For example, the subject information including the subject's body part obtained may be transferred to the consistency determination unit 204 via the CPU bus 230 and used to determine the consistency between the imaging order information and the subject information. The display control unit 207 can also display the subject information and analysis information obtained along the way on the display unit 235. Furthermore, the annotation unit 206 can annotate the subject information on the acquired radiographic image. The display control unit 207 can also display the radiographic image annotated with the subject information on the display unit 235.

[0138] As described above, the subject information determination unit 203 can determine subject information including information indicating the subject's region in the optical image using the subject's skeletal information in the optical image. With the above configuration, the information processing device 200 according to Prior Example 2 can determine the region of the subject (radiography subject) as subject information using skeletal information estimated from the optical image. Therefore, the information processing device 200 can more appropriately support the operator's judgment regarding the appropriateness of the subject's region using the determined subject information. Furthermore, as in Prior Example 1, the determined subject information can be used as additional information provided to the operator along with live images in radiography for medical examinations. Furthermore, as in Prior Example 1, the information processing device 200 can appropriately estimate skeletal information required in medical institutions and medical settings without collecting a large amount of training data. While Prior Example 2 determines subject information including information indicating the subject's region, information indicating the subject's laterality and region may also be determined as subject information.

[0139] (Prior example 3) A radiation imaging system, an information processing device, and an information processing method according to Prior Example 3 will be described below with reference to Fig. 15. Prior Example 3 describes customization (adjustment) processing for recognizing the laterality and part of the subject to be imaged from optical images acquired at a predetermined frame rate using a video camera as the camera 140, and outputting the results as subject information.

[0140] The configurations of the radiation imaging system and information processing device according to Prior Example 3 are similar to those of the radiation imaging system and information processing device according to Prior Example 1, and therefore the same reference numerals are used and the description thereof is omitted. Also, the description of the same processing as that described in detail in Prior Examples 1 and 2 is omitted.

[0141] (Processing flow) 15, a processing procedure for adjusting the rule-based algorithm related to the object information determination processing according to Prior Example 3 will be described. Fig. 15 is a flowchart showing the processing procedure according to Prior Example 3. When the adjustment processing of the rule-based algorithm related to the object information determination processing according to Prior Example 3 is started, the processing proceeds to step S1501.

[0142] (Step S1501) In step S1501, the optical image acquisition unit 201 controls the camera 140 to acquire an optical image of the radiation imaging site, including the subject being radiographed. In Prior Example 3, the camera 140 is a video camera attached to a radiation generator, and images the subject in an imaging position on the radiation detector 130 placed on a supine table, and outputs optical images at a predetermined frame rate. Here, as in Prior Example 2, a case will be described with reference to FIG. 10 where the radiation detector 130, head 1002, chest 1003, and abdomen 1004 are captured in the optical image 1000.

[0143] (Step S1502) In step S1502, the skeleton estimation unit 202 estimates skeletal information of the subject by using the optical image acquired by the optical image acquisition unit 201 as input data for the trained model. In Prior Example 3, an example will be described in which the skeleton estimation unit 202 applies whole-body skeleton estimation to the optical image 1000, estimating left and right eyes, left and right shoulder joints, left and right hip joints, left and right elbows, left and right wrists, left and right knees, and left and right ankles as skeletal information. As in Prior Example 2, the skeleton estimation unit 202 can detect left and right eyes 1101, 1102, left and right shoulder joints 1103, 1104, and left and right hip joints 1105, 1106 in the optical image 1000 as feature points, as shown in FIG.

[0144] Note that the trained model used in Prior Example 3 may be a trained model that has undergone additional training on a trained model used for general-purpose skeletal estimation so as to be able to output feature points required for each medical institution or medical site, similar to the trained model related to Prior Example 1. Furthermore, the training data may also be prepared in the same way as the training data related to Prior Example 1.

[0145] (Step S1503) In step S1503, the subject information determination unit 203 adjusts parameters of a rule-based algorithm for determining a region of the subject that is the subject of radiographic imaging, in response to an input from the operator via the operation unit 234. More specifically, the display control unit 207 causes the display unit 235 to display the skeletal information estimated by the skeleton estimation unit 202. Based on the skeletal information displayed on the display unit 235, the operator instructs the information processing device 200 via the operation unit 234 which region of the subject should be output as subject information. The subject information determination unit 203 adjusts various parameters of the rule-based algorithm as shown in FIG. 12 in response to an instruction from the operator.

[0146] Here, the parameters of the rule-based algorithm may include, for example, a threshold value for threshold processing applied to the probability in step S1201, and the number of feature points serving as a criterion for determining whether to branch the process in step S1202. Furthermore, the parameters of the rule-based algorithm may include, in step S1203, the positional relationship between the planned radiation irradiation region and the feature points to be retained, and the planned radiation irradiation region, and the correspondence relationship between the remaining feature points and the region to be output in step S1204. Note that the positional relationship between the feature points to be retained and the planned radiation irradiation region may include, for example, the order of proximity of the feature points to be retained to the planned radiation irradiation region (e.g., closest or second-closest).

[0147] For example, a case will be described in which, in order to adjust the rule-based algorithm as shown in FIG. 12 , the operator inputs an instruction via the operation unit 234 indicating that the region to be determined from the estimated skeletal information is the "chest." In this case, the object information determination unit 203 determines the planned radiation irradiation region and determines which of the left and right eyes 1101 and 1102, the left and right shoulder joints 1103 and 1104, and the left and right hip joints 1105 and 1106 is the closest feature point to the planned radiation irradiation region. In the above example, either the shoulder joints 1103 or 1104 is closest. Therefore, the object information determination unit 203 adjusts the parameters of the rule-based algorithm so that, when the shoulder joints are closest to the planned radiation irradiation region, the object information determination unit 203 determines and outputs the object information as the "chest." In this case, the object information determination unit 203 can adjust the positional relationship between the feature points to be retained and the planned radiation irradiation region, for example, in step S1203, so as to determine either the shoulder joints 1103 or 1104 as the feature point to be retained. Furthermore, the object information determination unit 203 can adjust the correspondence between the remaining feature points in step S1204 and the region names to be output.

[0148] Alternatively, subject information determination unit 203 may store, in storage unit 232, the coordinates of feature points with a probability equal to or greater than a threshold, among the skeletal information output by skeleton estimation unit 202, together with the region information input by the operator, without calculating the planned radiation irradiation region. In this case, information such as the coordinates of feature points when the region information is registered as the chest, and the coordinates of feature points when the region information is registered as the abdomen, is stored in storage unit 232. In such a case, when inputting an optical image to determine the region of the subject, subject information determination unit 203 may calculate the sum of the distances between the coordinates of feature points obtained by inputting the optical image and the coordinates of the stored feature points, and determine the region with the smallest sum as subject information.

[0149] Regarding a method for adjusting parameters of the rule-based algorithm, a configuration may be adopted in which multiple parameter combinations are prepared in advance and one of the parameter combinations is selected in response to an instruction from an operator. For example, the object information determination unit 203 may select a parameter combination by referring to a prepared lookup table in response to an instruction from an operator, and adjust the parameters. The parameter combinations may include, for example, a parameter combination related to processing as shown in FIG. 12, a parameter combination related to processing using coordinates of feature points equal to or greater than a threshold, and a parameter combination related to processing as shown in FIG. 14. The parameter combinations may also include, for example, a parameter combination related to processing for determining the laterality of the object as shown in FIG. 12. The instruction from the operator may specify object information to be output as described above, or may specify a parameter combination.

[0150] In step S1503, the parameters of the rule-based algorithm are adjusted, and the series of processes ends.

[0151] In Prior Art Example 3, an example was described in which parameters of a rule-based algorithm were adjusted for the process of determining the region of a subject as subject information. In contrast, for the process of determining the laterality of a subject as subject information, as in Prior Art Example 1, parameters of the rule-based algorithm can be adjusted in the same manner as in Prior Art Example 3. In this case, the parameters of the rule-based algorithm may include, for example, a threshold value for the threshold process applied to the probability in step S601 and the number of feature points serving as a criterion for determining whether to branch the process in step S602. Furthermore, the parameters of the rule-based algorithm may include the positional relationship between the region to be irradiated and the feature points to be retained in step S603 and the region to be irradiated, and the correspondence relationship between the remaining feature points and the laterality to be output in step S604. For the process of determining the region and laterality of a subject as subject information, parameters of the rule-based algorithm can also be adjusted in the same manner as in Prior Art Example 3.

[0152] As described above, the subject information determination unit 203 according to Prior Art Example 3 can adjust parameters of the rule-based processing based on skeletal information of the subject in the optical image and subject information acquired via the operation unit 234. Here, the parameters of the rule-based processing can include at least one of the threshold value of threshold processing related to selecting feature points of the subject in the optical image, the positional relationship between the feature points and the irradiation area to which radiation is irradiated, and the correspondence relationship between the feature points and at least one of the part and laterality of the subject. With the above configuration, the information processing device 200 according to Prior Art Example 3 can flexibly adapt the subject information determination process to the operations and knowledge of the medical institution or medical site by adjusting the rules according to the operations and knowledge of the medical institution or medical site.

[0153] In addition, in Prior Art Example 3, similarly to Prior Art Examples 1 and 2, the optical image acquisition unit 201 has been described as acquiring an optical image of a radiation imaging site. In contrast, the optical image used in the adjustment process of the rule-based algorithm may be an optical image acquired at a virtual radiation imaging site by imaging a human body phantom. That is, the subject information determination unit 203 can adjust the parameters of the rule-based processing based on skeletal information acquired using the optical image acquired by imaging the phantom and subject information acquired via the operation unit 234. In this case, an image that makes skeletal estimation easier in step S1502 can be acquired compared to using an actual subject, and more appropriate parameter adjustment can be performed in step S1503.

[0154] Furthermore, the optical image used in the adjustment process of the rule-based algorithm does not have to be an actual image. For example, a two-dimensional image on which three-dimensional coordinates representing virtual skeletal positions calculated from virtual subject data generated using a three-dimensional modeling tool are projected may be used. In this case, the subject information determination unit 203 may use skeletal information output by the skeleton estimation unit 202 based on the two-dimensional image for the parameter adjustment process. That is, the subject information determination unit 203 can adjust the parameters of the rule-based processing based on skeletal information obtained by projecting the three-dimensional coordinates of virtual subject data generated by the three-dimensional modeling tool onto two-dimensional image coordinates and subject information acquired via the operation unit 234. This method allows an image showing skeletal positions in a desired pose to be relatively easily acquired on the information processing device 200 without having to position the human body or phantom in an arbitrary pose. Note that such two-dimensional images may be used as input data for training data for additional training of a trained model used by the skeleton estimation unit 202.

[0155] In Prior Art Example 3, the subject information determination unit 203 adjusts parameters of a rule-based algorithm for determining subject information in response to instructions from the operator. Alternatively, the information processing device 200 may be provided with a programming environment, allowing the operator to write a program for handling the skeletal information output by the skeleton estimation unit 202 via the operation unit 234. Such a configuration may use a programming language such as C or Python. The information processing device 200 may also be configured using a no-code development platform that allows rule-based algorithms to be built using a graphical user interface. Furthermore, in recent years, it has become possible to generate programs by inputting instructions in natural language using an artificial intelligence chatbot, a type of generative AI. Therefore, the information processing device 200 may be configured to utilize a generative AI such as an artificial intelligence chatbot.

[0156] Furthermore, the training data for the trained model is not limited to data obtained using the camera 140 itself that actually takes the images, but may be data obtained using a camera of the same model or type, depending on the desired configuration. Note that the trained model for skeletal estimation according to the above-mentioned prior art is considered to extract, for example, the magnitude of the luminance values ​​of the optical image, the order and gradient of the bright and dark areas, position, distribution, continuity, etc. as part of the features and use them in the process of estimating skeletal information.

[0157] The trained model for skeleton estimation described above can be provided in the information processing device 200. The inference device (trained model) may be configured, for example, as a software module executed by a processor such as a CPU, MPU, GPU, or FPGA, or as a circuit performing a specific function such as an ASIC. The inference device may also be provided in a device such as a separate server connected to the information processing device 200. In this case, the information processing device 200 can use the inference device by connecting to a server equipped with the inference device via any network such as the Internet. Here, the server equipped with the inference device may be, for example, a cloud server, a fog server, or an edge server. Note that when a network within a facility, a site including a facility, or a region including multiple facilities is configured to enable wireless communication, the reliability of the network may be improved by using radio waves in a dedicated wavelength band allocated exclusively to the facility, site, region, or the like. The network may also be configured using wireless communication capable of high-speed communication, large-capacity communication, low-latency communication, and multiple simultaneous connections.

[0158] Example 1 In the prior art examples 1 to 3, methods for estimating the skeleton of a subject are described on the assumption that only one subject is captured in the optical image. However, when multiple subjects are captured in the optical image, for example, it may be impossible to determine which subject should have their skeleton estimated, and the process related to the skeleton estimation may be interrupted. As a result, it may be impossible to properly determine whether the subject's photographing position is appropriate.

[0159] Therefore, in the first embodiment, a method for appropriately determining whether the subject's photographing position is appropriate even when a plurality of subjects are captured in an optical image will be described.

[0160] 16 to 18, a radiation imaging system, an information processing device, and an information processing method according to a first embodiment of the present disclosure will be described. In this embodiment, when multiple subjects are captured in an optical image acquired at a predetermined frame rate using a camera 140, a process of determining the subject closest to an irradiation area as the imaging target (the subject to be imaged using radiation) will be described.

[0161] (Processing flow) A series of processing steps according to this embodiment will be described below with reference to FIG.

[0162] 16A is a flowchart showing the processing procedure according to this embodiment. When the processing procedure according to this embodiment starts, the process proceeds to step S1601.

[0163] (Step S1601) In step S1601, the optical image acquisition unit 201 controls the camera 140 to acquire an optical image of the radiation imaging site, including the subject who is the target of radiation imaging. In this embodiment, the camera 140 is a video camera attached to the radiation generator. The camera 140 images the subject who is in an imaging position above the radiation detector 130, which is placed in an upright position, and outputs the optical image at a predetermined frame rate.

[0164] 17, an example will be described in which the radiation detector 130, an object 1701, and an object 1702 are reflected in an optical image 1700. Fig. 17 shows an example of an optical image according to this embodiment.

[0165] (Step S1602) In step S1602, the human detection unit 208 and the image analysis unit 209 use the optical image acquired by the optical image acquisition unit 201 as input data for the trained model to determine whether there is one or multiple subjects that can be the subject of a radiological image capture.

[0166] If it is determined in step S1602 that there are a plurality of subjects who can be the subject of radiographic imaging, the process proceeds to step S1603.

[0167] On the other hand, if it is determined in step S1602 that there is only one subject that can be the subject of radiographic imaging, the process proceeds to step S1605.

[0168] In the example of FIG. 17, the object information determination unit 203 determines that there are two objects that can be the subject of radiographic imaging, and therefore the process proceeds to step S1603.

[0169] (Step S1603) In step S1603, the object information determination unit 203 determines the position of the radiation irradiation planned region (irradiation region 1703) in the optical image 1700, to which radiation is to be irradiated.

[0170] Here, the position of the irradiation area 1703 in the optical image 1700 is predicted based on information about the shooting distance included in the imaging order information. This is because, when the camera 140 is provided in the radiation generation device 120, the irradiation area 1703 in the optical image 1700 changes as the distance between the radiation generation device 120 and the radiation detector 130 changes. The information about the shooting distance is, for example, the source to image receptor distance (SID), which is the distance between the radiation generation device 120 and the radiation detector 130. Alternatively, it may be the intended use of the radiation detection device related to the SID (for use in upright imaging, for use in recumbent imaging, for portable imaging, etc.). The intended use of the radiation detection device is sometimes expressed as a position type.

[0171] The position of the irradiation area in the optical image 1700 may be calculated by taking into account the spatial positions of the radiation generating device 120 and the camera 140, as in the first embodiment. The position of the irradiation area 1703 may be the position of the irradiation area by the collimator lamp, or may be the center position of the optical image 1700. If it is necessary to determine the center position of image analysis when the image analysis unit 209 performs image analysis, the center position of the irradiation area may be used as the center position of the image analysis. The object information determination unit 203 is an example of a prediction unit that predicts the irradiation area of ​​radiation.

[0172] In the above example, since the object 1701 is closer to the irradiation area 1703 than the object 1702, it is determined that the object 1701 is the object to be imaged, and the process proceeds to step S1604. Whether the object 1701 is closer to the irradiation area 1703 is determined based on, for example, the distance between the center position of the irradiation area 1703 and the center of the detection position (detection frame) of the object.

[0173] (Step S1604) In step S1604, the object information determination unit 203 determines that the object 1701 closest to the position of the irradiation region 1703 in the optical image 1700 obtained in step S1603 is the object to be imaged.

[0174] Here, the display control unit 207 may highlight and display the subject 1701 determined to be an imaging target, in order to show the determination result of step S1604 to an operator (radiologist, etc.), as shown in Fig. 18. Fig. 18 shows an example of highlighting, in which a subject detection frame 1801 surrounding the subject 1701 is displayed.

[0175] (Step S1605) In step S1605, the consistency determination unit 204 determines whether the information of the subject to be imaged determined by the subject information determination unit 203 is consistent with the imaging order information of the radiographic image. If there is no consistency, as described above, an alert such as "Confirm: Imaged region" or "Confirm: Field of view position" is displayed on the display unit 235. If there is consistency, a message such as "Consistent" is displayed on the display unit 235. Note that if there is consistency, it is also possible not to display a message such as "Consistent" on the display unit 235. In other words, only if there is no consistency, a message indicating that there is no consistency may be displayed on the display unit 235.

[0176] A specific flow of step S1605 is shown in Fig. 16(b). In step S1606, the consistency determination unit 204 determines whether the imaging target and the imaging body part included in the imaging order information are consistent. More specifically, it determines whether the imaging target at the center position of image analysis and the imaging body part included in the imaging order information are consistent. If it is determined that there is a consistency, the consistency determination unit 204 selects whether to proceed to S1607, S1608, or end the process depending on the imaging body part.

[0177] For example, if the imaging region is a limb joint, the process proceeds to S1607 to determine the laterality (left or right), and the laterality is determined. In S1607, the consistency determination unit 204 determines whether the laterality of the imaging target is consistent with the laterality included in the imaging order information. Note that the laterality included in the imaging order information is an example of information related to imaging posture.

[0178] For example, if the imaging part is the chest, the process proceeds to S1608 to determine the direction (PA, AP, etc.). In S1608, the consistency determination unit 204 determines whether the direction of the imaging target is consistent with the direction included in the imaging order information. The direction included in the imaging order information is an example of information related to imaging posture.

[0179] For example, if the imaging region is the lumbar vertebrae, the process ends without determining the laterality or direction.

[0180] When the process of step S1605 is completed, the series of processes according to this embodiment ends.

[0181] As described above, the object information determination unit 203 can determine (determine) the imaging target even when multiple objects are captured in the optical image.

[0182] With the above configuration, the information processing device 200 according to this embodiment can determine (judge) the imaging target even when multiple subjects are captured in the optical image.

[0183] That is, the information processing device 200 according to this embodiment can appropriately determine whether the imaging position of the subject is appropriate even when a plurality of subjects are captured in the optical image.

[0184] Although the present embodiment describes a method for determining the imaging target based on the position of the irradiation area, the method for determining the position of the irradiation area is not limited to this. For example, the position of the irradiation area may be determined by image recognition of the radiation detector 130 reflected in the optical image. A neural network model that has been trained in advance may be used for image recognition of the radiation detector 130. If multiple radiation detectors are recognized, the radiation detector closest to the center of the optical image may be selected.

[0185] Furthermore, this embodiment may include an additional step of analyzing the subject's body movement. Body movement refers to, for example, a sudden movement caused by the subject sneezing. For example, after it is determined in S1605 that the subject's posture is consistent with the imaging order information, the process proceeds to a step of analyzing the subject's body movement. If body movement is detected in this step, an alert may be displayed to prompt the user to check the subject's posture, since there is a possibility that the subject's posture is not consistent with the imaging order information. If body movement is detected, for example, the flow of S1601 to S1605 is executed again. Furthermore, although this embodiment has been described with reference to a case where the subject is a human body, this embodiment may also be applicable to cases where the subject is not a human body. For example, this embodiment may be applicable to cases where the subject is baggage in baggage inspection using radiation. Alternatively, this embodiment may be applicable to cases where the subject is a product in product inspection using radiation (e.g., inspection of electrical circuit boards or piping). If the subject is baggage, the analysis unit may analyze, for example, whether the baggage contains any dangerous goods. When the object to be photographed is a product, the analysis unit analyzes, for example, whether there are defects in an electric circuit board or whether there are defects in a welded portion of a pipe.

[0186] Example 2 Second Embodiment Hereinafter, a radiation imaging system, an information processing apparatus, and an information processing method according to a second embodiment of the present disclosure will be described with reference to FIGS.

[0187] In this embodiment, when multiple objects are captured in an optical image acquired at a predetermined frame rate using the camera 140, a process of determining an object to be photographed by recognizing the characteristics of the objects will be described.

[0188] (Processing flow) A series of processing steps according to this embodiment will be described below with reference to Fig. 19. Fig. 19 is a flowchart showing the processing steps according to this embodiment. When the processing steps according to this embodiment start, the process proceeds to step S1901.

[0189] (Step S1901) In step S1901, the optical image acquisition unit 201 controls the camera 140 to acquire optical images of the radiation imaging site, including the subject of radiation imaging. In this embodiment, the camera 140 is a video camera attached to the radiation generator, and captures an image of the subject in an imaging position on the radiation detector 130, which is placed in an upright position, and outputs the optical images at a predetermined frame rate.

[0190] 20, an example will be described in which the radiation detector 130, an object 1701, and an object 1702 are reflected in an optical image 1700. Fig. 20 shows an example of an optical image according to this embodiment.

[0191] (Step S1902) In step S1902, the human detection unit 208 and the image analysis unit 209 use the optical image acquired by the optical image acquisition unit 201 as input data for the trained model to determine whether there is one or multiple subjects that can be the subject of a radiological image capture.

[0192] If it is determined in step S1902 that there are a plurality of subjects who can be the subject of radiographic imaging, the process proceeds to step S1903.

[0193] On the other hand, if it is determined in step S1902 that there is only one subject that can be the subject of radiographic imaging, the process proceeds to step S1906.

[0194] 20, the subject information determination unit 203 determines that there are two subjects that can be subjects for capturing radiographic images, and therefore the process proceeds to step S1903. Note that subjects 1701 and 1702 in Fig. 20 are examples of a first human body and a second human body.

[0195] (Step S1903) In step S1903, the object information determination unit 203 performs processing to recognize the features of each object in the optical image 1700.

[0196] Here, the process of recognizing the features of each object in the optical image 1700 is performed based on information for recognizing the features that is stored in advance in the storage unit 232 .

[0197] Information for recognizing features may include, for example, a facial image of the patient (subject), a facial image of the radiologist (photographer), information about a specific marker (wristband, etc.) attached to the patient, information about the features (color, shape, etc.) of the examination clothing, information about the features (color, shape, etc.) of the radiation protection clothing, etc.

[0198] In this embodiment, a facial image 2000 of the radiologist is stored in the storage unit 232 as information for recognizing features, and the facial image 2000 is used to perform the recognition process.

[0199] (Step S1904) In step S1904, the object information determination unit 203 determines whether the object has been recognized by the recognition process in step S1903.

[0200] If it is determined in step S1904 that the object has been recognized, the process proceeds to step S1905.

[0201] On the other hand, if it is determined in step S1904 that the object has not been recognized, the process proceeds to step S1903, where the recognition process is performed again.

[0202] In this embodiment, the object information determination unit 203 determines that the image 2000 of the radiological technician stored in the storage unit 232 has been recognized as the object 1702, and therefore the process proceeds to step S1905.

[0203] Here, the display control unit 207 may display a recognized detection frame 2001 surrounding the recognized face region of the subject 1702, as shown in FIG. 20, in order to show the operator the result of step S1904.

[0204] (Step S1905) In step S1905, the object information determination unit 203 determines the object to be imaged in the optical image 1700 based on the result recognized in step S1904.

[0205] In this embodiment, since the subject 1702 is determined to be a radiologist from the result of step S1904, the subject 1701 is determined to be the subject to be imaged. Note that in step S1903, a recognition process is performed using a face image of the subject 1701 (a face image of a patient), and when the subject 1701 is recognized in step S1904, the subject information determination unit 203 determines that the subject 1701 is the subject to be imaged.

[0206] (Step S1906) In step S1906, the consistency determining unit 204 determines whether the information on the subject to be imaged determined by the subject information determining unit 203 is consistent with the imaging order information for the radiographic images.

[0207] When the process of step S1906 is completed, the series of processes according to this embodiment ends.

[0208] As described above, the object information determination unit 203 can determine the imaging target even when multiple objects are captured in the optical image.

[0209] With the above configuration, the information processing device 200 according to this embodiment can appropriately determine whether the imaging position of the subject is appropriate even when multiple subjects are captured in the optical image.

[0210] Example 3 Hereinafter, a radiation imaging system, an information processing device, and an information processing method according to a third embodiment of the present disclosure will be described with reference to FIGS.

[0211] The radiation detector 130 of this embodiment has an automatic exposure control (AEC) function. The radiation detector 130 of this embodiment has multiple radiation detection areas (detection fields) on its detection surface. In radiography using the radiation detector 130 of this embodiment, when multiple subjects are captured in optical images acquired at a predetermined frame rate using the camera 140, a process for determining the object to be imaged based on the position of the detection field will be described. In radiography using automatic exposure control, the detection field to be used is selected from multiple detection fields. Figure 22 shows an example in which two detection fields 2203 are selected from nine detection fields. The selected detection fields 2203 can be considered to be the area to be imaged. In other words, the selected detection fields 2203 can be considered to be the irradiation area.

[0212] (Processing flow) A series of processing steps according to this embodiment will be described below with reference to FIG.

[0213] 21 is a flowchart showing the processing procedure according to this embodiment. When the processing procedure according to this embodiment starts, the process proceeds to step S2101.

[0214] (Step S2101) In step S2101, the optical image acquisition unit 201 controls the camera 140 to acquire an optical image of the radiation imaging site, including the subject of radiation imaging. In this embodiment, the camera 140 is a video camera attached to the radiation generator, which captures an image of the radiation detector 130 placed flat, and outputs an optical image at a predetermined frame rate.

[0215] Here, a case where a radiation detector 2201 and a radiation measurement field 2202 are reflected in an optical image 2200 will be described as an example with reference to Fig. 22. Fig. 22 shows an example of an optical image according to this embodiment.

[0216] (Step S2102) In step S2102, the subject information determination unit 203 acquires the position of the detection field to be used in automatic exposure control from the optical image acquired by the optical image acquisition unit 201 based on the information on the detection field to be used in automatic exposure control of the radiation detector 2201 included in the imaging order information.

[0217] Here, the display control unit 207 may highlight and display the position of the measurement field used in the automatic exposure control acquired in step S2102, as shown in Fig. 22. Fig. 22 shows an example in which the measurement field 2203 to be used is displayed by hatching.

[0218] (Step S2103) In step S2103, the human detection unit 208 and the image analysis unit 209 use the optical image acquired by the optical image acquisition unit 201 as input data for the trained model to determine whether there is one or multiple subjects that can be the subject of a radiological image capture.

[0219] 23, an example will be described in which a radiation detector 2201, a measurement field 2202, a subject 2301, and a subject 2302 are reflected in an optical image 2200. Fig. 23 shows an example of an optical image according to this embodiment.

[0220] If it is determined in step S2103 that there are a plurality of subjects who can be the subject of radiographic imaging, the process proceeds to step S2104.

[0221] On the other hand, if it is determined in step S2103 that there is only one subject that can be the subject of radiographic imaging, the process proceeds to step S2105.

[0222] In this embodiment, as shown in FIG. 23, the subject information determination unit 203 determines that there are two subjects 2301 and 2302 that can be subjects for capturing radiation images, and therefore the process proceeds to step S2104.

[0223] (Step S2104) The subject information determination unit 203 determines the subject that is closest to the position of the measurement field 2203 used in automatic exposure control, acquired in step S2102, as the subject to be imaged.

[0224] In this embodiment, as shown in FIG. 23, the object information determination unit 203 determines that the object 2301 closest to the measurement field 2203 used in automatic exposure control is the imaging target.

[0225] (Step S2105) In step S2105, the consistency determining unit 204 determines whether the information on the subject to be imaged determined by the subject information determining unit 203 is consistent with the imaging order information for the radiographic images.

[0226] When the process of step S2105 is completed, the series of processes ends.

[0227] As described above, the object information determination unit 203 can determine the imaging target even when multiple objects are captured in the optical image.

[0228] With the above configuration, the information processing device 200 according to this embodiment can appropriately determine whether the imaging position of the subject is appropriate even when multiple subjects are captured in the optical image.

[0229] Example 4 Hereinafter, a radiation imaging system, an information processing device, and an information processing method according to a fourth embodiment of the present disclosure will be described with reference to FIG.

[0230] In this embodiment, the information processing device 200 acquires optical images at a predetermined frame rate using the camera 140. Next, the information processing device 200 detects the object from the optical images. During the detection, the information processing device 200 determines the imaging target by comparing the current frame with a previous frame before the current frame. A specific processing flow performed by the information processing device 200 will be described with reference to FIG. 24 . The current frame is an example of a first optical image. The previous frame is an example of a second optical image.

[0231] (Step S2401) In step S2401, the optical image acquisition unit 201 controls the camera 140 to acquire an optical image of the radiation imaging site, including the subject of radiation imaging. The acquired optical image will be referred to as the current frame in the following description. In this embodiment, the camera 140 is a video camera attached to a radiation generator, and outputs optical images at a predetermined frame rate.

[0232] (Step S2402) In step S2402, the image analysis unit 209 checks frames older than the current frame stored in the storage unit 232. Specifically, the image analysis unit 209 acquires object information of the past frame and checks whether an object has been detected. If an object has been detected, the process proceeds to S2403. On the other hand, if an object has not been detected in S2402, the process proceeds to S2405.

[0233] (Step S2403) In step S2403, the image analysis unit 209 acquires the detection position (detection frame) of the subject in the past frame image stored in the storage unit 232. Next, the image analysis unit 209 analyzes the difference between the current frame image and the past frame image, and determines whether or not there is any movement of the subject between the frames.

[0234] Specifically, if there is a detection position (detection frame) where the subject was detected in a past frame, the image analysis unit 209 sets that detection position (detection frame) as the detection position (detection frame) in the current frame. Then, the image analysis unit 209 determines movement by comparing the average brightness and variance of the area of ​​the detection frame set in the current frame with the average brightness and variance of the area of ​​the detection frame in the past frame. Note that the detection position (detection frame) in the current frame is an example of a first area. Also, the detection position (detection frame) in the past frame is an example of a second area.

[0235] Note that edge detection or optical flow analysis may be used for analyzing the difference. In edge detection, edges can be enhanced using a Sobel filter or the Canny algorithm, making the movement boundary clearer. Here, the movement boundary is not limited to the movement boundary of the subject. For example, in a configuration in which the camera 140 is attached to the radiation generator 120, when the radiation generator 120 is moved by a radiologist, the background and the platform also move in addition to the subject. In other words, the movement boundary includes the movement of the background and the platform. In optical flow analysis, the movement of any point in the image can be calculated as a vector, making it possible to obtain more precise movement information.

[0236] Here, a comparison method using average brightness and variance will be described. The image analysis unit 209 compares the brightness value of each pixel within the subject's detection frame in the current frame with the brightness value of each pixel within the subject's detection frame in the previous frame, and calculates the difference. At this time, the presence or absence of overall movement within the detection frame is evaluated by calculating the sum or average of the differences in brightness values ​​of each pixel within the detection frame. In addition, the variance of the brightness values ​​of each pixel within the detection frame is calculated, and the changes are compared to determine whether there is local brightness fluctuation within the detection frame. This makes it possible to detect movement while suppressing the effects of minute movements and noise.

[0237] Furthermore, instead of comparing (taking the difference) each pixel, the image analysis unit 209 may divide the area within the detection frame of the subject into multiple blocks (divided areas) and calculate the average brightness for each block (each divided area). In the block division method (block division), the image analysis unit 209 compares the difference between the average brightness of each block in the current frame and the average brightness of each block in the past frame. Next, the presence or absence of movement is evaluated by calculating the percentage of blocks having a difference exceeding a certain threshold as a result of the comparison.

[0238] This method makes it possible to detect localized movements and changes in specific regions. Compared to pixel-by-pixel subtraction, block-based analysis has the advantage of being more robust against noise and minute variations between frames. For example, block-based analysis can detect even partial movement of the subject as a change in brightness within that block, improving the accuracy of motion determination. Furthermore, block-based analysis can also detect variations in brightness within a block by evaluating the brightness variance within each block, thereby enabling detailed understanding of the characteristics of movement. For example, a large brightness variance can be determined to indicate a high probability of motion occurring within that block. Conversely, a small brightness variance can be determined to indicate a high probability of no motion occurring within the block. This method is effective, for example, for detecting minute movements of the subject.

[0239] The image analysis unit 209 may detect motion using not only temporal changes but also spatial frequency information. Specifically, phase-only correlation (POC) in frequency space can be used to accurately determine motion between frames. POC is a method of extracting phase information between frames using Fourier transform and calculating the correlation to detect the magnitude and direction of motion. Techniques using POC can effectively detect motion that is difficult to detect using conventional differential comparison in the spatial domain, such as small motion or partially directional motion. Furthermore, because POC can accurately detect even minute changes in the position of an object, it may be effective in detecting subtle motion of the subject or motion relative to the background.

[0240] As described above, a combination of multiple methods can be used to determine whether or not there is motion between frames. Furthermore, the most recently analyzed frame (the frame one frame before the current frame) can be used as the past frame, but as long as no motion is detected in the most recently analyzed frame, a frame two or more frames before the current frame can also be used. In other words, the past frame is not necessarily limited to the immediately previous frame. This allows the presence or absence of motion to be evaluated over a broader range (with a wider temporal range).

[0241] Although the example of calculating the difference between the subject detection frame of the current frame and the subject detection frame of the previous frame has been described above, the difference may be calculated over the entire frame. The difference may also be calculated over a predetermined area (e.g., an irradiation area predicted based on the imaging order). As described in the description of step S1603, the irradiation area is predicted based on information about the imaging distance included in the imaging order information. The image analysis unit 209 may also calculate the difference between the irradiation area of ​​the previous frame and the irradiation area of ​​the current frame corresponding to the irradiation area of ​​the previous frame. This makes it possible to analyze overall movement, including movement other than that of the subject (e.g., movement indicating a change in the background).

[0242] (Step S2404) If the difference between the frames analyzed in step S2403 is equal to or less than a preset threshold, the process proceeds to step S2406. On the other hand, if the difference between the frames is greater than the threshold, the process proceeds to step S2405.

[0243] In step S2402, the image analysis unit 209 may acquire the reliability of the detection frame of the subject instead of the subject information in the past frame. In this case, if the reliability is low, the process proceeds from step S2402 to step S2405.

[0244] (Step S2405) In step S2405, the human detection unit 208 detects an object in the current frame. Human detection is the same as in the other embodiments described above, and a detailed description thereof will be omitted.

[0245] (Step S2406) In step S2406, the image analysis unit 209 uses the subject detection position (detection frame) in the previous frame stored in the storage unit 232 as the detection frame for the current frame. That is, the image analysis unit 209 outputs the detection frame (analysis result) of the previous frame as the detection frame (analysis result) for the current frame. This reduces the processing load for detecting people for each frame. Furthermore, depending on the lighting environment in the imaging room, the contrast between the background and the subject may decrease. In conventional methods, when the contrast between the background and the subject decreases, the recognition of the subject itself becomes unstable, resulting in unstable display of the person detection frame. An unstable display refers, for example, to the fact that the person detection frame is sometimes displayed and sometimes not, or the position of the person detection frame changes in a short period of time. In the method disclosed herein, when the difference between the current frame and the previous frame is small, the detection frame of the previous frame is used, thereby stabilizing the display of the person detection frame.

[0246] (Step S2407) In step S2407, the consistency determination unit 204 determines the photographing position. The operation in step S2407 is the same as in other embodiments, so a detailed description thereof will be omitted. Note that if it is determined in S2404 that there is no movement (the difference between the current frame and the previous frame is small), the determination result of the previous frame image stored in the storage unit 232 may also be used to determine the photographing position.

[0247] (Configuration 1) an acquisition unit that acquires radiography order information and optical images related to radiography; a prediction unit that predicts an irradiation area of ​​radiation using the radiography order information and the optical image; a detection unit that detects a human body part in the optical image; When a plurality of human body parts are detected in the optical image, an analysis unit that uses the optical image to analyze a human body part that is closest to the irradiation area; An information processing device comprising:

[0248] (Configuration 2) the photographing order information includes information regarding a photographing distance, 2. The information processing device according to configuration 1, wherein the prediction unit predicts the illumination area using information related to the shooting distance.

[0249] (Configuration 3) 3. The information processing device according to configuration 1 or 2, wherein the information about the imaging distance is a position type (application of use of the radiation detection device).

[0250] (Configuration 4) the radiography order information includes information about the radiography region, 4. The information processing device according to any one of configurations 1 to 3, wherein the analysis unit analyzes whether a body part closest to the irradiation region is consistent with information about the imaging region.

[0251] (Configuration 5) the photographing order information includes information regarding the photographing posture, 5. The information processing device according to any one of configurations 1 to 4, wherein the analysis unit analyzes whether the posture of the human body part closest to the irradiation region is consistent with information about the imaging posture.

[0252] (Configuration 6) the photographing order information includes information about a light field used in automatic exposure control; 6. The information processing device according to configuration 5, wherein the analysis unit analyzes whether the posture of the human body part closest to the measurement field is consistent with information about the imaging posture.

[0253] (Configuration 7) When a plurality of human body parts are detected in the optical image, 7. The information processing device according to any one of configurations 1 to 6, further comprising a display control unit that displays the human body part closest to the irradiation area in an emphasized manner.

[0254] (Configuration 8) an acquisition unit for acquiring optical images related to radiography; a detection unit that detects a human body in the optical image; When a first human body and a second human body are detected in the optical image, a recognition process is performed on the first human body and the second human body; When one of the first human body and the second human body is recognized as a photographer, an analysis of the other human body is performed using the optical image; an analysis unit that, when one of the first human body and the second human body is recognized as a subject, analyzes the one human body using the optical image; An information processing device comprising:

[0255] (Configuration 9) an acquisition unit that acquires radiography order information and optical images related to radiography; a prediction unit that predicts an irradiation area of ​​radiation using the radiography order information and the optical image; a detection unit that detects a subject in the optical image; When a plurality of subjects are detected in the optical image, an analysis unit that analyzes a subject that is closest to the illumination area using the optical image; An information processing device comprising:

[0256] (Configuration 10) a radiation detection device for detecting radiation; 10. A radiation imaging system comprising: an information processing device according to any one of configurations 1 to 9, which is communicably connected to the radiation detection device.

[0257] (Method 1) an acquisition step of acquiring radiography order information and an optical image; a prediction step of predicting an irradiation area of ​​radiation using the radiography order information and the optical image; a detecting step of detecting a human body part in the optical image; When a plurality of human body parts are detected in the optical image, an analyzing step of analyzing a body part closest to the irradiation area using the optical image; An information processing method comprising:

[0258] (Method 2) an acquisition step of acquiring an optical image for radiography; a detecting step of detecting a human body in the optical image; When a first human body and a second human body are detected in the optical image, a recognition process is performed on the first human body and the second human body; When one of the first human body and the second human body is recognized as a photographer, an analysis of the other human body is performed using the optical image; an analyzing step of analyzing one of the first human body and the second human body using the optical image when one of the first human body and the second human body is recognized as a subject; An information processing method comprising:

[0259] (Configuration 11) an acquisition unit for acquiring optical images related to radiography; a detection unit that detects an area including a human body part in the optical image; an analysis unit that outputs an analysis result of the region detected by the detection unit; An information processing device comprising: The analysis unit When a difference between a first optical image acquired by the acquisition unit and a second optical image acquired before the first optical image is large, the information processing device outputs an analysis result of a first region detected in the first optical image as an analysis result of the first optical image, and when the difference is small, outputs an analysis result of a second region detected in the second optical image as an analysis result of the first optical image.

[0260] (Configuration 12) 12. The information processing device according to configuration 11, wherein the difference is a difference between the second region in the second optical image and a region in the first optical image corresponding to the second region.

[0261] (Configuration 13) The information processing device according to configuration 11 or 12, wherein the difference is the difference between the average luminance or variance of the second region in the second optical image and the average luminance or variance of a region corresponding to the second region in the first optical image.

[0262] (Configuration 14) An information processing device according to any one of configurations 11 to 13, wherein the difference is the difference between the average brightness or variance of each divided region obtained by dividing the second region in the second optical image and the average brightness or variance of a divided region corresponding to each of the regions in the first optical image.

[0263] (Configuration 15) The information processing device according to any one of configurations 11 to 14, wherein the difference is a difference between a second illumination area in the second optical image, identified based on shooting order information, and an area in the first optical image corresponding to the second illumination area.

[0264] (Method 3) an acquisition step of acquiring an optical image for radiography; a detection step of detecting an area including a human body part in the optical image; an analysis step of outputting an analysis result of the region detected by the detection unit; An information processing device comprising: The analyzing step An information processing method in which, if there is a large difference between a first optical image acquired by the acquisition step and a second optical image acquired before the first optical image, an analysis result of a first region detected in the first optical image is output as an analysis result of the first optical image, and, if the difference is small, an analysis result of a second region detected in the second optical image is output as an analysis result of the first optical image.

[0265] (Program 1) A program that causes a computer to execute the information processing method according to any one of Methods 1 to 3.

[0266] (Other Examples) The present disclosure can also be realized by providing a program that implements one or more functions of the above-described embodiments and examples to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) that implements one or more functions. A computer may have one or more processors or circuits, and may include multiple separate computers or a network of multiple separate processors or circuits to read and execute computer-executable instructions.

[0267] The processor or circuitry may include a central processing unit (CPU), a microprocessing unit (MPU), a graphics processing unit (GPU), an application specific integrated circuit (ASIC), or a field programmable gateway (FPGA). The processor or circuitry may also include a digital signal processor (DSP), a data flow processor (DFP), or a neural processing unit (NPU).

[0268] Although the present invention has been described above with reference to embodiments and examples, the present invention is not limited to the above embodiments and examples. The present invention also includes inventions that have been modified within the scope of the present invention and inventions equivalent to the present invention. Furthermore, the above-described embodiments and examples can be combined as appropriate within the scope of the present invention. [Explanation of symbols]

[0269] 200 Information processing device 201 Optical image acquisition unit 202 Skeleton Estimation Department 203 Subject information determination section 208 Human detection unit 209 Image Analysis Unit

Claims

1. an acquisition unit that acquires radiography order information and optical images related to radiography; a prediction unit that predicts an irradiation area of ​​radiation using the radiography order information and the optical image; a detection unit that detects a human body part in the optical image; When a plurality of human body parts are detected in the optical image, an analysis unit that uses the optical image to analyze a human body part that is closest to the irradiation area; An information processing device comprising:

2. the photographing order information includes information regarding a photographing distance, The information processing apparatus according to claim 1 , wherein the prediction unit predicts the illumination area using information about the shooting distance.

3. The information processing apparatus according to claim 1 , wherein the information about the photographing distance is a position type (application of use of the radiation detection apparatus).

4. the radiography order information includes information about the radiography region, The information processing apparatus according to claim 1 , wherein the analysis unit analyzes whether the body part closest to the irradiation region is consistent with information about the imaging region.

5. the photographing order information includes information regarding the photographing posture, The information processing apparatus according to claim 1 , wherein the analysis unit analyzes whether the posture of the human body part closest to the irradiation region is consistent with information about the imaging posture.

6. the photographing order information includes information about a light field used in automatic exposure control; The information processing apparatus according to claim 5 , wherein the analysis unit analyzes whether the posture of the human body part closest to the measurement field is consistent with information about the imaging posture.

7. When a plurality of human body parts are detected in the optical image, The information processing device according to claim 1 , further comprising a display control unit that displays the human body part closest to the irradiation area in an emphasized manner.

8. an acquisition unit for acquiring optical images related to radiography; a detection unit that detects a human body in the optical image; When a first human body and a second human body are detected in the optical image, a recognition process is performed on the first human body and the second human body; When one of the first human body and the second human body is recognized as a photographer, the other human body is analyzed using the optical image; an analysis unit that, when one of the first human body and the second human body is recognized as a subject, analyzes the one human body using the optical image; An information processing device comprising:

9. an acquisition unit that acquires radiography order information and optical images related to radiography; a prediction unit that predicts an irradiation area of ​​radiation using the radiography order information and the optical image; a detection unit that detects a subject in the optical image; When a plurality of subjects are detected in the optical image, an analysis unit that analyzes a subject that is closest to the illumination area using the optical image; An information processing device comprising:

10. a radiation detection device for detecting radiation; A radiation imaging system comprising: the information processing device according to claim 1 , which is communicably connected to the radiation detection device.

11. an acquisition step of acquiring radiography order information and an optical image; a prediction step of predicting an irradiation area of ​​radiation using the radiography order information and the optical image; a detecting step of detecting a human body part in the optical image; When a plurality of human body parts are detected in the optical image, an analyzing step of analyzing a body part closest to the irradiation area using the optical image; An information processing method comprising:

12. an acquisition step of acquiring an optical image for radiography; a detecting step of detecting a human body in the optical image; When a first human body and a second human body are detected in the optical image, a recognition process is performed on the first human body and the second human body; When one of the first human body and the second human body is recognized as a photographer, the other human body is analyzed using the optical image; an analyzing step of analyzing one of the first human body and the second human body using the optical image when one of the first human body and the second human body is recognized as a subject; An information processing method comprising:

13. an acquisition unit for acquiring optical images related to radiography; a detection unit that detects an area including a human body part in the optical image; an analysis unit that outputs an analysis result of the region detected by the detection unit; An information processing device comprising: The analysis unit An information processing device that, when a difference between a first optical image acquired by the acquisition unit and a second optical image acquired before the first optical image is large, outputs an analysis result of a first region detected in the first optical image as an analysis result of the first optical image, and, when the difference is small, outputs an analysis result of a second region detected in the second optical image as an analysis result of the first optical image.

14. The information processing apparatus according to claim 13 , wherein the difference is a difference between the second region in the second optical image and a region in the first optical image corresponding to the second region.

15. 14. The information processing device according to claim 13, wherein the difference is the difference between the average brightness or variance of the second region in the second optical image and the average brightness or variance of a region corresponding to the second region in the first optical image.

16. 14. The information processing device of claim 13, wherein the difference is the difference between the average brightness or variance of each divided region obtained by dividing the second region in the second optical image and the average brightness or variance of the divided region corresponding to each of the regions in the first optical image.

17. 14. The information processing device according to claim 13, wherein the difference is a difference between a second illumination area in the second optical image, which is specified based on photographing order information, and an area in the first optical image corresponding to the second illumination area.

18. an acquisition step of acquiring an optical image for radiography; a detection step of detecting an area including a human body part in the optical image; an analysis step of outputting an analysis result of the region detected by the detection unit; An information processing device comprising: The analyzing step An information processing method in which, if there is a large difference between a first optical image acquired by the acquisition process and a second optical image acquired before the first optical image, an analysis result of a first region detected in the first optical image is output as an analysis result of the first optical image, and, if the difference is small, an analysis result of a second region detected in the second optical image is output as an analysis result of the first optical image.

19. A program that causes a computer to execute the information processing method according to any one of claims 11, 12, and 18.

Images