Information processing device, medical image imaging device, information processing method, and information processing program

Abstract

This console comprises a generation unit that, for a learning model trained in advance using, as input information, examination information relating to specifics of examinations performed on subjects and camera images that are obtained by imaging the subjects using a camera 7 and include depth information, and using, as output information, organ position information that indicates the positions of an organ of the subjects in the camera images, inputs a camera image and examination information relating to an actual subject to the learning model to generate organ position information corresponding to the actual subject.

Description

Information processing device, medical image capturing device, information processing method, and information processing program

[0001] The present disclosure relates to an information processing device, a medical imaging device, an information processing method, and an information processing program.

[0002] 2. Description of the Related Art In recent years, advances in medical equipment such as CT (Computed Tomography) devices and MRI (Magnetic Resonance Imaging) devices have led to the use of higher quality, higher resolution three-dimensional images for diagnostic imaging.

[0003] When imaging a subject using an imaging device such as a CT device or MRI device, a scanogram image (hereinafter referred to as "scanogram imaging") is captured prior to the actual imaging to obtain a three-dimensional image in order to determine the imaging range, and a two-dimensional positioning image (scanogram image) is obtained. The operator of the imaging device (technologist, etc.) sets the imaging range for the actual imaging while viewing the scanogram image. Note that scanogram images are also called scout images, topograms, etc., but will be referred to as "scanogram images" or "positioning images" below.

[0004] However, since imaging to obtain a scanogram image is performed using radiation, the subject is exposed to radiation. Therefore, a conventional technique that can be applied to set the imaging range during actual imaging without exposing the subject to radiation is the method disclosed in Japanese Patent Application Laid-Open No. 2019-080909.

[0005] The method for generating a positioning image includes determining an X-ray source / patient (S / P) relative position of a current X-ray imaging system geometry, determining a detector / patient (D / P) relative position with respect to the current X-ray imaging system geometry, determining a projection area on a detector plane of a detector based on the S / P relative position and the D / P relative position, registering an atlas image to a patient based on the projection area on the detector plane, and displaying a positioning image corresponding to the atlas image on a representation image of the patient, wherein the atlas image is obtained from an anatomical atlas library generated using machine learning.

[0006] The purpose of obtaining a positioning image is to allow a user, such as a technician or a doctor, to accurately set the imaging range during actual imaging so that the organ to be examined is reliably included. Therefore, it is extremely important that the positioning image is capable of accurately determining the position of the organ to be examined in an actual subject (hereinafter also referred to as the "real subject"). However, while the technology disclosed in JP 2019-080909 A can obtain a positioning image without performing radiography, it has a problem in that it does not necessarily allow the position of the organ in the real subject to be accurately determined.

[0007] The present disclosure has been made in consideration of the above points, and aims to provide an information processing device, a medical image capturing device, an information processing method, and an information processing program that are capable of accurately grasping the position of an organ in a real subject compared to conventional techniques.

[0008] An information processing device according to a first aspect of the present disclosure includes at least one processor, and the processor generates organ position information corresponding to an actual subject by inputting the camera image and examination information relating to the actual subject into a learning model that has been trained in advance using, as input information, a camera image obtained by photographing the subject with a camera, the camera image including depth information, and examination information relating to the examination content of the subject, and as output information, organ position information that is information indicating the position of the subject's organs in the camera image.

[0009] An information processing device according to a second aspect of the present disclosure is an information processing device according to the first aspect, in which the organ position information used in learning the learning model is information in which the size of the organ area is adjusted according to the body thickness of the subject indicated by the depth information.

[0010] An information processing device according to a third aspect of the present disclosure is an information processing device according to the first aspect, in which a processor adjusts the size of the organ area indicated by the organ position information generated by the learning model according to the body thickness of the actual subject indicated by the depth information included in the camera image of the actual subject.

[0011] An information processing device according to a fourth aspect of the present disclosure is an information processing device according to the first or second aspect, in which, when the table on which the actual subject is standing is moved to a position where the actual imaging of the actual subject is to be performed, the processor further performs processing to display, on the display unit, an organ position image indicating the position of the organ indicated by the organ position information, superimposed on the camera image using a predetermined object included in the camera image as a reference.

[0012] An information processing device according to a fifth aspect of the present disclosure is the information processing device according to the fourth aspect, wherein the predetermined object is at least one of a human body feature point of the actual subject and a table.

[0013] An information processing device according to a sixth aspect of the present disclosure is an information processing device according to the first or second aspect, in which the processor further performs processing to project an organ position image indicating the position of the organ indicated by the organ position information onto the actual subject.

[0014] An information processing device according to a seventh aspect of the present disclosure is an information processing device according to the first or second aspect, in which a processor changes and applies at least one of the position and size of an organ indicated by organ position information corresponding to an actual subject according to the posture of the actual subject.

[0015] An information processing device according to an eighth aspect of the present disclosure is an information processing device according to the first or second aspect, in which organ position information used in learning a learning model, at least one of the position and size of the organ indicated by the organ position information, is changed according to the posture of the subject.

[0016] An information processing device according to a ninth aspect of the present disclosure is an information processing device according to the first or second aspect, in which a processor pre-learns a learning model using camera images and examination information relating to a subject as input information and organ position information corresponding to the subject as output information.

[0017] A medical imaging device according to a tenth aspect of the present disclosure includes an information processing device of the present disclosure and a radiological imaging device that performs imaging of an actual subject using organ position information generated by the information processing device.

[0018] An information processing method according to an eleventh aspect of the present disclosure includes generating organ position information corresponding to an actual subject by inputting camera images and examination information relating to the actual subject into a learning model that has been trained in advance using, as input information, camera images obtained by photographing the subject with a camera, the camera images including depth information, and examination information relating to the examination content of the subject, and as output information, organ position information that is information indicating the positions of the subject's organs in the camera images.

[0019] An information processing program according to a twelfth aspect of the present disclosure is an information processing program for causing a computer to execute processing including generating organ position information corresponding to an actual subject by inputting camera images and examination information relating to an actual subject into a learning model that has been trained in advance using, as input information, camera images obtained by photographing a subject with a camera, the camera images including depth information and examination information relating to the examination content of the subject, and as output information, organ position information that is information indicating the positions of the subject's organs in the camera images.

[0020] According to the present disclosure, the positions of organs in a real subject can be grasped with higher accuracy than with conventional techniques.

[0021] FIG. 1 is a diagram showing a schematic configuration of a medical diagnosis apparatus according to an embodiment of the disclosed technology. FIG. 2 is a block diagram showing an example of a functional configuration of a console when executing a learning process according to an embodiment of the disclosed technology. FIG. 3 is a block diagram showing an example of a functional configuration of a console when executing a control process according to an embodiment of the disclosed technology. FIG. 4 is a schematic diagram showing an example of a configuration of a learning information database according to an embodiment of the disclosed technology. FIG. 5 is a schematic diagram provided for explaining a method for creating organ position information in learning information according to an embodiment of the disclosed technology. FIG. 6 is a schematic diagram showing an example of a configuration of an organ enlargement / reduction information database according to an embodiment of the disclosed technology. FIG. 7 is a flowchart showing an example of a flow of a learning process according to an embodiment of the disclosed technology. FIG. 8 is a flowchart showing an example of a flow of a control process according to an embodiment of the disclosed technology. FIG. 9 is a schematic diagram provided for explaining an example of an overall flow of learning processing and information processing by a console according to an embodiment of the disclosed technology. FIG. 11 is a flowchart showing another example of a flow of a control process according to an embodiment of the disclosed technology. FIG. 12 is a diagram provided for explaining the flow of the control process shown in FIG. 11 according to an embodiment of the disclosed technology. FIG. 13 is a flowchart showing another example of a flow of a control process according to an embodiment of the disclosed technology. FIG. 14 is a side view and a plan view of an imaging room provided for explaining the flow of the control process shown in FIG. 13 according to an embodiment of the disclosed technology. 15 is a flowchart illustrating another example of the flow of the control process according to the embodiment of the disclosed technology. FIG. 16 is a diagram illustrating an example of human body feature points and depth information provided for explaining the flow of the control process shown in FIG. 15 according to the embodiment of the disclosed technology.

[0022] Hereinafter, examples of embodiments of the present disclosure will be described with reference to the drawings. Note that the same reference numerals are used to designate the same or equivalent components and parts in each drawing. Furthermore, the dimensional proportions of the drawings are exaggerated for the sake of explanation and may differ from the actual proportions.

[0023] [First Embodiment] Fig. 1 is a diagram showing a schematic configuration of a medical diagnostic apparatus including an information processing apparatus and a medical imaging apparatus according to this embodiment. In this embodiment, the configuration of one embodiment of an X-ray CT (Computed Tomography) apparatus is shown as the medical diagnostic apparatus. Note that the medical diagnostic apparatus of the present disclosure is not limited to an X-ray CT apparatus, and can also be applied to other medical diagnostic apparatuses such as an MRI (Magnetic Resonance Imaging) apparatus.

[0024] The medical diagnostic apparatus according to this embodiment includes a scanner 1, a bed 3, and a console 4. The console 4 corresponds to the information processing apparatus of the present disclosure, the scanner 1 corresponds to the radiographic imaging apparatus of the present disclosure, and the combination of the scanner 1, the bed 3, and the console 4 corresponds to the medical imaging apparatus of the present disclosure. The bed 3 corresponds to the table of the present disclosure.

[0025] The scanner 1 is a part that performs CT scans. The scanner 1 includes a gantry 11, a rotating plate 12 that has an opening in its center and is rotatably supported by the gantry 11, an X-ray tube assembly 13 fixed to the rotating plate 12, a collimator 14 provided at an X-ray emission port of the X-ray tube assembly 13, an X-ray detector 15 disposed opposite the X-ray tube assembly 13 across the opening of the rotating plate 12, and a rotating plate drive unit 17 provided on the gantry 11. The rotating plate 12 of the scanner 1 also includes a collimator control unit 18 that controls the collimator 14 to change the X-ray irradiation field, a rotating plate drive control unit 19 that controls the drive of the rotating plate drive unit 17, an X-ray high-voltage generator 20 that supplies power to the X-ray tube assembly 13 for generating X-rays and controls the X-ray generation conditions, a data acquisition unit 16 that acquires the output of the X-ray detector 15, and a data transmission unit 21 that transmits the data acquired by the data acquisition unit 16. The supply of power and control signals to each unit provided on the rotating plate 12, and the extraction of data from each unit provided on the rotating plate 12, are carried out via a slip ring (not shown) provided between the base 11 and the rotating plate 12.

[0026] The bed 3 moves the subject between an imaging preparation position and an imaging position. The subject is placed on a tabletop 31. The bed 3 has a vertical movement mechanism and a front-to-back movement mechanism for the tabletop 31, which are not shown in the figure. The bed 3 is provided with a bed controller 32, a tabletop vertical movement controller 33, and a tabletop front-to-back movement controller 34 to control the operation of the vertical movement mechanism and the front-to-back movement mechanism of the tabletop 31.

[0027] The console 4 controls the medical diagnostic device, which is an X-ray CT device. The console 4 is an example of a computer of the present disclosure. The console 4 includes a central processing unit (CPU) 41, a read-only memory (ROM) 42, a random access memory (RAM) 43, and a storage 44. The console 4 also includes an input unit 45 and a display unit (display) 46.

[0028] The CPU 41, an example of a processor, is a central processing unit that executes various programs and controls each component. That is, the CPU 41 reads a program from the ROM 42 or storage 44 and executes the program using the RAM 43 as a work area. The CPU 41 controls the above-described components and performs various arithmetic processing in accordance with the program recorded in the ROM 42 or storage 44. In this embodiment, the ROM 42 or storage 44 stores a learning program that executes a learning process for learning a learning model 47 (see also FIG. 2 ), details of which will be described later, and a control program for the medical diagnostic apparatus (hereinafter simply referred to as the "control program") that includes information processing for determining the imaging range for actual imaging using the X-ray CT scanner. The program related to this information processing corresponds to the information processing program disclosed herein, and the processing method by this information processing corresponds to the information processing method disclosed herein.

[0029] The ROM 42 stores various programs and various data. The RAM 43 temporarily stores programs or data as a working area. The storage 44 is configured with a storage device such as an HDD (Hard Disk Drive), an SSD (Solid State Drive), or a flash memory, and stores various programs including the operating system and various data.

[0030] The input unit 45 includes a pointing device such as a mouse and a keyboard, and is used to input various information. The input unit 45 includes a bed operating unit that operates the height of the bed 3.

[0031] The display unit 46 is, for example, a liquid crystal display, and displays various information. The display unit 46 may be a touch panel type and function as the input unit 45.

[0032] In this embodiment, the console 4 is connected to a hospital information system 5 of a hospital in which an X-ray CT apparatus is installed. The hospital information system 5 includes a patient information management system 51 and an examination reservation system 52.

[0033] The patient information management system 51 is a database of personal information, medical data, examination image data, test data, medication data, etc. of patients who have previously received medical treatment at the hospital. The patient's personal information includes personal identification information such as name, ID (Identification), date of birth, age, and gender, as well as physical information such as height, weight, and body fat percentage, and the dates of previous examinations. This personal information is entered into the system when a subject visits the hospital as a patient and receives medical treatment. The patient's height and weight are collected by the patient's self-reporting in a medical questionnaire or by actual measurement, and then entered into the patient information management system 51. The medical data, examination image data, test data, and medication data are entered into the patient information management system 51 each time a patient is treated, and an in-hospital patient database is created.

[0034] On the other hand, the examination data includes information related to the examination of the corresponding patient, such as the region to be examined, the organ to be examined, and the scan protocol to be applied in the examination. Here, the region to be examined is information indicating a region such as the head, chest, abdomen, thoracic-abdominal region, etc., and the organ to be examined is information indicating the organ itself such as the lung field, stomach, etc. Furthermore, the scan protocol is information indicating setting conditions to be applied in the actual imaging, such as the imaging mode to be applied in the examination, the mAs value to be set in the X-ray tube device 13, the initial setting of the imaging range, etc. This examination data corresponds to the examination information in the present disclosure, and will be referred to hereinafter as "examination information."

[0035] The examination reservation system 52 creates an examination schedule for each day for each system of medical diagnostic equipment installed in the hospital based on the information stored or written in the patient information management system 51, and distributes the created examination schedule as data to each system. The examination schedule includes the order in which patients will be examined that day by the medical diagnostic equipment, as well as the personal identification information and physical information of each patient. The contents of the examination reservation system 52 are updated sequentially in accordance with the progress of medical treatment that day, and the updated data is sent to each system each time an update is made.

[0036] A camera 7 is mounted on the ceiling of an imaging room equipped with a medical diagnostic device, so as to be able to capture an image of an area within the movable range of the bed 3. The camera 7 according to this embodiment is a depth camera that can obtain depth information indicating the depth of each pixel in addition to an optical image within the imaging angle of view. Note that the camera 7 according to this embodiment uses a ToF (Time of Flight) sensor as the sensor for obtaining the depth information, but is not limited to this. For example, a stereo camera system may be used to obtain the depth information.

[0037] The camera 7 can detect the body thickness (hereinafter simply referred to as "body thickness") of a subject in a supine position on the top plate 31 of the bed 3. In this embodiment, the camera 7 acquires depth information (hereinafter referred to as "initial depth information") of the upper surface of the top plate 31 of the bed 3 in a state where the subject is not present and the top plate 31 of the bed 3 is positioned in the vertical direction relative to the subject during actual imaging. Then, in this embodiment, in order to actually perform actual imaging, the camera 7 acquires depth information of the upper surface of the top plate 31 (hereinafter referred to as "depth information at the time of imaging") when the subject is in a supine position on the top plate 31 of the bed 3, and subtracts the acquired depth information at the time of imaging from the initial depth information for each corresponding pixel to acquire information indicating the body thickness for each pixel (hereinafter referred to as "body thickness information").

[0038] As described above, in the medical diagnostic device according to this embodiment, a depth camera is used to acquire body thickness information, and the body thickness information is estimated using the depth camera image captured by the depth camera. However, this is not limited to this. For example, an optical camera may be used as the camera 7, and the body thickness information may be estimated by analyzing the subject image contained in the optical camera image captured by the optical camera. In this case, an example is provided in which the camera 7 is installed diagonally above or to the side of the bed 3, and the body thickness information of the subject is acquired from the optical camera image captured by the camera 7. In this configuration, if the camera 7 is installed diagonally above the bed 3, the body thickness indicated by the optical camera image captured by the camera 7 changes depending on the imaging angle of the camera 7 relative to the subject. Therefore, it is preferable to install the camera 7 to the side of the bed 3.

[0039] Furthermore, as described above, the camera 7 according to this embodiment is configured so that the movable range of the bed 3 during an examination of the subject falls within the imaging angle of view, and so that the entire body of the subject can be imaged during the examination. However, this is not limited to this configuration, and the camera 7 is required to be configured so that at least the area in which the part or organ to be examined is located falls within the imaging angle of view. Note that although the camera 7 according to this embodiment is configured to capture color still images as the optical images, this is not limited to this configuration. For example, the camera 7 may be configured to capture monochrome still images as optical images, or the camera 7 may be configured to capture color or monochrome moving images as optical images.

[0040] As described above, in conventional medical diagnostic devices, in order to set the imaging range for the actual imaging, a scanogram image obtained in advance is displayed, and a user such as a technician or doctor specifies the imaging range for the actual imaging on the scanogram image. However, this configuration results in the subject being exposed to radiation even when the actual imaging is not being performed.

[0041] To avoid this problem, the console 4 according to this embodiment generates organ position information, which is information indicating the positions of the organs of the actual subject, from a camera image obtained by optical photography using a learning model 47 (see also FIG. 2 ) based on AI (Artificial Intelligence) that has been trained in advance. Then, the console 4 according to this embodiment uses the organ position information to superimpose an image indicating the organs of the actual subject on the camera image and display it on the display unit 46. For this reason, the storage 44 according to this embodiment has registered therein the learning model 47 and also has registered therein a database 48 (see also FIG. 4 ) containing learning information, which is information for learning the learning model 47 (hereinafter referred to as the "learning information database").

[0042] The learning model 47 in this embodiment has been pre-trained using as input information a camera image including depth information obtained by photographing the subject with the camera 7, and the above-mentioned examination information regarding the examination content for the subject, and as output information organ position information, which is information indicating the position of the subject's organs in the camera image.

[0043] The learning model 47 according to this embodiment is a model based on generative AI, but is not limited to this. For example, other models such as a model based on a convolutional neural network (CNN) and / or a model based on a recurrent neural network (RNN) may be applied as the learning model 47.

[0044] The console 4 of this embodiment performs two types of processing related to determining the shooting range during actual shooting: the above-mentioned learning processing, which is processing in the phase of learning the learning model 47 (hereinafter referred to as the ``learning phase''), and the control processing, which is processing by the above-mentioned control program in the phase of operating the learned learning model 47 (hereinafter referred to as the ``operation phase'').

[0045] Next, the functional configuration of the console 4 according to this embodiment when the learning process is being executed (learning phase) will be described with reference to Fig. 2. Fig. 2 is a block diagram showing an example of the functional configuration of the console 4 according to this embodiment when the learning process is being executed.

[0046] 2, when the learning process is executed, the console 4 includes a learning information acquisition unit 41 A and a learning unit 41 B. The CPU 41 of the console 4 executes the learning program, thereby functioning as the learning information acquisition unit 41 A and the learning unit 41 B.

[0047] The learning information acquisition unit 41A according to this embodiment acquires the learning information, which is a combination of the camera images, examination information, and organ position information, by reading it from the learning information database 48 (see also FIG. 4 ), which will be described later. However, this is not limiting, and for example, the learning information may be acquired by registering the learning information database 48 in the hospital information system 5 and downloading it from the hospital information system 5.

[0048] The learning unit 41B according to this embodiment then learns the learning model 47 using the camera images and examination information in the acquired learning information as input information and the organ position information in the acquired learning information as output information.

[0049] In this embodiment, in order to generate virtual organ position information from a camera image, the camera image must include the organ to be examined, as indicated by the examination information. Under this condition, the camera image may be taken from the front (AP (Anterior-Posterior View) direction), the side (LAT (Lateral) direction), or another direction.

[0050] Next, a functional configuration of the console 4 according to this embodiment when the control process is executed (operation phase) will be described with reference to Fig. 3. Fig. 3 is a block diagram showing an example of the functional configuration of the console 4 according to this embodiment when the control process is executed.

[0051] 3, the console 4 during execution of the control process includes an acquisition unit 41C, a generation unit 41D, and a display control unit 41E. Each of these blocks corresponds to a block executed by the control program when executing the above-mentioned information processing. The console 4 during execution of the control process also includes an examination control unit 41G, an irradiation control unit 41H, a noise removal unit 41I, and a bed control unit 41J. Each of these blocks corresponds to a block executed by the control program when executing processing other than the above-mentioned information processing.

[0052] The CPU 41 of the console 4 executes the control program to function as an acquisition unit 41C, a generation unit 41D, a display control unit 41E, an examination control unit 41G, an irradiation control unit 41H, a noise removal unit 41I, and a bed control unit 41J.

[0053] The acquisition unit 41C according to this embodiment acquires camera images and examination information corresponding to an actual subject (hereinafter referred to as "actual subject"). Note that in this embodiment, the examination information is acquired by reading it from the patient information management system 51 of the hospital information system 5, but this is not limiting. For example, the examination information may be acquired by having the user of the console 4 input it via the input unit 45.

[0054] As described above, in this embodiment, the examination information includes the region of the subject to be examined, the organ to be examined, the scan protocol to be applied in the examination, etc. However, the present invention is not limited to this, and one type of information or a combination of two or more types of information may be applied as the examination information.

[0055] Furthermore, the generator 41D according to the present embodiment generates organ position information corresponding to the actual subject by inputting the acquired camera image and the acquired examination information into the learning model 47. Then, the display controller 41E according to the present embodiment displays, on the display unit 46, an image of the organ indicated by the organ position information generated by the generator 41D, superimposed on the camera image.

[0056] On the other hand, the examination control unit 41G controls the examination sequence of the subject by the medical diagnostic device.

[0057] The irradiation control unit 41H controls the amount of X-rays irradiated onto the subject from the X-ray tube assembly 13. For example, the shoulders require a larger dose, the lungs are filled with air and therefore have little attenuation, so a smaller dose is sufficient, and the liver has a large attenuation and therefore requires a larger dose. The irradiation control unit 41H controls the amount of X-rays irradiated onto the subject depending on the part of the subject.

[0058] When the amount of noise in an examination image obtained by irradiating a subject with X-rays exceeds the allowable amount for the desired amount of noise, the noise removal unit 41I performs noise removal (denoising) processing on the examination image. For example, if the dose modulation based on the subject's past dose data does not result in a desired image quality index (e.g., image noise) due to an increase in the subject's weight, and the image noise increases, the noise removal unit 41I increases the intensity level of the denoising processing to suppress the image noise and performs automatic reconstruction so that the desired image quality index is the same as that of the previous examination.

[0059] The bed control unit 41J controls the height of the bed 3. The bed control unit 41J may control the height of the bed 3 based on an operation by a technician or the like, or may control the height of the bed 3 based on data from the most recent examination of the subject.

[0060] Next, the learning information database 48 according to this embodiment will be described with reference to Fig. 4. Fig. 4 is a schematic diagram showing an example of the configuration of the learning information database 48 according to this embodiment.

[0061] The learning information database 48 according to this embodiment is where the above-mentioned learning information is registered. As shown in Fig. 4, the learning information database 48 according to this embodiment stores image IDs (Identification), camera images, examination information, and organ position information in association with each other.

[0062] The image ID is information that is assigned in advance as a unique ID in order to individually identify information such as camera images extracted in advance from past data for learning the learning model 47, and the camera image is information that indicates the camera image itself. The examination information is information that indicates the above-mentioned examination information itself regarding the subject when the corresponding camera image was obtained, and the organ position information is information that indicates the above-mentioned organ position information itself, which indicates the position of the organ in the corresponding camera image.

[0063] Here, a method for creating organ position information included in the learning information database 48 according to this embodiment will be described with reference to Fig. 5. Fig. 5 is a schematic diagram illustrating a method for creating organ position information in learning information according to this embodiment.

[0064] In this embodiment, during an examination using the medical diagnostic device of this embodiment, camera images obtained by photographing with the camera 7 and volume data 80 obtained during the examination are registered in the patient information management system 51 of the hospital information system 5.

[0065] 5 as an example, in this embodiment, a plurality of axial images 82 are created at different positions from the registered volume data 80. Then, from each of the created axial images 82, organ regions such as a lung field 90A, a heart 90B, etc. are extracted using a conventionally known object recognition technology, and the extracted organ regions are combined to identify the regions of each organ in the volume data 80.

[0066] In this way, in this embodiment, the region of each organ is extracted from each of the multiple axial images 82, but this is not limited to this form, and for example, the region of each organ may be extracted directly from the volume data 80.

[0067] As described above, in this embodiment, the regions of each organ are extracted using object recognition technology, but this is not limiting. For example, the regions of each organ may be extracted using a conventionally known segmentation technology such as semantic segmentation or instance segmentation. Alternatively, the axial image 82 may be displayed on the display unit 46, and the region of each organ may be extracted by allowing the user to specify the region of the organ on the displayed axial image 82.

[0068] Next, in this embodiment, an image of each extracted organ is created at the center position in the body thickness direction, and the size of the created image of each organ (in this embodiment, the size of the area surrounded by the contour line of the image of each organ) is adjusted using the body thickness obtained from the depth information included in the corresponding camera image 70. Then, in this embodiment, each of the images of each organ after this adjustment is aligned with the corresponding camera image 70, thereby creating organ position information in the same coordinate system as that of the camera image 70.

[0069] Here, a description will be given of how the size of the extracted image of the organ is adjusted using the body thickness of the subject prior to alignment with the camera image 70.

[0070] In this embodiment, the region of each organ obtained from the axial image or volume data is located at the center of the subject's body thickness, and therefore does not necessarily represent the maximum size of the organ in the image. For example, in a person with a standard build, the center of the lung field is located approximately at the center of the body thickness, but the center of the heart is shifted approximately 20% toward the camera 7 from the center of the body thickness, and the center of the liver is shifted approximately 5% toward the camera 7 from the center of the body thickness. The amount of this shift varies depending on the subject's body thickness.

[0071] Therefore, in this embodiment, the size of the region of each organ obtained from the axial image 82 etc. is enlarged or reduced depending on the body thickness of the subject. Hereinafter, this process of enlarging or reducing the region of the organ will be referred to as "organ enlargement / reduction process."

[0072] In this embodiment, as shown in Fig. 5 as an example, labeling data 84 is applied as organ position information for each type of corresponding organ, in which the area within the contour of each organ is assigned a value different from that of other areas. However, this is not limited to this. For example, as shown in Fig. 5 as an example, coordinate value data 86 indicating the position of the contour of each organ may be applied as organ position information for each type of corresponding organ. In other words, the organ position information according to this embodiment is not information indicating the position of a single point such as the center position or center of gravity of the corresponding organ, but information indicating the position of the contour of the organ.

[0073] As described above, in this embodiment, an organ scaling process is performed to scale up or down the size of the image area of ​​the organ indicated by the organ position information in the learning information in accordance with the subject's body thickness. For this reason, the storage 44 according to this embodiment also stores a database 49 (hereinafter referred to as an "organ scaling information database") containing information indicating the organ scaling ratio and reduction ratio in the organ scaling process (hereinafter referred to as "scaling ratio information"). (See also FIG. 6.)

[0074] Next, the organ enlargement / reduction information database 49 according to this embodiment will be described with reference to Fig. 6. Fig. 6 is a schematic diagram showing an example of the configuration of the organ enlargement / reduction information database 49 according to this embodiment.

[0075] As shown in FIG. 6, the organ enlargement / reduction information database 49 according to this embodiment stores information on shift amounts and enlargement / reduction rates in association with each other.

[0076] The shift amount is information indicating the distance of the organ from the center of the body thickness in the body thickness direction. In this embodiment, the shift amount is expressed as a + (plus) value when the organ is farther from the center of the body thickness toward the side where the camera 7 is installed, and as a - (minus) value when the organ is farther from the center of the body thickness toward the side where the camera 7 is not installed. In this embodiment, the shift amount is expressed as a percentage (%) of the distance from the center of the body thickness to the entire body thickness.

[0077] In the example shown in FIG. 6, an organ with a shift amount of +10% represents an increase in size to 104%, and an organ with a shift amount of −5% represents a decrease in size to 98%.

[0078] The medical diagnostic device according to this embodiment executes organ enlargement / reduction processing using an enlargement / reduction ratio corresponding to the amount of shift in the body thickness direction of the central position of each organ obtained from the volume data 80 from the center of the body thickness in the organ enlargement / reduction information database 49. As a result, the medical diagnostic device according to this embodiment automatically creates organ position information to be registered in the learning information database 48.

[0079] Next, the operation of the console 4 will be described.

[0080] First, the operation of the console 4 when executing the learning process according to this embodiment will be described with reference to Fig. 7. Fig. 7 is a flowchart showing an example of the flow of the learning process according to this embodiment.

[0081] The CPU 41 of the console 4 executes the learning program, thereby executing the learning process shown in Fig. 7. The learning process shown in Fig. 7 is executed when a command to start execution of the learning program is input by the user of the console 4 via the input unit 45. Note that, in order to avoid confusion, the following description will be given of a case where the number of pieces of information required to sufficiently learn the learning model 47 is registered in the learning information database 48.

[0082] In step S100, the CPU 41 reads all information (hereinafter referred to as "learning information") from the learning information database 48.

[0083] In step S102, the CPU 41 uses the camera images and examination information in the read learning information as input information and the organ position information in the read learning information as output information (correct answer information) to machine-train the learning model 47, and then terminates this learning process.

[0084] Through the above learning process, the learning model 47 is learned.

[0085] Next, the operation of the console 4 when executing the control process according to this embodiment will be described with reference to Fig. 8. Fig. 8 is a flowchart showing an example of the flow of the control process according to this embodiment.

[0086] The control process shown in Fig. 8 is executed by the CPU 41 of the console 4 executing the control program. The control process shown in Fig. 8 is executed when a command to start execution of the control program is input by the user of the console 4 via the input unit 45. In order to avoid confusion, the following description will be given assuming that the learning model 47 has already been trained, the target subject (hereinafter referred to as the "target subject") has been specified, and the target subject is correctly positioned at the imaging preparation position.

[0087] In step S200, the CPU 41 acquires the examination information corresponding to the target subject by reading it from the patient information management system 51 of the hospital information system 5.

[0088] In step S202, the CPU 41 moves the bed 3 to a predetermined position (in this embodiment, to an imaging preparation position) for capturing an optical image by the camera 7.

[0089] In step S204, the CPU 41 controls the camera 7 to perform optical photography, and acquires a camera image obtained by the optical photography. As described above, this camera image includes depth information.

[0090] In step S206, the CPU 41 inputs the acquired camera image and examination information to the learning model 47. When the camera image and examination information are input, the learning model 47 generates organ position information corresponding to the camera image and examination information. Then, the CPU 41 acquires the organ position information generated by the learning model 47.

[0091] In step S208, the CPU 41 performs control to superimpose an image of the organ indicated by the acquired organ position information (hereinafter referred to as an "organ position image") on the camera image and display it on the display unit 46.

[0092] In step S210, the CPU 41 accepts a user's designation of the imaging range for the actual imaging for the displayed image (hereinafter referred to as "organ superimposed image"), and determines the imaging range.

[0093] In step S212, the CPU 41 applies the determined imaging range to perform the main examination on the subject, and then ends this control process.

[0094] FIG. 9 is a schematic diagram illustrating an example of the overall flow of learning processing and information processing by the console 4 according to this embodiment.

[0095] As shown in Figure 9, in the console 4 of this embodiment, the learning model 47 is pre-learned using the camera image 70 obtained by photographing with the camera 7 and examination information related to the subject as input information, and the organ position information subjected to the organ enlargement / reduction processing described above as output information.

[0096] Furthermore, the console 4 according to this embodiment acquires organ position information corresponding to the actual subject by inputting a camera image 70 and examination information relating to the actual subject into a pre-trained learning model 47. Then, the console 4 according to this embodiment displays an organ position image 72 indicated by the organ position information superimposed on the camera image 70, thereby displaying an organ superimposed image 74 on the display unit 46.

[0097] As described above, according to this embodiment, camera images including depth information obtained by photographing the subject with a camera and examination information related to the examination content of the subject are used as input information, and organ position information, which is information indicating the positions of the subject's organs in the camera images, is used as output information to train a learning model in advance, and organ position information corresponding to the real subject is generated by inputting the camera images and examination information related to the real subject. Therefore, organ position information indicating the positions of the real subject's organs can be generated taking into account the body thickness of the real subject, and as a result, the positions of the real subject's organs can be grasped more accurately than with conventional techniques.

[0098] Furthermore, according to this embodiment, the organ position information used in the learning of the learning model is information in which the size of the organ region is adjusted according to the body thickness of the subject indicated by the depth information. Therefore, the accuracy of generating organ position information by the learning model can be further improved compared to when the organ position information is information in which the size of the organ region is not adjusted according to the body thickness of the subject.

[0099] Furthermore, according to this embodiment, an organ superimposed image is displayed in which an organ position image indicated by organ position information generated by a learning model is superimposed on a camera image, so that the user can intuitively grasp the positions of the organs of the actual subject by referring to the displayed organ superimposed image.

[0100] Furthermore, according to this embodiment, the learning model is trained in advance using camera images of the subject and examination information about the subject as input information, and organ position information corresponding to the subject as output information. Therefore, the learning model can be trained using existing information.

[0101] In the present embodiment, the case where information in which the size of the organ region is adjusted according to the body thickness of the subject indicated by the depth information is used as the organ position information used in learning the learning model 47 has been described, but the present invention is not limited to this. For example, the size of the organ region indicated by the organ position information generated by the learning model 47 may be adjusted according to the body thickness of the actual subject indicated by the depth information included in the camera image of the actual subject. According to this embodiment, the accuracy of the organ size in the organ superimposed image can be further improved compared to when this adjustment is not performed.

[0102] Furthermore, in this embodiment, the case where the camera 7 is installed on the ceiling of the imaging room has been described, but the present invention is not limited to this. For example, the camera 7 may be installed on the side of the bed 3. In this embodiment, the organ superimposed image 74 is displayed by superimposing an organ position screen showing the positions of the organs of the real subject as viewed from the side on a camera image obtained by photographing the real subject from the side. Furthermore, the installation position of the camera 7 may be a position other than the ceiling or the side of the bed 3 described above. In short, any position is applicable as long as it is a position where an area including the organ to be examined in the real subject can be photographed.

[0103] In addition, in the present embodiment, the case where all organs included in the volume data are targeted for extraction from the volume data has been described, but the present invention is not limited to this. For example, only the organs to be examined may be extracted from the volume data.

[0104] Furthermore, in this embodiment, a case has been described in which an image in which the entire body of the subject is included within the angle of view has been applied as the camera image 70. However, the present invention is not limited to this. For example, an image showing only a minimum area including the organ to be examined may be applied as the camera image 70.

[0105] [Second Embodiment] In the medical diagnostic apparatus according to the first embodiment, the learning model 47 generates organ position information indicating the positions of organs relative to the camera image 70 acquired when the bed 3 is positioned at the imaging preparation position, which is the timing immediately before the bed 3 is moved to perform the actual imaging. Therefore, if the bed 3 is moved after the camera image 70 is acquired to perform the actual imaging, the positions of the organs indicated by the organ position information will be misaligned with the positions of the organs in the actual subject. For example, if a user wants to stop the movement of the bed 3 at a desired position while the bed 3 is being moved to an automatically set position, it may be difficult to determine the approximate position to move the bed 3 to, and the bed 3 may not be moved to an appropriate position.

[0106] Therefore, in this embodiment, the positional relationship between a predetermined object in the camera image 70 and the position of the organ based on the organ position information is associated, and the organ position image is superimposed on the camera image 70 acquired after the bed 3 is moved based on the associated positional relationship.

[0107] That is, in the display control unit 41E according to this embodiment, when the bed 3 on which the actual subject is sitting is moved to a position where the actual imaging of the subject is to be performed, an organ position image 72 indicating the position of the organ indicated by the organ position information is superimposed on the camera image 70 using a predetermined object included in the camera image 70 as a reference and displayed on the display unit 46.

[0108] In this embodiment, at least one of the human body feature points of the actual subject and the bed 3 is applied as the predetermined object.

[0109] When applying the anatomical feature points of an actual subject (hereinafter referred to as "anatomical feature points") as the above-mentioned predetermined object, the anatomical feature points of the actual subject indicated by the organ position information generated by the learning model 47 may be associated with the anatomical feature points of the actual subject detected by analyzing the camera image 70 (matching each corresponding anatomical feature point).

[0110] Furthermore, when a bed 3 is applied as the above-mentioned predetermined object, the position of the bed 3 contained in the organ position information generated by the learning model 47 may be associated with position information indicating the position of the bed 3 contained in the corresponding camera image 70 (hereinafter referred to as "bed position information").

[0111] The method for detecting the human feature points of the actual subject may be, for example, a method for detecting the human feature points using a conventionally known learning model capable of detecting the human feature points from a camera image, or a method for detecting the human feature points of the actual subject using a conventionally known pattern matching technique.

[0112] FIG. 10 shows examples of anatomical feature points 76 of a real subject according to this embodiment. In the example shown in FIG. 10 , many anatomical feature points 76, such as the head, shoulders, and arms of the real subject, are illustrated. However, it goes without saying that it is not necessary to detect all of these anatomical feature points 76. Depending on the timing, the upper body of the real subject may be located inside the gantry 11. In such cases, the anatomical feature points 76 to be applied may be switched appropriately, such as using anatomical feature points 76 located on the lower body of the real subject. For example, depending on the subject of the examination or the situation of the real subject, a blanket may be placed over the real subject. In such cases, anatomical feature points 76 not covered by the blanket may be used. Furthermore, the tilt of the bed 3 or the real subject relative to the plane may change from the time the camera image 70 was acquired. In such cases, the tilt can be accommodated by simultaneously using two or more anatomical feature points 76.

[0113] In addition, as a method for acquiring bed position information, for example, a method may be applied in which the position of the bed 3 is managed as an absolute position or a relative position in the medical diagnostic device, and the bed position information is also acquired when acquiring the camera image 70.

[0114] 10, for example, when anatomical feature points of the actual subject are applied as the predetermined object, anatomical feature points are also detected for the actual subject in a camera image 70 acquired after the bed 3 is moved, and an organ position image 72 is superimposed and displayed so that the positions of the detected anatomical feature points match. When the bed 3 is applied as the predetermined object, an example can be given in which bed position information indicating the position of the bed 3 is acquired after the bed 3 is moved, and compared with the position of the bed 3 acquired when the association is made, and an organ position image 72 is superimposed and displayed so that it matches the difference in position.

[0115] Regarding the configuration of the medical diagnostic apparatus according to this embodiment, only the function of the display control unit 41E is different from that of the first embodiment, and the operation of the console 4 when executing the learning process is the same as that of the first embodiment, so a description of these will be omitted here. Hereinafter, the operation of the console 4 when executing the control process according to this embodiment will be described with reference to Fig. 11. Fig. 11 is a flowchart showing an example of the flow of the control process according to this embodiment, and steps that perform the same processes as those in Fig. 8 are assigned the same step numbers as in Fig. 8, and their description will be omitted.

[0116] 11, the CPU 41 starts control to display an organ superimposed image 74 on the display unit 46 by superimposing the organ position image 72 indicated by the organ position information acquired by the processing of step S206 on the camera image 70. At this time, as described above, the CPU 41 performs control to superimpose and display the organ position image 72 on the camera image 70 in real time so that the positions of predetermined objects match.

[0117] By this control, as shown in Fig. 12 as an example, the organ position image 72 displayed by the display unit 46 displays an organ superimposed image 74 that follows the movement of the actual subject accompanying the movement of the bed 3. Note that Fig. 12 is a diagram used to explain the flow of control processing according to this embodiment, with the left diagram showing an example of the organ superimposed image 74 before the movement of the bed 3 and the right diagram showing an example of the organ superimposed image 74 after the movement of the bed 3.

[0118] When the display unit 46 starts displaying the organ superimposed image 74, the user specifies the imaging range for the main imaging on the displayed organ superimposed image 74.

[0119] Therefore, in step S210B, the CPU 41 waits until the user specifies the imaging range, and in step S211, the CPU 41 stops displaying the organ superimposed image 74.

[0120] As described above, according to this embodiment, when the bed on which the real subject is placed is moved to a position where the actual imaging of the real subject is performed, an organ position image showing the positions of the organs indicated by the organ position information is superimposed on the camera image using a predetermined object included in the camera image as a reference and displayed on the display unit. Therefore, compared to when an organ superimposed image is displayed without using a predetermined object as a reference, it is possible to more easily display an organ superimposed image that follows the movement of the real subject.

[0121] In particular, according to this embodiment, at least one of the anatomical features of the actual subject and the bed is used as the predetermined object, and therefore, by using the applied object, it is possible to display an organ superimposed image that follows the movement of the actual subject.

[0122] [Third embodiment] When an organ position image 72 is superimposed on a camera image 70, when the bed 3 is moved, it is necessary to refer to both the organ superimposed image 74 on the display unit 46 and the actual subject on the bed 3, which may increase the effort required for confirmation by the user and lengthen the examination time.

[0123] Therefore, in this embodiment, organ position information generated by the learning model 47 is used to project an organ position image 72 onto the surface of the actual subject on the bed 3 using a projector, so that the organ position image 72 can be confirmed simply by referring to the actual subject on the bed 3.

[0124] That is, the display control unit 41E according to this embodiment performs a process of projecting an organ position image 72, which indicates the positions of organs indicated by the organ position information, onto the actual subject. At this time, the display control unit 41E deforms and projects the organ position image 72 so as to perform projection mapping on the surface of the body of the actual subject, depending on the unevenness of the body thickness of the actual subject obtained from the depth information.

[0125] Regarding the configuration of the medical diagnostic apparatus according to this embodiment, only the function of the display control unit 41E is different from that of the first embodiment, and the operation of the console 4 when executing the learning process is the same as that of the first embodiment, so a description of these will be omitted here. Hereinafter, the operation of the console 4 when executing the control process according to this embodiment will be described with reference to Fig. 13. Fig. 13 is a flowchart showing an example of the flow of the control process according to this embodiment, and steps that perform the same processes as those in Fig. 8 are assigned the same step numbers as in Fig. 8, and their description will be omitted.

[0126] In step S208C of FIG. 13, the CPU 41 controls the projector to project the organ position image 72 indicated by the organ position information acquired in the process of step S206 onto the surface of the actual subject on the bed 3.

[0127] 14 shows a side view and a plan view of a photography room for explaining the flow of control processing according to this embodiment. In the example shown in FIG. 14, the projector 8 is provided integrally with the camera 7.

[0128] 14, in the control process according to this embodiment, an organ position image 72 is projected onto the actual subject 6 under the control of the display control unit 41E. That is, the display controlled by the display control unit 41E according to this embodiment means a display by projecting an image.

[0129] As described above, according to this embodiment, an organ position image showing the positions of organs indicated by the organ position information is projected onto the actual subject, which makes it easier to grasp the positions of the organs of the actual subject compared to when the organ position image is not projected onto the actual subject.

[0130] In this embodiment, the case where the organ position image 72 is displayed by simply projecting the organ position image 72 onto the actual subject 6 has been described, but the present invention is not limited to this. For example, in addition to the display by projection, an organ superimposed image 74 may be displayed by the display unit 46, as in the aspects of the above-described embodiments.

[0131] In addition, in this embodiment, the projector 8 is installed integrally with the camera 7 on the ceiling of the imaging room, and the organ position image 72 is projected from in front of the real subject 6 lying on his / her back on the bed 3, but the present invention is not limited to this. For example, the projector 8 may be installed directly to the side of the real subject 6 lying on his / her back on the bed 3, and the organ position image 72 may be projected to the side of the real subject, or the installation position or projection direction of the projector 8 may be other.

[0132] Furthermore, in this embodiment, the case where only the organ position image 72 is projected has been described, but the present invention is not limited to this. For example, the organ superimposed image 74 may also be projected.

[0133] [Fourth embodiment] In each of the above embodiments, the actual subject lies supine on the bed 3, but the actual subject is not necessarily positioned horizontally at this time, and for example, one shoulder is often raised or the body is twisted along the body axis.

[0134] In the first and second embodiments described above, in such a case, it may not be possible to accurately align the organ position image 72 with the camera image 70 .

[0135] Therefore, in this embodiment, anatomical feature points 76 of the actual subject 6 are detected in the camera image 70, and the depth information of the camera 7 at the anatomical feature points 76 is used to three-dimensionally correct the state of the organ indicated by the organ position information.

[0136] That is, the display control unit 41E according to this embodiment changes and applies at least one of the position and size of the organ indicated by the organ position information corresponding to the real subject 6 according to the posture of the real subject 6. More specifically, the learning model 47 is the same as that in each of the above embodiments, and the display control unit 41E processes or aligns the organ position information generated by the learning model 47 using depth information for each anatomical feature point 76 of the real subject 6 in the acquired camera image 70.

[0137] Regarding the configuration of the medical diagnostic apparatus according to this embodiment, only the function of the display control unit 41E is different from that of the first embodiment, and the operation of the console 4 when executing the learning process is the same as that of the first embodiment, so a description of these will be omitted here. Hereinafter, the operation of the console 4 when executing the control process according to this embodiment will be described with reference to Fig. 15. Fig. 15 is a flowchart showing an example of the flow of the control process according to this embodiment, and steps that perform the same processes as those in Fig. 8 are assigned the same step numbers as in Fig. 8, and their description will be omitted.

[0138] In step S208D of Figure 15, before controlling the display of the organ superimposed image 74 on the display unit 46, the CPU 41 modifies the organ position information acquired by the processing of step S206 using the depth information contained in the camera image 70 as shown below.

[0139] First, as shown in Fig. 16 as an example, the CPU 41 detects human feature points 76 in an image of the actual subject 6 in a camera image 70. The detection of the human feature points 76 may be performed by a conventionally known method, and for example, image recognition technology using a rule base and / or AI may be applied. Fig. 16 is a diagram showing an example of the human feature points 76 and depth information 78 used to explain the flow of control processing according to this embodiment.

[0140] Next, the CPU 41 acquires the values ​​of the depth information 78 corresponding to the human feature points 76 and estimates the posture of the actual subject 6. For example, if the depth indicated by the depth information for the right shoulder position is greater than the depth indicated by the depth information for the left shoulder position (i.e., the distance from the camera 7 is greater), and the depth indicated by the depth information for the right hip position is less than the depth indicated by the depth information for the left hip position (i.e., the distance from the camera 7 is smaller), it is estimated that the actual subject 6 is twisted clockwise when viewed from the top of the head in the direction of the body axis.

[0141] Then, the CPU 41 processes at least one of the positions and sizes of the organs indicated by the organ position information in accordance with the estimated posture of the actual subject 6. As a method of this processing, for example, an image processing method such as affine transformation may be applied, or a technology that can output a change in a certain position in accordance with the movement of the human body model may be applied.

[0142] As described above, according to this embodiment, at least one of the positions and sizes of organs indicated by organ position information corresponding to a real subject is changed according to the posture of the real subject, and therefore the positions of the organs of the real subject can be grasped more accurately than when the change is not made.

[0143] In the present embodiment, at least one of the positions and sizes of the organs indicated by the organ position information generated by the learning model 47 is corrected according to the state of the actual subject 6, but the present invention is not limited to this. For example, as learning information for constructing the learning model 47, organ position information according to the posture of the subject in the camera image 70, which is estimated using depth information at the anatomical feature points 76, may be used.

[0144] That is, in this form, the organ position information used in learning the learning model 47 is information in which at least one of the position and size of the organ indicated by the organ position information is changed according to the posture of the subject.

[0145] In this case, since the learning information used when learning the learning model 47 is different from that used in the above embodiments, the organ position information generated by the learning model 47 is information that matches the twisting of the body of the actual subject 6, etc.

[0146] Although the embodiments of the present disclosure have been described in detail above with reference to the accompanying drawings, the technical scope of the present disclosure is not limited to such examples. It is clear that a person skilled in the art of the present disclosure can conceive of various modifications or alterations within the scope of the technical idea described in the claims, and it is understood that these modifications or alterations also naturally fall within the technical scope of the present disclosure.

[0147] Furthermore, the effects described in the above embodiments are explanatory or exemplary and are not limited to those described in the above embodiments. In other words, the technology according to the present disclosure may achieve other effects that are obvious to a person skilled in the art of the present disclosure from the description in the above embodiments, in addition to or instead of the effects described in the above embodiments.

[0148] For example, in the above embodiment, the information processing device according to the present disclosure is applied to an X-ray CT scanner, but the present disclosure is not limited to this. The information processing device according to the present disclosure may be applied to an MRI scanner or the like, as long as the scanner is an imaging device that sets an imaging range using a scanogram image.

[0149] In the above embodiment, the case where information previously obtained in the medical diagnostic device that actually operates the learning model 47 is applied as learning information for training the learning model 47 has been described, but the present invention is not limited to this form. For example, information obtained by a medical diagnostic device other than this medical diagnostic device may be applied as learning information used for training the learning model 47.

[0150] In each of the above embodiments, the following various processors can be used as the hardware structure of processing units that perform various processes, such as the learning information acquisition unit 41A, the learning unit 41B, the acquisition unit 41C, the generation unit 41D, the display control unit 41E, the examination control unit 41G, the irradiation control unit 41H, the noise removal unit 41I, and the bed control unit 41J. As described above, the various processors include a CPU, which is a general-purpose processor that executes software (programs) and functions as various processing units, as well as dedicated electrical circuits, such as a programmable logic device (PLD), a processor whose circuit configuration can be changed after manufacture, such as an FPGA (Field Programmable Gate Array), and an ASIC (Application Specific Integrated Circuit), which is a processor having a circuit configuration specifically designed to perform specific processes.

[0151] A single processing unit may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, multiple processing units may be configured with a single processor.

[0152] Examples of configuring multiple processing units with a single processor include: first, a form in which one processor is configured with a combination of one or more CPUs and software, and this processor functions as multiple processing units, as typified by computers such as client and server; second, a form in which a processor is used to realize the functions of an entire system including multiple processing units with a single IC (Integrated Circuit) chip, as typified by systems on chips (SoCs); and thus, various processing units are configured using one or more of the above-mentioned various processors as a hardware structure.

[0153] Furthermore, more specifically, the hardware structure of these various processors can be an electric circuit that combines circuit elements such as semiconductor elements.

[0154] In addition, in the above-described embodiments, the learning program and the control program are pre-stored (installed) in the ROM 42 or storage 44 of the console 4, but this is not limiting. The above-described programs may be provided in a form recorded on a recording medium such as a CD-ROM (Compact Disc Read Only Memory), a DVD-ROM (Digital Versatile Disc Read Only Memory), or a USB (Universal Serial Bus) memory. The above-described programs may also be downloaded from an external device via a network.

[0155] From the above description, the invention described in the following appendix can be understood.

[0156] [Supplementary Note 1] An information processing device comprising at least one processor, wherein the processor generates organ position information corresponding to an actual subject by inputting the camera image and examination information of the actual subject into a learning model that has been trained in advance using camera images including depth information obtained by photographing the subject with a camera and examination information related to the examination content of the subject as input information and organ position information that is information indicating the positions of the organs of the subject in the camera image as output information.

[0157] [Supplementary Note 2] The information processing device according to Supplementary Note 1, wherein the organ position information used in learning the learning model is information in which the size of the organ region is adjusted according to the body thickness of the subject indicated by the depth information.

[0158] [Supplementary Note 3] The information processing device described in Supplementary Note 1, wherein the processor adjusts the size of the organ area indicated by the organ position information generated by the learning model according to the body thickness of the actual subject indicated by the depth information included in the camera image of the actual subject.

[0159] [Supplementary Note 4] The information processing device described in any one of Supplementary Note 1 to Supplementary Note 3, wherein when a table on which the actual subject is standing is moved to a position where the actual imaging of the actual subject is to be performed, the processor further performs processing to display, on a display unit, an organ position image indicating the position of the organ indicated by the organ position information, superimposed on the camera image using a predetermined object included in the camera image as a reference.

[0160] [Supplementary Note 5] The information processing device according to Supplementary Note 4, wherein the predetermined object is at least one of a human body feature of the actual subject and the table.

[0161] [Supplementary Note 6] The information processing device according to any one of Supplementary Note 1 to Supplementary Note 5, wherein the processor further performs processing to project an organ position image indicating the position of the organ indicated by the organ position information onto the actual subject.

[0162] [Appendix 7] The information processing device described in any one of Appendices 1 to 6, wherein the processor changes and applies at least one of the position and size of the organ indicated by the organ position information corresponding to the actual subject according to the posture of the actual subject.

[0163] [Appendix 8] The information processing device described in any one of Appendices 1 to 7, wherein the organ position information used in learning the learning model is changed in accordance with the posture of the subject, with respect to at least one of the position and size of the organ indicated by the organ position information.

[0164] [Supplementary Note 9] The information processing device according to any one of Supplementary Note 1 to Supplementary Note 8, wherein the processor pre-learns the learning model using the camera images and the examination information relating to the subject as input information and the organ position information corresponding to the subject as output information.

[0165] [Supplementary Note 10] A medical imaging device including: an information processing device according to any one of Supplementary Note 1 to Supplementary Note 9; and a radiological imaging device that performs imaging of a real subject using organ position information generated by the information processing device.

[0166] [Supplementary Note 11] An information processing method including: generating organ position information corresponding to an actual subject by inputting the camera image and the examination information of an actual subject into a learning model that has been trained in advance using, as input information, camera images obtained by photographing a subject with a camera, the camera images including depth information, and examination information related to the examination content of the subject, and as output information, organ position information that is information indicating the positions of the organs of the subject in the camera image.

[0167] [Supplementary Note 12] An information processing program for causing a computer to execute a process including generating organ position information corresponding to an actual subject by inputting the camera image and examination information regarding an actual subject into a learning model that has been trained in advance using, as input information, camera images obtained by photographing a subject with a camera, the camera images including depth information and examination information regarding examination details for the subject, and as output information, organ position information that is information indicating the positions of the subject's organs in the camera image.

[0168] In the above, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed by connecting them with "and / or."

Claims

1. An information processing device comprising a processor, wherein the processor receives as input information camera images obtained by photographing a subject with a camera, the camera images including depth information, and examination information regarding the examination content of the subject, and outputs as output information organ position information indicating the positions of the subject's organs in the camera images, and inputs the camera images and examination information regarding an actual subject into a learning model that has been trained in advance, thereby generating the organ position information corresponding to the actual subject.

2. The information processing device according to claim 1, wherein the organ position information used in learning the learning model is information in which the size of the organ area is adjusted according to the body thickness of the subject indicated by the depth information.

3. The information processing device described in claim 1, wherein the processor adjusts the size of the organ area indicated by the organ position information generated by the learning model according to the body thickness of the actual subject indicated by the depth information contained in the camera image of the actual subject.

4. An information processing device as described in claim 1 or claim 2, wherein the processor further performs processing to display on a display an organ position image indicating the position of the organ indicated by the organ position information on the camera image, superimposed on the camera image using a predetermined object contained in the camera image as a reference, when a table on which the actual subject is standing is moved to a position where the actual photographing of the subject is to be performed.

5. The information processing device according to claim 4, wherein the predetermined object is at least one of a physique feature of the actual subject and the table.

6. An information processing device according to claim 1 or claim 2, wherein the processor further performs processing to project an organ position image showing the position of the organ indicated by the organ position information onto the actual subject.

7. An information processing device as described in claim 1 or claim 2, wherein the processor changes and applies at least one of the position and size of the organ indicated by the organ position information corresponding to the actual subject according to the posture of the actual subject.

8. An information processing device as described in claim 1 or claim 2, wherein the organ position information used in learning the learning model is changed in accordance with the posture of the subject, in terms of at least one of the position and size of the organ indicated by the organ position information.

9. An information processing device as described in claim 1 or claim 2, wherein the processor pre-learns the learning model using the camera images and examination information regarding the subject as input information and the organ position information corresponding to the subject as output information.

10. A medical imaging device comprising: an information processing device according to claim 1 or 2; and a radiological imaging device that performs imaging of an actual subject using organ position information generated by the information processing device.

11. An information processing method comprising: generating organ position information corresponding to an actual subject by inputting the camera image and examination information relating to the actual subject into a learning model that has been trained in advance using, as input information, camera images obtained by photographing a subject with a camera, the camera images including depth information, and examination information relating to the examination content of the subject, and as output information, organ position information that indicates the positions of the subject's organs in the camera image.

12. An information processing program for causing a computer to execute a process including generating organ position information corresponding to an actual subject by inputting the camera image and examination information regarding an actual subject into a learning model that has been trained in advance using, as input information, camera images obtained by photographing a subject with a camera, the camera images including depth information, and examination information regarding the examination content of the subject, and as output information, organ position information that indicates the position of the subject's organs in the camera image.

Images