Image processing device, medical image imaging device, image processing method, and image processing program

Abstract

In the present invention, a console is provided with a generation unit for generating a virtual positioning image corresponding to a real examinee by inputting examinee information and camera images related to the real examinee into a learning model 48 that has been trained in advance using: as input information, examinee information related to examinees and camera images obtained by imaging the examinees with a camera 7; and, as output information, a positioning image which is an image displayed in order to receive the imaging range for primary imaging.

Description

Image processing device, medical image capturing device, image processing method, and image processing program

[0001] The present disclosure relates to an image processing device, a medical imaging device, an image processing method, and an image processing program.

[0002] 2. Description of the Related Art In recent years, advances in medical equipment such as CT (Computed Tomography) devices and MRI (Magnetic Resonance Imaging) devices have led to the use of higher quality, higher resolution three-dimensional images for diagnostic imaging.

[0003] When imaging a subject using an imaging device such as a CT device or MRI device, a scanogram image (hereinafter referred to as "scanogram imaging") is captured prior to the actual imaging to obtain a three-dimensional image in order to determine the imaging range, and a two-dimensional positioning image (scanogram image) is obtained. The operator of the imaging device (technologist, etc.) sets the imaging range for the actual imaging while viewing the scanogram image. Note that scanogram images are also called scout images, topograms, etc., but will be referred to as "scanogram images" or "positioning images" below.

[0004] However, since imaging to obtain a scanogram image is performed using radiation, it involves exposure of the subject to radiation. Therefore, a medical image diagnostic device disclosed in Japanese Patent Application Laid-Open No. 2020-110444 is a conventional technology that can be applied to obtain a scanogram image without the radiation exposure.

[0005] This medical image diagnostic device includes an acquisition unit that acquires information about the area to be scanned and an optical image of the subject, and a processing unit that inputs the acquired information about the area to be scanned and the optical image of the subject into a trained model that outputs the scan range of the subject during scanning based on the information about the area to be scanned and the optical image of the subject, and outputs the acquired information about the area to be scanned and the scan range of the subject related to the aligned image of the subject.

[0006] However, the technology disclosed in JP 2020-110444 A automatically sets the imaging range (scan range) for the actual imaging, and although the imaging range can be easily set, a scanogram image cannot be generated. In other words, this technology sets the imaging range for the actual imaging without human intervention, which poses a problem in that the accuracy with which the imaging range for the actual imaging is set is not necessarily high enough to satisfy requirements.

[0007] The present disclosure has been made in consideration of the above points, and aims to provide an image processing device, a medical image capturing device, an image processing method, and an image processing program that can improve the accuracy of setting the capturing range for the actual capture compared to conventional techniques.

[0008] An image processing device according to a first aspect of the present disclosure includes at least one processor, which takes as input information a camera image obtained by photographing the subject with an optical camera and subject information about the subject, and generates a virtual positioning image corresponding to the actual subject by inputting the camera image and subject information about the actual subject into a pre-trained learning model that uses as output information a positioning image, which is an image displayed to accept the photographing range for the actual photographing.

[0009] An image processing device according to a second aspect of the present disclosure is an image processing device according to the first aspect, in which the learning model is a model that has been pre-trained using camera images, subject information, and test information related to the test contents for the subject as input information and positioning images as output information, and a processor inputs camera images, subject information, and test information of an actual subject into the learning model, thereby generating a virtual positioning image corresponding to the actual subject.

[0010] An image processing device according to a third aspect of the present disclosure is an image processing device according to the first or second aspect, in which a processor accepts a designation of a display target area for a virtual positioning image, and displays the virtual positioning image on a display unit with the accepted display target area as the display target.

[0011] An image processing device according to a fourth aspect of the present disclosure is an image processing device according to the third aspect, in which a processor displays a camera image on a display unit and accepts a specification from a user regarding the displayed camera image, thereby accepting a specification of a display target area.

[0012] An image processing device according to a fifth aspect of the present disclosure is an image processing device according to the third aspect, in which a processor accepts a selection specification from a plurality of pre-set display target areas from a user, thereby accepting the specification of the display target area.

[0013] An image processing device according to a sixth aspect of the present disclosure is an image processing device according to the first aspect, in which the learning model is a model that has been pre-trained using multiple camera images obtained by photographing the subject from multiple directions with an optical camera and subject information as input information, and at least one of a raysum image and a three-dimensional image created from an axial image obtained by photographing the subject as output information, and a processor inputs the multiple camera images and subject information of the actual subject into the learning model, thereby generating a three-dimensional virtual positioning image corresponding to the actual subject.

[0014] An image processing device according to a seventh aspect of the present disclosure is an image processing device according to the first aspect, in which a processor pre-learns a learning model using camera images obtained by photographing the subject with an optical camera and subject information about the subject as input information, and a positioning image as output information.

[0015] A medical imaging device according to an eighth aspect of the present disclosure includes an image processing device according to the present disclosure and a radiological imaging device that performs actual imaging within an imaging range set by a virtual positioning image generated by the image processing device.

[0016] In an image processing method according to a ninth aspect of the present disclosure, a processor takes as input information a camera image obtained by photographing the subject with an optical camera and subject information about the subject, and outputs as output information a positioning image, which is an image displayed to accept the shooting range of the actual photograph, to a pre-trained learning model, thereby generating a virtual positioning image corresponding to the actual subject.

[0017] An image processing program according to a tenth aspect of the present disclosure is a program that is trained in advance using a camera image obtained by photographing a subject with an optical camera and subject information related to the subject as input information, and a positioning image that is an image to be displayed to accept a photographing range for main photography as output information.

[0018] This is an image processing program that causes a computer to execute a process of generating a virtual positioning image corresponding to an actual subject by inputting camera images and subject information about the actual subject into a learning model.

[0019] According to the present disclosure, it is possible to improve the accuracy of setting the shooting range for main photography compared to conventional techniques.

[0020] FIG. 1 is a diagram showing a schematic configuration of a medical diagnostic apparatus according to an embodiment of the disclosed technology. FIG. 2 is a block diagram showing an example of a functional configuration of a console when executing a learning process according to an embodiment of the disclosed technology. FIG. 3 is a block diagram showing an example of a functional configuration of a console when executing a control process according to an embodiment of the disclosed technology. FIG. 4 is a schematic diagram showing an example of a configuration of a learning information database according to an embodiment of the disclosed technology. FIG. 5 is a flowchart showing an example of a flow of a learning process according to an embodiment of the disclosed technology. FIG. 6 is a flowchart showing an example of a flow of a control process according to an embodiment of the disclosed technology. FIG. 7 is a schematic diagram provided for explaining an example of an overall flow of a learning process and image processing by a console according to an embodiment of the disclosed technology. FIG. 8 is a schematic diagram showing another example of the configuration of a learning information database according to an embodiment of the disclosed technology. FIG. 9 is a schematic diagram provided for explaining another example of an overall flow of a learning process and image processing by a console according to an embodiment of the disclosed technology. FIG. 10 is a diagram showing a specific example of a virtual positioning image according to an embodiment of the disclosed technology, where the left half is an example of a virtual positioning image obtained when a learning model is trained without using test information, and the right half is an example of a virtual positioning image obtained when a learning model is trained using test information. FIG. 11 is a block diagram showing another example of a functional configuration of a console when executing a control process according to an embodiment of the disclosed technology. FIG. 12 is a flowchart showing another example of a flow of a control process according to an embodiment of the disclosed technology. 1 is a diagram showing an example of a scan planning screen displayed by a console according to an embodiment of the disclosed technology, and is a schematic diagram showing an example of a placement position of a camera in a medical diagnostic apparatus according to an embodiment of the disclosed technology, an image of a raysum image generated from an axial image, and an example of a state in which the raysum image is displayed on a display unit as a virtual positioning image.

[0021] Hereinafter, examples of embodiments of the present disclosure will be described with reference to the drawings. Note that the same reference numerals are used to designate the same or equivalent components and parts in each drawing. Furthermore, the dimensional proportions of the drawings are exaggerated for the sake of explanation and may differ from the actual proportions.

[0022] [First Embodiment] Fig. 1 is a diagram showing a schematic configuration of a medical diagnostic apparatus including an image processing apparatus and a medical imaging apparatus according to this embodiment. In this embodiment, the configuration of one embodiment of an X-ray CT (Computed Tomography) apparatus is shown as the medical diagnostic apparatus. Note that the medical diagnostic apparatus of the present disclosure is not limited to an X-ray CT apparatus, and can also be applied to other medical diagnostic apparatuses such as an MRI (Magnetic Resonance Imaging) apparatus.

[0023] The medical diagnostic apparatus according to this embodiment includes a scanner 1, a bed 3, and a console 4. The console 4 corresponds to the image processing apparatus of the present disclosure, the scanner 1 corresponds to the radiographic imaging apparatus of the present disclosure, and the combination of the scanner 1, the bed 3, and the console 4 corresponds to the medical imaging apparatus of the present disclosure.

[0024] The scanner 1 is a part that performs CT scans. The scanner 1 includes a gantry 11, a rotating plate 12 that has an opening in its center and is rotatably supported by the gantry 11, an X-ray tube assembly 13 fixed to the rotating plate 12, a collimator 14 provided at an X-ray emission port of the X-ray tube assembly 13, an X-ray detector 15 disposed opposite the X-ray tube assembly 13 across the opening of the rotating plate 12, and a rotating plate drive unit 17 provided on the gantry 11. The rotating plate 12 of the scanner 1 also includes a collimator control unit 18 that controls the collimator 14 to change the X-ray irradiation field, a rotating plate drive control unit 19 that controls the drive of the rotating plate drive unit 17, an X-ray high-voltage generator 20 that supplies power to the X-ray tube assembly 13 for generating X-rays and controls the X-ray generation conditions, a data acquisition unit 16 that acquires the output of the X-ray detector 15, and a data transmission unit 21 that transmits the data acquired by the data acquisition unit 16. The supply of power and control signals to each unit provided on the rotating plate 12, and the extraction of data from each unit provided on the rotating plate 12, are carried out via a slip ring (not shown) provided between the base 11 and the rotating plate 12.

[0025] The bed 3 moves the subject between an imaging preparation position and an imaging position. The subject is placed on a tabletop 31. The bed 3 has a vertical movement mechanism and a front-to-back movement mechanism for the tabletop 31, which are not shown in the figure. The bed 3 is provided with a bed controller 32, a tabletop vertical movement controller 33, and a tabletop front-to-back movement controller 34 to control the operation of the vertical movement mechanism and the front-to-back movement mechanism of the tabletop 31.

[0026] The console 4 controls the medical diagnostic device, which is an X-ray CT device. The console 4 is an example of a computer of the present disclosure. The console 4 includes a central processing unit (CPU) 41, a read-only memory (ROM) 42, a random access memory (RAM) 43, and a storage 44. The console 4 also includes an input unit 45 and a display unit (display) 46.

[0027] The CPU 41, an example of a processor, is a central processing unit that executes various programs and controls each component. That is, the CPU 41 reads a program from the ROM 42 or storage 44 and executes the program using the RAM 43 as a work area. The CPU 41 controls the above-described components and performs various arithmetic processing in accordance with the program recorded in the ROM 42 or storage 44. In this embodiment, the ROM 42 or storage 44 stores a learning program that executes a learning process for learning a learning model 48 (see also FIG. 2 ), details of which will be described later, and a control program (hereinafter simply referred to as the "control program") for the medical diagnostic apparatus, which includes image processing for determining the imaging range for actual imaging using the X-ray CT scanner. The program related to this image processing corresponds to the image processing program disclosed herein, and the processing method for this image processing corresponds to the image processing method disclosed herein.

[0028] The ROM 42 stores various programs and various data. The RAM 43 temporarily stores programs or data as a working area. The storage 44 is configured with a storage device such as an HDD (Hard Disk Drive), an SSD (Solid State Drive), or a flash memory, and stores various programs including the operating system and various data.

[0029] The input unit 45 includes a pointing device such as a mouse and a keyboard, and is used to input various information. The input unit 45 includes a bed operating unit that operates the height of the bed 3.

[0030] The display unit 46 is, for example, a liquid crystal display, and displays various information. The display unit 46 may be a touch panel type and function as the input unit 45.

[0031] In this embodiment, the console 4 is connected to a hospital information system 5 of a hospital in which an X-ray CT apparatus is installed. The hospital information system 5 includes a patient information management system 51 and an examination reservation system 52.

[0032] The patient information management system 51 is a database of personal information, medical data, examination image data, test data, medication data, etc., of patients who have previously received medical treatment at the hospital. The patient's personal information includes personal identification information such as name, ID (Identification), date of birth, age, and gender, as well as physical information such as height, weight, and body fat percentage, and past consultation dates. This personal information is entered into the system when a subject visits the hospital as a patient and receives medical treatment. The patient's height and weight are collected by the patient's self-reporting in a medical questionnaire or by actual measurements, and then entered into the patient information management system 51. The medical data, examination image data, test data, and medication data are entered into the patient information management system 51 each time the patient receives medical treatment, creating an in-hospital patient database. The above-mentioned patient's personal information corresponds to the subject information of the present disclosure, and this personal information will be referred to as "subject information" hereinafter.

[0033] On the other hand, the examination data includes information related to the examination of the corresponding patient, such as the region to be examined, the organ to be examined, and the scan protocol to be applied in the examination. Here, the region to be examined is information indicating a region such as the head, chest, abdomen, thoracic-abdominal region, etc., and the organ to be examined is information indicating the organ itself such as the lung field, stomach, etc. Furthermore, the scan protocol is information indicating setting conditions to be applied in the actual imaging, such as the imaging mode to be applied in the examination, the mAs value to be set in the X-ray tube device 13, the initial setting of the imaging range, etc. This examination data corresponds to the examination information in the present disclosure, and will be referred to hereinafter as "examination information."

[0034] The examination reservation system 52 creates an examination schedule for each day for each system of medical diagnostic equipment installed in the hospital based on the information stored or written in the patient information management system 51, and distributes the created examination schedule as data to each system. The examination schedule includes the order in which patients will be examined that day by the medical diagnostic equipment, as well as the personal identification information and physical information of each patient. The contents of the examination reservation system 52 are updated sequentially in accordance with the progress of medical treatment that day, and the updated data is sent to each system each time an update is made.

[0035] A camera 7 is provided on the ceiling of the imaging room in which the medical diagnostic device is installed, in a state in which it can capture an image of an area within the movable range of the bed 3. The camera 7 according to this embodiment is capable of obtaining an optical image (hereinafter referred to as a "camera image") within the imaging angle of view by imaging. Note that, although the camera 7 in this embodiment is configured to capture color still images, the present invention is not limited to this. For example, the camera 7 may be configured to capture monochrome still images, or may be configured to capture color or monochrome moving images.

[0036] As described above, in conventional medical diagnostic devices, in order to set the imaging range for the actual imaging, a scanogram image obtained in advance is displayed, and a user such as a technician or doctor specifies the imaging range for the actual imaging on the scanogram image. However, this configuration results in the subject being exposed to radiation even when the actual imaging is not being performed.

[0037] To avoid this problem, the console 4 according to this embodiment generates a scanogram image from a camera image obtained by optical photography using a learning model 48 (see also FIG. 2 ) based on AI (Artificial Intelligence) that has been trained in advance. For this reason, the storage 44 according to this embodiment has registered therein the learning model 48, as well as a database 49 (see also FIG. 4 ) that includes learning information that is information used for learning the learning model 48 (hereinafter referred to as the "learning database").

[0038] The learning model 48 according to this embodiment is learned in advance using, as input information, a camera image obtained by photographing the subject with the camera 7 and subject information relating to the subject, and as output information, a positioning image, which is an image displayed to accept the photographing range for the actual photographing. Note that the positioning image corresponds to a scanogram image, but will be referred to collectively below as the "positioning image."

[0039] The learning model 48 according to the present embodiment is a model based on generative AI, but is not limited to this. For example, other models such as a convolutional neural network (CNN) model or a recurrent neural network (RNN) model may be applied as the learning model 48.

[0040] The console 4 of this embodiment performs two types of processing related to determining the shooting range during actual shooting: the above-mentioned learning processing, which is processing in the phase of learning the learning model 48 (hereinafter referred to as the "learning phase"), and the control processing, which is processing by the above-mentioned control program in the phase of operating the learned learning model 48 (hereinafter referred to as the "operation phase").

[0041] Next, the functional configuration of the console 4 according to this embodiment when the learning process is being executed (learning phase) will be described with reference to Fig. 2. Fig. 2 is a block diagram showing an example of the functional configuration of the console 4 according to this embodiment when the learning process is being executed.

[0042] 2, when the learning process is executed, the console 4 includes a learning information acquisition unit 41 A and a learning unit 41 B. The CPU 41 of the console 4 executes the learning program, thereby functioning as the learning information acquisition unit 41 A and the learning unit 41 B.

[0043] The learning information acquisition unit 41A according to this embodiment acquires the learning information, which is a combination of the camera image, subject information, and positioning image, by reading it from a learning information database 49 (see also FIG. 4 ), which will be described later. However, this is not limiting, and for example, the learning information database 49 may be registered in the hospital information system 5, and the learning information may be acquired by downloading it from the hospital information system 5.

[0044] The learning unit 41B according to this embodiment then learns the learning model 48 using the camera image and subject information in the acquired learning information as input information and the positioning image in the acquired learning information as output information.

[0045] In this embodiment, in order to generate a virtual positioning image (hereinafter referred to as a "virtual positioning image") from a camera image, the camera image and the positioning image must be images in which at least a portion of the image overlaps. In other words, the camera image and the positioning image must be images that can be aligned. If this condition is met, the camera image may be captured from the front (AP (Anterior-Posterior View) direction), from the side (LAT (Lateral) direction), or from another direction.

[0046] Next, a functional configuration of the console 4 according to this embodiment when the control process is executed (operation phase) will be described with reference to Fig. 3. Fig. 3 is a block diagram showing an example of the functional configuration of the console 4 according to this embodiment when the control process is executed.

[0047] 3, the console 4 during execution of the control process includes an acquisition unit 41C, a generation unit 41D, and a display control unit 41E. Each of these blocks corresponds to a block executed by the control program when the above-described image processing is performed. The console 4 during execution of the control process also includes an examination control unit 41G, an irradiation control unit 41H, a noise reduction unit 41I, and a bed control unit 41J. Each of these blocks corresponds to a block executed by the control program when processing other than the above-described image processing is performed.

[0048] The CPU 41 of the console 4 executes the control program to function as an acquisition unit 41C, a generation unit 41D, a display control unit 41E, an examination control unit 41G, an irradiation control unit 41H, a noise removal unit 41I, and a bed control unit 41J.

[0049] The acquisition unit 41C according to this embodiment acquires camera images and subject information corresponding to an actual subject (hereinafter referred to as "actual subject"). Note that in this embodiment, the subject information is acquired by reading it from the patient information management system 51 of the hospital information system 5, but this is not limiting. For example, the subject information may be acquired by having the user of the console 4 input the information via the input unit 45.

[0050] As described above, in this embodiment, the subject information includes the subject's sex, height, weight, age, and body fat percentage, etc. However, the present invention is not limited to this, and one type of information or a combination of two or more types of information may be used as the subject information.

[0051] Furthermore, the generator 41D according to the present embodiment generates a virtual positioning image corresponding to the actual subject by inputting the acquired camera image and the acquired subject information into the learning model 48. Then, the display controller 41E according to the present embodiment displays the virtual positioning image generated by the generator 41D on the display unit 46.

[0052] On the other hand, the examination control unit 41G controls the examination sequence of the subject by the medical diagnostic device.

[0053] The irradiation control unit 41H controls the amount of X-rays irradiated onto the subject from the X-ray tube assembly 13. For example, the shoulders require a larger dose, the lungs are filled with air and therefore have little attenuation, so a smaller dose is sufficient, and the liver has a large attenuation and therefore requires a larger dose. The irradiation control unit 41H controls the amount of X-rays irradiated onto the subject depending on the part of the subject.

[0054] When the amount of noise in an examination image obtained by irradiating a subject with X-rays exceeds the allowable amount for the desired amount of noise, the noise removal unit 41I performs noise removal (denoising) processing on the examination image. For example, if the dose modulation based on the subject's past dose data does not result in a desired image quality index (e.g., image noise) due to an increase in the subject's weight, and the image noise increases, the noise removal unit 41I increases the intensity level of the denoising processing to suppress the image noise and performs automatic reconstruction so that the desired image quality index is the same as that of the previous examination.

[0055] The bed control unit 41J controls the height of the bed 3. The bed control unit 41J may control the height of the bed 3 based on an operation by a technician or the like, or may control the height of the bed 3 based on data from the most recent examination of the subject.

[0056] Next, the learning information database 49 according to this embodiment will be described with reference to Fig. 4. Fig. 4 is a schematic diagram showing an example of the configuration of the learning information database 49 according to this embodiment.

[0057] The learning information database 49 according to this embodiment is a database in which the various types of learning information described above are registered. As shown in Fig. 4, the learning information database 49 according to this embodiment stores information such as image IDs (Identifications), camera images, subject information, and positioning images in association with each other.

[0058] The image ID is information that is assigned in advance as a unique ID in order to individually identify information such as camera images extracted in advance from past data in order to learn the learning model 48, and the camera image is information that indicates the camera image itself. The subject information is information that indicates the above-mentioned subject information itself regarding the subject when the corresponding camera image was obtained, and the positioning image is information that indicates the scanogram image itself that was displayed when the actual imaging of the subject was performed.

[0059] Next, the operation of the console 4 will be described.

[0060] First, the operation of the console 4 when executing the learning process according to this embodiment will be described with reference to Fig. 5. Fig. 5 is a flowchart showing an example of the flow of the learning process according to this embodiment.

[0061] The CPU 41 of the console 4 executes the learning program, thereby executing the learning process shown in Fig. 5. The learning process shown in Fig. 5 is executed when a command to start execution of the learning program is input by the user of the console 4 via the input unit 45. Note that, in order to avoid confusion, the following description will be given of a case where the number of pieces of information required to sufficiently learn the learning model 48 is registered in the learning information database 49.

[0062] In step S100, the CPU 41 reads all information from the learning information database 49 (hereinafter referred to as "learning information").

[0063] In step S102, the CPU 41 uses the camera image and subject information in the read learning information as input information and the read positioning image as output information (correct answer information) to machine-train the learning model 48, and then terminates this learning process.

[0064] Through the above learning process, the learning model 48 is learned.

[0065] Next, the operation of the console 4 when executing the control process according to this embodiment will be described with reference to Fig. 6. Fig. 6 is a flowchart showing an example of the flow of the control process according to this embodiment.

[0066] The control process shown in Fig. 6 is executed by the CPU 41 of the console 4 executing the control program. The control process shown in Fig. 6 is executed when a command to start execution of the control program is input by a user of the console 4 via the input unit 45. Note that, in order to avoid confusion, the following description will be given assuming that the learning model 48 has already been trained, the target subject (hereinafter referred to as the "target subject") has been specified, and the target subject is correctly positioned at the imaging preparation position.

[0067] In step S200, the CPU 41 acquires the subject information corresponding to the target subject by reading it from the patient information management system 51 of the hospital information system 5.

[0068] In step S202, the CPU 41 moves the bed 3 to a predetermined position where the camera 7 captures an optical image.

[0069] In step S204, the CPU 41 controls the camera 7 to perform optical photography, and acquires a camera image obtained by the optical photography.

[0070] In step S206, the CPU 41 inputs the acquired camera image and the read-out subject information to the learning model 48. When the camera image and the subject information are input, the learning model 48 generates a virtual positioning image according to the camera image and the subject information. Then, the CPU 41 acquires the virtual positioning image generated by the learning model 48.

[0071] In step S208, the CPU 41 controls the display of the acquired virtual positioning image on the display unit 46, and determines the shooting range by accepting a shooting range specification from the user for the displayed virtual positioning image.

[0072] In step S210, the CPU 41 applies the determined imaging range to perform the main examination on the subject, and then ends this control process.

[0073] FIG. 7 is a schematic diagram illustrating an example of the overall flow of learning processing and image processing by the console 4 according to this embodiment.

[0074] As shown in Figure 7, in the console 4 of this embodiment, the learning model 48 is pre-trained using the camera image 70 obtained by photographing with the camera 7 and subject information regarding the subject as input information, and a positioning image (scanogram image), which is an image displayed to accept the photographing range for the actual photographing, as output information.

[0075] In addition, in the console 4 according to this embodiment, a camera image 70 and subject information of an actual subject are input into a pre-trained learning model 48, thereby generating a virtual positioning image 72 corresponding to the actual subject.

[0076] As described above, according to this embodiment, a camera image of the subject obtained by photographing the subject with an optical camera and subject information about the subject are used as input information, and a positioning image, which is an image displayed to accept the photographing range for the actual photographing, is used as output information to train a learning model in advance. By inputting the camera image and subject information about the actual subject to the training model, a virtual positioning image corresponding to the actual subject is generated. Therefore, compared to conventional techniques that automatically set the photographing range for the actual photographing, the accuracy of setting the photographing range for the actual photographing can be improved.

[0077] Furthermore, according to this embodiment, the learning model is trained in advance using camera images of the subject captured by an optical camera and subject information relating to the subject as input information, and the positioning image as output information, so that the learning model can be trained using existing information.

[0078] [Second embodiment] However, since the organs or tissues shown in the positioning image differ depending on the examination, depending on the gradation of the pixel values ​​or brightness values ​​of the virtual positioning image, it may be difficult to plan a scan to determine the imaging range during actual imaging.

[0079] Therefore, in this embodiment, in addition to the camera image, subject information, and positioning image, examination information regarding the examination content for the subject is also used as learning information, and by inputting the camera image, subject information, and examination information, a virtual positioning image with an appropriate style is generated according to the examination information such as the examination area or the organ to be examined.

[0080] In other words, the learning model 48 in this embodiment is pre-trained with camera images, subject information, and test information as input information and positioning images as output information, and by inputting camera images, subject information, and test information of an actual subject into the learning model 48, a virtual positioning image corresponding to the actual subject is generated.

[0081] In this embodiment, the examination information applied includes the region of the subject to be examined, the organ to be examined, the scan protocol to be applied in the examination, etc. Therefore, the learning information database 49 according to this embodiment differs from the learning information database 49 according to the first embodiment only in that it includes examination information, as shown in FIG.

[0082] In this embodiment, the examination information includes the region to be examined, the organ to be examined, the scan protocol to be applied in the examination, and the like, but is not limited to this. For example, one or a combination of these pieces of information may be applied as the examination information. Also, in this embodiment, the examination information is acquired by reading it from the patient information management system 51 of the hospital information system 5, but is not limited to this. For example, the examination information may be acquired by having the user of the console 4 input the information via the input unit 45.

[0083] The learning process according to this embodiment differs from the first embodiment only in that test information is applied in addition to the camera image and subject information as input information to the learning model 48 in the learning information, and therefore a description thereof will be omitted here. Also, the control process according to this embodiment differs from the first embodiment only in that test information of the actual subject is applied in addition to the camera image and subject information of the actual subject as information to be input to the learning model 48 in the processing of step S206, and therefore a description thereof will be omitted here.

[0084] 9 is a schematic diagram illustrating an example of the overall flow of learning processing and image processing by the console 4 according to this embodiment. As shown in Fig. 9, the overall flow by the console 4 according to this embodiment differs from the first embodiment only in that test information is added to the input information for learning information (referred to as "example data" in Fig. 9).

[0085] 10 shows a specific example of a virtual positioning image 72 according to this embodiment. The left half of FIG. 10 is an example of a virtual positioning image 72 obtained when the learning model 48 is trained without using test information, and the right half is an example of a virtual positioning image 72 obtained when the learning model 48 is trained using test information.

[0086] As shown in FIG. 10, when a virtual positioning image 72 is generated by applying a learning model 48 that has been learned including the examination information, the image is clearer and easier to see.

[0087] As described above, according to this embodiment, a pre-trained model is applied in which the camera image, subject information, and examination information related to the examination content of the subject are used as input information for the learning model, and the positioning image is used as output information, and the camera image, subject information, and examination information of the actual subject are input to the learning model to generate a virtual positioning image corresponding to the actual subject. Therefore, compared to a case in which the examination image is not included in the input information for the learning model, the virtual positioning image can be made clearer and easier to see, thereby further improving the accuracy of setting the imaging range for the actual imaging.

[0088] [Third embodiment] Incidentally, the virtual positioning image 72 illustrated in Figures 7 and 9 is an image including the entire body of the actual subject. In this case, the area to be examined (hereinafter referred to as the "area to be examined") or the organ to be examined (hereinafter referred to as the "organ to be examined") may be relatively small, making it difficult to specify the imaging range during actual imaging.

[0089] Therefore, in this embodiment, the user can specify the area to be displayed as the virtual positioning image.

[0090] The functional configuration of the console 4 according to this embodiment when the learning process is being executed is the same as that according to the first embodiment shown in Fig. 2, so a description thereof will be omitted here, and the functional configuration of the console 4 according to this embodiment when the control process is being executed will be described with reference to Fig. 11. Fig. 11 is a block diagram showing an example of the functional configuration of the console 4 according to this embodiment when the control process is being executed, and blocks that perform the same or approximately the same processes as those in Fig. 3 are assigned the same reference numerals as those in Fig. 3.

[0091] As shown in FIG. 11, the console 4 during execution of the control process differs from the functional configuration of the first embodiment only in that a reception unit 41F has been added and in the processing content of a display control unit 41E.

[0092] The receiving unit 41F according to the present embodiment receives a designation of a display target area of ​​the virtual positioning image, and the display control unit 41E according to the present embodiment displays the virtual positioning image on the display unit 46, with the display target area received by the receiving unit 41F as the display target.

[0093] Here, the receiving unit 41F according to the present embodiment receives a selection specification from a plurality of preset display target regions from the user, thereby receiving the specification of the display target region.

[0094] Next, the operation of the console 4 when executing the control process according to this embodiment will be described with reference to Fig. 12. Fig. 12 is a flowchart showing an example of the flow of the control process according to this embodiment, and steps that perform the same or substantially the same processes as the control process according to the first embodiment shown in Fig. 6 are assigned the same step numbers as in Fig. 6.

[0095] As shown in FIG. 12, the control process according to this embodiment differs from the control process according to the first embodiment only in that the process of step S205 is added and the process content of step S206.

[0096] In step S205, the CPU 41 performs a process of acquiring from the user a range for generating a virtual positioning image (hereinafter referred to as a "generation range acquisition process"). Fig. 13 shows an example of a scan planning screen 80 that is displayed on the display unit 46 by the console 4 when the generation range acquisition process and the process of step S206 are performed. The generation range acquisition process and the process of step S206 according to this embodiment will be described below with reference to Fig. 13. An example of the above-mentioned scan protocol table is shown at the bottom of Fig. 13.

[0097] First, the CPU 41 displays the camera image 70 acquired by the processing of step S204 on the display unit 46, and also displays preset buttons 82, each displaying the name of a predetermined examination target region, on the display unit 46. In the example shown in Fig. 13, the preset buttons 82 displayed include a button displaying "Head" indicating the head, a button displaying "Chest" indicating the chest, and the like. When the camera image 70 and the preset buttons 82 are displayed, the user designates, via the input unit 45, the preset button 82 that corresponds to the range for which a desired virtual positioning image is to be generated.

[0098] When the user designates one of the preset buttons 82, the CPU 41 acquires an area including the range of the part corresponding to the designated preset button 82 as a display target area.

[0099] Then, in step S206, the CPU 41 extracts an image of the acquired display target area from the camera image 70, and inputs the extracted image and the subject information of the actual subject as input information into the learning model 48, thereby generating a virtual positioning image 72 corresponding to the extracted range.

[0100] The user specifies the final imaging range for the actual imaging via the input unit 45 on the virtual positioning image 72 displayed on the scan planning screen 80 .

[0101] As described above, according to this embodiment, the designation of the display target area of ​​the virtual positioning image is accepted, and the virtual positioning image is displayed on the display unit with the accepted display target area as the display target. Therefore, only the desired area can be set as the display target area of ​​the virtual positioning image.

[0102] In particular, according to this embodiment, the designation of the display target area is accepted by accepting a selection designation from a plurality of preset display target areas from the user, which improves user convenience compared to when the user is required to directly designate the display target area itself.

[0103] However, this is not limited to this form, and the display control unit 41E may display the camera image 70 on the display unit 46, and the reception unit 41F may receive specifications from the user regarding the displayed camera image, thereby accepting the specification of the display target area.

[0104] In this case, for example, the camera image 70 may be displayed on the scan planning screen 80, and the range in which the virtual positioning image 72 is to be displayed on the camera image 70 may be displayed by a line 74 as shown in FIG. 13, for example, and the user may specify the position by operating the line 74 with an input unit 45 such as a mouse and / or keyboard, thereby acquiring the display target area.

[0105] In addition, in the present embodiment, the case where the virtual positioning image 72 itself generated by the learning model 48 is within the range specified by the user has been described, but the present invention is not limited to this. For example, the learning model 48 may generate a virtual positioning image 72 including the entire body of the subject, and then extract an image within the range specified by the user from the virtual positioning image 72 and use it as the final virtual positioning image 72.

[0106] Furthermore, in the present embodiment, the case where the user is prompted to specify the display target area has been described, but the present invention is not limited to this. For example, a configuration may be adopted in which an inspection target portion is identified from inspection information, and an area including the identified inspection target portion is automatically applied as the display target area.

[0107] Fourth Embodiment In each of the above embodiments, a camera image 70 is obtained by a single camera 7, and a two-dimensional virtual positioning image 72 is generated using the camera image 70. However, it may be difficult to accurately set the imaging range for the actual imaging using the two-dimensional virtual positioning image 72. For example, if the organ to be examined is the lung field, part of the lung field is hidden behind other organs using only the virtual positioning image 72 based on the camera image 70 obtained by imaging in the AP direction, making it difficult to accurately set the imaging range for the actual imaging.

[0108] Therefore, in this embodiment, the learning information is at least one of camera images captured from multiple directions, subject information, and ray sum images created from axial images and three-dimensional images (ground truth data) created from the axial images. Then, in this embodiment, a three-dimensional virtual positioning image is generated by inputting the camera images obtained by capturing images of a real subject from multiple directions and the subject information of the real subject.

[0109] That is, the learning model 48 in this embodiment is a model that has been pre-trained using multiple camera images obtained by photographing the subject from multiple directions using the camera 7 and subject information as input information, and at least one of a raysum image and a three-dimensional image created from an axial image obtained by photographing the subject as output information.

[0110] The generating unit 41D according to this embodiment inputs a plurality of camera images of the actual subject and subject information into the learning model 48, thereby generating a three-dimensional virtual positioning image corresponding to the actual subject.

[0111] 14 is a schematic diagram showing an example of the arrangement position of the camera 7 in the medical diagnostic apparatus according to this embodiment, an image of a raysum image generated from an axial image, and an example of a state in which the raysum image is displayed on the display unit 46 as a virtual positioning image 72. Note that the example shown in Fig. 14 illustrates a case in which the cameras 7 are provided at five locations around the bed 3 (in the example shown in Fig. 14, the position in front of the subject 6 lying supine on the bed 3 is set to 0 (zero) degrees, and five locations at 0 degrees, ±45 degrees, and ±90 degrees).

[0112] In this configuration, the ray sum image used as training information is an image that matches the shooting direction of the camera image in the training information and can be aligned with the camera image. This configuration improves the accuracy of setting the shooting range during actual imaging without increasing the X-ray exposure dose that would be required to create a three-dimensional positioning image.

[0113] As described above, according to this embodiment, a pre-trained model is applied using, as input information, multiple camera images obtained by photographing the subject from multiple directions with an optical camera and subject information, and at least one of a raysum image and a three-dimensional image created from axial images obtained by photographing the subject as output information, and the multiple camera images of the actual subject and the subject information are input to the training model to generate a three-dimensional virtual positioning image corresponding to the actual subject. Therefore, the accuracy of setting the imaging range for the actual imaging can be further improved compared to when a three-dimensional virtual positioning image is not generated.

[0114] Although the embodiments of the present disclosure have been described in detail above with reference to the accompanying drawings, the technical scope of the present disclosure is not limited to such examples. It is clear that a person skilled in the art of the present disclosure can conceive of various modifications or alterations within the scope of the technical idea described in the claims, and it is understood that these modifications or alterations also naturally fall within the technical scope of the present disclosure.

[0115] Furthermore, the effects described in the above embodiments are explanatory or exemplary and are not limited to those described in the above embodiments. In other words, the technology according to the present disclosure may achieve other effects that are obvious to a person skilled in the art of the present disclosure from the description in the above embodiments, in addition to or instead of the effects described in the above embodiments.

[0116] For example, in the above embodiment, the image processing device according to the present disclosure is applied to an X-ray CT scanner, but the present disclosure is not limited to this. The image processing device according to the present disclosure may be applied to an MRI scanner or the like, as long as the scanner uses a scanogram image to set the imaging range.

[0117] In the above embodiment, the case where information previously obtained in the medical diagnostic device that actually operates the learning model 48 is applied as learning information for training the learning model 48 has been described, but the present invention is not limited to this form. For example, information obtained by a medical diagnostic device other than this medical diagnostic device may be applied as learning information used for training the learning model 48.

[0118] In each of the above embodiments, the following various processors can be used as the hardware structure of processing units that perform various processes, such as the learning information acquisition unit 41A, the learning unit 41B, the acquisition unit 41C, the generation unit 41D, the display control unit 41E, the reception unit 41F, the examination control unit 41G, the irradiation control unit 41H, the noise removal unit 41I, and the bed control unit 41J. As described above, the various processors include a CPU, which is a general-purpose processor that executes software (programs) and functions as various processing units, as well as dedicated electrical circuits, such as a programmable logic device (PLD), a processor whose circuit configuration can be changed after manufacture, such as an FPGA (Field Programmable Gate Array), and an ASIC (Application Specific Integrated Circuit), which is a processor having a circuit configuration specifically designed to perform specific processes.

[0119] A single processing unit may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, multiple processing units may be configured with a single processor.

[0120] Examples of configuring multiple processing units with a single processor include: first, a form in which one processor is configured with a combination of one or more CPUs and software, and this processor functions as multiple processing units, as typified by computers such as client and server; second, a form in which a processor is used to realize the functions of an entire system including multiple processing units with a single IC (Integrated Circuit) chip, as typified by systems on chips (SoCs); and thus, various processing units are configured using one or more of the above-mentioned various processors as a hardware structure.

[0121] Furthermore, more specifically, the hardware structure of these various processors can be an electric circuit that combines circuit elements such as semiconductor elements.

[0122] Furthermore, in each of the above embodiments, the learning program and the control program are pre-stored (installed) in the ROM 42 or the storage 44 of the console 4, but the present invention is not limited to this.

[0123] The above programs may be provided in a form recorded on a recording medium such as a CD-ROM (Compact Disc Read Only Memory), a DVD-ROM (Digital Versatile Disc Read Only Memory), a USB (Universal Serial Bus) memory, etc. Furthermore, the above programs may be provided in a form downloaded from an external device via a network.

[0124] From the above description, the invention described in the following appendix can be understood.

[0125] [Supplementary Note 1] An image processing device comprising at least one processor, wherein the processor takes as input information a camera image obtained by photographing a subject with an optical camera and subject information about the subject, and generates a virtual positioning image corresponding to the actual subject by inputting the camera image and subject information about the actual subject into a learning model that has been trained in advance and uses as output information a positioning image that is displayed to accept the photographing range of the actual photographing.

[0126] [Appendix 2] The learning model is a model that has been trained in advance using the camera image, the subject information, and test information regarding the test content for the subject as input information and the positioning image as output information, and the processor generates the virtual positioning image corresponding to the actual subject by inputting the camera image, the subject information, and the test information of the actual subject into the learning model. This is the image processing device described in Appendix 1.

[0127] [Supplementary Note 3] The image processing device according to Supplementary Note 1 or Supplementary Note 2, wherein the processor accepts designation of a display target area of ​​the virtual positioning image, and displays the virtual positioning image on a display unit with the accepted display target area as the display target.

[0128] [Supplementary Note 4] The image processing device according to Supplementary Note 3, wherein the processor displays the camera image on the display unit, and accepts a designation of the display target area by accepting a designation from a user for the displayed camera image.

[0129] [Supplementary Note 5] The image processing device according to Supplementary Note 3, wherein the processor accepts the designation of the display target area by accepting, from a user, a selection designation from a plurality of preset display target areas.

[0130] [Appendix 6] The learning model is a model that has been trained in advance using multiple camera images obtained by photographing the subject from multiple directions using an optical camera and the subject information as input information, and at least one of a raysum image and a three-dimensional image created from an axial image obtained by photographing the subject as output information, and the processor generates a three-dimensional virtual positioning image corresponding to the actual subject by inputting the multiple camera images and the subject information of the actual subject into the learning model.

[0131] [Supplementary Note 7] The image processing device according to any one of Supplementary Note 1 to Supplementary Note 6, wherein the processor pre-learns the learning model using a camera image obtained by photographing the subject with an optical camera and subject information about the subject as input information, and the positioning image as output information.

[0132] [Supplementary Note 8] A medical image capturing device including: the image processing device according to any one of Supplementary Note 1 to Supplementary Note 7; and a radiographic image capturing device that performs actual imaging within an imaging range set by a virtual positioning image generated by the image processing device.

[0133] [Supplementary Note 9] An image processing method in which a processor takes a camera image obtained by photographing a subject with an optical camera and subject information about the subject as input information, and generates a virtual positioning image corresponding to the actual subject by inputting the camera image and subject information about the actual subject into a learning model that has been trained in advance and uses a positioning image, which is an image to be displayed to accept the shooting range of the actual photograph, as output information.

[0134] [Supplementary Note 10] An image processing program that causes a computer to execute a process of generating a virtual positioning image corresponding to an actual subject by inputting a camera image obtained by photographing the subject with an optical camera and subject information about the subject into a learning model that has been trained in advance and that uses a positioning image, which is an image to be displayed to accept the photographing range of the actual photographing, as output information.

[0135] In the above, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed by connecting them with "and / or."

Claims

1. An image processing device comprising a processor, which takes as input information a camera image obtained by photographing a subject with an optical camera and subject information about the subject, and outputs as output information a positioning image, which is an image displayed to accept the photographing range of the actual photograph, to a pre-trained learning model, thereby generating a virtual positioning image corresponding to the actual subject.

2. The image processing device described in claim 1, wherein the learning model is a model that has been trained in advance using the camera image, the subject information, and test information regarding the test content for the subject as input information and the positioning image as output information, and the processor generates the virtual positioning image corresponding to the actual subject by inputting the camera image, the subject information, and the test information of the actual subject into the learning model.

3. An image processing device according to claim 1 or claim 2, wherein the processor accepts designation of a display target area of ​​the virtual positioning image, and displays the virtual positioning image on a display with the accepted display target area as the display target.

4. The image processing device according to claim 3, wherein the processor displays the camera image on the display and accepts a user's specification for the displayed camera image, thereby accepting the specification of the display target area.

5. The image processing device according to claim 3, wherein the processor accepts the designation of the display target area by accepting a selection designation from a plurality of preset display target areas from a user.

6. The image processing device described in claim 1, wherein the learning model is a model that has been trained in advance using multiple camera images obtained by photographing the subject from multiple directions with an optical camera and the subject information as input information, and at least one of a raysum image and a three-dimensional image created from an axial image obtained by photographing the subject as output information, and wherein the processor generates a three-dimensional virtual positioning image corresponding to the actual subject by inputting the multiple camera images of the actual subject and the subject information into the learning model.

7. The image processing device described in claim 1, wherein the processor pre-learns the learning model using camera images obtained by photographing the subject with an optical camera and subject information regarding the subject as input information, and the positioning image as output information.

8. A medical imaging device comprising: the image processing device according to claim 1; and a radiographic imaging device that performs actual imaging within an imaging range set by a virtual positioning image generated by said image processing device.

9. An image processing method including generating a virtual positioning image corresponding to an actual subject by inputting a camera image obtained by photographing the subject with an optical camera and subject information about the subject into a learning model that has been trained in advance using as input information a positioning image, which is an image displayed to accept the shooting range of the actual photograph, and the camera image and subject information about the actual subject.

10. An image processing program for causing a computer to execute a process including generating a virtual positioning image corresponding to an actual subject by inputting a camera image obtained by photographing the subject with an optical camera and subject information about the subject into a learning model that has been pre-trained using a positioning image, which is an image displayed to accept the shooting range of the actual photograph, as output information.

Images