Information processing device, information processing method, and information processing program

Abstract

This information processing device is provided with at least one processor that performs: a process for acquiring at least one optical image obtained by optically capturing an image of a subject; extracting feature points of the subject according to the optical image; specifying, on the basis of the feature points, a target imaging region in the optical image that is a target when radiographic imaging is performed on the subject from a direction substantially the same as the imaging direction of the optical imaging; generating a superimposed image obtained by superimposing the target imaging area on the optical image; and performing control to display the superimposed image on a display.

Description

Technical Field

[0001] The present invention relates to an information processing device, an information processing method and an information processing program. Background Art

[0002] Conventionally, there are known technologies for assisting positioning in radiography based on optical images obtained by optically imaging a subject. For example, Japanese Patent Application Laid-Open No. 2014-117368 discloses a method for guiding a radiograph using an optical image of the subject and imaging conditions at a past time point so that the radiograph can be reproduced at the current time point under the same imaging conditions and positioning as the past radiograph. Summary of the Invention

[0003] Technical issues to be solved by the invention

[0004] In recent years, the demand for obtaining high-quality radiographic images that accurately capture the imaging area of ​​the subject has increased significantly in radiography. Consequently, the alignment of the radiation source, radiation detector, and subject has become crucial. Furthermore, the subject's posture must be precisely positioned as pre-determined using guide wires or other methods.

[0005] The present invention provides an information processing device, an information processing method, and an information processing program capable of supporting high-quality radiography.

[0006] Means for solving technical problems

[0007] The first embodiment of the present invention is an information processing device comprising at least one processor, which performs the following processing: acquiring at least one optical image obtained by optically photographing a subject; extracting feature points of the subject based on the optical image; determining, based on the feature points, a target photographic area in the optical image that serves as a target when performing radiographic photography of the subject from a direction approximately the same as the photographic direction of the optical photography; generating an overlapping image obtained by overlapping the target photographic area on the optical image; and controlling the display of the overlapping image on a display.

[0008] In the first aspect, the processor may perform the following processing: extracting a plurality of feature points of the subject based on the optical image; and determining the target imaging region based on the relative positional relationship of the plurality of feature points.

[0009] In the first aspect, the processor may perform the following processing: determining at least one predetermined reference feature point among the plurality of feature points; and determining the target imaging area based on the reference feature point.

[0010] In the first aspect, the processor may perform the following processing: acquiring imaging part information indicating an imaging part of radiography; and determining a reference feature point corresponding to the imaging part information.

[0011] In the first aspect, the processor may generate an overlapped image using a learned model that has been previously learned to take at least one of an optical image and a feature point as input and to output a target photographic area or an overlapped image.

[0012] In the first aspect, the processor may perform processing such that, when a feature point other than a predetermined feature point among the plurality of feature points is included in the target photographic area or is located within a predetermined range from the target photographic area, a warning is issued.

[0013] In the first aspect, the processor may determine whether the subject is located at a predetermined position based on the positional relationship between the plurality of feature points, and issue a warning if it is determined that the subject is not located at the predetermined position.

[0014] In the above-mentioned first method, the processor can perform the following processing: obtain at least one optical image for determination obtained by optically photographing the subject from a direction different from the photographing direction of the optical image; extract multiple feature points for determination of the subject based on the optical image for determination; determine whether the subject is in a predetermined position based on the positional relationship of the multiple feature points for determination; and issue a warning if it is determined that the subject is not in the predetermined position.

[0015] In the first aspect described above, the optical image may be at least one of a visible light image and a distance image indicating the distance to the subject.

[0016] The second embodiment of the present invention is an information processing method, which includes the following processing: acquiring at least one optical image obtained by optically photographing a subject; extracting feature points of the subject based on the optical image; determining, based on the feature points, a target photographic area in the optical image that becomes the target when performing radiographic photography of the subject from a direction approximately the same as the photographic direction of the optical photography; generating an overlapping image obtained by overlapping the target photographic area on the optical image; and controlling the display of the overlapping image on a display.

[0017] The third embodiment of the present invention is an information processing program that causes a computer to perform the following processing: acquiring at least one optical image obtained by optically photographing a subject; extracting feature points of the subject based on the optical image; determining, based on the feature points, a target photographic area in the optical image that serves as a target when performing radiographic photography of the subject from a direction approximately the same as the photographic direction of the optical photography; generating an overlapping image obtained by overlapping the target photographic area on the optical image; and controlling the display of the overlapping image on a display.

[0018] Effects of the Invention

[0019] According to the above aspects, the information processing device, information processing method, and information processing program of the present invention can support high-quality radiography. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 This is a diagram showing an example of a schematic configuration of a photography system.

[0021] Figure 2 This is a schematic diagram showing an example of how a photographic device is used.

[0022] Figure 3 This is a block diagram showing an example of the hardware configuration of a console.

[0023] Figure 4 This is a block diagram showing an example of the functional structure of the console.

[0024] Figure 5 This is a diagram showing an example of an optical image.

[0025] Figure 6 This is a diagram showing an example of feature points.

[0026] Figure 7 This is a diagram showing an example of a target photography area.

[0027] Figure 8 This is a diagram showing an example of an optical image for determination.

[0028] Figure 9 This is an example of a screen displayed on a monitor.

[0029] Figure 10 This is a flowchart showing an example of information processing. DETAILED DESCRIPTION

[0030] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. Figure 1 , the structure of the photographic system 1 is described. Figure 1 1 is a diagram showing a schematic configuration of the photographic system 1. Figure 1As shown, the imaging system 1 includes an imaging device 10 and a console 50. The imaging device 10 and the console 50, and the console 50 and an external RIS (Radiology Information System) 6 are configured to be connectable via a wired or wireless network.

[0031] The console 50 receives imaging instructions from the RIS 6 and controls the imaging device 10 according to the received imaging instructions and user instructions. The imaging device 10 captures radiographic images of the subject H under the control of the console 50. The console 50 is an example of an information processing device of the present invention.

[0032] Next, refer to Figure 2 , the photographic device 10 is described. Figure 2 1 is a diagram showing a schematic structure of the photographing device 10. Figure 2 As shown in FIG, the imaging device 10 includes a radiation irradiation unit 12, a radiation detector 20, a first optical camera 26, and a second optical camera 28. Figure 2 , as an example, a case where radiography is performed on the chest of the subject H as the imaging part is shown.

[0033] The radiation irradiation unit 12 includes a radiation source 13 for irradiating radiation R such as X-rays. The radiation irradiation unit 12 also includes a collimator (not shown) and the like to change the irradiation field (by adjusting the angle) of the radiation R irradiated from the radiation source 13. Figure 2 The type of the radiation source 13 is not particularly limited, and for example, a hot cathode type or a cold cathode type radiation source can be appropriately applied.

[0034] For example, the radiation irradiation unit 12 may be a so-called ceiling-mounted irradiation unit that is held on a support suspended from the ceiling of the radiographic studio. The ceiling-mounted irradiation unit is mounted on a guide rail around the ceiling on a support that is extendable in the vertical direction (Z direction) via wheels, and is movable in the horizontal direction (X direction and Y direction) within the radiographic studio. By the horizontal movement and vertical extension of the support, the radiation irradiation unit 12 also translates in the horizontal and vertical directions. Furthermore, the radiation irradiation unit 12 may be rotatable around a rotation axis extending in the horizontal direction, or may be rotatable around a rotation axis extending in the vertical direction.

[0035] Furthermore, for example, the radiation irradiation unit 12 may be a portable irradiation unit. Portable irradiation units can be used, for example, for simple radiological examinations in medical facilities, radiological examinations during home medical treatment, radiological examinations outdoors, and for home visits to disaster-stricken areas or areas with limited medical resources. Furthermore, for example, the radiation irradiation unit 12 may be a fixed irradiation unit installed in an imaging studio.

[0036] The radiation detector 20 detects radiation R that has passed through the subject H at its detection surface 20A, generates a radiographic image based on the detected radiation R, and outputs image data representing the generated radiographic image. The radiation detector 20 may be, for example, a portable electronic cassette, or may be a device that is placed on a stand or carried by the subject H. Specifically, the radiation detector 20 may be movable to any position in the horizontal (X and Y) and vertical (Z) directions relative to the radiation irradiation unit 12. Furthermore, for example, the radiation detector 20 may be a fixed radiation detector that is located within a radiography table in a radiography studio.

[0037] The type of radiation detector 20 is not particularly limited. For example, it may be an indirect conversion radiation detector that converts radiation R into light and then converts the converted light into charge, or it may be a direct conversion radiation detector that directly converts radiation R into charge.

[0038] The first optical camera 26 and the second optical camera 28 are optical digital cameras that use visible light and are configured using, for example, a CMOS (Complementary Metal Oxide Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor. The first optical camera 26 and the second optical camera 28 are capable of capturing still and / or moving images.

[0039] The first optical camera 26 captures the irradiation field ( Figure 2 The wider area ( Figure 2 The range shown by the single dotted line is obtained, and the optical image 30 is generated. The field angle ω of the first optical camera 26 is stored in advance in the storage unit 52. Figure 2 As shown, the imaging direction of the optical imaging by the first optical camera 26 is substantially the same as the imaging direction of the radiographic imaging using the radiation irradiation unit 12 and the radiation detector 20. Here, the substantially same direction may include a deviation to the extent that the optical image 30 can be aligned with the radiographic image by performing image correction (geometric transformation) such as affine transformation and projective transformation on the optical image 30.

[0040] In addition, the position of the first optical camera 26 is not particularly limited, for example, Figure 2As shown, the first optical camera 26 can be mounted on a surface substantially flush with the radiation R irradiation opening of the radiation irradiation unit 12, or on a wall surface of the imaging room, etc. However, in order to identify the joint points described later, it is preferable to be able to optically image the entire subject H. Therefore, the first optical camera 26 is preferably mounted on a surface substantially flush with the radiation R irradiation opening and located below the radiation R irradiation opening. Furthermore, the optical axis Ao of the first optical camera 26 is preferably substantially parallel to the irradiation axis Ar of the radiation R irradiated from the radiation source 13.

[0041] Furthermore, the positional relationship between the radiation source 13 and the first optical camera 26 is preset. Figure 2 As shown, the positional relationship is represented by, for example, the Z-direction distance dz and the X-direction distance dx (not shown) between the irradiation axis Ar of the radiation R emitted from the radiation source 13 and the optical axis Ao of the first optical camera 26, and the Y-direction distance dy between the radiation source 13 and the first optical camera 26. The distances dx, dy, and dz representing these positional relationships, as well as the field of view angle ω of the first optical camera 26, are pre-stored in the storage unit 52.

[0042] The second optical camera 28 optically photographs the subject H from a direction different from the photographing direction of the optical image 30 based on the first optical camera 26, and generates a judgment optical image 34 (details will be described later) for judging whether the subject H is in a predetermined position. Figure 2 , an example is shown in which the first optical camera 26 takes an image from the back side of the subject H, whereas the second optical camera 28 takes an image from the upper side of the subject H's head.

[0043] The imaging device 10 may also include a control device (not shown) that controls the overall operation of the imaging device 10 based on instructions from the console 50 and the user. Specifically, the control device acquires image data representing a radiographic image generated by the radiation detector 20 and outputs it to the console 50. Furthermore, the control device acquires an optical image 30 of the subject H captured by the first optical camera 26 and an optical image 34 for determination of the subject H captured by the second optical camera 28, and outputs them to the console 50.

[0044] The control device includes, for example, a CPU (Central Processing Unit), ROM (Read Only Memory), RAM (Random Access Memory), a storage medium, an I / F (Interface) unit, and an operating unit (not shown). The control device transmits and receives various information to and from the console 50 via the I / F unit.

[0045] In radiography, it is desirable to obtain high-quality radiographic images that appropriately capture the imaging site of the subject H. Therefore, alignment of the radiation irradiation unit 12 (radiation source 13), radiation detector 20, and subject H is crucial. This alignment is particularly important when, as mentioned above, at least one of the radiation irradiation unit 12 and radiation detector 20 is movable. Furthermore, the subject H must be positioned as previously determined using guide wires or the like.

[0046] Therefore, the console 50 according to the present embodiment supports high-quality radiography using the optical image 30 obtained by the first optical camera 26 and the determination optical image 34 obtained by the second optical camera 28 .

[0047] First, refer to Figure 3 , an example of the hardware structure of the console 50 is described. Figure 3 As shown, the console 50 includes a CPU (Central Processing Unit) 51 , a nonvolatile storage unit 52 , and a memory 53 as a temporary storage area.

[0048] The console 50 includes a display 54 such as a liquid crystal display, an operation unit 55 such as a touch panel, a keyboard, and a mouse, and an interface (I / F) unit 56. The I / F unit 56 performs wired or wireless communication with the imaging device 10, the RIS 6, and other external devices.

[0049] The CPU 51 , the storage unit 52 , the memory 53 , the display 54 , the operation unit 55 , and the I / F unit 56 are connected to one another via a bus 58 such as a system bus and a control bus so as to be able to transmit and receive various information to and from one another.

[0050] The storage unit 52 is implemented using a storage medium such as an HDD (Hard Disk Drive), an SSD (Solid State Drive), or flash memory. The storage unit 52 stores an information processing program 57 for the console 50. The CPU 51 reads the information processing program 57 from the storage unit 52, expands it into the memory 53, and executes the expanded information processing program 57. The CPU 51 is an example of a processor of the present invention. For example, a personal computer, a server computer, a smartphone, a tablet terminal, a wearable terminal, etc. can be appropriately used as the console 50.

[0051] Next, refer to Figure 4 , an example of the functional structure of the console 50 is described. Figure 4As shown, the console 50 includes an acquisition unit 60, an extraction unit 61, a determination unit 62, a generation unit 63, a determination unit 64, and a control unit 65. When the CPU 51 executes the information processing program 57, the CPU 51 functions as each functional unit of the acquisition unit 60, the extraction unit 61, the determination unit 62, the generation unit 63, the determination unit 64, and the control unit 65.

[0052] (Determination of target photography area)

[0053] First, a method for determining a target imaging area for positioning the radiation irradiation unit 12 (radiation source 13), the radiation detector 20, and the subject H will be described. The target imaging area refers to an area including the part to be radiographed. For example, when imaging the chest, the area near the chest becomes the target imaging area; when imaging the knee joint, the area near the knee becomes the target imaging area; and when imaging the head, the area near the head becomes the target imaging area.

[0054] The acquisition unit 60 acquires at least one optical image 30 obtained by optically photographing the subject H using the first optical camera 26 . Figure 5 Shown in Figure 2 An example of an optical image 30 captured by the first optical camera 26. Figure 5 In the optical image 30 , the subject H is photographed from the back side.

[0055] The extraction unit 61 extracts feature points of the subject H based on the optical image 30 acquired by the acquisition unit 60. Figure 6 In the figure, black dots represent Figure 5 Multiple feature points P1L to P6L and P1R to P6R are extracted from the optical image 30. Feature points P1L to P6L and P1R to P6R correspond to the joints of the subject H: the ear, shoulder, elbow, wrist, waist, and knee, respectively. Hereinafter, when the multiple feature points P1L to P6L and P1R to P6R are not distinguished, they are simply referred to as "feature points P," and the feature points corresponding to each joint are referred to as "feature points of (joint name)." Known posture estimation techniques, etc., can be appropriately applied as a method for extracting feature points P (joints).

[0056] The identifying unit 62 identifies the target imaging region 90 in the optical image 30, which is the target when performing radiographic imaging of the subject H from a direction substantially the same as the imaging direction of optical imaging, based on the feature points P extracted by the extracting unit 61. Figure 7 In the figure, the solid rectangle represents the Figure 7 The target photographing area 90 is determined by the feature points P in the optical image 30 .

[0057] Specifically, if Figure 6As shown, the extraction unit 61 extracts multiple feature points P of the subject H from the optical image 30, and the determination unit 62 determines the target imaging area 90 based on the relative positional relationship of the multiple feature points P. For example, the determination unit 62 first determines at least one pre-set reference feature point from the multiple feature points P1L to P6L and P1R to P6R. When imaging the chest as in this embodiment, the determination unit 62 determines the shoulder feature points P2L and P2R and the waist feature points P5L and P5R as reference feature points. Based on the relative positional relationship of the multiple feature points P, it is possible to determine which feature point P is the shoulder feature points P2L and P2R, and which is the waist feature points P5L and P5R.

[0058] Furthermore, the feature points P used as reference feature points can be pre-set, arbitrarily set by the user, or can be reference feature points corresponding to the imaging site. For example, in head imaging, reference feature points such as the eye and ear are appropriate. In knee joint imaging, reference feature points such as the waist, knee, and ankle are appropriate. Therefore, the determination unit 62 can obtain imaging site information representing the imaging site for radiographic imaging, included in the imaging instruction received from the RIS 6, and determine the reference feature points corresponding to the obtained imaging site information. The types of reference feature points corresponding to the imaging site information can be pre-stored in the storage unit 52, for example.

[0059] Next, the determination unit 62 determines the target imaging area 90 based on the determined reference feature points. Figure 6 In the example of , the determination unit 62 calculates the distance d in the head-to-tail direction (Z direction) between the midpoint of the line segment (illustrated by a dotted line) connecting the characteristic points P2L and P2R of the shoulders and the midpoint of the line segment (illustrated by a dotted line) connecting the characteristic points P5L and P5R of the waist. Furthermore, the determination unit 62 calculates the distance Zd obtained by multiplying the distance d by a predetermined coefficient, and determines a point Q (illustrated by an asterisk) separated from the midpoint of the line segment connecting the characteristic points P5L and P5R of the waist in the head direction by the distance Zd. The coefficient used to calculate the distance Zd can be arbitrarily set by the user, for example, anatomically and / or statistically. Furthermore, for example, it can also be derived by a machine learning model that is pre-learned by unsupervised learning using a combination of the learning optical image 30 and the target imaging area 90 as learning data. In addition, in the case of photographing the chest, point Q corresponds to the vertebra prominence.

[0060] Then, the determination unit 62 sets the determined point Q as the center of the upper side and determines a rectangular area of ​​a size corresponding to the detection surface 20A of the radiation detector 20 as the target imaging area 90. In addition, the size of the target imaging area 90 (i.e., the size corresponding to the detection surface 20A of the radiation detector 20) is obtained by geometric calculation using the distance between the radiation source 13 and the detection surface 20A of the radiation detector 20, i.e., the SID (Source to Image Receptor Distance) (refer to Figure 2 The SID value may be, for example, a value stored in the storage unit 52 or the like, on the premise that the imaging device 10 is used while ensuring a pre-set appropriate SID.

[0061] Furthermore, for example, the SID value can be measured using a measured value obtained by a distance measuring sensor such as LIDAR (Laser Imaging Detection and Ranging), a TOF (Time Of Flight) camera, or a stereo camera. LIDAR and TOF cameras are cameras that emit light such as infrared and visible light and measure the distance based on the time it takes to receive the reflected light or the phase change between the emitted light and the received light. LIDAR measures the distance to the object being measured by arranging multiple laser beam emitters in the vertical direction and scanning (rotating) each emitter horizontally. TOF cameras measure the distance to the object being measured by irradiating diffuse light. Stereo cameras use the principle of triangulation to measure the distance to the object being measured based on multiple images of the object being measured obtained by photographing the object from different directions.

[0062] The generating unit 63 generates a superimposed image 32 obtained by superimposing the target imaging area 90 determined by the determining unit 62 on the optical image 30. Figure 7 As shown by the middle dotted line, the generating unit 63 can further superimpose the irradiation field 92 of the radiation R emitted from the radiation irradiation unit 12 on the superimposed image 32. The irradiation field 92 of the radiation R is obtained by, for example, geometric calculation using the value of SID, the positional relationship (intervals dx, dy, and dz) between the radiation source 13 and the first optical camera 26 stored in the storage unit 52, and the angle of view ω of the first optical camera 26 (see FIG. Figure 2 ).

[0063] The control unit 65 controls the display 54 to display the superimposed image 32 generated by the generating unit 63 . Figure 9FIG. 5 shows an example of a screen D1 displayed on the display 54 by the control unit 65. The screen D1 includes a superimposed image 32 obtained by superimposing the target imaging area 90 and the irradiation field 92 of the radiation R on the optical image 30. Figure 9 In the example shown in FIG. 1 , the target imaging region 90 is offset from the irradiation field 92. If radiography is performed in this state, an appropriate radiographic image cannot be obtained. The user confirms the screen D1 and moves at least one of the radiation irradiation unit 12, the radiation detector 20, and the subject H so that the target imaging region 90 overlaps the irradiation field 92 for alignment.

[0064] Furthermore, the control unit 65 can perform the following control: in the overlapping image 32, the coordinates of the target photographic area 90 are compared with the coordinates of the illumination field 92, and when the difference between the coordinates is greater than a preset threshold value (that is, when the target photographic area 90 and the illumination field 92 deviate significantly), a warning is issued. Figure 9 The screen D1 includes a warning message indicating that the target photographic area 90 and the exposure field 92 are deviated.

[0065] If the feature point P cannot be extracted and the target imaging area 90 cannot be determined at this point in time, it is considered that the positions of the radiation irradiation unit 12, the radiation detector 20, and the subject H, as well as the posture and positioning of the subject H, are inappropriate. Therefore, the control unit 65 can promote appropriate alignment and positioning by, for example, performing control such as displaying a notification on the display 54.

[0066] (Determination of the appropriateness of positioning)

[0067] Next, a method for determining whether the posture of the subject H is positioned as previously set using guide wires or the like will be described.

[0068] The determination unit 64 determines whether the subject H is located in a predetermined position based on the positional relationship of the plurality of feature points P extracted by the extraction unit 61. Figure 6 In the example shown in FIG. 2 , if the length of the line segment connecting the shoulder feature points P2L and P2R is less than a predetermined threshold, the subject H may not be facing the radiation detector 20 but may be tilted. Furthermore, if the line segment connecting the shoulder feature points P2L and P2R is tilted at an angle greater than a predetermined threshold, the subject H may not be facing the front of the radiation detector 20 but may be tilted. Therefore, the determination unit 64 determines whether the subject H is in the predetermined position by determining whether the plurality of feature points P extracted by the extraction unit 61 maintain a predetermined positional relationship.

[0069] Furthermore, the suitability of positioning can be determined based on at least one determination optical image 34 obtained by optically photographing the subject H with the second optical camera 28 from a direction different from the photographing direction of the optical image 30. In this case, the acquisition unit 60 acquires the determination optical image 34 obtained by the second optical camera 28. Figure 8 Shown in Figure 2 An example of the determination optical image 34 captured by the second optical camera 28. Figure 8 In the determination optical image 34 , the subject H is photographed from the top of the head.

[0070] The extraction unit 61 extracts a plurality of feature points for determination of the subject H based on the optical image for determination 34 acquired by the acquisition unit 60. Figure 8 In the figure, black dots represent multiple determination feature points J1L to J3L and J1R to J3R extracted from the determination optical image 34. The determination feature points J1L to J3L and J1R to J3R correspond to the ear, shoulder, and elbow joints of the subject H, respectively. Hereinafter, when the multiple determination feature points J1L to J3L and J1R to J3R are not distinguished, they are simply referred to as "determination feature points J," and the determination feature points corresponding to the respective joints are referred to as "determination feature points of (joint name)."

[0071] Furthermore, as a method for extracting the determination feature points J (joint points), a well-known posture estimation technique or the like can be appropriately applied.

[0072] The determination unit 64 determines whether the subject H is in a predetermined position based on the positional relationship of the plurality of determination feature points J extracted by the extraction unit 61. For example, in chest imaging, a guide line or the like is used to define a posture such that the shoulders and elbows are as close as possible to the detection surface 20A of the radiation detector 20. Therefore, if the line segment connecting the determination feature points J2L (J2R) of the shoulder and the determination feature points J3L (J3R) of the elbow is tilted at an angle greater than a predetermined threshold, there is a possibility that the subject H has not brought the elbow sufficiently close to the detection surface 20A. Therefore, the determination unit 64 determines whether the subject H is in the predetermined position by determining whether the plurality of determination feature points J extracted by the extraction unit 61 maintain the predetermined positional relationship.

[0073] The control unit 65 performs control such that, when the determination unit 64 determines that the subject H is not positioned in a predetermined position, a warning is issued based on at least one of the feature point P and the determination feature point J. Figure 9 The screen D1 includes the following warning: Assuming that there is a possibility that the subject H has not brought his elbow sufficiently close to the detection surface 20A, a warning is issued to bring his elbow close to the radiation detector 20.

[0074] Furthermore, the determination unit 64 may determine whether feature points other than the predetermined feature points among the plurality of feature points P are included in the target imaging region 90 or are located within a predetermined range (i.e., close to) the target imaging region 90. For example, if an unnecessary portion overlaps with a portion desired for radiography, an appropriate radiographic image may not be obtained.

[0075] For example, in Figure 7 In the example shown in FIG, the fingertips of the subject H are excessively included in the target imaging area 90, which is an undesirable state. When imaging the chest, the determination unit 64 determines whether the feature points other than the feature points P2L and P2R of the shoulder and the feature points P5L and P5R of the waist are included in or close to the target imaging area 90. As a result, it is determined that the feature points P4L and P4R of the wrist are within a predetermined range from the target imaging area 90 (i.e., close to the target imaging area 90).

[0076] The control unit 65 performs control such that, when the determination unit determines that a feature point other than the predetermined feature point among the plurality of feature points P is included in the target imaging area 90 or is within a predetermined range from the target imaging area 90, a warning is issued. Figure 9 The screen D1 includes a warning message indicating that the right hand (feature point P4R of the wrist) and the left hand (feature point P4L of the wrist) may intrude into the target imaging area 90 .

[0077] Alternatively, instead of determining whether the feature point P is included in the target imaging area 90 or is located within a predetermined range from the target imaging area 90, the determination unit 64 may utilize known image recognition technology to determine whether the target imaging area 90 includes an unnecessary portion. For example, if the target imaging area 90 in the optical image 30 should only include the color of the examination gown, but the color of the skin is detected, it may be determined that the target imaging area 90 includes an unnecessary portion.

[0078] The above-described target imaging region determination process and positioning adequacy determination process are repeatedly performed each time the optical image 30 is updated.

[0079] Next, refer to Figure 10 The function of the console 50 according to this embodiment will be described. In the console 50, the CPU 51 executes the information processing program 57 to execute Figure 10 The information processing shown is executed when the user issues an instruction to start the execution via the operation unit 55, for example.

[0080] In step S10, the acquisition unit 60 acquires at least one optical image 30 obtained by optically photographing the subject H using the first optical camera 26. In step S12, the extraction unit 61 extracts characteristic points P of the subject H based on the optical image 30 acquired in step S10. In step S14, the determination unit 62 determines, based on the characteristic points P extracted in step S12, a target imaging region 90 in the optical image 30 that serves as a target when performing radiographic imaging of the subject H from a direction substantially identical to the imaging direction of the optical imaging.

[0081] In step S16, the generator 63 generates a superimposed image 32 by superimposing the target imaging area 90 determined in step S14 on the optical image 30 acquired in step S10. In step S18, the controller 65 controls the display 54 to display the superimposed image 32 generated in step S16, and terminates the information processing.

[0082] As described above, the console 50 involved in one embodiment of the present invention has at least one processor, which performs the following processing: obtaining at least one optical image 30 obtained by optically photographing the subject H; extracting the feature points P of the subject H based on the optical image 30; determining the target photographing area 90 in the optical image 30 that becomes the target when performing radiographic photography of the subject H from a direction approximately the same as the photographic direction of the optical photography; generating an overlapping image 32 obtained by overlapping the target photographing area 90 on the optical image 30; and controlling the display of the overlapping image 32 on the display 54.

[0083] That is, according to the console 50 of this embodiment, it is possible to determine the target imaging area 90 for aligning the radiation irradiation unit 12 (radiation source 13), the radiation detector 20, and the subject H. Therefore, the radiation irradiation unit 12 (radiation source 13), the radiation detector 20, and the subject H can be appropriately aligned, thereby supporting high-quality radiography.

[0084] Furthermore, in the above embodiment, the optical image 30 obtained by the first optical camera 26 and the determination optical image 34 obtained by the second optical camera 28 are described as visible light images, but the present invention is not limited thereto. For example, at least one of the optical image 30 obtained by the first optical camera 26 and the determination optical image 34 obtained by the second optical camera 28 may be a distance image indicating the distance to the subject H. In this case, LIDAR (Laser Imaging Detection and Ranging) cameras, TOF (Time of Flight) cameras, stereo cameras, and the like can be appropriately used as the first optical camera 26 and the second optical camera 28.

[0085] It is also possible to extract the feature points P and the determination feature points J from the range image. Therefore, it is also possible to determine the target imaging area 90 using these and determine whether the positioning is appropriate or not.

[0086] Furthermore, a combination of the first optical camera 26 and the second optical camera 28 may be a digital camera that obtains visible light images, and a three-dimensional camera that obtains range images. Furthermore, both a digital camera that obtains visible light images and a three-dimensional camera that obtains range images may be used as the first optical camera 26, and both the visible light image and the range image may be used to determine the target imaging area and determine whether positioning is appropriate.

[0087] Furthermore, in the above embodiment, the determination unit 62 determines the target imaging area 90 by using the relative positional relationship of the feature points P and a calculation formula of preset coefficients, and the generation unit 63 generates the image 32 , but the present invention is not limited thereto.

[0088] For example, the determination unit 62 may determine the target imaging area 90 using a learned model that has been previously learned by taking at least one of the optical image 30 and the feature points P as input and outputting the target imaging area 90. In this case, the generation unit 63 may generate the superimposed image 32 by superimposing the target imaging area 90 determined using the learned model on the optical image 30. In this case, the optical image used as input may be a visible light image, a range image, or both.

[0089] Furthermore, for example, the determination of the target imaging area 90 by the determination unit 62 and the generation of the superimposed image 32 can be performed integrally. For example, the generation unit 63 can generate the superimposed image 32 using a learned model that has been previously learned to take at least one of the optical image 30 and the feature points P as input and to output the superimposed image 32. In this case, the optical image used as input can be a visible light image, a range image, or both.

[0090] Furthermore, in each of the above-described embodiments, for example, various processors described below can be used as the hardware configuration of the processing units (processing units) that perform various processes, such as the acquisition unit 60, extraction unit 61, determination unit 62, generation unit 63, determination unit 64, and control unit 65. As described above, the various processors described above include general-purpose processors (CPUs) that execute software (programs) and function as various processing units, as well as processors (programmable logic devices (PLDs)) whose circuit configuration can be modified after manufacturing, such as FPGAs (Field Programmable Gate Arrays), and processors (special-purpose circuits) with circuit configurations specifically designed to perform specific processes, such as ASICs (Application Specific Integrated Circuits).

[0091] A processing unit may be composed of one of these various processors, or a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs or a combination of a CPU and an FPGA). Furthermore, multiple processing units may be composed of one processor.

[0092] Examples of multiple processing units composed of a single processor include the following: First, as exemplified by computers such as clients and servers, a processor is constructed by combining one or more CPUs and software, with the processor functioning as multiple processing units. Second, as exemplified by system-on-chips (SoCs), a processor is used that implements the functions of an entire system including multiple processing units on a single IC (Integrated Circuit). In this manner, various processing units are constructed using one or more of the aforementioned processors as a hardware configuration.

[0093] Furthermore, more specifically, as the hardware configuration of these various processors, a circuit (Circuitry) formed by combining circuit elements such as semiconductor elements can be used.

[0094] Furthermore, in the above embodiment, the information processing program 57 in the console 50 is pre-stored in the storage unit 52, but the present invention is not limited to this. The information processing program 57 can be provided by recording it on a recording medium such as a CD-ROM (Compact Disc Read Only Memory), a DVD-ROM (Digital Versatile Disc Read Only Memory), or a USB (Universal Serial Bus) memory device. Furthermore, the information processing program 57 can also be downloaded from an external device via a network. Furthermore, the technology disclosed herein relates not only to programs but also to storage media that non-temporarily store programs.

[0095] The technology of the present invention can also be appropriately combined with the above-mentioned embodiments and examples. The above-mentioned descriptions and illustrations are detailed descriptions of the parts involved in the technology of the present invention and are merely examples of the technology of the present invention. For example, the descriptions related to the above-mentioned structure, function, action, and effect are descriptions related to an example of the structure, function, action, and effect of the parts involved in the technology of the present invention.

[0096] Therefore, unnecessary parts may be deleted or new elements may be added or substituted for the above-described descriptions and drawings without departing from the technical spirit of the present invention.

[0097] Regarding the above-mentioned embodiment, the following supplementary notes are further disclosed.

[0098] [Supplementary Note 1]

[0099] An information processing device comprising at least one processor,

[0100] The above processor performs the following processing:

[0101] acquiring at least one optical image obtained by optically photographing the subject;

[0102] extracting feature points of the subject based on the optical image;

[0103] determining a target imaging region in the optical image that is a target when performing radiographic imaging of the subject from a direction substantially the same as the imaging direction of the optical imaging, based on the characteristic points;

[0104] generating an overlapping image obtained by overlapping the target imaging area on the optical image; and

[0105] Control is performed to display the superimposed image on a display.

[0106] [Supplementary Note 2]

[0107] The information processing device according to Supplementary note 1, wherein:

[0108] The above processor performs the following processing:

[0109] extracting a plurality of feature points of the subject based on the optical image; and

[0110] The target photography area is determined based on the relative position relationship of the plurality of feature points.

[0111] [Supplementary Note 3]

[0112] The information processing device according to Supplementary note 2, wherein:

[0113] The above processor performs the following processing:

[0114] Determining at least one pre-set reference feature point among the plurality of feature points; and

[0115] Based on the above-mentioned reference feature points, the above-mentioned target photography area is determined.

[0116] [Supplementary Note 4]

[0117] The information processing device according to Supplementary note 2 or Supplementary note 3, wherein:

[0118] The above processor performs the following processing:

[0119] acquiring imaging part information indicating an imaging part for the radiography; and

[0120] The reference feature points corresponding to the imaging part information are determined.

[0121] [Supplementary Note 5]

[0122] The information processing device according to any one of Supplementary Notes 2 to 4, wherein:

[0123] The above processor performs the following processing:

[0124] When a feature point other than the predetermined feature point among the plurality of feature points is included in the target imaging area or is located within a predetermined range from the target imaging area, a warning is issued.

[0125] [Supplementary Note 6]

[0126] The information processing device according to any one of Supplementary Notes 2 to 5, wherein:

[0127] The above processor performs the following processing:

[0128] Determining whether the subject is located at a predetermined position based on the positional relationship of the plurality of feature points; and

[0129] If it is determined that the subject is not in the preset position, a warning is issued.

[0130] [Supplementary Note 7]

[0131] The information processing device according to any one of Supplementary Notes 1 to 6, wherein:

[0132] The above processor performs the following processing:

[0133] The superimposed image is generated using a learned model that is previously learned so as to take at least one of the optical image and the feature points as input and to output the target imaging area or the superimposed image.

[0134] [Supplementary Note 8]

[0135] The information processing device according to any one of Supplementary Notes 1 to 7, wherein:

[0136] The above processor performs the following processing:

[0137] acquiring at least one determination optical image obtained by optically photographing the subject from a direction different from a photographing direction of the optical image;

[0138] extracting a plurality of feature points for determination of the subject based on the optical image for determination;

[0139] determining whether the subject is located at a predetermined position based on the positional relationship of the plurality of determination feature points; and

[0140] If it is determined that the subject is not in the preset position, a warning is issued.

[0141] [Supplementary Note 9]

[0142] The information processing device according to any one of Supplementary Notes 1 to 8, wherein:

[0143] The optical image is at least one of a visible light image and a distance image indicating a distance from the subject.

[0144] [Supplementary Note 10]

[0145] An information processing method, comprising the following steps:

[0146] acquiring at least one optical image obtained by optically photographing the subject;

[0147] extracting feature points of the subject based on the optical image;

[0148] determining a target imaging region in the optical image that is a target when performing radiographic imaging of the subject from a direction substantially the same as the imaging direction of the optical imaging, based on the characteristic points;

[0149] generating an overlapping image obtained by overlapping the target imaging area on the optical image; and

[0150] Control is performed to display the superimposed image on a display.

[0151] [Supplementary Note 11]

[0152] An information processing program that causes a computer to execute the following processing:

[0153] acquiring at least one optical image obtained by optically photographing the subject;

[0154] extracting feature points of the subject based on the optical image;

[0155] determining a target imaging region in the optical image that is a target when performing radiographic imaging of the subject from a direction substantially the same as the imaging direction of the optical imaging, based on the characteristic points;

[0156] generating an overlapping image obtained by overlapping the target imaging area on the optical image; and

[0157] Control is performed to display the superimposed image on a display.

[0158] The disclosure of Japanese Patent Application No. 2023-025466, filed on February 21, 2023, is hereby incorporated by reference in its entirety. All documents, patent applications, and technical standards described in this specification are hereby incorporated by reference to the same extent as if each individual document, patent application, or technical standard were specifically and individually indicated to be incorporated by reference.

Claims

1. An information processing device comprising at least one processor, The processor performs the following processing: acquiring at least one optical image obtained by optically photographing the subject; extracting feature points of the subject according to the optical image; determining, based on the feature points, a target imaging region in the optical image that is a target when radiographing the subject from a direction substantially the same as an imaging direction of the optical imaging; generating an overlapping image obtained by overlapping the target photographic area on the optical image; and Control is performed to display the superimposed image on a display.

2. The information processing device according to claim 1, wherein The processor performs the following processing: extracting a plurality of feature points of the subject based on the optical image; and The target photographing area is determined according to the relative position relationship of the plurality of feature points.

3. The information processing device according to claim 2, wherein: The processor performs the following processing: Determining at least one pre-set reference feature point among the plurality of feature points; and The target photographing area is determined based on the reference feature points.

4. The information processing device according to claim 3, wherein: The processor performs the following processing: acquiring imaging part information indicating an imaging part of the radiography; and The reference feature points corresponding to the imaging part information are determined. The information processing device according to claim 1 , wherein: The processor performs the following processing: The superimposed image is generated using a learned model that is previously learned so as to take at least one of the optical image and the feature points as input and to output the target imaging area or the superimposed image. The information processing apparatus according to claim 2 , wherein: The processor performs the following processing: When a feature point other than the predetermined feature point among the plurality of feature points is included in the target imaging area or is located within a predetermined range from the target imaging area, a warning is issued.

7. The information processing apparatus according to claim 2, wherein: The processor performs the following processing: determining whether the subject is located at a predetermined position based on the positional relationship of the plurality of feature points; and If it is determined that the subject is not in the preset position, a warning is issued.

8. The information processing apparatus according to claim 1, wherein: The processor performs the following processing: acquiring at least one determination optical image obtained by optically photographing the subject from a direction different from a photographing direction of the optical image; extracting a plurality of feature points for determination of the subject based on the optical image for determination; determining whether the subject is located at a predetermined position based on a positional relationship of the plurality of determination feature points; and If it is determined that the subject is not in the preset position, a warning is issued.

9. The information processing apparatus according to claim 1, wherein: The optical image is at least one of a visible light image and a distance image indicating a distance from the subject.

10. An information processing method, comprising the following processing: acquiring at least one optical image obtained by optically photographing the subject; extracting feature points of the subject according to the optical image; determining, based on the feature points, a target imaging region in the optical image that is a target when radiographing the subject from a direction substantially the same as an imaging direction of the optical imaging; generating an overlapping image obtained by overlapping the target photographic area on the optical image; and Control is performed to display the superimposed image on a display.

11. An information processing program that causes a computer to execute the following processing: acquiring at least one optical image obtained by optically photographing the subject; extracting feature points of the subject according to the optical image; determining, based on the feature points, a target imaging region in the optical image that is a target when radiographing the subject from a direction substantially the same as an imaging direction of the optical imaging; generating an overlapping image obtained by overlapping the target photographic area on the optical image; and Control is performed to display the superimposed image on a display.

Images