Image processing device, image processing method, and program

Abstract

To provide an image processing device, an image processing method, and a program that can support positioning of a subject performed when capturing a radiation image without performing irradiation with a radiation ray.SOLUTION: An image processing device includes at least one processor. The processor acquires an optical image of a subject, extracts a plurality of feature points of a subject's body on the basis of the optical image, generates a line segment determined on the basis of any one of the plurality of feature points, which is the information indicating the posture of the subject, as posture information, generates a line segment corresponding to the line segment as the posture information, which is the information for guiding the posture to be taken by the subject, as guide information, and displays the posture information and the guide information.SELECTED DRAWING: Figure 6

Description

[Technical field]

[0001] The disclosed technology relates to an image processing device, an image processing method, and a program. [Background technology]

[0002] The following techniques are known as techniques for estimating the posture of a subject from an image. For example, Patent Document 1 describes a technique in which a posture determination unit detects the position of the pixel located at the top in the vertical direction among the pixels corresponding to the human part for each pixel row in the vertical direction of a binary image, and detects the shoulder line or the center position of the body of the person being photographed based on the change when the detected pixel positions are plotted in the horizontal direction.

[0003] Patent document 2 describes an exercise menu providing system that includes an imaging means for imaging a user's posture, a specific part extraction means for extracting specific part images showing multiple specific parts from the user image captured by the imaging means, a specific part analysis means for analyzing the posture of the specific part corresponding to each specific part image based on the multiple specific part images extracted by the specific part extraction means, and an exercise menu providing means for providing an exercise menu corresponding to the posture of each specific part analyzed by the specific part analysis means.

[0004] Patent Document 3 describes that an analysis unit analyzes captured images obtained by a camera and detects skeletal information. In detecting skeletal information, each part of each person (head, shoulders, hands, feet, etc.) is recognized from the captured images, and the coordinate position of each part is calculated. The skeletal information includes information such as characteristic angles formed by two or more line segments connecting points on the skeleton. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2016-178531 A [Patent Document 2] JP 2017-047105 A [Patent Document 3] International Publication No. 2022 / 244298 Summary of the Invention [Problem to be solved by the invention]

[0006] A videofluoroscopic examination of swallowing is known as one of the examinations using radiation images. A videofluoroscopic examination of swallowing is an examination to evaluate the process and state of a subject swallowing a sample. In a videofluoroscopic examination of swallowing, a video consisting of a series of radiation images taken by continuously irradiating the subject with radiation is used.

[0007] When taking a radiological image, the subject is positioned so that the subject is in a posture specified by the radiography menu, etc. Currently, whether the subject is in the posture specified is confirmed by the radiological image. In other words, the subject is irradiated with radiation for positioning.

[0008] The disclosed technology has been made in consideration of the above-mentioned points, and aims to assist in positioning of a subject when capturing a radiological image, without irradiating the subject with radiation. [Means for solving the problem]

[0009] The image processing device according to the disclosed technology has at least one processor. The processor acquires an optical image of a subject, extracts a plurality of feature points of the subject's body based on the optical image, generates information indicating the posture of the subject, which is a line segment determined based on any one of the plurality of feature points, as posture information, generates information guiding the posture the subject should take, which is a line segment corresponding to the line segment as the posture information, as guide information, and displays the posture information and the guide information.

[0010] The processor may display the posture information and the guide information by superimposing them on the optical image. The processor may cause the posture information and the guide information to follow a change in posture of the subject. The processor may generate, as the guide information, a line segment having a starting point at a feature point close to a support part supporting the body of the subject, among feature points present at both ends of the line segment as the posture information.

[0011] The processor may set the direction of the line segment as the guide information based on designation information that designates a posture that the subject should take. The processor may acquire designation information included in an imaging menu for a radiographic image to be captured of the subject. The processor may acquire designation information input by operating an input means or by voice.

[0012] The processor may extract the subject's ears as first feature points and the subject's eyes as second feature points based on the optical image, and generate a line segment connecting the first feature point and the second feature point as posture information. The processor may extract the midpoints of the subject's left and right shoulder joints as third feature points and the midpoints of the subject's left and right greater trochanters as fourth feature points based on the optical image, and generate a line segment connecting the third feature point and the fourth feature point as posture information.

[0013] The processor may set the orientation of the line segment as the guide information based on the designation information specifying the posture the subject should take, acquire a series of radiographic images of the subject that are continuously captured with the guide of the guide information, and associate the designation information with the series of radiographic images. The processor may set the orientation of each of the line segments as each of the multiple pieces of guide information based on multiple pieces of designation information that respectively specify postures the subject should take, acquire a series of radiographic images that are continuously captured with the guide of each of the multiple pieces of guide information, and perform a process for each of the multiple pieces of designation information to associate one of the multiple pieces of designation information with a part of the series of radiographic images that are captured with the guide of the guide information generated based on the one piece of designation information.

[0014] The processor may identify a shooting direction of a radiological image taken of the subject based on at least a portion of the plurality of feature points, and may issue an alert if the identified shooting direction differs from a specified direction.

[0015] The image processing method related to the disclosed technology is a method in which at least one processor provided in the image processing device acquires an optical image of a subject, extracts multiple feature points of the subject's body based on the optical image, generates information indicating the subject's posture, which is a line segment determined based on any of the multiple feature points, as posture information, generates information guiding the posture the subject should take, which is a line segment corresponding to the line segment as the posture information, as guide information, and displays the posture information and the guide information.

[0016] The program related to the disclosed technology is a program for causing at least one processor provided in an image processing device to execute a process of acquiring an optical image of a subject, extracting multiple feature points of the subject's body based on the optical image, generating information indicating the subject's posture, which is a line segment determined based on any of the multiple feature points, as posture information, generating information guiding the posture the subject should take, which is a line segment corresponding to the line segment as the posture information, as guide information, and displaying the posture information and the guide information. Effect of the Invention

[0017] According to the disclosed technology, it is possible to assist in positioning of a subject when capturing a radiological image, without irradiating the subject with radiation. [Brief description of the drawings]

[0018] [Figure 1] 1 is a diagram illustrating an example of a configuration of a radiation image capturing system according to an embodiment of the disclosed technique. [Diagram 2] 1 is a diagram illustrating an example of a configuration of a radiological image capturing device according to an embodiment of the disclosed technique. [Diagram 3] FIG. 1 is a diagram illustrating an example of a hardware configuration of an image processing device according to an embodiment of the disclosed technique. [Figure 4] 1 is a functional block diagram illustrating an example of a functional configuration of an image processing device according to an embodiment of the disclosed technique. [Diagram 5] 11A to 11C are diagrams illustrating an example of a posture information generating process according to an embodiment of the disclosed technology. [Figure 6] FIG. 11 is a diagram illustrating an example of a guide information generating process according to an embodiment of the disclosed technology. [Figure 7] 11 is a flowchart showing an example of a flow of processing performed by a CPU according to an embodiment of the disclosed technique by executing an image processing program. [Figure 8] FIG. 13 is a diagram showing posture information and guide information when positioning is completed. [Figure 9] FIG. 11 is a functional block diagram showing an example of a functional configuration of an image processing device according to another embodiment of the disclosed technique. [Figure 10] 11 is a diagram illustrating an example of a structure of a data file generated by an association processing unit according to an embodiment of the disclosed technique. [Figure 11] 13 is a flowchart showing an example of a flow of processing performed by a CPU according to another embodiment of the disclosed technology by executing an image processing program. [Figure 12] 13 is a flowchart showing an example of a flow of processing performed by a CPU according to another embodiment of the disclosed technology by executing an image processing program. [Figure 13] FIG. 11 is a diagram illustrating an example of association processing according to an embodiment of the disclosed technique. [Figure 14] FIG. 11 is a functional block diagram showing an example of a functional configuration of an image processing device according to another embodiment of the disclosed technique. [Figure 15] 13 is a flowchart showing an example of a flow of processing performed by a CPU according to another embodiment of the disclosed technology by executing an image processing program. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0019] Hereinafter, an example of an embodiment of the disclosed technology will be described with reference to the drawings. In each drawing, the same or equivalent components and parts are given the same reference numerals, and duplicated descriptions will be omitted.

[0020] [First embodiment] 1 is a diagram showing an example of the configuration of a radiographic imaging system 1 according to an embodiment of the disclosed technique. In the following description, a case where the radiographic imaging system 1 is used for a swallowing radiography test will be described as an example. The radiographic imaging system 1 has an image processing device 10, a radiographic imaging device 20, and an optical camera 30. The optical camera 30 may be provided integrally with the radiographic imaging device 20, or may be independent from the radiographic imaging device 20.

[0021] The radiological image capturing device 20 is capable of capturing radiological images using radiation such as X-rays. It is also possible to display a series of radiological images captured by continuously irradiating radiation in a time series, thereby displaying the series of radiological images in sequence, thereby displaying a moving image. The capturing of a series of radiological images for displaying a moving image is called fluoroscopic imaging. In a swallowing contrast examination, fluoroscopic imaging is performed to observe the process in which a subject swallows a sample. Note that "continuous irradiation of radiation" includes both a case where radiation is continuously irradiated for a certain period of time, and a case where pulsed radiation is irradiated multiple times within a certain period of time. The radiological images captured by the radiological image capturing device 20 are immediately transmitted to the image processing device 10.

[0022] 2 is a diagram showing an example of the configuration of the radiographic image capturing device 20. The radiographic image capturing device 20 has an imaging table 21. The imaging table 21 is supported on the floor surface by a stand 22. A radiation generating unit 24 is attached to the imaging table 21 via a support 23. The radiation generating unit 24 is configured to include a radiation source 25 and a collimator 26. A radiation detector 27 is built into the imaging table 21.

[0023] The radiation source 25 has a radiation tube (not shown) and irradiates the subject P with radiation. The collimator 26 limits the irradiation field of the radiation emitted from the radiation tube. The radiation detector 27 has a plurality of pixels that generate signal charges according to the radiation that has passed through the subject P. The radiation detector 27 is called an FPD (Flat Panel Detector).

[0024] The radiation generating unit 24 can be moved back and forth together with the support 23 along the long side direction of the imaging table 21 by a moving mechanism such as a motor (not shown). The radiation detector 27 can be moved back and forth along the long side direction of the imaging table 21 in conjunction with the movement of the radiation generating unit 24. The imaging table 21 and the support 23 can be rotated between a standing state shown in FIG. 2 and a lying state (not shown) by a rotating mechanism such as a motor (not shown). In the lying state, the surface of the imaging table 21 is parallel to the floor surface, and the extension direction of the support 23 is perpendicular to the floor surface. On the other hand, in the standing state, the surface of the imaging table 21 is perpendicular to the floor surface, and the extension direction of the support 23 is parallel to the floor surface. In the standing state, it is possible to perform fluoroscopic imaging on a subject P sitting on a wheelchair 50 as shown in FIG. 2.

[0025] The optical camera 30 is an optical digital camera including a CMOS (Complementary Metal Oxide Semiconductor) type image sensor or a CCD (Charge Coupled Device) type image sensor, and is capable of capturing optical images using visible light. The optical camera 30 is capable of capturing still images and moving images.

[0026] The optical camera 30 is disposed so that a body part required for estimating the posture of the subject falls within the imaging field of view. The optical camera 30 may be attached to the housing of the radiation source 25, for example. The direction in which the optical camera 30 captures an optical image may be substantially the same as the direction in which the radiographic image is captured by the radiographic image capturing device 20. The optical camera 30 may be provided separately from the radiographic image capturing device 20.

[0027] 3 is a diagram showing an example of a hardware configuration of the image processing device 10. The image processing device 10 includes a CPU (Central Processing Unit) 101, a RAM (Random Access Memory) 102, a non-volatile memory 103, an input device 104 including a keyboard and a mouse, a display 105, and a communication interface 106. These hardware components are connected to a bus 108.

[0028] The display 105 may be a touch panel display. The communication interface 106 is an interface for the image processing device 10 to communicate with the radiation image capturing device 20 and the optical camera 30. The communication method may be either wired or wireless. For the wireless communication, a method conforming to existing wireless communication standards such as Wi-Fi (registered trademark) and Bluetooth (registered trademark) can be applied.

[0029] The non-volatile memory 103 is a non-volatile storage medium such as a hard disk or a flash memory. An image processing program 110 is stored in the non-volatile memory 103. The RAM 102 is a work memory for the CPU 101 to execute processing. The CPU 101 loads the image processing program 110 stored in the non-volatile memory 103 into the RAM 102, and executes processing in accordance with the image processing program 110. The CPU 101 is an example of a "processor" in the disclosed technology.

[0030] 4 is a functional block diagram showing an example of a functional configuration of the image processing device 10. The image processing device 10 has an image acquisition unit 11, a posture information generation unit 12, a designation information acquisition unit 13, a guide information generation unit 14, and a display control unit 15. The CPU 101 executes the image processing program 110, so that the image processing device 10 functions as the image acquisition unit 11, the posture information generation unit 12, the designation information acquisition unit 13, the guide information generation unit 14, and the display control unit 15.

[0031] The image acquisition unit 11 acquires optical images captured by the optical camera 30. The optical images are a series of still images or videos continuously captured during the positioning process of a subject in a videofluoroscopic swallowing examination.

[0032] The posture information generating unit 12 extracts a plurality of feature points of the subject's body based on the optical image acquired by the image acquiring unit 11. The posture information generating unit 12 extracts characteristic parts of the subject's body such as the eyes, ears, nose, shoulder joints, and greater trochanters as feature points using known face recognition and posture estimation technologies. The posture information generating unit 12 generates, as posture information, a line segment determined based on any one of the plurality of feature points extracted from the optical image.

[0033] 5 is a diagram showing an example of posture information generation processing by the posture information generating unit 12. The posture information generating unit 12 extracts the eyes, ears, nose, mouth, joints, and other characteristic parts of the subject P captured in the optical image 200, as well as points geometrically determined from the characteristic parts, as feature points. For example, the posture information generating unit 12 extracts feature points Q corresponding to the ears. 1 and the feature point Q corresponding to the nose 2 The posture information generating unit 12 extracts the feature points Q 1 and Q 2 Line segment A connecting 1 The first orientation information is generated as follows: 1 indicates the face direction of subject P.

[0034] In addition, for example, the posture information generating unit 12 may set the midpoints of the left and right shoulder joints as feature points Q 3 The midpoints of the left and right greater trochanters are extracted as feature points Q 4 The posture information generating unit 12 extracts the feature point Q 3 and Q 4 Line segment A connecting 2 The second pose information is generated as follows. 2indicates the orientation of the spine of subject P. Note that in the optical image 200 shown in Fig. 5, the right shoulder joint and right greater trochanter of subject P, which are located on the far side in the depth direction, are hidden and not captured, but the posture information generating unit 12 can extract feature points even for parts that are hidden and not captured in the optical image.

[0035] The optical image 200 is a real-time image of the subject P, and the processing by the posture information generating unit 12 is a real-time process. That is, the posture information generated by the posture information generating unit 12 indicates the current posture of the subject P. Note that the above-mentioned first posture information and second posture information are merely examples, and the feature point Q 1 ~Q 4 It is also possible to use a line segment defined by feature points other than the above as the posture information.

[0036] When the position of the feature point moves with a change in the posture of the subject P, the posture information generating unit 12 extracts the feature point after the movement and generates a line segment determined from the feature point after the movement as new posture information. That is, the posture information generating unit 12 makes the posture information follow the posture change of the subject P.

[0037] The designation information acquisition unit 13 acquires designation information that designates a posture that the subject should take in the swallowing contrast examination. The designation information can also be said to be information that designates the direction (angle) of a line segment as guide information generated by the guide information generation unit 14. Details of the guide information will be described later.

[0038] The radiography menu for radiographic images taken of a subject in a swallowing contrast examination includes designation information for designating a posture that the subject should take (for example, the direction (angle) of the face and the direction (angle) of the spine). The designation information acquisition unit 13 may acquire the designation information included in the radiography menu.

[0039] Furthermore, the designated information can be input to the image processing device 10 by a person who takes a radiological image, such as a doctor or a radiologist, by voice or by operating the input device 104. The designated information is input by voice through a microphone. The designated information acquisition unit 13 may acquire the designated information input by voice or by operating the input device 104.

[0040] The guide information generating unit 14 generates guide information for guiding a posture that a subject should take in a swallowing contrast examination. The guide information generating unit 14 generates a line segment corresponding to a line segment as posture information as guide information. The line segment corresponding to the line segment as posture information may be, for example, a line segment having a tangent point with the line segment as posture information.

[0041] 6 is a diagram showing an example of a guide information generation process by the guide information generating unit 14. The guide information generating unit 14 generates a line segment A 1 Characteristic points Q on both ends of 1 and Q 2 Among them, the feature point Q closest to the support part supporting the subject's body is 1 Line segment B starting from 1 As illustrated in FIG. 6, when the subject P is sitting on the seat 300, the seat 300 becomes the "support part supporting the subject's body." Based on the designation information acquired by the designation information acquisition unit 13, the guide information generation unit 14 generates the line segment B 1 The angle of the line segment as the guide information may be, for example, based on the horizontal direction (0°). For example, when the specification information specifies that the subject's face should be oriented at 30°, the guide information generating unit 14 sets the direction (angle) of the feature point Q 1 Line segment B starts at and is inclined 30 degrees to the horizontal 1 is generated as the first guide information.

[0042] The guide information generating unit 14 generates a line segment A as second posture information indicating the current orientation of the spine of the subject P. 2Characteristic points Q on both ends of 3 and Q 4 Among them, the feature point Q closest to the support part supporting the subject's body is 4 Line segment B starting from 2 The guide information generating unit 14 generates the line segment B 2 For example, when the specification information specifies that the orientation of the subject's spine should be 120°, the guide information generating unit 14 sets the orientation (angle) of the feature point Q 4 Line segment B starts at and is inclined 120 degrees to the horizontal 2 as the second guide information. Note that the above-mentioned first guide information and second guide information are merely examples, and the guide information generating unit 14 generates guide information according to the posture information.

[0043] When the position of the feature point moves with a change in the posture of the subject P, the guide information generating unit 14 generates a line segment having the feature point after the movement as a starting point as new guide information. That is, the guide information generating unit 14 causes the guide information to follow the change in the posture of the subject P.

[0044] The display control unit 15 controls the display 105 to display the posture information generated by the posture information generation unit 12 and the guide information generated by the guide information generation unit 14. For example, as shown in FIG. 6, the display control unit 15 displays a line segment A 1 and A 2 And line B as guide information 1 and B. 2 may be displayed superimposed on the optical image of the subject P. The line segments as posture information and the line segments as guide information may be displayed in different manners. For example, the line segments as posture information and the line segments as guide information may differ from each other in at least one of line type, color, and thickness. This makes it possible to easily identify whether each line segment indicates posture information or guide information.

[0045] FIG. 7 is a flowchart showing an example of the flow of processing carried out by the CPU 101 executing the image processing program 110.

[0046] In step S1, the image acquisition unit 11 acquires optical images captured by the optical camera 30. The optical images are a series of still images or videos continuously captured during the positioning process of a subject in a videofluoroscopic swallowing examination.

[0047] In step S2, the posture information generating unit 12 extracts a plurality of feature points of the subject's body based on the optical image acquired in step S1, and generates a line segment determined based on any one of the extracted feature points as posture information.

[0048] In step S3, the designation information acquisition unit 13 acquires designation information that designates a posture that the subject should take in the swallowing contrast examination. The designation information can also be said to be information that designates the direction (angle) of a line segment as guide information generated by the guide information generation unit 14.

[0049] In step S4, the guide information generating unit 14 generates guide information for guiding the posture that the subject should take in the swallowing contrast examination. The guide information generating unit 14 generates a line segment corresponding to the line segment as the posture information as the guide information. The guide information generating unit 14 sets the direction (angle) of the line segment as the guide information based on the designation information acquired in step S3.

[0050] In step S5, the display control unit 15 controls the display 105 to display the posture information generated in step S2 and the guide information generated in step S4. For example, as shown in FIG. 6, the display control unit 15 displays a line segment A 1 and A 2 And line B as guide information 1 and B. 2 may be displayed superimposed on the optical image of the subject P.

[0051] As described above, the image processing device 10 according to the embodiment of the disclosed technology acquires an optical image of a subject, extracts multiple feature points of the subject's body based on the optical image, generates information indicating the posture of the subject in the form of a line segment determined based on any of the multiple feature points as posture information, generates information guiding the posture the subject should adopt in the form of a line segment corresponding to the line segment as the posture information as guide information, and displays the posture information and the guide information.

[0052] The swallowing contrast examination is performed by reproducing the subject's usual eating posture. For example, if the subject is bedridden, the examination is performed with the subject lying down. In addition, subjects with swallowing disorders have a habit of eating with their face turned slightly upward, so the examination is performed to see if they will swallow food when their face is turned slightly upward. When taking a radiographic image, the subject is positioned to reproduce the subject's usual eating posture. Currently, it is confirmed by the radiographic image whether the subject is in the specified posture. In other words, the subject is irradiated with radiation for positioning.

[0053] According to the image processing device 10 of this embodiment, information guiding the posture that the subject should take is displayed together with posture information indicating the current posture of the subject, so that it is possible to support the positioning of the subject without irradiation of radiation. For example, as shown in FIG. 6, while posture information and guide information are displayed, the subject is positioned so that the posture of the subject becomes the posture indicated by the guide information. FIG. 8 is a diagram showing posture information and guide information in a state where positioning is completed. According to the image processing device 10 of this embodiment, it is easy to perform positioning so as to accurately match the specified posture. This makes it possible to ensure the accuracy and reproducibility of the examination.

[0054] Furthermore, the image processing device 10 displays the posture information and guide information superimposed on the optical image. This makes it possible to more clearly show the subject's current posture and the posture the subject should take. Furthermore, the image processing device 10 makes the posture information and guide information follow changes in the subject's posture. This makes it possible to more effectively support the subject's positioning.

[0055] Furthermore, the image processing device 10 generates a line segment as guide information, starting from a feature point close to the support part supporting the subject's body among the feature points present at both ends of the line segment as posture information. When the posture of the subject changes, the subject often moves around a feature point close to the support part supporting the subject's body. Therefore, among the feature points present at both ends of the line segment as posture information, the feature point close to the support part supporting the subject's body often moves less when the posture of the subject changes. By using a line segment starting from a feature point close to the support part supporting the subject's body as a line segment as guide information, it is possible to provide guide information that is suited to the characteristics of the subject's movement.

[0056] [Second embodiment] 9 is a functional block diagram showing an example of a functional configuration of an image processing device 10A according to a second embodiment of the disclosed technique. The image processing device 10A differs from the image processing device 10 according to the above-described first embodiment (see FIG. 4) in that the image processing device 10A further includes an association processing unit 16.

[0057] In the image processing device 10A according to the second embodiment, the image acquisition unit 11 acquires not only optical images but also a series of radiation images continuously captured with the guidance of guide information. The series of radiation images are captured by continuously irradiating radiation in the radiation image capturing device 20. The series of radiation images are fluoroscopic images of the process in which a subject chews and swallows a sample in a swallowing contrast examination. When capturing the radiation images, the subject is positioned based on the guide information.

[0058] The association processing unit 16 associates the designation information acquired by the designation information acquisition unit 13 with the series of radiation images. The designation information is information that designates the posture that the subject should take. The designation information designates, for example, the face direction or spine direction of the subject when capturing the radiation images. The series of radiation images captured with the guidance of the guide information are fluoroscopic images of the subject in a posture designated by the designation information. The association processing unit 16 may associate the series of radiation images with the designation information by generating, for example, image group data 61 including the series of radiation images and a data file 60 including designation information 62, as shown in FIG. 10.

[0059] 11 is a flowchart showing an example of the flow of processing performed by the CPU 101 in the image processing device 10A according to the second embodiment by executing the image processing program 110. Note that the processing from step S11 to step S15 is similar to the processing from step S1 to step S5 in the flowchart (see FIG. 7) according to the first embodiment described above, and therefore a description of these processing will be omitted.

[0060] In step S16, the image acquisition unit 11 acquires a series of radiographic images captured successively under the guidance of the guide information. The series of radiographic images are fluoroscopic images of the subject in a posture designated by the designation information acquired in step S13.

[0061] In step S17, the association processing unit 16 associates the designation information acquired in step S13 with the series of radiation images acquired in step S16.

[0062] In step S18, the association processing unit 16 stores a data file in which the specification information and the series of radiation images are associated with each other in a recording medium. The recording medium in which the data file is stored may be the non-volatile memory 103 included in the image processing device 10A, or may be a recording medium included in an external image server (not shown).

[0063] As described above, the image processing device 10A according to the second embodiment of the disclosed technology sets the direction of a line segment as guide information based on designation information that specifies the posture that the subject should take, acquires a series of radiological images of the subject that are successively captured with guidance from the guide information, and associates the designation information with the series of radiological images.

[0064] In a swallowing contrast examination, radiation images are taken for each case where the posture of the subject is changed, and a data file of the radiation image may be generated for each posture. In this case, since a large number of data files are generated, it is desirable to be able to know which data file corresponds to which posture. It is troublesome to manually associate the radiation images with the postures, and there is also a possibility that the association may be performed incorrectly. According to the image processing device 10A of this embodiment, the association processing unit 16 associates the designated information with a series of radiation images, so that accurate association between the radiation images and the postures can be performed without imposing a burden on the user.

[0065] In the above description, a case where one piece of designation information is associated with a series of radiation images has been exemplified, but when the posture of the subject is changed sequentially while the radiation images are continuously captured, a plurality of pieces of designation information may be associated with the series of radiation images. That is, a certain piece of designation information is associated with some frames of a moving image composed of a series of radiation images, and another piece of designation information is associated with some other frames. In the following, a case where the image processing device 10A associates a plurality of pieces of designation information with a series of radiation images will be described.

[0066] The designated information acquisition unit 13 sequentially acquires a plurality of pieces of designated information each designating a posture that the subject should take during the period of capturing a series of radiation images. The plurality of pieces of designated information can also be said to be information designating the direction (angle) of a line segment as guide information generated by the guide information generation unit 14.

[0067] The guide information generating unit 14 sets the direction (angle) of each of the line segments as each of the plurality of pieces of guide information, based on the plurality of pieces of designation information.

[0068] The display control unit 15 performs control to sequentially display a plurality of pieces of guide information on the display 105 together with the posture information.

[0069] The association processing unit 16 performs a process for associating one of the plurality of pieces of designation information with a part of the series of radiation images captured with the guide of the guide information generated based on the one piece of designation information for each of the plurality of pieces of designation information. The association processing unit 16 stores a data file in which the plurality of pieces of designation information and the series of radiation images are associated with each other in the recording medium.

[0070] 12 is a flowchart showing an example of the flow of processing performed in the image processing device 10A by the CPU 101 executing the image processing program 110. The processing from step S21 to step S25 is similar to the processing from step S1 to step S5 in the flowchart (see FIG. 7) according to the above-described first embodiment, and therefore a description of these processing will be omitted.

[0071] In step S23, one of the multiple pieces of designation information is acquired. In step S24, one piece of guide information is generated based on the one piece of designation information acquired in step S23. In step S25, the one piece of guide information generated in step S24 is displayed on the display 105 together with the posture information.

[0072] In step S26, the image acquisition unit 11 acquires a series of radiographic images captured successively while being guided by the guide information generated in step S24. The series of radiographic images are obtained by fluoroscopically capturing the subject in a posture designated by the designation information acquired in step S23.

[0073] In step S27, CPU 101 determines whether or not radiography has been completed for the current posture. A radiographer, such as a doctor or a radiologist, can input status information indicating whether or not radiography has been completed for the current posture, for example, by operating input device 104. CPU 101 may determine whether or not radiography has been completed for the current posture based on the status information. If it is determined that radiography has been completed for the current posture, the process proceeds to step S28, and if it is determined that radiography has not been completed for the current posture, the process returns to step S26.

[0074] In step S28, the CPU 101 determines whether or not radiographic images have been captured for all postures. A radiographic image capturer, such as a doctor or a radiologist, can input status information indicating whether or not radiographic images have been captured for all postures by, for example, operating the input device 104. The CPU 101 may determine whether or not radiographic images have been captured for all postures based on the status information. If it is determined that radiographic images have been captured for all postures, the process proceeds to step S29, and if it is determined that radiographic images have not been captured for all postures, the process returns to step S21. That is, each process from step S21 to step S27 is repeatedly performed until radiographic images have been captured for all postures.

[0075] In step S23, which is performed from the second time onwards, new designation information different from the designation information previously acquired is acquired. In step S24, new guide information is generated based on the new designation information. In step S25, the new guide information is displayed on the display 105 together with the posture information. In step S26, a series of radiation images are acquired that are continuously captured with the guidance of the new guide information. Radiation images are continuously captured until capturing of radiation images is completed for all postures.

[0076] In step S29, the association processing unit 16 performs a process of associating one of the plurality of pieces of designation information with a part of the series of radiographic images captured with the guide of the guide information generated based on the one piece of designation information for each of the plurality of pieces of designation information. That is, the association processing unit 16 performs a process of associating the designation information acquired in step S23 with the series of radiographic images acquired in step S26 for each repetition unit in the repetition process in which the processes from step S21 to step S27 are the repetition units.

[0077] In step S30, the association processing unit 16 stores a data file in which a plurality of pieces of designation information and a series of radiation images are associated with each other in the recording medium.

[0078] Fig. 13 is a diagram showing an example of the association process by the association processing unit 16. A series of radiation images acquired by the image acquisition unit 11 constitutes each frame of a moving image (hereinafter referred to as an inspection moving image) constituted by a series of radiation images. Fig. 13 illustrates an example in which first designation information 71 is associated with each frame from the first frame to the third frame of the inspection moving image, and second designation information 72 is associated with each frame from the fourth frame to the sixth frame of the inspection moving image.

[0079] The first designation information 71 is information indicating that the subject's face should be oriented at 0° and the spine should be oriented at 90°. The association processing unit 16 associates the first designation information 71 with each of the first to third frames captured with the guide of the guide information generated based on the first designation information 71. The second designation information 72 is information indicating that the subject's face should be oriented at 10° and the spine should be oriented at 90°. The association processing unit 16 associates the second designation information 72 with each of the fourth to sixth frames captured with the guide of the guide information generated based on the second designation information 72.

[0080] As described above, the image processing device 10A according to this embodiment sets the orientation of each of the line segments as each of the multiple guide information based on multiple designation information each of which specifies the posture that the subject should take, acquires a series of radiological images captured consecutively with guidance from each of the multiple guide information, and performs processing for each of the multiple designation information to correspond one of the multiple designation information to a part of the series of radiological images captured with guidance from guide information generated based on that one designation information.

[0081] In a swallowing contrast examination, radiographic images may be taken continuously while changing the posture of the subject, and multiple postures may be mixed in one examination video. In this case, it is desirable to accurately match the posture of the subject to the frame related to that posture so that it is possible to know which part of a series of radiographic images corresponds to which posture.

[0082] According to the image processing device 10A of the present embodiment, a process of associating one of a plurality of pieces of designation information with a part of a series of radiographic images captured with the guide of the guide information generated based on the one piece of designation information is performed for each of the plurality of pieces of designation information, so that accurate association between the frame and the posture can be performed without imposing a burden on the user. According to the image processing device 10A of the present embodiment, for example, when playing back an examination video, it is possible to cue up a part related to a desired posture.

[0083] In the above description, the case where the designation information and the radiological image are associated with each other is exemplified, but the disclosed technology is not limited to this. Information indicating the direction (angle) of the line segment as posture information at the time when the positioning of the subject is completed may be associated with the radiological image instead of or together with the designation information. The information indicating the direction (angle) of the line segment as posture information indicates the actual posture of the subject at that time.

[0084] [Third embodiment] 14 is a functional block diagram showing an example of a functional configuration of an image processing device 10B according to a third embodiment of the disclosed technique. The image processing device 10B differs from the image processing device 10 according to the above-described first embodiment (see FIG. 4) in that the image processing device 10B further includes an imaging direction identification unit 17 and a notification unit 18.

[0085] The imaging direction specifying unit 17 specifies the imaging direction of a radiological image to be captured of the subject based on at least a part of the multiple feature points extracted by the posture information generating unit 12. The imaging direction specifying unit 17 may specify the imaging direction based on, for example, the positional relationship of feature points corresponding to the left and right eyes of the subject. It is also possible to specify the imaging direction based on the positional relationship of feature points corresponding to the left and right ears or the positional relationship of feature points corresponding to the left and right shoulder joints. The imaging direction specifying unit 17 specifies, for example, whether the imaging direction is a direction in which the subject is imaged from the front or a direction in which the subject is imaged from the side.

[0086] It should be noted that the feature points are extracted from the optical image captured by the optical camera 30. If the relationship between the direction in which the optical image is captured by the optical camera 30 and the direction in which the radiographic image is captured by the radiographic image capturing device 20 is known, it is possible to identify the direction in which the radiographic image is captured based on the feature points.

[0087] The notification unit 18 issues an alert to notify that the imaging direction of the radiographic image is different from the specified imaging direction when the imaging direction specified by the imaging direction specifying unit 17 is different from the specified imaging direction. The notification unit 18 may issue an alert by, for example, displaying a predetermined message on the display 105, or outputting a predetermined sound from a speaker (not shown). The designation information acquired by the designation information acquisition unit 13 includes information designating the imaging direction of the radiographic image. The notification unit 18 issues an alert when the imaging direction specified by the imaging direction specifying unit 17 is different from the imaging direction indicated by the designation information.

[0088] 15 is a flow chart showing an example of the flow of processing performed by the CPU 101 executing the image processing program 110 in the image processing device 10B according to the third embodiment. Note that the processes from step S31 to step S35 are similar to the processes from step S1 to step S5 in the process according to the first embodiment (see FIG. 7), and therefore descriptions of these processes are omitted. Note that the designation information acquired in step S33 includes information designating the direction (angle) of the line segment as guide information and information designating the imaging direction of the radiographic image.

[0089] In step S36, the imaging direction specifying unit 17 specifies the imaging direction of a radiographic image to be captured of the subject, based on at least a part of the plurality of feature points extracted in step S32.

[0090] In step S37, the notification unit 18 determines whether the shooting direction identified in step S36 matches the shooting direction indicated by the designation information acquired in step S33. If it is determined that the shooting direction identified in step S36 matches the shooting direction indicated by the designation information acquired in step S33, the process ends. On the other hand, if it is determined that the shooting direction identified in step S36 is different from the shooting direction indicated by the designation information acquired in step S33, the process proceeds to step S38.

[0091] In step S38, the notification unit 18 issues an alert to notify that the imaging direction of the radiographic image is different from the designated imaging direction.

[0092] According to the image processing device 10 of this embodiment, an alert is issued when the shooting direction of the radiographic image identified based on the feature points differs from the specified shooting direction, so that it is possible to prevent the need to reshoot the radiographic image.

[0093] In the above first to third embodiments, the disclosed technology is applied to a swallowing contrast examination, but the disclosed technology is not limited to this embodiment. The disclosed technology can be applied to any situation in which the posture of the subject is specified.

[0094] The hardware for executing the processes in each functional unit of the image processing device 10, 10A, 10B may be various processors as shown below. The processor may be a CPU that executes software (programs) and functions as various processing units. The processor may also be a programmable logic device (PLD) that can change the circuit configuration, such as an FPGA. The processor may also have a circuit configuration designed specifically for executing specific processes, such as an application specific integrated circuit (ASIC).

[0095] Each functional unit of the image processing device 10, 10A, 10B may be configured with one of the various processors described above, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, multiple functional units may be configured with one processor.

[0096] In the above embodiment, the image processing program 110 is pre-stored (installed) in the non-volatile memory 103, but the present invention is not limited to this. The image processing program 110 may be provided in a form recorded on a recording medium such as a CD-ROM (Compact Disc Read Only Memory), a DVD-ROM (Digital Versatile Disc Read Only Memory), or a USB (Universal Serial Bus) memory. The image processing program 110 may be downloaded from an external device via a network.

[0097] The following supplementary notes are further disclosed regarding the above first to fourth embodiments. (Appendix 1) 1. An image processing device having at least one processor, The processor, Acquiring an optical image of the subject; Extracting a plurality of feature points of the subject's body based on the optical image; generating, as posture information, a line segment that is determined based on any one of the plurality of feature points, which is information indicating a posture of the subject; generating, as guide information, a line segment corresponding to the line segment as the posture information, which is information for guiding the posture to be taken by the subject; The posture information and the guide information are displayed. Image processing device.

[0098] (Appendix 2) The processor displays the posture information and the guide information superimposed on the optical image. 2. An image processing device as described in claim 1.

[0099] (Appendix 3) The processor causes the posture information and the guide information to follow a change in posture of the subject. 3. An image processing device according to claim 1 or 2.

[0100] (Appendix 4) The processor generates, as the guide information, a line segment having a starting point at a feature point that is close to a support part supporting the body of the subject, among feature points present at both ends of the line segment as the posture information. 4. An image processing device according to claim 1,

[0101] (Appendix 5) The processor sets a direction of the line segment as the guide information based on designation information that designates a posture that the subject should take. 5. An image processing device according to claim 1,

[0102] (Appendix 6) The processor acquires the designation information included in an imaging menu for imaging a radiation image of the subject. 6. The image processing device according to claim 5.

[0103] (Appendix 7) The processor acquires the designated information input by operating an input means or by voice.

[0104] 6. The image processing device according to claim 5.

[0105] (Appendix 8) The processor, extracting an ear of the subject as a first feature point and an eye of the subject as a second feature point based on the optical image; A line segment connecting the first feature point and the second feature point is generated as the posture information. 8. An image processing device according to any one of claims 1 to 7.

[0106] (Appendix 9) The processor, extracting a midpoint of the left and right shoulder joints of the subject as a third feature point based on the optical image, and extracting a midpoint of the left and right greater trochanters of the subject as a fourth feature point; A line segment connecting the third feature point and the fourth feature point is generated as the posture information. 9. An image processing device according to any one of claims 1 to 8.

[0107] (Appendix 10) The processor, setting a direction of the line segment as the guide information based on designation information that designates a posture that the subject should take; A series of radiological images of the subject are continuously captured while being guided by the guide information; Corresponding the specified information to the series of radiation images 10. An image processing device according to any one of claims 1 to 9.

[0108] (Appendix 11) The processor, setting a direction of each of the line segments as each of the plurality of pieces of guide information based on a plurality of pieces of designation information each designating a posture to be taken by the subject; acquiring a series of radiation images successively captured with guidance by each of the plurality of pieces of guide information; A process of associating one of the plurality of pieces of designation information with a part of the series of radiation images that have been captured with the guide by the guide information generated based on the one piece of designation information is performed for each of the plurality of pieces of designation information. 11. An image processing device according to any one of claims 1 to 10.

[0109] (Appendix 12) The processor, identifying a direction of imaging of a radiation image of the subject based on at least a part of the plurality of feature points; Issue an alert if the identified shooting direction differs from the specified direction. 12. An image processing device according to any one of claims 1 to 11. [Explanation of symbols]

[0110] 1 Radiography system 10, 10A, 10B Image processing device 11 Image acquisition section 12 Posture information generation section 13 Specified information acquisition section 14 Guide information generation unit 15 Display control section 16 Correspondence Processing Unit 17 Shooting direction identification unit 18. Notification Department 20 Radiation imaging device 21 Photo stand 22 Stand 23 Pillar 24 Radiation Generation Unit 25 Radiation source 26 Collimator 27 Radiation Detector 30 Optical Camera 50 Wheelchair 60 Data Files 61 Image Group Data 62 Specification information 71 First Designation Information 72 Second Designation Information 101 CPU 102 RAM 103 Non-volatile memory 104 Input Devices 105 Display 106 Communication Interface 108 Bus 110 Image Processing Program 200 Optical Images Q 1 , Q 2 , Q 3 , Q 4 Feature points A 1 , A 2 , B 1 , B 2 line segment

Claims

1. 1. An image processing device having at least one processor, The processor, Acquiring an optical image of the subject; Extracting a plurality of feature points of the subject's body based on the optical image; generating, as posture information, a line segment that is determined based on any one of the plurality of feature points, which is information indicating a posture of the subject; generating, as guide information, a line segment corresponding to the line segment as the posture information, which is information for guiding the posture to be taken by the subject; The posture information and the guide information are displayed. Image processing device.

2. The processor displays the posture information and the guide information superimposed on the optical image. The image processing device according to claim 1 .

3. The processor causes the posture information and the guide information to follow a change in posture of the subject. The image processing device according to claim 1 .

4. The processor generates, as the guide information, a line segment having a starting point at a feature point that is close to a support part supporting the body of the subject, among feature points present at both ends of the line segment as the posture information. The image processing device according to claim 1 .

5. The processor sets a direction of the line segment as the guide information based on designation information that designates a posture that the subject should take. The image processing device according to claim 1 .

6. The processor acquires the designation information included in an imaging menu for imaging a radiation image of the subject. The image processing device according to claim 5 .

7. The processor acquires the designated information input by operating an input means or by voice. The image processing device according to claim 5 .

8. The processor, extracting an ear of the subject as a first feature point and an eye of the subject as a second feature point based on the optical image; A line segment connecting the first feature point and the second feature point is generated as the posture information. The image processing device according to claim 1 .

9. The processor, extracting a midpoint of the left and right shoulder joints of the subject as a third feature point based on the optical image, and extracting a midpoint of the left and right greater trochanters of the subject as a fourth feature point; A line segment connecting the third feature point and the fourth feature point is generated as the posture information. The image processing device according to claim 1 .

10. The processor, setting a direction of the line segment as the guide information based on designation information that designates a posture that the subject should take; A series of radiological images of the subject are continuously captured while being guided by the guide information; Corresponding the specified information to the series of radiation images The image processing device according to claim 1 .

11. The processor, setting a direction of each of the line segments as each of the plurality of pieces of guide information based on a plurality of pieces of designation information each designating a posture to be taken by the subject; acquiring a series of radiation images successively captured with guidance by each of the plurality of pieces of guide information; A process of associating one of the plurality of pieces of designation information with a part of the series of radiation images captured with the guide by the guide information generated based on the one piece of designation information is performed for each of the plurality of pieces of designation information. The image processing device according to claim 1 .

12. The processor, identifying a direction of imaging of a radiation image of the subject based on at least a part of the plurality of feature points; Issue an alert if the identified shooting direction differs from the specified direction. The image processing device according to claim 1 .

13. Acquiring an optical image of the subject; Extracting a plurality of feature points of the subject's body based on the optical image; generating, as posture information, a line segment that is determined based on any one of the plurality of feature points, which is information indicating a posture of the subject; generating, as guide information, a line segment corresponding to the line segment as the posture information, which is information for guiding the posture to be taken by the subject; The posture information and the guide information are displayed. An image processing method in which processing is executed by at least one processor included in an image processing device.

14. Acquiring an optical image of the subject; Extracting a plurality of feature points of the subject's body based on the optical image; generating, as posture information, a line segment that is determined based on any one of the plurality of feature points, which is information indicating a posture of the subject; generating, as guide information, a line segment corresponding to the line segment as the posture information, which is information for guiding the posture to be taken by the subject; The posture information and the guide information are displayed. A program for causing at least one processor included in the image processing device to execute the process.

Images