Dynamic image display device, dynamic image display method and program

Abstract

To provide a dynamic image display device capable of further accurately comparing dynamic images of different frame rates.SOLUTION: A dynamic image display device (a diagnostic console 3) includes a dynamic image acquisition unit (a control unit 31) for acquiring a first dynamic image captured at a first frame rate and a second dynamic image captured at a second frame rate lower than the first frame rate, an interpolated image generation unit (the control unit 31) for generating an interpolated image between respective frame images of the second dynamic image and increasing a frame rate of the second dynamic image, and a display unit 34 for displaying the first dynamic image and the second dynamic image including the interpolated image in a comparable manner.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a moving image display device, a moving image display method, and a program.

Background Art

[0002] Conventionally, there are cases where users such as doctors compare a moving image captured at a first frame rate with another moving image captured at a second different frame rate for case inspection.

[0003] Patent Document 1 describes a method of matching the frame rates between moving images by thinning out the frames of one of the moving images when comparing moving images with different frame rates.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] In the method described in Patent Document 1, the frame rate between moving images is adjusted to the lower frame rate. Therefore, even if the frame rate of each moving image is improved, the accuracy of comparing the low-frame-rate moving image does not improve, and the accuracy of case inspection does not improve.

[0006] An object of the present invention is to provide a moving image display device, a moving image display method, and a program that can compare moving images with different frame rates more accurately.

Means for Solving the Problems

[0007] To solve the above problems, a moving image display device according to the present invention is A moving image acquisition unit that acquires a first moving image captured at a first frame rate and a second moving image captured at a second frame rate lower than the first frame rate; An interpolation image generation unit that generates an interpolation image between each frame image of the second moving image and increases the frame rate of the second moving image; A display unit that displays the first moving image and the second moving image including the interpolation image in a comparable manner; It is provided with.

[0008] Also, in order to solve the above problems, the moving image display method according to the present invention is In a moving image display device that displays a moving image, A moving image acquisition step of acquiring a first moving image captured at a first frame rate and a second moving image captured at a second frame rate lower than the first frame rate; An interpolation image generation step of generating an interpolation image between each frame image of the second moving image and increasing the frame rate of the second moving image; A display step of displaying the first moving image and the second moving image including the interpolation image in a comparable manner; It includes.

[0009] Also, in order to solve the above problems, the program according to the present invention is A computer of a moving image display device that displays a moving image is A moving image acquisition unit that acquires a first moving image captured at a first frame rate and a second moving image captured at a second frame rate lower than the first frame rate; An interpolation image generation unit that generates an interpolation image between each frame image of the second moving image and increases the frame rate of the second moving image; A display control unit that causes the display unit to display the first moving image and the second moving image including the interpolation image in a comparable manner; Function as.

Effect of the Invention

[0010] According to the present invention, dynamic images with different frame rates can be compared with higher accuracy.

Brief Description of the Drawings

[0011]

Figure 1

Figure 2

Figure 3

Figure 4

Modes for Carrying Out the Invention

[0012] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. However, the scope of the invention is not limited to the illustrated examples.

[0013] 〔Configuration of Dynamic Image Display System 100〕 FIG. 1 shows the overall configuration of the dynamic image display system 100 in the present embodiment. As shown in FIG. 1, the dynamic image display system 100 is configured such that the imaging device 1 and the imaging console 2 are connected by a communication cable or the like, and the imaging console 2 and the diagnostic console 3 as the dynamic image display device are connected via a communication network NT such as a LAN (Local Area Network).

[0014] 〔Configuration of Imaging Device 1〕 The imaging device 1 is, for example, imaging means for imaging the dynamics of a subject having periodicity, such as the morphological changes of the lungs during breathing movement, the expansion and contraction, and the pulsation of the heart. Dynamic imaging refers to acquiring a plurality of images by irradiating a subject with radiation such as X-rays in a pulsed manner at predetermined time intervals (pulse irradiation) or continuously irradiating at a low dose rate without interruption (continuous irradiation). That is, dynamic imaging is to continuously perform radiation imaging of the dynamics of a target site having periodicity along the time axis. Note that for dynamic imaging, imaging may be performed not only using radiation such as X-rays, but also using ultrasonic waves or magnetism. Dynamic imaging includes video imaging, but does not include imaging of still images while displaying a video. Also, a series of images obtained by dynamic imaging is called a dynamic image. Also, a dynamic image can be obtained, for example, by imaging using a semiconductor image sensor such as an FPD (Flat Panel Detector). Also, a dynamic image includes a video, but does not include an image obtained by imaging a still image while displaying a video. Also, each of the plurality of images constituting a dynamic image is called a frame image. Note that in the following embodiments, the case of performing dynamic imaging by pulsed irradiation will be taken as an example for explanation. Also, in the following embodiments, the case where the subject M is the chest of the subject will be taken as an example for explanation, but it is not limited thereto.

[0015] The radiation source 11 is arranged at a position facing the radiation detection unit 13 with the subject M interposed therebetween, and irradiates the subject M with radiation (X-rays) according to the control of the radiation irradiation control device 12. The radiation irradiation control device 12 is connected to the imaging console 2, and controls the radiation source 11 based on the radiation irradiation conditions input from the imaging console 2 to perform radiation imaging. The radiation irradiation conditions are, for example, pulse rate, pulse width, pulse interval, number of imaging frames per imaging, value of X-ray tube current, value of X-ray tube voltage, additional filter type, etc. The pulse rate is the number of radiation irradiations per second and is consistent with the frame rate described later. The pulse width is the radiation irradiation time per radiation irradiation. The pulse interval is the time from the start of one radiation irradiation to the start of the next radiation irradiation and is consistent with the frame interval described later.

[0016] The radiation detection unit 13 is composed of a semiconductor image sensor such as an FPD. The FPD has, for example, a glass substrate or the like, and at a predetermined position on the substrate, a plurality of detection elements (pixels) are arranged in a matrix, which detect the radiation irradiated from the radiation source 11 and transmitted through at least the subject M according to its intensity, convert the detected radiation into an electrical signal, and accumulate it. Each pixel is configured to include a switching unit such as a TFT (Thin Film Transistor). The FPD includes an indirect conversion type that converts X-rays into an electrical signal by a photoelectric conversion element via a scintillator, and a direct conversion type that directly converts X-rays into an electrical signal, and either type may be used. In the present embodiment, the pixel value (signal value) of the image data generated in the radiation detection unit 13 is a density value, and it is assumed that the higher the radiation transmission amount, the higher the value. The radiation detection unit 13 is provided so as to face the radiation source 11 with the subject M interposed therebetween.

[0017] The reading control device 14 is connected to the imaging console 2. The reading control device 14 controls the switching unit of each pixel of the radiation detection unit 13 based on the image reading conditions input from the imaging console 2, switches the reading of the electrical signal accumulated in each pixel, and reads the electrical signal accumulated in the radiation detection unit 13 to acquire image data. This image data is a frame image. Then, the reading control device 14 assigns an identification ID and a frame number, and outputs the acquired frame image to the imaging console 2. The image reading conditions are, for example, frame rate, frame interval, pixel size, image size (matrix size), etc. The frame rate is the number of frame images acquired per second and coincides with the pulse rate. The frame interval is the time from the start of the acquisition operation of one frame image to the start of the acquisition operation of the next frame image and coincides with the pulse interval.

[0018] Here, the radiation irradiation control device 12 and the reading control device 14 are connected to each other and exchange synchronization signals with each other to synchronize the radiation irradiation operation and the image reading operation.

[0019] 〔Configuration of the imaging console 2〕 The imaging console 2 outputs radiation irradiation conditions and image reading conditions to the imaging device 1 and controls the radiation imaging and radiation image reading operations performed by the imaging device 1. As shown in FIG. 1, the imaging console 2 includes a control unit 21, a storage unit 22, an operation unit 23, a display unit 24, and a communication unit 25, and each unit is connected by a bus 26.

[0020] The control unit 21 is composed of a CPU (Central Processing Unit), a RAM (Random Access Memory), etc. The CPU of the control unit 21 reads out the system program and various processing programs stored in the storage unit 22 according to the operation of the operation unit 23 and expands them in the RAM, and executes various processes including imaging control processing according to the expanded programs, and centrally controls the operations of each part of the imaging console 2, and the radiation irradiation operation and reading operation of the imaging device 1.

[0021] The storage unit 22 is composed of a non-volatile semiconductor memory, a hard disk, etc. The storage unit 22 stores various programs executed by the control unit 21, parameters necessary for executing the processing by the programs, or data such as processing results. For example, the storage unit 22 stores a program for executing imaging control processing. The various programs are stored in the form of readable program codes, and the control unit 21 sequentially executes operations according to the program codes. Specifically, the storage unit 22 stores a series of frame images (moving images) with identification IDs and frame numbers attached, which are output from the imaging device 1. In addition, the storage unit 22 stores imaging order information. The imaging order information is attached to a series of frame images (moving images) and stored in the storage unit 22. The imaging order information includes radiation irradiation conditions (described above), image reading conditions (described above), subject information, examination information, etc. The subject information is, for example, the subject's name, height, weight, age, gender, etc. The examination information is, for example, the imaging site (such as the chest) and the diagnosis target (such as ventilation, pulmonary blood flow, etc.).

[0022] The operation unit 23 is composed of a keyboard equipped with cursor keys, numeric input keys, and various function keys, and a pointing device such as a mouse. It outputs the instruction signal input by a key operation on the keyboard or a mouse operation to the control unit 21. Further, the operation unit 23 may be provided with a touch panel on the display screen of the display unit 24. In this case, it outputs the instruction signal input via the touch panel to the control unit 21. The imaging performer inputs the above imaging order information using the operation unit 23.

[0023] The display unit 24 is composed of a monitor such as an LCD (Liquid Crystal Display) or a CRT (Cathode Ray Tube), and displays the input instruction and data from the operation unit 23 according to the instruction of the display signal input from the control unit 21.

[0024] The communication unit 25 is equipped with a LAN adapter, a modem, a TA (Terminal Adapter), etc., and controls the data transmission and reception with each device connected to the communication network NT.

[0025] 〔Configuration of the Diagnostic Console 3〕 The diagnostic console 3 (dynamic image display device) acquires and displays the dynamic image from the imaging console 2. As described above, the imaging order information is attached to the dynamic image. As shown in FIG. 1, the diagnostic console 3 is composed of a control unit 31, a storage unit 32, an operation unit 33, a display unit 34, and a communication unit 35, and each unit is connected by a bus 36.

[0026] The control unit 31 is composed of a CPU, a RAM, etc. The CPU of the control unit 31 reads out the system program and various processing programs stored in the storage unit 32 according to the operations of the operation unit 33, expands them in the RAM, and executes various processes according to the expanded programs. Also, the CPU of the control unit 31 reads out the program 32a stored in the storage unit 32, expands it in the RAM, and executes the image display process described later according to the expanded program 32a. Also, the control unit 31 functions as a moving image acquisition unit that acquires a first moving image captured at a first frame rate and a second moving image captured at a second frame rate lower than the first frame rate. Also, the control unit 31 functions as an interpolation image generation unit that generates interpolation images between the frame images of the second moving image and makes the frame rate of the second moving image the same as the first frame rate. Also, the control unit 31 functions as a display control unit that causes the display unit to display the first moving image and the second moving image including the interpolation images in a comparable manner.

[0027] The storage unit 32 is composed of a non-volatile semiconductor memory, a hard disk, etc. The storage unit 32 stores a program 32a for executing image display processing by the control unit 31, various programs, parameters necessary for executing processing by the programs, or data such as processing results. These various programs are stored in the form of readable program codes, and the control unit 31 sequentially executes operations according to the program codes. Also, the storage unit 32 stores the moving image acquired from the shooting console 2 and the accompanying shooting order information.

[0028] The operation unit 33 is configured to include a keyboard having cursor keys, numeric input keys, and various function keys, and a pointing device such as a mouse, and outputs an instruction signal input by a key operation on the keyboard or a mouse operation to the control unit 31. Also, the operation unit 33 may include a touch panel on the display screen of the display unit 34, and in this case, outputs an instruction signal input via the touch panel to the control unit 31.

[0029] The display unit 34 is composed of a monitor such as an LCD or a CRT, and performs various displays according to the instructions of the display signal input from the control unit 31. The display unit 34 functions as a display unit that can display and compare a first moving image and a second moving image including an interpolation image.

[0030] The communication unit 35 includes a LAN adapter, a modem, a TA, etc., and controls data transmission and reception with each device connected to the communication network NT.

[0031] 〔Image display process〕 Next, with reference to FIG. 2, the image display process in the diagnostic console 3 will be described. The image display process is a process of generating a correction image between each frame of the moving image with a lower frame rate and displaying the moving images with different frame rates in a comparable manner. Let one moving image be a first moving image captured at a first frame rate. Let the other moving image be a second moving image captured at a second frame rate lower than the first frame rate. Also, it is desirable that the interpolation image be generated so that the frame rate of the second moving image becomes the same as the first frame rate. Even if they are not the same, the interpolation image may be generated so as to increase the frame rate of the second moving image.

[0032] First, the control unit 31 acquires the first moving image from the storage unit 32 (step S1). Next, the control unit 31 acquires the second moving image from the storage unit 32 (step S2).

[0033] Next, the control unit 31 generates an interpolation image for the second moving image (step S3). Then, the control unit 31 adds the interpolation image to the second moving image.

[0034] Here, a specific interpolation method will be described. For example, an interpolation method using a sine wave will be described. First, plot the pixel values of each frame image on a graph with the vertical axis representing the pixel values of each frame image and the horizontal axis representing the time (shooting time) at which each frame image was shot. Next, approximate the change in pixel values over time using a sine wave. This is because in a moving image, since the images are continuously shot at a certain frame rate, the pixel values change smoothly over time. Finally, an interpolated image is generated by obtaining the pixel values at a certain time between each shooting time and arranging the obtained pixel values.

[0035] Also, for example, linear interpolation will be described. Linear interpolation means that between each frame image, it is assumed that the pixel values change linearly, and an interpolated image is generated by obtaining the pixel values at a certain time between each shooting time and arranging the obtained pixel values. In this case, the processing becomes lighter than interpolation using a sine wave.

[0036] Also, for example, when the frame rate of the first moving image is twice the frame rate of the second moving image, an interpolated image between those frame images can be generated by averaging the pixel values of adjacent frame images. In this case, the processing becomes lighter than interpolation using a sine wave or linear interpolation.

[0037] Also, for example, interpolation may be performed using machine learning. A model that has previously learned the characteristics of the change in pixel values of each frame image is created and stored in the storage unit 32, and that model is used to calculate an interpolated image between each frame image.

[0038] Next, the control unit 31 causes the display unit 34 to display a second moving image including the first moving image and the interpolated image (step S4). Specifically, when the control unit 31 displays the second moving image including the interpolated image, it displays that it is the second moving image including the interpolated image. Furthermore, when the image (still image) at the timing when the moving image is stopped is an interpolated image, it is displayed that the image is an interpolated image. For example, the control unit 31 causes the display unit 34 to display as shown in FIGS. 3 and 4. FIGS. 3 and 4 are image diagrams of a display screen D1 that displays side by side the moving image of the current day's inspection and the moving image of the past inspection. Here, the moving image of the current day's inspection is the first moving image, and the moving image of the past inspection is the second moving image. FIG. 3 shows a case where the actually captured frame image in the second moving image at a certain timing is displayed. FIG. 4 shows a case where an interpolated image in the second moving image at a certain timing is displayed. In FIG. 4, a display I indicating that the image is an interpolated image is displayed on the display screen D1. Thereby, the user can confirm whether the comparison target is an actually captured frame image or an interpolated image.

[0039] 〔Others〕 In the above, in step S1, the case of acquiring the first moving image from the storage unit 32 has been described, but it is not limited to this. For example, the first moving image during or after shooting may be acquired from the shooting console 2. Also, as in the configuration of the moving image display system 100 shown in FIG. 1, a plurality of shooting consoles 2 may be connected to the communication network NT, and the first moving image and the second moving image may be acquired from different shooting consoles 2.

[0040] Also, in step S4, it is displayed on the display unit 34, but it is not limited to this. For example, it may be displayed on another display device connected via the communication network NT. Also, the user may set the interpolation ratio. For example, when the first frame rate is 40 FPS and the second frame rate is 30 FPS, when the interpolation ratio is 50%, the second frame rate becomes 35 FPS, and when the interpolation ratio is 100%, the second frame rate becomes 40 FPS.

[0041] 〔Effect〕 As described above, the moving image display device (diagnostic console 3) includes a moving image acquisition unit (control unit 31) that acquires a first moving image captured at a first frame rate and a second moving image captured at a second frame rate lower than the first frame rate, an interpolation image generation unit (control unit 31) that generates interpolation images between each frame image of the second moving image to increase the frame rate of the second moving image, and a display unit 34 that displays the first moving image and the second moving image including the interpolation images in a comparable manner. Further, the interpolation image generation unit (control unit 31) may generate interpolation images so as to be the same as the first frame rate. Therefore, moving images with different frame rates can be compared with higher accuracy. Also, the second moving image is a moving image captured before the first moving image, and even if the frame rate is low, it can be compared.

[0042] Also, the first moving image and the second moving image are moving images captured by different imaging devices. Therefore, it can be compatible with various modes of the moving image display system 100.

[0043] The moving image display method includes a moving image acquisition step (steps S1 and S2) of acquiring a first moving image captured at a first frame rate and a second moving image captured at a second frame rate lower than the first frame rate in a moving image display device (diagnostic console 3) that displays moving images, an interpolation image generation step (step S3) of generating interpolation images between each frame image of the second moving image to increase the frame rate of the second moving image, and a display step (step S4) of displaying the first moving image and the second moving image including the interpolation images in a comparable manner. Therefore, moving images with different frame rates can be compared with higher accuracy.

[0044] In addition, the program causes a computer of a moving image display device (diagnostic console 3) that displays a moving image to function as a moving image acquisition unit (control unit 31) that acquires a first moving image captured at a first frame rate and a second moving image captured at a second frame rate lower than the first frame rate, an interpolation image generation unit (control unit 31) that generates an interpolation image between each frame image of the second moving image to increase the frame rate of the second moving image, and a display control unit (control unit 31) that causes the display unit to display the first moving image and the second moving image including the interpolation image in a comparable manner. Therefore, moving images with different frame rates can be compared with higher accuracy.

[0045] Note that the description in the present embodiment is an example of a suitable moving image display system according to the present invention and is not limited thereto.

[0046] For example, in the above embodiment, the case where the present invention is applied to a moving image of the chest is described as an example, but it is not limited thereto, and it may be applied to a moving image of another part.

[0047] In addition, in the above description, an example in which a hard disk, a semiconductor non-volatile memory, etc. are used as a computer-readable medium of the program according to the present invention is disclosed, but it is not limited to this example. As other computer-readable media, portable recording media such as CD-ROM can be applied. In addition, a carrier wave is also applied as a medium for providing the data of the program according to the present invention via a communication line.

[0048] In addition, regarding the detailed configuration and detailed operation of each device constituting the moving image display system 100, it can be appropriately changed within a range not departing from the gist of the present invention.

Explanation of Reference Numerals

[0049] 100 Moving image display system 1 Imaging device 11 Radiation source 12 Radiation Irradiation Control Device 13 Radiation Detection Unit 14 Reading Control Device 2 Imaging Console 21 Control Unit 22 Memory Unit 23 Operation Unit 24 Display Unit 25 Communication Unit 26 Bus 3 Diagnostic Console (Dynamic Image Display Device) 31 Control Unit (Dynamic Image Acquisition Unit, Interpolation Image Generation Unit, Display Control Unit) 32 Memory Unit 33 Operation Unit 34 Display Unit 35 Communication Unit 36 Bus

Claims

1. A moving image acquisition unit that acquires a first moving image captured at a first frame rate and a second moving image captured at a second frame rate lower than the first frame rate; An interpolation image generation unit that generates interpolation images between each frame image of the second moving image to increase the frame rate of the second moving image; A display unit that displays the first moving image and the second moving image including the interpolation image in a comparable manner; A moving image display device comprising:

2. The moving image display device according to claim 1, wherein the interpolation image generation unit generates an interpolation image so as to be the same as the first frame rate.

3. The moving image display device according to claim 1, wherein the interpolation image generation unit generates the interpolation image based on the front and rear frame images.

4. The moving image display device according to claim 1, wherein the first moving image and the second moving image are moving images captured by different imaging devices.

5. The moving image display device according to claim 1, wherein when the interpolation image is displayed as a still image, the display unit displays a message indicating that the image is an interpolation image.

6. In a moving image display device for displaying a moving image, A moving image acquisition step of acquiring a first moving image captured at a first frame rate and a second moving image captured at a second frame rate lower than the first frame rate; An interpolation image generation step of generating interpolation images between each frame image of the second moving image to increase the frame rate of the second moving image; A display step of displaying the first moving image and the second moving image including the interpolation image in a comparable manner; A moving image display method including:

7. A computer of a moving image display device that displays a moving image, A moving image acquisition unit that acquires a first moving image captured at a first frame rate and a second moving image captured at a second frame rate lower than the first frame rate, An interpolation image generation unit that generates an interpolation image between each frame image of the second moving image and increases the frame rate of the second moving image, A display control unit that causes a display unit to display the first moving image and the second moving image including the interpolation image in a comparable manner, A program that functions as such.

Images