IMAGE PROCESSING APPARATUS, IMAGE PROCESSING METHOD, IMAGE DISPLAY SYSTEM AND PROGRAM

Abstract

An image processing device (shooting console 2) comprising an acquisition unit (control unit 21) that acquires frame images that constitute a moving image, an interpolated frame image generation unit (control unit 21) that generates an interpolated frame image to be inserted between an unacquired frame image acquired after the multiple frame images and an immediately preceding frame image acquired immediately before the unacquired frame image based on the multiple frame images acquired by the acquisition unit, and an output unit (control unit 21) that outputs the immediately preceding frame image and the interpolated frame image.

Description

[Technical field]

[0001] The present invention relates to an image processing device and an image processing method. , Image display system and programs. [Background technology]

[0002] Conventionally, in capturing moving images, frame interpolation is performed by generating an interpolation frame image between frame images acquired by capturing the moving images, which directly improves the frame rate of the displayed moving images.

[0003] Patent Document 1 describes a technology in which X-rays are irradiated intermittently at a predetermined exposure interval, frame images are acquired in accordance with the exposure, and interpolation frame images are generated by interpolation for timing when frame images are missing, and a moving image with the interpolated frames is displayed. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] International Publication No. 2016 / 021240 Summary of the Invention [Problem to be solved by the invention]

[0005] In conventional frame interpolation, when displaying an interpolated frame image between a first frame image and a second frame image, it is necessary to display the interpolated frame image before the second frame image, as shown in Fig. 7. Therefore, after image processing of the second frame image, the display of the moving image is delayed by at least the time T0 required for the generation processing of the interpolated frame image. Furthermore, in Patent Document 1, the display of moving images is also delayed.

[0006] An object of the present invention is to reduce a display delay of a moving image when frame interpolation of the moving image is performed. [Means for solving the problem]

[0007] In order to solve the above problems, the image processing device according to the present invention comprises: an acquisition unit for acquiring frame images constituting a moving image; Based on the plurality of frame images acquired by the acquisition unit, A first interpolation frame image between the last frame image of the plurality of acquired frame images and a yet-to-be-acquired frame image to be acquired immediately after the last frame image. Statue an interpolation frame image generating unit for generating an interpolation frame image; The frame image Statue an output unit that outputs to a display device; Equipped with 、 The output unit outputs the first interpolation frame image after the last frame image. do. Further, the image processing device according to the present invention comprises: an acquisition unit for acquiring frame images constituting a moving image; Based on the plurality of frame images acquired by the acquisition unit, an interpolation frame image generating unit that generates a second interpolation frame image between a first frame image and a second frame image that are consecutive, from among unacquired frame images that are to be acquired after the plurality of acquired frame images; an output unit that outputs the frame image to a display device; Equipped with The output section outputs the second interpolated frame image after the acquired first frame image.

[0008] In order to solve the above problems, an image processing method according to the present invention includes: In the image processing device, an acquisition step of acquiring frame images constituting a moving image; Based on the plurality of frame images acquired by the acquisition step, A first interpolation frame image between the last frame image of the plurality of acquired frame images and a yet-to-be-acquired frame image to be acquired immediately after the last frame image. Statue an interpolation frame image generating step; The frame image Statue an output step of outputting the image to a display device; Including fruit, The output step outputs the first interpolation frame image after the last frame image. . Further, an image processing method according to the present invention comprises: In the image processing device, an acquisition step of acquiring frame images constituting a moving image; Based on the plurality of frame images acquired by the acquisition step, an interpolated frame image generating step of generating a second interpolated frame image between a first frame image and a second frame image that are consecutive, among unacquired frame images to be acquired after the plurality of acquired frame images; an output step of outputting the frame image to a display device; Including, The output step outputs the second interpolated frame image after the acquired first frame image.

[0009] In order to solve the above problems, the program according to the present invention comprises: The computer of the image processing device, an acquisition unit for acquiring frame images constituting a moving image; Based on the plurality of frame images acquired by the acquisition unit, A first interpolation frame image between the last frame image of the plurality of acquired frame images and a yet-to-be-acquired frame image to be acquired immediately after the last frame image. Statue an interpolation frame image generating unit for generating the interpolation frame image; The frame image Statue an output unit for outputting to a display device; Function as 、 The output unit outputs the first interpolation frame image after the last frame image. do. In addition, the program according to the present invention is The computer of the image processing device, an acquisition unit for acquiring frame images constituting a moving image; Based on the plurality of frame images acquired by the acquisition unit, an interpolation frame image generating unit that generates a second interpolation frame image between a first frame image and a second frame image that are consecutive, from among unacquired frame images that are to be acquired after the plurality of acquired frame images; an output unit that outputs the frame image to a display device; Function as a The output section outputs the second interpolated frame image after the acquired first frame image. In order to solve the above problems, the image display system according to the present invention comprises: An image display system including an image processing device and a display device, The image processing device includes: an acquisition unit for acquiring frame images constituting a moving image; Based on the plurality of frame images acquired by the acquisition unit, A first interpolation frame image between the last frame image of the plurality of acquired frame images and a yet-to-be-acquired frame image to be acquired immediately after the last frame image. Statue an interpolation frame image generating unit for generating an interpolation frame image; The frame image Statue an output unit that outputs to the display device, the output unit outputs the first interpolation frame image after the last frame image; The display device displays the frame image output by the output unit. Statue Display. Further, the image display system according to the present invention comprises: An image display system including an image processing device and a display device, The image processing device includes: an acquisition unit for acquiring frame images constituting a moving image; Based on the plurality of frame images acquired by the acquisition unit, an interpolation frame image generating unit that generates a second interpolation frame image between a first frame image and a second frame image that are consecutive, from among unacquired frame images that are to be acquired after the plurality of acquired frame images; an output unit that outputs the frame image to the display device, the output unit outputs the second interpolation frame image after the acquired first frame image; The display device displays the frame image output by the output section. Effect of the Invention

[0010] According to the present invention, when frame interpolation of a moving image is performed, a delay in displaying the moving image can be reduced. [Brief description of the drawings]

[0011] [Figure 1] 1 is a diagram showing an overall configuration of a moving image display system according to an embodiment of the present invention. [Diagram 2] 13 is a flowchart showing a frame image output process. [Diagram 3] 10 is a timing chart showing a frame image output process. [Figure 4] 13 is a flowchart showing a frame image output process. [Diagram 5] 13 is a flowchart showing a frame image output process. [Figure 6] 13 is a flowchart showing a frame image output process. [Figure 7]1 is a timing chart showing a conventional frame image output process. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. However, the scope of the invention is not limited to the illustrated examples.

[0013] [Moving image display system 100] FIG. 1 shows the overall configuration of a moving image display system 100 according to this embodiment. As shown in FIG. 1, the moving image display system 100 is configured such that an imaging device 1 and an imaging console 2 (image processing device) are connected by a communication cable or the like, and the imaging console 2 and a diagnostic console 3 are connected via a communication network NT such as a LAN (Local Area Network).

[0014] [Photographing device 1] The photographing device 1 is a photographing means for photographing a moving image of a subject having periodicity, such as changes in shape of the expansion and contraction of the lungs accompanying respiratory movement, or heartbeat. Video photography refers to obtaining multiple images by repeatedly irradiating a subject with pulsed radiation such as X-rays at a specified time interval (pulsed irradiation) or by continuously irradiating the subject with a low dose rate without interruption (continuous irradiation). In other words, video photography refers to continuously radiographing a moving image of a periodic target area along the time axis. Note that the video may be captured using not only radiation such as X-rays, but also ultrasound or magnetism. A series of images obtained by moving image capture is called a “moving image.” Specifically, the moving image is a medical moving image, such as an X-ray moving image or an echo moving image. Furthermore, moving images can be acquired by capturing images using a semiconductor image sensor such as a flat panel detector (FPD). Each of the multiple images constituting the moving image is called a frame image. In the following embodiment, a case where moving images are captured by pulse irradiation is described as an example. In the following embodiment, a case where the subject M is the chest of a subject is described as an example, but the present invention is not limited to this.

[0015] The radiation source 11 is disposed at a position facing the radiation detection unit 13 with the subject M in between, and irradiates the subject M with radiation (X-rays) under the control of the radiation irradiation control device 12. The radiation irradiation control device 12 is connected to the imaging console 2, and controls the radiation source 11 based on radiation irradiation conditions input from the imaging console 2 to perform radiation imaging. The radiation irradiation conditions include, for example, a pulse rate, a pulse width, a pulse interval, the number of frames per imaging, the value of the X-ray tube current, the value of the X-ray tube voltage, and the type of additional filter. The pulse rate is the number of radiation exposures per second, and corresponds to the frame rate described below. The pulse width is the radiation exposure time per radiation exposure. The pulse interval is the time from the start of one radiation exposure to the start of the next radiation exposure, and corresponds to the frame interval described below.

[0016] The radiation detection unit 13 is composed of a semiconductor image sensor such as an FPD. The FPD has, for example, a glass substrate or the like, and a plurality of detection elements (pixels) are arranged in a matrix at predetermined positions on the substrate. The detection elements (pixels) detect radiation that is irradiated from a radiation source 11 and has passed through at least a subject M according to its intensity, and convert the detected radiation into an electrical signal and accumulate the signal. Each pixel is configured to include a switching section such as a TFT (Thin Film Transistor). FPDs include an indirect conversion type in which X-rays are converted into an electric signal by a photoelectric conversion element via a scintillator, and a direct conversion type in which X-rays are directly converted into an electric signal, and either type may be used. In this embodiment, the pixel values ​​(signal values) of the image data generated by the radiation detection unit 13 are density values, and the greater the amount of transmitted radiation, the higher the density values. The radiation detection unit 13 is disposed opposite the radiation source 11 with the subject M interposed therebetween.

[0017] The reading control device 14 is connected to the imaging console 2 . The read control device 14 controls the switching units of each pixel of the radiation detection unit 13 based on the image reading conditions input from the radiography console 2, switches the reading of the electrical signals accumulated in each pixel, and obtains image data by reading the electrical signals accumulated in the radiation detection unit 13. This image data is a frame image. Then, the reading control device 14 assigns an identification ID and a frame number to the acquired frame image, and outputs the acquired frame image to the console 2 for radiography. The image reading conditions include, for example, the frame rate, the frame interval, the pixel size, the image size (matrix size), and the like. The frame rate is the number of frame images acquired per second, which is equal to the pulse rate. The frame interval is the time from the start of acquisition of one frame image to the start of acquisition of the next frame image, which is equal to the pulse interval.

[0018] The radiation irradiation control device 12 and the reading control device 14 are connected to each other and exchange synchronous signals with each other to synchronize the radiation irradiation operation and the image reading operation.

[0019] [Filming console 2] The imaging console 2 outputs radiation irradiation conditions and image reading conditions to the imaging device 1 to control the operations of the imaging device 1 for radiography and reading of radiographic images. The imaging console 2 generates an interpolated frame image as soon as it acquires a frame image from the imaging device 1, and outputs a moving image consisting of the frame image and the interpolated frame image. The imaging console 2 outputs and displays the moving image on the display unit 24, so that the user who is the photographer can check the frame images being captured as a moving image. As shown in FIG. 1, the radiography console 2 is configured to include a control unit 21, a storage unit 22, an operation unit 23, a display unit 24, and a communication unit 25, and each unit is connected by a bus .

[0020] The control unit 21 is composed of a CPU (Central Processing Unit), a RAM (Random Access Memory), etc. The CPU of the control unit 21 reads out the system program and various processing programs stored in the storage unit 22 in response to the operation of the operation unit 23, expands them in the RAM, executes various processes including imaging control processing according to the expanded programs, and centrally controls the operation of each part of the imaging console 2 and the radiation irradiation operation and reading operation of the imaging device 1. The control unit 21 functions as an acquisition unit that acquires frame images that constitute a moving image. The control unit 21 functions as an interpolation frame image generation unit that generates an interpolation frame image to be inserted between an unacquired frame image acquired after the multiple frame images and an immediately preceding frame image acquired immediately before the unacquired frame image, based on the multiple frame images acquired by the acquisition unit. The control unit 21 functions as an interpolation processing unit that performs interpolation processing based on the multiple frame images acquired by the acquisition unit to output an interpolated frame image that should be inserted between an unacquired frame image acquired after the multiple frame images and the immediately preceding frame image acquired immediately before it, together with the immediately preceding frame image. The control unit 21 functions as an output unit that outputs the previous frame image and the interpolated frame image. The control unit 21 functions as an image processing unit that performs noise reduction processing or image processing using a motion compensation filter on moving images. The control unit 21 functions as a first determination unit that determines whether to perform interpolation processing based on predetermined information. The control unit 21 functions as a second determination unit that determines whether to output an interpolated frame image based on predetermined information.

[0021] The storage unit 22 is composed of a non-volatile semiconductor memory, a hard disk, or the like. The storage unit 22 stores various programs executed by the control unit 21, parameters necessary for executing processing by the programs, or data such as processing results. For example, the storage unit 22 stores a program for executing imaging control processing. The various programs are stored in the form of readable program codes, and the control unit 21 sequentially executes operations according to the program codes. Specifically, the storage unit 22 stores a frame interpolation model used for interpolation processing. The frame interpolation model is, for example, an extrapolation machine learning model. The frame interpolation model does not have to use machine learning, and may be, for example, a model using optical flow. The extrapolation machine learning model is a frame interpolation model that predicts a frame image at an arbitrary later time from a plurality of frame images captured at an arbitrary time. For example, the machine learning model predicts a frame image at a time after t1 from frame images captured at different times t0 and t1 (t0 < t1). In addition, the storage unit 22 stores a series of frame images (moving images) with identification IDs and frame numbers attached, output from the imaging device 1. In addition, the storage unit 22 stores imaging order information. The imaging order information is attached to a series of frame images (moving images) and stored in the storage unit 22. The imaging order information includes radiation irradiation conditions (described above), image reading conditions (described above), subject information, examination information, and the like. The subject information is, for example, the subject's name, height, weight, age, gender, and the like. The examination information is, for example, the imaging site (such as the chest) and the diagnosis target (such as ventilation, pulmonary blood flow, etc.).

[0022] The operation unit 23 is configured with a keyboard equipped with cursor keys, numeric input keys, various function keys, etc., and a pointing device such as a mouse, and outputs instruction signals input by key operations on the keyboard or mouse operations to the control unit 21. The operation unit 23 may also be equipped with a touch panel on the display screen of the display unit 24, and in this case, outputs instruction signals input via the touch panel to the control unit 21. The radiographer uses the operation unit 23 to input the radiographing order information.

[0023] The display unit 24 is composed of a monitor such as an LCD (Liquid Crystal Display) or a CRT (Cathode Ray Tube), and displays input instructions and data from the operation unit 23 according to instructions of a display signal input from the control unit 21.

[0024] The communication unit 25 includes a LAN adapter, a modem, a TA (Terminal Adapter), etc., and controls data transmission and reception between each device connected to the communication network NT.

[0025] [Diagnostic Console 3] The diagnostic console 3 is a device that acquires moving images from the radiography console 2 and displays them on a monitor such as an LCD or a CRT.

[0026] [Frame image output processing] Next, the frame image output process in the radiography console 2 will be described with reference to FIG. The frame image output process is a process of acquiring frame images, generating interpolated frame images, and outputting a moving image. Here, the interpolation process in the frame image output process generates an interpolated frame image (N+1.5th frame image) between the N+1th frame image (previous frame image) and the unacquired N+2th frame image (unacquired frame image) from the Nth frame image and the N+1th frame image. Note that N starts from 1.

[0027] First, the control unit 21 acquires the N-th frame image from the photographing device 1 using the communication unit 25 (step S1; acquisition step). Specifically, the control unit 21 acquires the first frame image.

[0028] Next, the control unit 21 uses the communication unit 25 to acquire the (N+1)th frame image from the photographing device 1 (step S2; acquisition step). Specifically, the control unit 21 acquires the second frame image.

[0029] Next, the control unit 21 outputs the (N+1)th frame image (step S3). For example, the control unit 21 outputs the second frame image to the display unit 24, and causes the display unit 24 to display it.

[0030] Next, the control unit 21 generates an interpolated frame image, which is an (N+1.5)th frame image, from the Nth frame image and the (N+1)th frame image (step S4; interpolation processing step). For example, the control unit 21 generates a 2.5th interpolated frame image.

[0031] Next, the control unit 21 outputs the 2.5th interpolated frame image (step S5; output step). For example, the control unit 21 outputs the 2.5th interpolated frame image to the display unit 24, and causes the display unit 24 to display it.

[0032] Next, the control unit 21 determines whether there are any frame images that have not been obtained from the photographing device 1 (step S6). If there are any (step S6; YES), the control unit 21 adds 1 to N and advances the frame image output process to step S1. If there are no frame images (step S6; NO), the control unit 21 ends the frame image output process.

[0033] The flow of the frame image output process explained above will be explained using the timing chart (horizontal axis: time t) showing the frame image output process shown in Fig. 3. Note that Fig. 7 is a timing chart showing conventional frame image output processing. Time T0 is the time required for the interpolation process (step S4). Time T1 is a frame interval. Time T2 is the time required for image processing. Time T3 is the time it takes from acquisition of the (N+1)th frame image to output of the immediately following frame image. In this frame image output process, time T3 is equal to time T2. On the other hand, in the past, time T3 was equal to time T2 plus time T0. That is, in this frame image output process, the display delay is determined only by the time required for image processing. Therefore, according to this frame image output process, when frame interpolation of a moving image is performed, the display delay of the moving image is reduced.

[0034] In the above description, the (N+1.5)th interpolated frame image is generated from the Nth frame image and the (N+1)th frame image, but the present invention is not limited to this. As shown in FIG. 4, an N+2.5th interpolated frame image between the N+2nd frame image (the immediately preceding frame image) and the unacquired N+3rd frame image (unacquired frame image) may be generated from the Nth, N+1th, and N+2nd frame images. First, the control unit 21 acquires the N-th frame image from the photographing device 1 using the communication unit 25 (step S11; acquisition step). Specifically, the control unit 21 acquires the first frame image. Next, the control unit 21 uses the communication unit 25 to acquire the (N+1)th frame image from the photographing device 1 (step S12; acquisition step). Specifically, the control unit 21 acquires the second frame image. Next, the control unit 21 uses the communication unit 25 to acquire the (N+2)th frame image from the photographing device 1 (step S13; acquisition step). Specifically, the control unit 21 acquires the third frame image. Next, the control unit 21 outputs the (N+2)th frame image (step S14). For example, the control unit 21 outputs the third frame image to the display unit 24, and causes the display unit 24 to display it. Next, the control unit 21 generates an interpolated frame image, which is an (N+2.5)th frame image, from the Nth, (N+1)th, and (N+2)th frame images (step S15; interpolation processing step). For example, the control unit 21 generates a 3.5th interpolated frame image. Next, the control unit 21 outputs the 3.5th interpolated frame image (step S16; output step). For example, the control unit 21 outputs the 3.5th interpolated frame image to the display unit 24, and causes the display unit 24 to display it. Next, the control unit 21 determines whether there are any frame images that have not been obtained from the photographing device 1 (step S17). If there are any (step S17; YES), the control unit 21 adds 1 to N and advances the frame image output process to step S11. If there are no frame images (step S17; NO), the control unit 21 ends the frame image output process.

[0035] In addition, in the above-described interpolation process, an interpolated frame image is generated between the immediately preceding frame image at the end of the multiple acquired frame images and the unacquired frame image acquired immediately after that, but this is not limited to this. For example, from the acquired Nth frame image and the acquired N+1th frame image, an N+2.5th interpolated frame image between the unacquired N+2th frame image (previous frame image) and the unacquired N+3rd frame image (unacquired frame image) or an N+3.5th interpolated frame image between the unacquired N+3rd frame image (previous frame image) and the unacquired N+4th frame image (unacquired frame image) may be generated.

[0036] Furthermore, the control unit 21 may execute various types of image processing, such as sharpening and / or smoothing, before outputting the frame image.

[0037] [Determination process] Next, a frame image output process including a determination process in the imaging console 2 will be described with reference to Figures 5 and 6. The determination process is a process for determining whether or not to generate or output an interpolated frame image.

[0038] 5 shows the frame image output process including a process in which the control unit 21 determines whether or not to generate an interpolated frame image (step S24; first determination step). If it is determined that an interpolated frame image is to be generated (step S24; YES), the control unit 21 advances the frame image output process to step S25. If it is determined that an interpolated frame image is not to be generated (step S24; NO), the control unit 21 advances the frame image output process to step S27. The processing contents of the other steps are the same as those of FIG.

[0039] Specifically, the control unit 21 determines whether or not to generate an interpolated frame image from at least one acquired frame image using a predetermined determination algorithm. That is, the control unit 21 determines whether or not to perform an interpolation process based on the analysis result of the frame image. For example, the determination algorithm determines the degree of correlation between a plurality of adjacent frame images, and determines to generate an interpolated frame image only when the degree of correlation is equal to or greater than a threshold value. Also, for example, the determination algorithm determines that an interpolated frame image is to be generated only when the composition ratio of low frequency components among the spatial frequency components contained in at least one or more frame images is less than a threshold value. Also, for example, the determination algorithm determines to generate an interpolated frame image only when the enlargement magnification of at least one frame image is less than a threshold value. The enlargement magnification is the enlargement ratio of the display area to the original image when enlarging and displaying a region of interest (ROI).

[0040] Also, for example, the determination algorithm detects the magnitude of the motion of the subject M from the optical flow of multiple adjacent frame images, and determines to generate an interpolated frame image only when the magnitude of the motion is less than a threshold value. This is particularly effective when the subject M is a subject that moves a lot, such as a fetus. Also, for example, the determination algorithm determines that an interpolated frame image is to be generated only when the observation magnification in the imaging console 2 is less than a threshold value. Also, for example, the determination algorithm determines, based on the subject information input to the console 2 for radiography, that if the subject is a child, an interpolated frame image is not to be generated.

[0041] Specifically, the control unit 21 determines whether or not to generate an interpolated frame image using a predetermined determination algorithm based on information from an auxiliary imaging device such as an optical camera attached to the imaging console 2 and / or a sensor attached to the imaging device 1. In other words, the control unit 21 determines whether or not to perform an interpolation process based on setting information of a device for generating frame images. For example, the determination algorithm detects the motion of a subject from a moving image captured by an auxiliary imaging device, and determines to generate an interpolated frame image only if the magnitude of the motion is less than a threshold value. The magnitude of the motion can be calculated using a known motion detection technique such as optical flow. Also, for example, the judgment algorithm detects the movement of the X-ray generating unit or the X-ray detector from sensor information attached to the radiation source 11 or the radiation detection unit 13 of the imaging device 1, and judges to generate an interpolated frame image only when the magnitude of the movement is less than a threshold value.

[0042] 6 shows the frame image output process including a process in which the control unit 21 determines whether or not to output an interpolated frame image (step S35; second determination step). If it is determined that the image is to be output (step S35; YES), the control unit 21 advances the frame image output process to step S36. If it is determined that the image is not to be output (step S35; NO), the control unit 21 advances the frame image output process to step S37. The processing contents of the other steps are the same as those of FIG.

[0043] Specifically, the control unit 21 determines whether or not to output an interpolated frame image using a predetermined determination algorithm, based on at least one acquired frame image and the generated interpolated frame image. For example, the determination algorithm determines the degree of correlation between an interpolated frame image and the frame image immediately preceding it, and determines to output the interpolated frame image only when the degree of correlation is equal to or greater than a threshold value.

[0044] Also, for example, the determination algorithm may be the same as the determination algorithm used to determine whether to generate the above-mentioned interpolated frame image. Note that the description of "at least one or more acquired frame images" may be read as "at least one or more frame images" or "interpolated frame images," and "generation" may be read as "output," etc.

[0045] 〔others〕 In the above, the control unit 21 outputs the frame image or the interpolated frame image as is, but the control unit 21 may output the frame image or the interpolated frame image after correcting it by various noise reduction processes. The noise reduction processes are processes using a recursive filter or persistence technology, for example. The control unit 21 may take a weighted average of at least two or more temporally consecutive images to obtain a frame image or an interpolated frame image, and then correct and output the frame image or the interpolated frame image. Note that in the weighted average, the frame image or the interpolated frame image may be weighted before the weighted average is obtained. The control unit 21 may use a known motion compensation filter to modify and output the frame image or the interpolated frame image.

[0046] Furthermore, the control unit 21 may switch between the interpolation process by interpolation and the interpolation process by extrapolation at a predetermined timing. In this case, a frame interpolation model for interpolation needs to be stored in the storage unit 22. The frame interpolation model for interpolation is a frame interpolation model that predicts an interpolated image between a plurality of frame images captured at an arbitrary time from the frame images. Specifically, the predetermined timing is the timing of completion of video shooting. In other words, while the video shooting device 1 and the video shooting console 2 are working together to shoot a video, the control unit 21 generates an interpolated frame image using an extrapolation frame interpolation model, and after the video shooting is completed, generates an interpolated frame image using an interpolation frame interpolation model. In this way, display delays can be reduced during video capture, and more accurate display can be achieved after video capture is completed.

[0047] [Effect 1] As described above, the image processing device (the shooting console 2) comprises an acquisition unit (control unit 21) that acquires frame images that constitute a moving image, an interpolated frame image generation unit (control unit 21) that generates an interpolated frame image to be inserted between an unacquired frame image acquired after the multiple frame images and an immediately preceding frame image acquired immediately before the unacquired frame image based on the multiple frame images acquired by the acquisition unit, and an output unit (control unit 21) that outputs the immediately preceding frame image and the interpolated frame image. This makes it possible to reduce the display delay of moving images when frame interpolation of the moving images is performed.

[0048] The image processing method also includes an acquisition step (steps S1 and S2) in an image processing device (shooting console 2) for acquiring frame images that constitute a moving image, an interpolated frame image generation step (step S4) for generating an interpolated frame image to be inserted between an unacquired frame image acquired after the multiple frame images and an immediately preceding frame image acquired immediately before the unacquired frame image based on the multiple frame images acquired by the acquisition step, and an output step (step S5) for outputting the immediately preceding frame image and the interpolated frame image. This makes it possible to reduce the display delay of moving images when frame interpolation of the moving images is performed.

[0049] The program also causes the computer of the image processing device (the shooting console 2) to function as an acquisition unit (control unit 21) that acquires frame images that constitute a moving image, an interpolated frame image generation unit (control unit 21) that generates an interpolated frame image to be inserted between an unacquired frame image acquired after the multiple frame images and an immediately preceding frame image acquired immediately before the unacquired frame image based on the multiple frame images acquired by the acquisition unit, and an output unit (control unit 21) that outputs the immediately preceding frame image and the interpolated frame image. This makes it possible to reduce the display delay of moving images when frame interpolation of the moving images is performed.

[0050] [Effect 2] As described above, the image processing device (the shooting console 2) comprises an acquisition unit (control unit 21) that acquires frame images that constitute a moving image, an interpolation processing unit (control unit 21) that performs interpolation processing based on the multiple frame images acquired by the acquisition unit to output an interpolated frame image that should be inserted between an unacquired frame image acquired after the multiple frame images and the immediately preceding frame image acquired immediately before it, together with the immediately preceding frame image, and a first judgment unit (control unit 21) that judges whether or not to perform interpolation processing based on specified information. This makes it possible to reduce delays in displaying moving images and prevent degradation of image quality when frame interpolation of moving images is performed.

[0051] The image processing device (shooting console 2) also includes an acquisition unit (control unit 21) that acquires frame images that constitute a moving image, an interpolation processing unit (control unit 21) that performs interpolation processing based on the multiple frame images acquired by the acquisition unit to output an interpolated frame image that should be inserted between an unacquired frame image acquired after the multiple frame images and the immediately preceding frame image acquired immediately before it, together with the immediately preceding frame image, and a second judgment unit (control unit 21) that judges whether or not to output the interpolated frame image based on specified information. This makes it possible to reduce delays in displaying moving images and prevent degradation of image quality when frame interpolation of moving images is performed.

[0052] The image processing method also includes an acquisition step (steps S21 and S22) in which frame images constituting a moving image are acquired in an image processing device (the shooting console 2), an interpolation processing step (step S25) in which, based on the multiple frame images acquired in the acquisition step, an interpolation frame image to be inserted between an unacquired frame image acquired after the multiple frame images and an immediately preceding frame image acquired immediately before the unacquired frame image is performed, and output together with the immediately preceding frame image, and a first judgment step (step S24) in which it is judged whether or not to perform the interpolation processing based on specified information. This makes it possible to reduce delays in displaying moving images and prevent degradation of image quality when frame interpolation of moving images is performed.

[0053] The image processing method also includes an acquisition step (steps S31 and S32) in which frame images constituting a moving image are acquired in an image processing device (the shooting console 2), an interpolation processing step (step S34) which performs an interpolation process based on the multiple frame images acquired in the acquisition step to output an interpolated frame image that should be inserted between an unacquired frame image acquired after the multiple frame images and an immediately preceding frame image acquired immediately before the unacquired frame image, together with the immediately preceding frame image, and a second judgment step (step S35) which judges whether or not to output the interpolated frame image based on specified information. This makes it possible to reduce delays in displaying moving images and prevent degradation of image quality when frame interpolation of moving images is performed.

[0054] The program also causes the computer of the image processing device (shooting console 2) to function as an acquisition unit (control unit 21) that acquires frame images that constitute a moving image, an interpolation processing unit (control unit 21) that performs interpolation processing based on the multiple frame images acquired by the acquisition unit to output an interpolated frame image that should be inserted between an unacquired frame image acquired after the multiple frame images and the immediately preceding frame image acquired immediately before it, together with the immediately preceding frame image, and a first judgment unit (control unit 21) that judges whether or not to perform interpolation processing based on specified information. This makes it possible to reduce delays in displaying moving images and prevent degradation of image quality when frame interpolation of moving images is performed.

[0055] The program also causes the computer of the image processing device (shooting console 2) to function as an acquisition unit (control unit 21) that acquires frame images that constitute a moving image, an interpolation processing unit (control unit 21) that performs interpolation processing based on the multiple frame images acquired by the acquisition unit to output an interpolated frame image that should be inserted between an unacquired frame image acquired after the multiple frame images and the immediately preceding frame image acquired immediately before it, together with the immediately preceding frame image, and a second judgment unit (control unit 21) that judges whether or not to output the interpolated frame image based on specified information. This makes it possible to reduce delays in displaying moving images and prevent degradation of image quality when frame interpolation of moving images is performed.

[0056] It should be noted that the description of this embodiment is merely an example of a suitable moving image display system according to the present invention, and the present invention is not limited to this.

[0057] For example, in the above description, a hard disk or a non-volatile semiconductor memory is used as a computer-readable medium for the program according to the present invention, but the present invention is not limited to this example. As other computer-readable media, a portable recording medium such as a CD-ROM can be used. Furthermore, a carrier wave can be used as a medium for providing data for the program according to the present invention via a communication line.

[0058] Additionally, the detailed configurations and operations of the devices constituting the moving image display system 100 may be modified as appropriate without departing from the spirit of the present invention. [Industrial Applicability]

[0059] The present invention can be utilized in an image processing device, an image processing method, and a program. [Explanation of symbols]

[0060] 100 Video display system 1. Imaging device 11 Radiation source 12 Radiation exposure control device 13 Radiation detection unit 14 Read control device 2. Shooting console 21 control unit (acquisition unit, interpolation frame image generation unit, interpolation processing unit, output unit, image processing unit, first judgment unit, second judgment unit) 22 Memory section 23 Control section 24 Display section 25 Communications Department 26 Bus 3 Diagnostic Console

Claims

1. an acquisition unit for acquiring frame images constituting a moving image; Based on the plurality of frame images acquired by the acquisition unit, an interpolation frame image generating unit that generates a first interpolation frame image between a last frame image of the plurality of acquired frame images and a yet-to-be-acquired frame image that is to be acquired immediately after the last frame image; an output unit that outputs the frame image to a display device; Equipped with the output unit outputs the first interpolation frame image after the last frame image. Image processing device.

2. The image processing device according to claim 1 , wherein the interpolated frame image generation section generates the first interpolated frame image by using a machine learning model as a frame interpolation model.

3. an acquisition unit for acquiring frame images constituting a moving image; Based on the plurality of frame images acquired by the acquisition unit, an interpolation frame image generating unit that generates a second interpolation frame image between a first frame image and a second frame image that are consecutive, from among unacquired frame images that are to be acquired after the plurality of acquired frame images; an output unit that outputs the frame image to a display device; Equipped with the output unit outputs the second interpolation frame image after the acquired first frame image. Image processing device.

4. The image processing device according to claim 3 , wherein the interpolated frame image generation section generates the second interpolated frame image by using a machine learning model as a frame interpolation model.

5. The image processing apparatus according to claim 1 , wherein the moving image is a medical moving image.

6. 6. The image processing apparatus according to claim 5, wherein the medical moving image is an X-ray moving image.

7. 6. The image processing apparatus according to claim 5, wherein the medical moving image is an echo moving image.

8. The image processing device according to claim 1 , further comprising an image processing unit that performs noise reduction processing on the moving image.

9. The image processing device according to claim 8 , wherein the noise reduction process uses a recursive filter or persistence.

10. 4. The image processing device according to claim 1, further comprising an image processing unit that performs image processing on the moving image using a motion compensation filter.

11. 4. The image processing device according to claim 1, wherein the interpolation frame image generating section switches between the interpolation process by interpolation and the interpolation process by extrapolation at a predetermined timing.

12. In the image processing device, an acquisition step of acquiring frame images constituting a moving image; Based on the plurality of frame images acquired by the acquisition step, an interpolated frame image generating step of generating a first interpolated frame image between a last frame image of the plurality of acquired frame images and an unacquired frame image to be acquired immediately after the last frame image; an output step of outputting the frame image to a display device; Including, The image processing method, in which the output step outputs the first interpolation frame image after the last frame image.

13. In the image processing device, an acquisition step of acquiring frame images constituting a moving image; Based on the plurality of frame images acquired by the acquisition step, an interpolated frame image generating step of generating a second interpolated frame image between a first frame image and a second frame image that are consecutive, among unacquired frame images to be acquired after the plurality of acquired frame images; an output step of outputting the frame image to a display device; Including, The image processing method, wherein the output step outputs the second interpolated frame image after the acquired first frame image.

14. The computer of the image processing device, an acquisition unit for acquiring frame images constituting a moving image; Based on the plurality of frame images acquired by the acquisition unit, an interpolation frame image generating unit that generates a first interpolation frame image between a last frame image of the plurality of acquired frame images and an unacquired frame image to be acquired immediately after the last frame image; an output unit that outputs the frame image to a display device; Function as a The output unit is a program that outputs the first interpolation frame image after the last frame image.

15. The computer of the image processing device, an acquisition unit for acquiring frame images constituting a moving image; Based on the plurality of frame images acquired by the acquisition unit, an interpolated frame image generating unit that generates a second interpolated frame image between a first frame image and a second frame image that are consecutive among unacquired frame images that are to be acquired after the plurality of acquired frame images; an output unit that outputs the frame image to a display device; Function as a The output unit is a program that outputs the second interpolation frame image after the acquired first frame image.

16. An image display system including an image processing device and a display device, The image processing device includes: an acquisition unit for acquiring frame images constituting a moving image; Based on the plurality of frame images acquired by the acquisition unit, an interpolation frame image generating unit that generates a first interpolation frame image between a last frame image of the plurality of acquired frame images and a yet-to-be-acquired frame image that is to be acquired immediately after the last frame image; an output unit that outputs the frame image to the display device, the output unit outputs the first interpolation frame image after the last frame image; The display device displays the frame image output by the output section.

17. An image display system including an image processing device and a display device, The image processing device includes: an acquisition unit for acquiring frame images constituting a moving image; Based on the plurality of frame images acquired by the acquisition unit, an interpolation frame image generating unit that generates a second interpolation frame image between a first frame image and a second frame image that are consecutive, from among unacquired frame images that are to be acquired after the plurality of acquired frame images; an output unit that outputs the frame image to the display device, the output unit outputs the second interpolation frame image after the first frame image obtained; The display device displays the frame image output by the output section.