Image processing apparatus and program

Abstract

To reduce the stress and the wasteful stand-by time of a user when generating a composite static image from a plurality of frame images constituting a dynamic image.SOLUTION: An image processing device (console 2) for processing a dynamic image obtained with dynamic imaging for imaging the dynamic state of a subject by irradiating the subject with radiation comprises: an acquisition unit (control unit 21) which acquires a plurality of frame images constituting a dynamic image; a generation unit (control unit 21) which generates a first image being a composite static image obtained by combining at least two or more frame images in the plurality of frame images; and an output control unit (control unit 21) which outputs the first image to an output unit. The output control unit outputs a second image which is the static image based on at least one or more frame images in the plurality of frame images and has the lower image quality than the first image before generating the first image.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present invention relates to an image processing apparatus and a program.

Background Art

[0002] Conventionally, a technique for synthesizing a plurality of frame images constituting a moving image to generate a single synthesized still image has been known. For example, Patent Document 1 describes that a synthesized still image with increased resolution is automatically generated without the user specifying a reference frame image when generating a synthesized still image with increased resolution from a plurality of frame images constituting a moving image.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in the technique described in Patent Document 1, after the generation process of the synthesized still image is completely finished and the synthesized still image is displayed, the user can first check the content of the synthesized still image. Therefore, if the processing time from the start of the generation process of the synthesized still image to the display of the synthesized still image is long, the user's waiting time also becomes long, and the user feels time stress. Also, until the synthesized still image is displayed, the user cannot check the progress of whether the generation process of the synthesized still image is being executed normally. Furthermore, even if the synthesized still image is displayed, if it is an inappropriate image, for example, the automatically selected reference frame image is not what the user desires, reprocessing or reshooting is required, and as a result, the waiting time required to generate the synthesized still image is wasted.

[0005] An object of the present invention is to reduce the stress of the user and the wasted waiting time when generating a composite still image from a plurality of frame images constituting a moving image.

Means for Solving the Problems

[0006] To solve the above problems, the image processing apparatus according to claim 1 is An image processing apparatus that processes a moving image obtained by dynamic imaging that irradiates a subject with radiation to capture the dynamics of the subject, An acquisition unit that acquires a plurality of frame images constituting the moving image, A generation unit that generates a first image that is a composite still image obtained by synthesizing at least two or more of the plurality of frame images, An output control unit that outputs the first image to an output unit, and Before generating the first image, the output control unit outputs a second image that is a still image based on at least one or more of the plurality of frame images and has a lower image quality than the first image. and the generation unit generates a composite still image obtained by combining at least two or more of the plurality of frame images as the second image, the generation unit performs alignment for a plurality of resolutions in order from low resolution to high resolution, and generates a composite still image at the time when alignment at each resolution is completed as the second image It is characterized by doing so.

[0007] Claim 1 1 The program described in On a computer used in an image processing apparatus that processes a moving image obtained by dynamic imaging that irradiates a subject with radiation to capture the dynamics of the subject, A function of acquiring a plurality of frame images constituting the moving image, A function of generating a first image that is a composite still image obtained by synthesizing at least two or more of the plurality of frame images, A function of outputting the first image to an output unit is realized, The function of outputting to the output unit outputs a second image that is a still image based on at least one or more of the plurality of frame images and has a lower image quality than the first image before generating the first image.and the function of generating the image generates a composite still image obtained by combining at least two or more of the plurality of frame images as the second image, the function of generating the image performs alignment for a plurality of resolutions in order from low resolution to high resolution, and generates a composite still image at the time when alignment at each resolution is completed as the second image It is characterized by doing so.

Advantages of the Invention

[0008] According to the present invention, when generating a composite still image from a plurality of frame images constituting a moving image, it is possible to reduce the stress of the user and the wasted waiting time.

Brief Description of the Drawings

[0009]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Modes for Carrying Out the Invention

[0010] Hereinafter, embodiments of the present invention will be described with reference to the drawings. However, the scope of the invention is not limited to the illustrated examples.

[0011] <First Embodiment> 〔Configuration of Image Processing System 100〕 First, the configuration of the first embodiment will be described. FIG. 1 is a diagram showing an overall configuration example of an image processing system 100 according to the present embodiment. As shown in FIG. 1, the image processing system 100 is configured by connecting a photographing device 1 and a console 2 so that data can be transmitted and received therebetween.

[0012] The photographing device 1 is a photographing device that irradiates a subject with radiation and performs still image photographing or dynamic photographing. In the present embodiment, the dynamic photographing is performed, for example, on a diagnostic target part of the human body having a periodicity (cycle), such as the form of expansion and contraction of the lungs accompanying the respiratory movement and the pulsation of the heart, as the subject. A series of images obtained by continuously photographing the subject by dynamic photographing is called a dynamic image. Each of the plurality of images constituting the dynamic image is called a frame image. Note that the dynamic photographing includes video photographing, but does not include those that photograph still images while displaying a video. Also, the dynamic image includes a video, but does not include an image obtained by photographing a still image while displaying a video.

[0013] The photographing device 1 includes a radiation detector P, a photographing table 11 capable of loading the radiation detector P, and a radiation generator 12. The photographing table 11 is configured to be able to load the radiation detector P in its holder 11a.

[0014] The radiation detector P is composed of a semiconductor image sensor such as an FPD (Flat Panel Detector), and is provided so as to face the radiation generating device 12 with the subject H interposed therebetween. The radiation detector P has, for example, a glass substrate or the like, and at a predetermined position on the substrate, radiation (X-rays) irradiated from the radiation generating device 12 and transmitted through at least the subject H is detected according to its intensity, and a plurality of detection elements (pixels) that convert the detected radiation into an electrical signal and accumulate it are arranged in a matrix. Each pixel is configured to include a switching unit such as a TFT (Thin Film Transistor). The radiation detector P controls the switching unit of each pixel based on the image reading conditions input from the console 2, switches the reading of the electrical signal accumulated in each pixel, and reads the electrical signal accumulated in each pixel to acquire an image as image data. Then, the radiation detector P outputs the acquired image to the console 2.

[0015] The radiation generating device 12 is arranged at a position facing the radiation detector P with the subject H interposed therebetween, and irradiates the radiation detector P loaded in the holder 11a with radiation through the patient who is the subject H based on the radiation irradiation conditions input from the console 2 to perform imaging.

[0016] The console 2 outputs imaging conditions such as radiation irradiation conditions and image reading conditions to the imaging device 1 to control the radiation imaging and the operation of reading a radiation image (image) by the imaging device 1, and functions as an image processing device that performs image processing on the image acquired by the imaging device 1. As shown in FIG. 2, the console 2 includes a control unit 21, a storage unit 22, an operation unit 23, a display unit 24, and a communication unit 25, and each unit is connected by a bus 26.

[0017] The control unit 21 is composed of a CPU (Central Processing Unit), a RAM (Random Access Memory), etc. The CPU of the control unit 21 reads out the system program and various processing programs stored in the storage unit 22 according to the operation of the operation unit 23, expands them in the RAM, and centrally controls the operations of each part of the console 2, and the radiation irradiation operation and reading operation of the imaging device 1 according to the expanded programs. Further, the control unit 21 executes various processes including a composite still image generation process A described later using the simple captured image or moving image transmitted from the radiation detector P of the imaging device 1, and functions as an acquisition unit, a generation unit, and an output control unit.

[0018] Here, the outline of the composite still image generation process will be described below with reference to FIGS. 3 and 4. FIG. 3 shows the outline of the composite still image generation process. The composite still image generation process is a process of generating a still image capable of morphological diagnosis from a moving image. The composite still image refers to a still image obtained by synthesizing frame images.

[0019] First, the control unit 21 selects a reference frame image from a plurality of frame images of the moving image. This selection of the reference frame image may be automatic or manual. In the case of automatic selection, for example, like the prior art of Japanese Unexamined Patent Application Publication No. 2019-212138, the control unit 21 of the console 2 analyzes each of the plurality of frame images to calculate feature amounts, and determines the reference frame image based on the calculated feature amounts. In the example of FIG. 3, the frame image indicated by the thick frame and arrow is used as the reference frame image.

[0020] Next, the control unit 21 performs alignment of the frame images to be subjected to the synthesis process. In FIG. 3, the control unit 21 determines the three frame images before and after the reference frame image as the frame images to be subjected to the synthesis process, and shows an example in which alignment is performed with respect to the reference frame image.

[0021] Finally, the control unit 21 synthesizes the aligned frame images and ends the synthesized still image generation process. Here, as a method for synthesizing a plurality of frame images, for example, there is a method of taking the average value from pixels at the same spatial position. By performing the synthesized still image generation process, an effect of improving granularity can be obtained compared to a single frame image, and a still image with image quality equivalent to that of a simple photographed image can be created. That is, since it is not necessary to additionally perform simple photography after dynamic image shooting, a reduction in the radiation dose to the patient and an improvement in the workflow of the technician are expected. Here, the image quality includes at least one of resolution, fineness, granularity, and sharpness. In addition, factors that affect "image quality" include, for example, the number of frame images to be synthesized, the resolution, and the alignment accuracy between the frame images to be synthesized.

[0022] Here, the alignment accuracy depends on settings such as the algorithm of coordinate transformation, the similarity evaluation function for evaluating whether the transformed image matches the target image described later, the pixel block size referred to in the similarity evaluation, and the resolution level of the image. Among these, taking an example at the resolution level of the image, the result of alignment is different depending on whether alignment is performed with the image at the original magnification (high resolution) or whether alignment is first performed after reducing the image to a low resolution once. In the former case, since alignment is performed in a narrow range while referring to each original pixel, alignment in a narrow area is prioritized, and the alignment accuracy for fine structures such as blood vessels becomes high. On the other hand, in the latter case, since alignment is performed after thinning out the pixels with the reduction, alignment in a wide area is prioritized, and the alignment accuracy for large structures such as ribs and the diaphragm becomes high. Generally, alignment using multiple resolutions can achieve highly accurate alignment in both wide and narrow areas by first performing rough alignment at a low resolution and then performing alignment at a high resolution. At this time, if a plurality of frame images are synthesized in a state where only alignment for the low-resolution image is performed, in the synthesized still image, the sharpness of fine structures such as blood vessels that have not been aligned is in a reduced state.

[0023] Figure 4 shows an overview of the alignment process of the frame image to be synthesized in the synthetic still image generation process. In Figure 4, the alignment between two images is shown as an example. Note that the alignment process described later is in a form where the alignment for multiple resolutions is performed in order from low resolution to high resolution. In this embodiment, an example of alignment in two stages is shown. The target image is the image to be aligned. Specifically, it is the reference frame image. The target image is the image on which coordinate transformation is performed by alignment. Specifically, it is the frame image to be synthesized. The deformation information is the information used for alignment. Specifically, it is the information used for coordinate transformation.

[0024] First, the control unit 21 downsamples the target image and the target image. The control unit 21 obtains the target image and the target image that have been downsampled to a low resolution by downsampling.

[0025] Next, the control unit 21 aligns the target image and the target image at low resolution. The control unit 21 obtains the deformation information (low resolution) used for coordinate transformation of the target image.

[0026] Next, the control unit 21 upsamples the deformation information (low resolution). The control unit 21 obtains the upsampled deformation information. By upsampling the deformation information (low resolution), in the alignment of the target image and the target image at high resolution, the control unit 21 can use the upsampled deformation information as the initial deformation information.

[0027] Next, the control unit 21 uses the upsampled deformation information as the initial deformation information to align the target image and the target image at high resolution and ends the alignment process. The control unit 21 obtains the deformation information (high resolution).

[0028] In FIG. 4, alignment between two images is shown as an example. When there are a plurality of frame images to be subjected to the compositing process, the alignment process shown in FIG. 4 is repeatedly executed. This concludes the description of the outline of the composite still image generation process.

[0029] The storage unit 22 is constituted by a nonvolatile semiconductor memory, a hard disk, or the like. The storage unit 22 stores various programs executed by the control unit 21, parameters necessary for executing the processes by the programs, or data such as process results. The various programs are stored in the form of readable program codes, and the control unit 21 sequentially executes operations according to the program codes.

[0030] Also, the storage unit 22 stores imaging conditions (radiation irradiation conditions and image reading conditions) corresponding to the imaging site. Further, the storage unit 22 stores imaging order information transmitted from a RIS (Radiology Information System) or the like (not shown). The imaging order information includes patient information, examination information (examination ID, imaging site (including imaging direction), examination date, type of still image imaging or moving image imaging, etc.).

[0031] Also, the storage unit 22 stores the captured simple captured images, moving images, and images generated by image processing in association with patient information and examination information.

[0032] The operation unit 23 includes a keyboard having cursor keys, numeric input keys, and various function keys, and a pointing device such as a mouse, and outputs an instruction signal input by a key operation on the keyboard or a mouse operation to the control unit 21. Further, the operation unit 23 may include a touch panel on the display screen of the display unit 24, and in this case, outputs an instruction signal input via the touch panel to the control unit 21. Furthermore, the operation unit 23 is provided with an exposure switch for instructing dynamic imaging to the radiation generator 12.

[0033] The display unit 24 is composed of a monitor such as an LCD (Liquid Crystal Display) or a CRT (Cathode Ray Tube), and displays input instructions, data, etc. from the operation unit 23 according to the instructions of the display signal input from the control unit 21, functioning as an output unit.

[0034] The communication unit 25 has an interface for data transmission and reception with the radiation generator 12 and the radiation detector P. Note that the communication between the console 2, the radiation generator 12, and the radiation detector P may be wired communication or wireless communication. Also, the communication unit 25 includes a LAN adapter, a modem, a TA (Terminal Adapter), etc., and controls data transmission and reception with a RIS (not shown) connected to a communication network.

[0035] 〔Operation of the image processing system 100〕 When imaging order information of an imaging target is selected by the operation unit 23 on the console 2 with the radiation detector P set in the holder 11a in the imaging device 1, imaging conditions (radiation irradiation conditions and radiation image reading conditions) corresponding to the selected imaging order information are read from the storage unit 22 and transmitted to the imaging device 1 and set in the imaging device 1. When the positioning of the subject H is performed and the exposure switch is pressed, in the imaging device 1, radiation is irradiated by the radiation generator 12, and a plurality of frame images constituting a simple imaging image or a dynamic image are acquired by the radiation detector P and transmitted to the console 2.

[0036] On the console 2, when the control unit 21 functions as an acquisition unit and acquires a plurality of frame images constituting a simple imaging image or a dynamic image from the radiation detector P via the communication unit 25, the control unit 21 stores the acquired plurality of frame images constituting the simple imaging image or the dynamic image in the storage unit 22 in association with patient information and examination information. Further, the control unit 21 executes a composite still image generation process A based on the acquired dynamic image.

[0037] FIG. 5 is a flowchart showing the flow of the synthetic still image generation process A. The synthetic still image generation process A is executed in cooperation with the control unit 21 and the program stored in the storage unit 22.

[0038] First, the control unit 21 acquires a moving image including a plurality of frame images via the communication unit 25 (step S11).

[0039] Next, the control unit 21 selects a reference frame image from the plurality of frame images (step S12). Note that the selection of the reference frame image may be either automatic selection or manual selection as described above.

[0040] Next, the control unit 21 determines the frame images before and after the reference frame image by N frames as the frame images to be subjected to the synthesis process (step S13). Note that the number of frames (N) may be set in advance before the start of the synthetic still image generation process A, or may be set after the start of the synthetic still image generation process A. When setting after the start of the synthetic still image generation process A, for example, there is a method of setting according to the magnitude of the movement of the frame images before and after the reference frame image. Specifically, when there are many frames with large body movement and low correlation with the reference frame before and after, the number of frames (N) is decreased. Conversely, when there are many frames with high correlation with the reference frame before and after, the number of frames (N) is set to be larger.

[0041] Next, the control unit 21 aligns all of the frame images (before and after N frames) to be subjected to the synthesis process with respect to the reference frame image at a low resolution (step S14). Specifically, the low-resolution frame images obtained by reducing the resolution of the acquired frame images are aligned with each other, and deformation information a, which is information used for coordinate transformation, is obtained. Note that the alignment at a low resolution is processed at a higher speed than the alignment at a high resolution because the number of pixels to be aligned is small.

[0042] Next, when the alignment at low resolution is completed, the control unit 21 functions as a generation unit, performs coordinate transformation on the frame images (N frames before and after) using the deformation information a obtained in step S14, synthesizes the reference frame image and the frame images (N frames before and after) to be processed for synthesis, and generates a preview image (second image) (step S15).

[0043] Next, the control unit 21 causes the display unit 24, which functions as an output unit, to display the preview image (step S16). Also, confirmation items are displayed together with the preview image. The confirmation items are, for example, the correctness of the reference frame and the noise reduction effect of the image. Note that the image quality of the preview image is lower than that of the main image (first image) described later. Also, when synthesizing using the deformation information a, the preview image will be of low resolution, but if synthesized using the deformation information b obtained by upsampling the deformation information a in step S15, the resolution of the preview image will be the same as that of the main image described later. Also, if the generation of the preview image by the control unit 21 fails, or if the quality of the preview image generated by the control unit 21 does not meet a specific standard, the control unit 21 may output warning information to the display unit 24. Thereby, the user can immediately recognize the problem if there is an obvious problem with the preview image.

[0044] Next, the user checks the preview image, and if there is a problem, uses the operation unit 23 to send a processing stop request to the control unit 21 (step S17). Then, the control unit 21 aborts the alignment in step S18 described later (step S111). If the processing is aborted, the composite still image generation process A ends.

[0045] In parallel with the generation of the preview image in step S15, the generation process of the main image (steps S18 and S19) is started. First, the control unit 21 performs alignment of the frame image to be subjected to the composition process with respect to the reference frame image at high resolution (step S18). In the alignment, the deformation information b obtained by upsampling the deformation information a obtained in step S14 is used as the initial deformation information to obtain new deformation information c. In addition, if the upsampled deformation information b has been obtained in step S15, the deformation information b can be diverted as the initial deformation information.

[0046] Next, when the alignment at high resolution is completed, the control unit 21 functions as a generation unit, and using the deformation information c obtained in step S18, the frame images (N frames before and after) are subjected to coordinate transformation, and the reference frame image and the frame images (N frames before and after) to be subjected to the composition process are combined to generate the main image (the first image) (step S19).

[0047] Next, the control unit 21 causes the main image to be displayed on the display unit 24 that functions as an output unit (step S110), and ends the composite still image generation process A.

[0048] The main image generated by the composite still image generation process A is stored in the storage unit 22 in association with the patient information and examination information of the original moving image.

[0049] In this way, since the alignment at low resolution is processed faster than the alignment at high resolution, once a composite still image for preview is created and pre-displayed on the console when the alignment at low resolution is completed, the user can early confirm the correctness of the reference frame and the noise reduction effect of the image.

[0050] <Second Embodiment> Hereinafter, a second embodiment of the present invention will be described. In the first embodiment, the control unit 21 performs alignment processing at a low resolution on all the frame images (N frames before and after) to be subjected to synthesis processing, once performs preview image display, and then performs alignment processing at a high resolution and performs main image display. In contrast, in the second embodiment, the control unit 21 performs alignment processing on a certain number of frame images (n frames before and after, n < N) among the frame images to be subjected to synthesis processing, performs preview image display, and then performs alignment processing on the remaining frame images (N - n pieces) to be subjected to synthesis processing and performs main image display.

[0051] In the second embodiment, a program for executing a composite still image generation process B described later is stored in the storage unit 22.

[0052] Other configurations in the second embodiment are the same as those described with reference to FIGS. 1 to 2 in the first embodiment, and thus the description thereof is omitted. Also, the operation of the image processing system 100 in the second embodiment is the same as that in the first embodiment for the imaging device 1, and thus the description thereof is omitted. Hereinafter, the composite still image generation process B executed on the console 2 based on the moving image acquired from the imaging device 1 will be described.

[0053] FIG. 6 is a flowchart showing the flow of the composite still image generation process B. The composite still image generation process B is executed in cooperation with the control unit 21 and the program stored in the storage unit 22. In FIG. 6, steps S21 to S23 are the same as steps S11 to S13 in FIG. 5, and thus the description thereof is omitted. Note that in the composite still image generation process B, the alignment described later may be alignment at a plurality of resolutions.

[0054] The control unit 21 performs alignment of the frame images to be subjected to the synthesis process with respect to the reference frame image for a certain number of frame images (n frames before and after) (step S24). Specifically, among the acquired frame images, a certain number of frame images are aligned with each other, and deformation information d, which is information used for coordinate transformation, is obtained. Note that the alignment performed for a certain number of frame images (n frames before and after) is performed for a smaller number of frame images (n < N) than the alignment performed for all frame images (N frames before and after), so the processing is performed at high speed. Note that the number of frames (n) may be set in advance before the start of the composite still image generation process B, or may be set after the start of the composite still image generation process B. An example of setting after the start of the composite still image generation process B is the same as the example of the composite still image generation process A described above.

[0055] Next, when the alignment for a certain number of frame images is completed, the control unit 21 functions as a generation unit, performs coordinate transformation on the frame images (n frames before and after) using the deformation information d obtained in step S24, synthesizes the reference frame image and the frame images to be subjected to the synthesis process (n frames before and after), and generates a preview image (second image) (step S25).

[0056] Next, the control unit 21 causes the display unit 24, which functions as an output unit, to display the preview image (step S26).

[0057] Next, the user checks the preview image, and if there is a problem, uses the operation unit 23 to send a process stop request to the control unit 21 (step S27). Then, the control unit 21 aborts the alignment in step S28 described later (step S211). If the process is aborted, the composite still image generation process B ends.

[0058] In parallel with the generation of the preview image in step S25, the generation process of the main image (steps S28 and S29) is started. First, the control unit 21 performs alignment of the frame images to be subjected to the synthesis process with respect to the reference frame image for the remaining frame images (N - n sheets) other than a certain number of frame images to be subjected to the synthesis process (step S28). In the alignment, deformation information e is obtained.

[0059] Next, when the alignment is completed for all the frame images to be subjected to the synthesis process, the control unit 21 functions as a generation unit, performs coordinate transformation using the deformation information d and the deformation information e obtained in steps S24 and S28, synthesizes the reference frame image and the frame images to be subjected to the synthesis process, and generates the main image (the first image) (step S29).

[0060] Next, the control unit 21 causes the display unit 24, which functions as an output unit, to display the main image (step S210), and ends the composite still image generation process B.

[0061] The composite still image generated by the composite still image generation process B is stored in the storage unit 22 in association with the patient information and the examination information of the original moving image.

[0062] In this way, since the alignment with a small number of sheets is processed at a higher speed than the alignment with all the numbers of sheets to be subjected to the synthesis process, once a composite still image for preview is created and pre-displayed on the console when the alignment with a small number of sheets is completed, the user can early confirm the deterioration of the image quality that occurs when the alignment fails between a small number of frames due to the correctness of the reference frame and the influence of body movement. If it is possible to confirm the deterioration of the image quality that occurs when the alignment fails between a small number of frames due to the influence of body movement, since it is unlikely to ensure high image quality even if the remaining frames (N - n sheets) are synthesized, if there is a problem with the preview image, the process can be interrupted at that time, and reprocessing or reshooting can be determined.

[0063] As described above, an image processing apparatus (console 2) that processes a moving image obtained by irradiating a subject with radiation to capture the dynamics of the subject includes an acquisition unit (control unit 21) that acquires a plurality of frame images constituting the moving image, a generation unit (control unit 21) that generates a first image that is a composite still image obtained by combining at least two or more frame images among the plurality of frame images, and an output control unit (control unit 21) that outputs the first image to an output unit. Before generating the first image, the output control unit outputs a second image that is a still image based on at least one or more frame images among the plurality of frame images and has lower image quality than the first image, so that the user can determine whether reprocessing or reshooting is necessary at an early stage by checking the second image. Therefore, when generating a composite still image from a plurality of frame images constituting a moving image, it is possible to reduce the stress on the user and the wasted waiting time.

[0064] Note that the description content in the above embodiment is a preferred example of the present invention and is not limited thereto.

[0065] For example, in a form in which alignment is performed in order from low resolution to high resolution at a plurality of resolutions, there may be more than two steps of low resolution and high resolution. The flow of alignment in this case will be described. As a specific example, the alignment of two frame images will be described. First, low-resolution frame images obtained by reducing the resolution of the acquired frame images are aligned with each other to obtain deformation information a, which is information used for coordinate transformation. Next, using the deformed information b obtained by upsampling the deformed information a as the initial deformed information, the frame images with increased resolution are aligned with each other to obtain new deformed information c. Next, using the deformed information d obtained by upsampling the deformed information c as the initial deformed information, the frame images with further increased resolution are aligned with each other to obtain new deformed information e. That is, using the deformation information obtained by alignment at a low resolution as the starting point (initial deformation information), alignment at a high resolution is repeatedly performed. For example, the resolution is gradually increased as 1 / 8 times → 1 / 4 times → 1 / 2 times → full scale, and alignment is performed. Also, when alignment at multiple resolutions is performed in order from low resolution to high resolution, a composite still image may be generated at the time when alignment at each resolution is completed, and the composite still image may be sequentially reflected on the second image. In this case, the user can check the composite images at each resolution. Also, the timing of preview display can be set. For example, preview display is performed at the timing when alignment at 1 / 4 times the resolution is completed. Also, the preview image may be displayed multiple times. For example, preview display is performed at the timing when alignment at 1 / 8 times and 1 / 2 times the resolution is completed. Also, each magnification and step of the resolution gradually increased can be set.

[0066] Also, when a preview image or the main image is being displayed on the display unit 24, an icon or comment indicating that the preview image or the main image is being displayed may be displayed on the display unit 24.

[0067] Also, when steps S18, S19, S28, and S29 that are executed in parallel with step S15 and step S25 are being executed, the progress may be displayed on the display unit 24. For example, it can be mentioned that a comment indicating 50% completion is displayed, or using the horizontal axis with a maximum value of 100% to show the progress on that axis.

[0068] Also, when a preview image is being displayed on the display unit 24, image adjustment and image processing (tone processing), etc. may be made inoperable. Note that image adjustment and image processing (tone processing), etc. are operable when the main image is being displayed on the display unit 24.

[0069] Also, in step S16 of the first embodiment and step S26 of the second embodiment, a reference frame image may be displayed as a preview image. In this case, the flow branches immediately after step S12 or step S22, and the generation process of the main image (steps S13-14 and steps S18-19, steps S23-24 and steps S28-29) and the generation process of the preview image (steps S15-17, steps S25-27) are executed in parallel. Thereby, the user can confirm the reference frame image as a preview image earlier and can confirm the correctness of the reference frame at an early stage.

[0070] Note that the main image is generated and displayed based on the reference frame image and at least one frame image other than the reference frame image.

[0071] Also, when generating the preview image, at least one of the frame images used for generating the main image may be used.

[0072] Also, the configuration for generating the preview image is not limited to the generation unit (control unit 21) of the image processing apparatus (console 2). For example, a processing apparatus or a server other than the image processing apparatus (console 2) can be mentioned.

[0073] Also, for example, in the above description, an example in which a hard disk, a semiconductor non-volatile memory, etc. are used as a computer-readable medium of the program according to the present invention is disclosed, but it is not limited to this example. As other computer-readable media, portable recording media such as CD-ROM can be applied. Also, a carrier wave is applied as a medium for providing the data of the program according to the present invention via a communication line.

[0074] In addition, regarding the detailed configuration and detailed operation of each device constituting the image processing system, it can be appropriately changed within a range not departing from the gist of the present invention.

Explanation of Signs

[0075] 100 Image processing system 1 Imaging device 11 Photography stage 11a Holder 12 Radiation generator P Radiation detector 2 Console (image processing device) 21 Control unit (acquisition unit, generation unit, output control unit) 22 Memory unit 23 Operation unit 24 Display unit (output unit) 25 Communication unit 26 Bus

Claims

1. An image processing apparatus that processes a moving image obtained by a moving image shooting that shoots the movement state of a subject by irradiating the subject with radiation, an acquisition unit that acquires a plurality of frame images constituting the moving image, a generation unit that generates a first image that is a composite still image obtained by synthesizing at least two or more of the plurality of frame images, and an output control unit that outputs the first image to an output unit, wherein the output control unit outputs, before generating the first image, a second image that is a still image based on at least one or more of the plurality of frame images and has lower image quality than the first image, the generation unit generates a composite still image obtained by synthesizing at least two or more of the plurality of frame images as the second image, the generation unit performs a plurality of alignments in order from a low resolution to a high resolution for a plurality of resolutions, and generates, as the second image, a composite still image at the time when the alignment at each resolution is completed. An image processing apparatus characterized by that.

2. The image processing apparatus according to claim 1, wherein the second image is a composite still image obtained by synthesizing at least two or more of the plurality of frame images.

3. The image processing apparatus according to claim 2, wherein the second image is generated under a condition that the alignment accuracy at the time of synthesis is lower than that of the first image.

4. The image processing apparatus according to any one of claims 1 to 3, wherein the second image is generated from a smaller number of frame images than the first image.

5. The image processing apparatus according to any one of claims 1 to 4, wherein the generation unit starts alignment at the next resolution during the generation of the second image.

6. The image processing apparatus according to any one of claims 1 to 5, wherein the generation unit generates the first image using information obtained when generating the second image.

7. The image processing apparatus according to any one of claims 1 to 6, wherein the output control unit outputs warning information to the output unit when generation of the second image by the generation unit fails or when the quality of information of the second image generated by the generation unit does not meet a specific standard.

8. The generation unit generates the first image based on one reference frame image among the plurality of frame images and at least one frame image other than the reference frame image. The image processing apparatus according to any one of claims 1 to 7, wherein the second image is the reference frame image. **Claim 9** The image processing apparatus according to any one of claims 1 to 8, wherein the generation unit generates the second image using at least one of the frame images used for generating the first image. **Claim 10** The image processing apparatus according to any one of claims 1 to 9, wherein the image quality includes at least one of resolution, sharpness, granularity, and edge sharpness. **Claim 11** A computer used in an image processing apparatus that processes a moving image obtained by a moving image shooting that irradiates a subject with radiation to shoot the dynamics of the subject, A function of acquiring a plurality of frame images constituting the moving image, A function of generating a first image that is a composite still image obtained by synthesizing at least two or more of the plurality of frame images, A function of outputting the first image to an output unit is realized, The function of outputting to the output unit outputs, before generating the first image, a still image based on at least one or more of the plurality of frame images, and the second image has a lower image quality than the first image. The function of generating the image generates a composite still image obtained by synthesizing at least two or more of the plurality of frame images as the second image. A program, wherein the function of generating the image performs a plurality of alignments in order from a low resolution to a high resolution for a plurality of resolutions, and generates a composite still image at the time when the alignment at each resolution is completed as the second image.

Images