Image processing apparatus and image processing method

Abstract

To provide a technique for suppressing a decrease in the effect of noise reduction and a decrease in visibility due to the amplitude of the effect, even when frames determined to be the same image are input across a plurality of frames in the noise reduction using the plurality of frames.SOLUTION: An image processing apparatus acquires a plurality of frames based on a determination result as to whether or not a frame is the same image as a previous frame input before the frame being sequentially input, and performs noise reduction based on the acquired plurality of frames. The image processing apparatus outputs the result of the noise reduction.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present invention relates to noise reduction technology. [Background technology]

[0002] Generally, various noises are mixed into images captured by a photographing device such as a camera, reducing the visibility of the image. Therefore, a process for reducing noise by image processing (hereinafter referred to as noise reduction, abbreviated as NR) is essential. In this NR, when frames (images) such as a moving image are input sequentially, there is a method using multiple frames. For example, in recent years, there is a process for performing NR by inputting multiple frames using deep learning (Patent Document 1).

[0003] In addition, in a cyclic NR method that combines an input frame and an NR result based on a cyclic coefficient, there is a method in which NR is performed by determining whether the input frame and the previous frame are identical, and then changing the combination ratio of the input frame and the NR result (hereinafter referred to as the cyclic coefficient) (Patent Document 2). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent Publication No. 2023-35928 [Patent Document 2] JP2010-226646Publication Summary of the Invention [Problem to be solved by the invention]

[0005] In the field of surveillance cameras, there is a function (hereinafter referred to as slow shutter) that shoots a very dark scene with an exposure time longer than the frame rate of the image. In the image shot with this slow shutter, the same frame is output over multiple frames in order to maintain the frame rate at the time of output. For example, at a frame rate of 30 fps, in the case of a shutter speed of 1 / 3 seconds, the same frame is output over 10 frames. In the NR that inputs multiple frames as in Patent Document 1, when an image shot with such a slow shutter is provided, the exact same frame is input. As a result, the NR effect changes depending on the number of different frames in the reference frame, and the NR effect decreases when the number of different frames decreases. For example, as shown in FIG. 10, when three frames I1, I2, and I3 are referenced, when the slow shutter frames are input sequentially, an image with three identical frames or two identical frames becomes the input in the NR. As a result, the NR effect decreases compared to when the three frames are different, and the NR effect is further amplified, causing a decrease in visibility. In addition, the method of Patent Document 2 determines whether two frames are the same, and if they are the same, outputs the previous NR result, and if they are different, combines the previous NR result and the input frame with a cyclic coefficient to reduce noise. However, this method cannot perform NR using an input of three or more frames.

[0006] The present invention provides a technology for suppressing a decrease in the effect of noise reduction and a decrease in visibility due to the amplitude of the effect, even when frames that are determined to be the same image across multiple frames are input in noise reduction using multiple frames. [Means for solving the problem]

[0007] One aspect of the present invention is characterized in that it comprises a processing means for acquiring a plurality of frames based on a determination result of whether or not a frame which is input sequentially is the same image as a previous frame which was input before the frame, and performing noise reduction based on the acquired plurality of frames, and an output means for outputting a result of the noise reduction performed by the processing means. Effect of the Invention

[0008] According to the present invention, in noise reduction using multiple frames, even if frames that are determined to be the same image across multiple frames are input, it is possible to suppress a decrease in the noise reduction effect and a decrease in visibility due to the amplitude of the effect. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram showing an example of a system configuration. [Diagram 2] FIG. 1 is a block diagram showing an example of the hardware configuration of an image processing device 100. [Diagram 3] FIG. 2 is a block diagram showing an example of the arrangement of an image processing unit 207. [Figure 4] 4 is a flowchart of a noise reduction process performed by the image processing device 100. [Diagram 5] 13A and 13B are diagrams for explaining a modified example of a method for generating a difference image. [Figure 6] FIG. 11 is a diagram showing an example of the process in step S409. [Figure 7] FIG. 2 is a block diagram showing an example of the arrangement of an image processing unit 207. [Figure 8] 4 is a flowchart of a noise reduction process performed by the image processing device 100. [Figure 9] FIG. 13 is a diagram showing an example of an outline of an operation. [Figure 10] FIG. 1 is a diagram for explaining a technical problem. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, the embodiments will be described in detail with reference to the attached drawings. Note that the following embodiments do not limit the invention according to the claims. Although the embodiments describe a number of features, not all of these features are essential to the invention, and the features may be combined in any manner. Furthermore, in the attached drawings, the same reference numbers are used for the same or similar configurations, and duplicated descriptions are omitted.

[0011] [First embodiment] The image processing device according to the present embodiment acquires a plurality of frames according to a result of a judgment as to whether or not a frame inputted in sequence is the same as a previous frame inputted before the frame, and performs NR (noise reduction) based on the acquired plurality of frames. More specifically, when a frame inputted in sequence is an image different from a previous frame inputted before the frame, the image processing device buffers the frame, and when the buffered frame reaches a prescribed frame number, performs NR using the buffered frame. In addition, when the input frame is the same as the previous frame, the image processing device outputs the result of the previous noise reduction. In addition, when the input frame is an image different from the previous frame and the buffered frame does not reach the prescribed frame number, the image processing device outputs the result of the previous noise reduction. In this embodiment, a case where the prescribed frame number=3 will be described, but the prescribed frame number is not limited to a specific number.

[0012] First, a configuration example of a system including an image processing device according to this embodiment will be described with reference to Fig. 1. As shown in Fig. 1, the system according to this embodiment includes an imaging device 101, an image processing device 100, and a display device 103. The imaging device 101 and the image processing device 100, and the image processing device 100 and the display device 103 are connected via a video transmission cable 102 such as an HDMI or SDI (serial digital interface) cable.

[0013] First, the imaging device 101 will be described. The imaging device 101 may be a moving image imaging device that captures moving images, or may be a still image imaging device that captures still images periodically or irregularly. When the imaging device 101 is a moving image imaging device, the imaging device 101 outputs images of each frame in a moving image as a captured image. When the imaging device 101 is a still image imaging device, the imaging device 101 outputs still images of frames captured periodically or irregularly as a captured image. The captured images output from the imaging device 101 are input to the image processing device 100 via a video transmission cable 102.

[0014] The imaging device 101 may be an imaging device capable of controlling PTZ (pan-tilt-zoom), may be an imaging device whose position is movable, or may be a portable imaging device such as a handheld camera.

[0015] Next, a description will be given of the display device 103. The display device 103 has a liquid crystal screen or a touch panel screen, and displays images and characters output from the image processing device 100 and accepts user operations such as touch operations and swipe operations from the user.

[0016] Next, the image processing device 100 will be described. The image processing device 100 is a computer device such as a PC (personal computer), a smartphone, or a tablet terminal device. An example of the hardware configuration of the image processing device 100 will be described with reference to the block diagram of Fig. 2. The hardware configuration shown in Fig. 2 is an example of a hardware configuration applicable to the image processing device 100, and can be modified / changed as appropriate.

[0017] The CPU 201 executes various processes using computer programs and data stored in the RAM 202. As a result, the CPU 201 controls the operation of the entire image processing device 100, and executes or controls various processes described as processes performed by the image processing device 100.

[0018] The RAM 202 has an area for storing computer programs and data loaded from the ROM 203 or the storage device 204. The RAM 202 also has an area for storing "captured images received by the image input unit 206 from the imaging device 101 via the communication I / F 205." The RAM 202 also has a work area used when the CPU 201 or the image processing unit 207 executes various processes. In this way, the RAM 202 can provide various areas as appropriate.

[0019] The ROM 203 stores setting data for the image processing device 100, computer programs and data related to the startup of the image processing device 100, computer programs and data related to the basic operations of the image processing device 100, and the like.

[0020] The storage device 204 is a memory device such as an SSD or HDD. The storage device 204 stores an OS (operating system), computer programs and data for causing the CPU 201 and image processing unit 207 to execute or control various processes described as processes performed by the image processing device 100, and the like. The computer programs and data stored in the storage device 204 are loaded into the RAM 202 as appropriate under the control of the CPU 201, and become targets for processing by the CPU 201. The image processing device 100 may further be a device into which a recording medium such as an SD card can be inserted, and in this case, the image processing device 100 may also store various types of information in the recording medium.

[0021] The communication I / F 205 is an interface such as HDMI or SDI, and functions as a communication interface for performing data communication with the image capture device 101 and the display device 103 via the video transmission cable 102 .

[0022] The image input unit 206 receives (inputs) the captured image output from the imaging device 101 via the video transmission cable 102 via the communication I / F 205. The image processing unit 207 performs various types of image processing on the captured image received by the image input unit 206.

[0023] The operation unit 209 is a user interface such as a keyboard, a mouse, and a touch panel screen, and the user can input various information and instructions to the image processing device 100 by operating it.

[0024] The image output unit 210 outputs various information such as images and characters to the "display device 103 connected to the image processing device 100 via the video transmission cable 102" via the communication I / F 205. This causes the information to be displayed on the display device 103.

[0025] The CPU 201 , RAM 202 , ROM 203 , storage device 204 , communication I / F 205 , image input unit 206 , image processing unit 207 , operation unit 209 , and image output unit 210 are all connected to a system bus 211 .

[0026] A configuration example of the image processing unit 207 is shown in the block diagram of Fig. 3. In this embodiment, a case will be described in which each functional unit of the image processing unit 207 is implemented by hardware, but one or more of the functional units of the image processing unit 207 may be implemented by software (computer program). In this case, the computer program is executed by the CPU 201, thereby realizing the function of the corresponding functional unit. The process related to noise reduction performed by the image processing device 100 will be described with reference to the flowchart of Fig. 4.

[0027] In step S401, the CPU 201 determines whether or not an instruction to start imaging has been input. The method of inputting the instruction to start imaging is not limited to a specific input method. For example, the user may input the instruction to start imaging by pressing a button for starting imaging provided on the imaging device 101, or the user may input the instruction to start imaging by operating the operation unit 209.

[0028] If it is determined that an instruction to start imaging has been input, the process proceeds to step S402; if it is determined that an instruction to start imaging has not been input, the process waits in step S401.

[0029] In step S402, the CPU 201 performs initialization processing related to the following processing, such as initializing variables i and n to 0, which are used in the following processing.

[0030] In step S403, the image input unit 206 receives (inputs) the captured image output from the imaging device 101 via the video transmission cable 102 as frame I(i) via the communication I / F 205, and stores the frame I(i) in the memory unit 301.

[0031] In step S404, the determination unit 304 determines whether or not the frame I(i) is the same image as the frame I(i-1) stored in the storage unit 301. Here, when i=0, for example, an image in which the pixel values ​​of all pixels are 0 may be used as the frame I(i-1). There are various methods for determining whether the frame I(i) and the frame I(i-1) are the same image, and the method is not limited to a specific method.

[0032] For example, the determination unit 304 generates a difference image DI(i) between frame I(i) and frame I(i-1). Here, the difference value (absolute value) between the pixel value of the pixel at pixel position (x, y) in frame I(i) and the pixel value of the pixel at pixel position (x, y) in frame I(i-1) becomes the pixel value of the pixel at pixel position (x, y) in the difference image DI(i). Then, if the number of pixels in the difference image DF(i) whose pixel value is greater than the threshold is equal to or less than a specified number, the determination unit 304 determines that "frame I(i) and frame I(i-1) are the same image". On the other hand, if the number of pixels in the difference image DF(i) whose pixel value is greater than the threshold is greater than the specified number, the determination unit 304 determines that "frame I(i) and frame I(i-1) are not the same image".

[0033] In other words, in this embodiment, the determination that "frame I(i) and frame I(i-1) are the same image" is not limited to the determination that "frame I(i) and frame I(i-1) are exactly the same image", but the threshold value and specified number are adjusted to allow for some flexibility in the determination.

[0034] Note that the method is not limited to obtaining a difference image between the entire region in one frame and the entire region in the other frame. In other words, instead of the entire region in the frame, for example, as shown in Fig. 5, a difference image may be generated using a difference calculation region 503 excluding an on-screen display region (OSD region) 502 in a frame 501.

[0035] If it is determined that frame I(i) and frame I(i-1) are the same image, the process proceeds to step S405. On the other hand, if it is determined that frame I(i) and frame I(i-1) are not the same image, the process proceeds to step S406.

[0036] In step S405, the output unit 306 acquires O(i-1), which is the previous NR output result, from the storage unit 303, and copies the acquired O(i-1) to O(i), which indicates the current NR output result. Note that when i=0, frame I(i), for example, may be used as O(i-1). Meanwhile, in step S406, the determination unit 304 stores frame I(i) in the storage unit 302 as NR frame B(n).

[0037] In step S407, CPU 201 increments variable n (buffer position of NR frame B(n) in storage unit 302) by 1. In step S408, determination unit 307 determines whether the value of variable n is 3. If the result of this determination is that the value of variable n is 3, determination unit 307 determines that the number of frames required for noise reduction (NR frames) have been stored in storage unit 302, and processing proceeds to step S409. On the other hand, if the value of variable n is less than 3, determination unit 307 determines that the number of frames required for noise reduction (NR frames) have not yet been stored in storage unit 302, and processing proceeds to step S412.

[0038] In step S409, the NR processor 305 performs noise reduction using the NR frames B(1), B(2), and B(3) stored in the storage unit 302 to generate O(i) indicating the current NR output result. Details of the process in step S409 will be described later.

[0039] In step S410, the CPU 201 initializes the value of a variable n to 0. In step S411, the CPU 201 increments the value of a variable i (position of the processing frame) by one.

[0040] In step S412, the output unit 306 acquires O(i-1), which is the previous NR output result, from the storage unit 303, and copies the acquired O(i-1) to O(i), which indicates the current NR output result.

[0041] In step S413, the output unit 306 outputs O(i) indicating the current NR output result to the image output unit 210. The image output unit 210 outputs O(i) to the display device 103 via the communication I / F 205.

[0042] In step S414, the CPU 201 determines whether or not an instruction to end image capture has been input. The method of inputting the instruction to end image capture is not limited to a specific input method. For example, the user may input the instruction to end image capture by pressing a button for ending image capture provided on the image capture device 101, or the user may input the instruction to end image capture by operating the operation unit 209.

[0043] If it is determined as a result of such a determination that an instruction to end image capture has been input, the processing according to the flowchart in FIG. 4 ends, and if it is determined that an instruction to end image capture has not been input, the processing proceeds to step S403.

[0044] Next, an example of the process in step S409 above will be described with reference to FIG. 6. The NR processing unit 305 generates an input concatenated image 601 by concatenating NR frames B(0), B(1), and B(2) stored in the storage unit 302 in the channel direction. Here, the channel direction refers to the direction in which pixels at the same coordinates in the NR frames B(0), B(1), and B(2) are stacked, and this direction is perpendicular to the height and width of the NR frame. For example, in the case where the NR frame has three channels of RGB, if the height of the NR frame is H and the width is W, the input concatenated image 601 has a data structure of H×W×3×3. Next, the NR processing unit 305 inputs the input concatenated image 601 to the CNN 602 and performs the calculation of the CNN 602 (by repeating the convolution calculation and nonlinear calculation of the filter), thereby outputting an output concatenated image 603 in which degradation has been restored. The CNN 602 is a CNN that has been trained for noise reduction.

[0045] The output concatenated image 603 is an image of the same shape as the input concatenated image 601, and corresponding channels in the output concatenated image 603 and the input concatenated image 601 are images at the same time. The NR processor 305 then outputs, from the output concatenated image 603, an image at the same time (same frame) as frame I(i) as the NR output result O(i).

[0046] Here, the CNN 602 has an input layer 604, a hidden layer 605 consisting of multiple layers, and an output layer 606. The input layer 604 and the output layer 606 have the same shape. In this embodiment, the hidden layer 605 is made smaller in size (height and width) than the input / output layers, and has a larger number of channels. This is generally a technique for acquiring information in an image over a wide range and for enhancing expressiveness.

[0047] In this embodiment, the area for calculating the difference image is the area excluding the OSD area, but since the same frame is input during slow shutter, only one pixel may be used as the area.

[0048] In addition, in the present embodiment, an example of using the NR method using deep learning has been described, but the NR method is not limited to this. For example, an NR method that performs block matching in the frame direction and synthesizes blocks according to similarity, or an NR method that synthesizes blocks according to luminance differences for each pixel may be used.

[0049] In this way, according to this embodiment, even if the same frame continues to be input as in the case of a slow shutter, the previous NR result is output until the different frames are held by the "number of frames required for NR". In other words, in the frames input sequentially, if the frame is a different image from the previous frame input before the frame, the frame is buffered, and when the buffered frames reach a specified number of frames, noise reduction is performed using the buffered frames, and the result of the noise reduction is output. By performing such processing control, it is possible to suppress the NR effect reduction and the amplitude of the NR effect, and to suppress the reduction in visibility due to noise.

[0050] [Second embodiment] In this embodiment, the difference from the first embodiment will be described, and unless otherwise specified below, it will be assumed that the present embodiment is the same as the first embodiment. In the first embodiment, a method for outputting the previous NR result until the number of frames required for NR is held when an input in which the same frames continue, such as a slow shutter, is described. In contrast, in this embodiment, an example of performing NR even before the number of frames required for NR are held will be described.

[0051] An example of the arrangement of the image processing unit 207 according to this embodiment is shown in the block diagram of Fig. 7. In Fig. 7, the same reference numerals are given to the same functional units as those shown in Fig. 3, and the description of these functional units will be omitted.

[0052] The noise reduction process performed by the image processing device 100 will be described with reference to the flowchart in Fig. 8. In Fig. 8, the same process steps as those shown in Fig. 4 are given the same step numbers, and the description of these process steps will be omitted.

[0053] In step S804, the determination unit 304 determines whether or not a slow shutter has started. For example, the determination unit 304 determines whether or not the frame I(i) and the frame I(i-1) stored in the storage unit 301 are the same image. If the determination unit 304 determines that the frame I(i) and the frame I(i-1) are the same image, it determines that the slow shutter has started, and the process proceeds to step S813. On the other hand, if the determination unit 304 determines that the frame I(i) and the frame I(i-1) are not the same image, it determines that the slow shutter has not started, and the process proceeds to step S406.

[0054] However, the method for determining whether or not the slow shutter has started is not limited to a specific method in the imaging device 101. For example, the determination unit 304 may determine that the slow shutter has started when the shutter speed is longer than the frame frequency calculated from the frame rate in the meta tag of frame I(i).

[0055] In this embodiment, in step S406, the selection unit 707 stores the frame I(i) as the NR frame B(n) in the storage unit 302. In step S813, the CPU 201 initializes the value of the variable m to 1.

[0056] In step S814, the CPU 201 determines whether the value of the variable m is equal to or less than the slow shutter frame number (the specified frame number) that is preset as the number of frames in which the same image continues in the slow shutter. In the following, a case where the slow shutter frame number=3 will be described, but the slow shutter frame number is not limited to 3.

[0057] If the result of this determination is that the value of variable m (i.e., the number of frames input after it is determined that frame I(i) and frame I(i-1) are not the same image) is 3 or less, the process proceeds to step S815. On the other hand, if the value of variable m is not 3 or less, the process proceeds to step S816.

[0058] In step S815, the CPU 201 increments the value of the variable m by 1. In step S816, the selection unit 707 determines whether the frame I(i) and the NR frame B(2), which is the most recent NR frame among the NR frames stored in the storage unit 302, are the same image in the same manner as in step S404. Note that different determination criteria may be used in steps S404 and S816.

[0059] If it is determined that frame I(i) and NR frame B(2) are the same image, the process proceeds to step S412. On the other hand, if it is determined that frame I(i) and NR frame B(2) are not the same image, the process proceeds to step S817.

[0060] In step S817, the selection unit 707 stores NR frame B(1) as NR frame B(0), stores NR frame B(2) as NR frame B(1), and stores frame I(i) as NR frame B(2).

[0061] In step S822, the CPU 201 determines whether the slow shutter has ended. The method for determining whether the slow shutter has ended is not limited to a specific method. For example, a method may be used in which it is determined that the slow shutter has ended when it is determined that the same image frame continues for three frames. Also, a method may be used in which it is determined that the slow shutter has ended when the shutter speed is shorter than the frame frequency calculated from the frame rate of the meta tag of frame I(i).

[0062] If it is determined that the slow shutter has ended as a result of this determination, the process proceeds to step S414, and if it is not determined that the slow shutter has ended, the process proceeds to step S814.

[0063] If it is determined in step S414 that an instruction to end imaging has been input, the process according to the flowchart in FIG. 8 ends, and if an instruction to end imaging has not been input, the process proceeds to step S804.

[0064] Fig. 9 shows an example of the outline of the operation in this embodiment. As shown in Fig. 9, when the slow shutter starts, frames of the same image are input, and NR processing is executed until the number of frames of the same image reaches the slow shutter frame number (time = 0 to 2). After the number of frames of the same image reaches the slow shutter frame number, the end of the buffer used for NR is compared with the input frame, and if the frames are different, the buffer is updated and NR is executed, and if the frames are the same, the previous NR result is output. As a result, the NR effect gradually changes.

[0065] According to this embodiment, by performing the processing control described above, when performing NR using different frames for an input in which frames of the same image follow one another, such as with a slow shutter, a smooth transition can be achieved when switching between normal shutter and slow shutter by gradually increasing the number of different frames, outputting the previous NR result when the frame is identical to the previous frame, and NR that allows overlap within the frame.

[0066] In the first embodiment, the previous NR result is output until the number of frames required for NR is held, and in the present embodiment, a process of performing NR even when the number of frames is not sufficient for NR according to the slow shutter state is described. However, the present embodiment is not limited to this as long as the method controls the output result according to the slow shutter state and the NR effect. For example, a method of performing weighted addition of the previous NR result and the NR result in a state where the number of frames is not sufficient according to the slow shutter state may be used.

[0067] In addition, in each of the above embodiments, the imaging device 101, the image processing device 100, and the display device 103 are separate devices, but two or more of these may be integrated into one device. For example, by integrating the imaging device 101 and the image processing device 100 into one device, it is possible to configure a device having an imaging function and a noise reduction function.

[0068] The numerical values, processing timing, processing order, processing subject, data (information) acquisition method / destination / source / storage location, etc. used in each of the above embodiments are given as examples to provide a concrete explanation, and are not intended to be limited to these examples.

[0069] In addition, a part or all of the embodiments described above may be used in appropriate combination. In addition, a part or all of the embodiments described above may be used selectively.

[0070] (Other embodiments) The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions.

[0071] The invention of this specification includes the following image processing device, image processing method, and computer program. (Item 1) a processing means for acquiring a plurality of frames in accordance with a result of a determination as to whether or not a frame is the same image as a previous frame inputted before the frame, among the frames that are sequentially inputted, and performing noise reduction based on the acquired plurality of frames; an output means for outputting a result of the noise reduction performed by the processing means; An image processing device comprising: (Item 2) 2. The image processing device according to item 1, wherein the processing means buffers a frame when it is determined that the frame is a different image from a previous frame input before the frame, and when the number of buffered frames reaches a specified number of frames, performs noise reduction based on the buffered frames. (Item 3) 3. The image processing device according to item 2, characterized in that the output means outputs the result of the previous noise reduction when the frame is determined to be a different image from the previous frame and the number of buffered frames has not reached a specified number of frames. (Item 4) 4. The image processing device according to any one of items 1 to 3, wherein the output means outputs a result of a previous noise reduction when the frame is the same image as the previous frame. (Item 5) 3. The image processing device according to claim 2, wherein when the processing means determines that a frame is the same image as a previous frame input before the frame in question, and the number of frames input after the determination is equal to or less than a specified number of frames, the processing means performs noise reduction based on the input frame and buffered frames. (Item 6) The image processing device according to item 2, characterized in that when a frame is judged to be the same image as a previous frame input before the frame, and the number of frames input after the judgment exceeds a specified number of frames, and it is judged that the input frame and the most recently buffered frame are different images, the processing means performs noise reduction based on the input frame and the buffered frame. (Item 7) The image processing device according to item 2, characterized in that when a frame is judged to be the same image as a previous frame input before the frame in question, the number of frames input after the judgment exceeds a specified number of frames, and the input frame and the most recently buffered frame are judged to be the same image, the processing means outputs the result of the previous noise reduction. (Item 8) moreover, An imaging means is provided, 8. The image processing device according to any one of items 1 to 7, wherein the sequentially input frames are frames captured by the imaging means. (Item 9) An image processing method performed by an image processing device, comprising: a processing step in which a processing means of the image processing device acquires a plurality of frames according to a result of a judgment as to whether or not a frame is the same image as a previous frame inputted before the frame, for each of the frames inputted sequentially, and performs noise reduction based on the acquired plurality of frames; an output step in which an output means of the image processing device outputs a result of the noise reduction performed by the processing step; An image processing method comprising: (Item 10) A computer program for causing a computer to function as each of the means of the image processing device according to any one of items 1 to 7.

[0072] The invention is not limited to the above-described embodiments, and various modifications and variations are possible without departing from the spirit and scope of the invention. Accordingly, the following claims are appended to apprise the public of the scope of the invention. [Explanation of symbols]

[0073] 301: Storage unit 302: Storage unit 303: Storage unit 304: Determination unit 305: Noise reduction processing unit 306: Output unit 307: Determination unit

Claims

1. a processing means for acquiring a plurality of frames in accordance with a result of a determination as to whether or not a frame is the same image as a previous frame inputted before the frame, among the frames that are sequentially inputted, and performing noise reduction based on the acquired plurality of frames; an output means for outputting a result of the noise reduction performed by the processing means; An image processing device comprising:

2. 2. The image processing device according to claim 1, wherein the processing means buffers a frame when it is determined that a frame is a different image from a previous frame input before the frame, and when the number of buffered frames reaches a specified number of frames, performs noise reduction based on the buffered frames.

3. 3. The image processing device according to claim 2, wherein the output means outputs the result of the previous noise reduction when the frame is determined to be a different image from the previous frame and the number of buffered frames has not reached a specified number of frames.

4. 2. The image processing apparatus according to claim 1, wherein said output means outputs the result of the previous noise reduction when said frame is the same image as said previous frame.

5. 3. The image processing device according to claim 2, wherein when the processing means determines that a frame inputted in sequence is the same image as a previous frame inputted before the frame, and the number of frames inputted after the determination is equal to or less than a specified number of frames, the processing means performs noise reduction based on the input frame and buffered frames.

6. 3. The image processing device according to claim 2, wherein when a frame is judged to be the same image as a previous frame input before the frame, and the number of frames input after the judgment exceeds a specified number of frames, and the input frame and the most recently buffered frame are judged to be different images, the processing means performs noise reduction based on the input frame and the buffered frame.

7. 3. The image processing device according to claim 2, wherein the processing means outputs the result of a previous noise reduction when it determines that a frame, among the frames that are sequentially input, is the same image as a previous frame that was input before the frame, and when the number of frames input after the determination exceeds a specified number of frames and it is determined that the input frame and the most recently buffered frame are the same image.

8. moreover, An imaging means is provided, 2. The image processing apparatus according to claim 1, wherein the sequentially input frames are frames captured by the imaging means.

9. An image processing method performed by an image processing device, comprising: a processing step in which a processing means of the image processing device acquires a plurality of frames according to a result of a judgment as to whether or not a frame is the same image as a previous frame inputted before the frame, for each of the frames inputted sequentially, and performs noise reduction based on the acquired plurality of frames; an output step in which an output means of the image processing device outputs a result of the noise reduction performed by the processing step; An image processing method comprising:

10. A computer program for causing a computer to function as each of the means of the image processing device according to any one of claims 1 to 7.

Images