IMAGE PROCESSING APPARATUS, IMAGING APPARATUS, AND METHOD AND PROGRAM FOR CONFIRMING BLUR

Abstract

To provide an image processing device, an imaging apparatus, a blur confirmation method, and a program that can accurately confirm whether or not a photographed image is blurred.SOLUTION: A processor 240 of a camera main body 200 constituting an imaging apparatus 10 executes: processing for detecting two directions which are a first direction that is a maximum blur direction and a second direction intersecting the first direction on the basis of an output of a blur detection sensor 270 detected while photographing a photographed image; blur evaluation value calculation processing for performing frequency analysis for two directions to respectively calculate a first blur evaluation value indicating a result of the frequency analysis for the first direction and a second blur evaluation value indicating a result of the frequency analysis for the second direction on the basis of an image in an evaluation object area set in the photographed image; and display processing for causing an LCD290 to display the first blur evaluation value and the second blur evaluation value.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present invention relates to an image processing device, an imaging device, a blur checking method and a program, and more particularly to a technique for checking whether or not a captured image is blurred. [Background technology]

[0002] Patent Document 1 describes a method for analyzing image data, calculating a first figure of merit of the digital image in a first direction (X), calculating a second figure of merit of the digital image in a second direction (Y) substantially perpendicular to the first direction, and determining that the digital image has motion blur if the ratio (first ratio) between the first figure of merit and the second figure of merit is greater than a threshold. Note that the magnitude of the amplitude of a particular spatial frequency is proportional to the figure of merit. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2003-78812 A Summary of the Invention [Problem to be solved by the invention]

[0004] One embodiment of the technique disclosed herein provides an image processing device, an imaging device, a blur checking method, and a program that are capable of checking with high accuracy whether a captured image is blurred. [Means for solving the problem]

[0005] The invention according to a first aspect is an image processing device including a shake detection sensor and a processor, in which the processor performs a process of detecting two directions, a first direction which is the maximum shake direction and a second direction intersecting the first direction, based on an output of the shake detection sensor detected during shooting of a captured image, a shake evaluation value calculation process of performing frequency analysis in the two directions based on an image within an evaluation target area set in the captured image, and calculating a first shake evaluation value indicating the result of the frequency analysis in the first direction and a second shake evaluation value indicating the result of the frequency analysis in the second direction, and a display process of displaying the first shake evaluation value and the second shake evaluation value on a display.

[0006] The invention according to a second aspect is an image processing device including a shake detection sensor and a processor, in which the processor performs a process of detecting two directions, a first direction which is the maximum shake direction and a second direction intersecting the first direction, based on an output of the shake detection sensor detected during shooting of a captured image, a shake evaluation value calculation process of performing frequency analysis in the two directions based on an image within an evaluation target area set in the captured image, and calculating a first shake evaluation value indicating the result of the frequency analysis in the first direction and a second shake evaluation value indicating the result of the frequency analysis in the second direction, and a first shake judgment process of comparing the first shake evaluation value and the second shake evaluation value to make a first shake judgment of the captured image.

[0007] In the image processing device according to the third aspect of the present invention, the second direction is preferably a direction perpendicular to the first direction.

[0008] In the image processing device according to the fourth aspect of the present invention, it is preferable that the processor performs a process of acquiring a first blur angle in a first direction and a second blur angle in a second direction based on an output of the shake detection sensor detected during shooting of the captured image, and a second blur determination process of comparing the first blur angle with the second blur angle to perform a second blur determination of the captured image.

[0009] In the image processing device according to the fifth aspect of the present invention, it is preferable that the processor performs notification processing for notifying the result of at least one of the first and second blur determinations.

[0010] In the image processing device according to the sixth aspect of the present invention, it is preferable that the blur evaluation value calculation process creates a first graph showing the first blur evaluation value versus spatial frequency in the first direction, and a second graph showing the second blur evaluation value versus spatial frequency in the second direction.

[0011] In the image processing device according to the seventh aspect of the present invention, it is preferable that the blur evaluation value calculation process includes a process of removing high frequency components from the first blur evaluation value and the second blur evaluation value to create the first graph and the second graph.

[0012] In the image processing device according to the eighth aspect of the present invention, it is preferable that the first blur judgment process calculates a difference or a ratio between the first blur evaluation value and the second blur evaluation value, and performs the first blur judgment based on the difference or the ratio and a threshold value.

[0013] In the image processing device according to the ninth aspect of the present invention, the display process preferably causes the first graph and the second graph to be displayed on a display.

[0014] In the image processing device according to the tenth aspect of the present invention, it is preferable that the processor performs an automatic extraction process for automatically extracting, as an evaluation target region, a region that is a focused region of the captured image and has a uniform texture in two different directions.

[0015] In the image processing device according to the eleventh aspect of the present invention, it is preferable that the processor performs a receiving process of displaying the captured image on the display and receiving setting of an evaluation target area within the captured image through a user operation.

[0016] In the image processing device according to the twelfth aspect of the present invention, it is preferable that the processor causes the in-focus area of ​​the captured image to be visibly displayed on the display, and performs an acceptance process for accepting an instruction to select an evaluation target area within the in-focus area by a user operation.

[0017] In the image processing device according to the thirteenth aspect of the present invention, it is preferable that the processor performs a storage process of storing in memory the captured image and the output of the shake detection sensor detected during shooting of the captured image or the detected maximum shake direction in association with each other.

[0018] In the image processing device according to the fourteenth aspect of the present invention, it is preferable that the first blur evaluation value and the second blur evaluation value are power spectra.

[0019] A fifteenth aspect of the invention is an imaging device including the image processing device according to any one of the first to fourteenth aspects.

[0020] The invention according to a 16th aspect is an imaging device comprising the image processing device according to the 2nd or 8th aspect and a memory, wherein the processor performs a storage process to store captured images determined to be not blurred by a first blur determination process in a first folder in the memory, and to store captured images determined to be blurred in a second folder in the memory.

[0021] The invention according to a seventeenth aspect is a blur confirmation method including the steps of acquiring a captured image, acquiring an output of a shake detection sensor while capturing the captured image, detecting two directions, a first direction which is the direction of maximum shake and a second direction intersecting the first direction, based on the output of the shake detection sensor, performing frequency analysis in the two directions based on an image within an evaluation target area set in the captured image, and calculating a first blur evaluation value indicating the result of the frequency analysis in the first direction and a second blur evaluation value indicating the result of the frequency analysis in the second direction, and displaying the first blur evaluation value and the second blur evaluation value on a display, wherein a processor executes processing of each step.

[0022] An invention according to an 18th aspect is a blur confirmation method including the steps of acquiring a captured image, acquiring an output of a shake detection sensor while capturing the captured image, detecting two directions, a first direction which is the direction of maximum shake and a second direction intersecting the first direction, based on the output of the shake detection sensor, performing frequency analysis in the two directions based on an image within an evaluation target area set in the captured image, and calculating a first blur evaluation value indicating the result of the frequency analysis in the first direction and a second blur evaluation value indicating the result of the frequency analysis in the second direction, and comparing the first blur evaluation value with the second blur evaluation value to make a first blur judgment of the captured image, wherein a processor executes processing of each step.

[0023] It is preferable that the blur confirmation method according to the nineteenth aspect of the present invention includes the steps of acquiring a first blur angle in a first direction and a second blur angle in a second direction based on an output of a blur detection sensor detected during shooting of a captured image, and comparing the first blur angle with the second blur angle to make a second blur determination of the captured image.

[0024] In the blur checking method according to the twentieth aspect of the present invention, it is preferable that when blur of the captured image cannot be confirmed by the second blur determination, blur of the captured image is determined by the first blur determination.

[0025] The invention according to a 21st aspect is a blur confirmation program that causes a computer to realize a function of acquiring a captured image, a function of acquiring an output of a shake detection sensor while capturing the captured image, a function of detecting two directions, a first direction which is the direction of maximum shake and a second direction intersecting the first direction, based on the output of the shake detection sensor, a function of performing frequency analysis in the two directions based on an image within an evaluation target area set in the captured image, and calculating a first blur evaluation value indicating the result of the frequency analysis in the first direction and a second blur evaluation value indicating the result of the frequency analysis in the second direction, and a function of displaying the first blur evaluation value and the second blur evaluation value on a display.

[0026] The invention according to a 22nd aspect is a blur confirmation program that causes a computer to realize a function of acquiring a captured image, a function of acquiring an output of a shake detection sensor while capturing the captured image, a function of detecting two directions, a first direction which is the direction of maximum shake and a second direction intersecting the first direction, based on the output of the shake detection sensor, a function of performing frequency analysis in the two directions based on an image within an evaluation target area set in the captured image, and calculating a first blur evaluation value indicating the result of the frequency analysis in the first direction and a second blur evaluation value indicating the result of the frequency analysis in the second direction, and a function of comparing the first blur evaluation value and the second blur evaluation value to make a first blur judgment of the captured image. [Brief description of the drawings]

[0027] [Figure 1] FIG. 1 is a perspective view of an imaging device including an image processing device according to the present invention, as viewed obliquely from the front. [Diagram 2] FIG. 2 is a rear view of the imaging device shown in FIG. [Diagram 3] FIG. 3 is a block diagram showing an embodiment of the internal configuration of the imaging device shown in FIG. [Figure 4] FIG. 4 is an example of an image captured by the imaging device. [Diagram 5] FIG. 5 is a diagram showing a focused area in a captured image and an evaluation target area set in the captured image. [Figure 6] FIG. 6 is a diagram showing an image of the evaluation target area extracted from a captured image (extracted image), a first direction which is the maximum shake direction of the camera body, and a second direction which intersects with the first direction. [Figure 7] FIG. 7 shows an extracted image, a graph (first graph) showing an example of the spatial frequency-power spectrum in a first direction of the extracted image, and a graph (second graph) showing an example of the spatial frequency-power spectrum in a second direction. [Figure 8] FIG. 8 is a diagram showing an example of an LCD screen, and in particular a display example when shaking occurs in the camera body during shooting. [Figure 9]FIG. 9 is a diagram showing another example of the LCD screen, and in particular a display example when no shaking occurs in the camera body during shooting. [Figure 10] FIG. 10 is a diagram showing an embodiment in which an area for performing frequency analysis is further selected from an image (extracted image) within the evaluation target area. [Figure 11] FIG. 11 is a flowchart showing a first embodiment of the blur checking method according to the present invention, and is a flowchart showing processing during shooting. [Figure 12] FIG. 12 is a flowchart showing a first embodiment of the blur checking method according to the present invention, and is a flowchart showing the blur checking process during playback. [Figure 13] FIG. 13 is a flowchart showing a variation of the flowchart showing the blur confirmation process during playback shown in FIG. [Figure 14] FIG. 14 is a flow chart showing a second embodiment of the blur checking method according to the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0028] Hereinafter, preferred embodiments of an image processing device, an imaging device, a blur confirmation method, and a program according to the present invention will be described with reference to the accompanying drawings.

[0029] [Imaging device] <External appearance of imaging device> FIG. 1 is a perspective view of an imaging device including an image processing device according to the present invention, as viewed obliquely from the front, and FIG. 2 is a rear view of the imaging device shown in FIG.

[0030] As shown in FIG. 1, the imaging device 10 is a mirrorless digital single-lens camera that is composed of an interchangeable lens 100 and a camera body 200 to which the interchangeable lens 100 is detachable.

[0031] 1, a body mount 262 on which an interchangeable lens 100 is attached is provided on the front surface of the camera body 200, and a shutter release button 22, a shutter speed dial 23, an exposure compensation dial 24, a power lever 25, etc. are mainly provided on the top surface of the camera body 200. Note that 262A is a contact point connected to the interchangeable lens 100.

[0032] As shown in FIG. 2, the rear surface of the camera body 200 is provided with an EVF (Electronic View Finder) 26, a MENU / OK key 27, a cross key 28, a playback button 29, and an LCD (Liquid Crystal Display) 290, etc.

[0033] The LCD 290 functions as a display that displays a live view image in shooting mode, plays back and displays a captured image in playback mode, and displays various menu screens. In shooting mode, if you bring your eye close to the EVF 26, an eye sensor (not shown) automatically switches the display to the EVF 26, and if you move your eye away, the display switches to the LCD 290.

[0034] The MENU / OK key 27 is an operation key that functions both as a menu button for issuing a command to display a menu on the screen of the LCD 290 and as an OK button for issuing a command to confirm and execute a selection.

[0035] The cross key 28 is an operation unit for inputting instructions in four directions, up, down, left and right, and functions as a button for selecting an item from a menu screen and for instructing the selection of various setting items from each menu. In addition, the up and down keys of the cross key 28 function as a zoom switch during shooting or a playback zoom switch in playback mode, and the left and right keys function as frame-by-frame (forward and backward) buttons in playback mode. The playback button 29 is a button for switching to a playback mode in which a captured and recorded still image or video is displayed on the LCD 290.

[0036] In addition, a shake detection sensor 270 is provided in the camera body 200 .

[0037] The shake detection sensor 270 is composed of a gyro sensor (angular velocity sensor), and as shown in FIG. 2, if the horizontal direction of camera body 200 (left and right direction in FIG. 2) is the X-axis and the vertical direction of camera body 200 (up and down direction in FIG. 2) is the Y-axis, it detects the angular velocity around the X-axis (tilt direction T) and the angular velocity around the Y-axis (pan direction P).

[0038] <Internal configuration of the imaging device> FIG. 3 is a block diagram showing an embodiment of the internal configuration of the imaging device shown in FIG.

[0039] 3, the camera body 200 constituting the imaging device 10 includes an image sensor 210, an AFE (Analog Front End) 220, a sensor driver 230, a processor 240, a display driver 250, a memory 260, a shake detection sensor 270, an operation unit 280, the EVF 26, and an LCD 290.

[0040] The image sensor 210 is configured by a CMOS (Complementary Metal-Oxide Semiconductor) type color image sensor. Note that the image sensor 210 is not limited to the CMOS type, and may be a CCD (Charge Coupled Device) type image sensor.

[0041] The image sensor 210 has red (R), green (G) and blue (B) color filters arranged in a periodic color array (e.g., Bayer array, X-Trans (registered trademark), etc.) on a plurality of pixels composed of photoelectric conversion elements (photodiodes) arranged two-dimensionally in the x direction (horizontal direction) and y direction (vertical direction), and a microlens is arranged on each photodiode.

[0042] An optical image of a subject formed on the light receiving surface of the image sensor 210 by the imaging optical system of the interchangeable lens 100 is converted into an electrical signal by the image sensor 210. Charges corresponding to the amount of incident light are accumulated in each pixel of the image sensor 210, and an electrical signal corresponding to the amount of charge (signal charge) accumulated in each pixel is read out from the image sensor 210 as an image signal.

[0043] The processor 240 is composed of a CPU (Central Processing Unit) and the like, and performs overall control of each section of the camera body 200 and various types of processing in accordance with user operations using an operation section 280 .

[0044] The operation unit 280 includes the shutter release button 22, the shutter speed dial 23, the exposure compensation dial 24, the power lever 25, the MENU / OK key 27, the cross key 28, the playback button 29, etc., shown in FIGS.

[0045] The various processes by the processor 240 include a process for detecting two directions, that is, a first direction which is the maximum blur direction and a second direction which intersects with the first direction, based on the output of the shake detection sensor 270 detected during shooting of the captured image, a blur evaluation value calculation process for performing frequency analysis in the two directions based on an image in an evaluation target area set in the captured image, and calculating a first blur evaluation value which indicates the result of the frequency analysis in the first direction and a second blur evaluation value which indicates the result of the frequency analysis in the second direction, and a process for calculating the first blur evaluation value and the second blur evaluation value. The process includes a display process for displaying the image on the display (EVF 26, LCD 290), a first blur determination process for performing a first blur determination of the captured image by comparing the first blur evaluation value with the second blur evaluation value, a process for acquiring a first blur angle in a first direction and a second blur angle in a second direction based on the output of the blur detection sensor 270 detected during shooting of the captured image, a second blur determination process for comparing the first blur angle with the second blur angle and performing a second blur determination of the captured image, and a notification process for notifying the result of at least one of the first and second blur determinations. Details of these processes will be described later.

[0046] The memory 260 includes a flash memory, a ROM (Read-only Memory), a RAM (Random Access Memory), etc. The memory 260 also includes a memory card that is detachable from the camera body 200.

[0047] The flash memory and ROM are non-volatile memories that store firmware, various programs including the shake confirmation program according to the present invention, and captured images (still images, video images), etc.

[0048] The RAM functions as a working area for processing by the processor 240, and also temporarily stores firmware, a blur confirmation program, etc. stored in the non-volatile memory. Note that the processor 240 may incorporate a part of the memory 260 (RAM).

[0049] The sensor driver 230 controls the reading of image signals from the image sensor 210 in accordance with instructions from the processor 240. The sensor driver 230 also has an electronic shutter function that, in response to an electronic shutter control signal from the processor 240, discharges (resets) the electric charge accumulated in each pixel of the image sensor 210 and starts exposure.

[0050] The AFE 220 performs various analog signal processing on an analog image signal obtained by capturing an image of a subject with the image sensor 210, and converts the image signal after the analog processing into a digital image signal. The analog processing in the AFE 220 includes, for example, color separation processing, AGC (Automatic Gain Control), etc. The AGC functions as a sensitivity adjustment unit that adjusts the sensitivity (ISO sensitivity (ISO: International Organization for Standardization)) during shooting, and adjusts the gain of an amplifier that amplifies the input image signal so that the signal level of the image signal falls within an appropriate range.

[0051] When capturing a still image or a moving image, image data for each pixel of RGB (mosaic image data) output via the image sensor 210 and the AFE 220 is input to and temporarily stored in the memory 260. When the image sensor 210 is a CMOS image sensor, the AFE 220 is often built into the image sensor 210.

[0052] The processor 240 also functions as a digital signal processing unit that performs various types of digital signal processing on image data temporarily stored in the memory 260. That is, the processor 240 performs digital signal processing such as offset processing, gain control processing including sensitivity correction, gamma correction processing, demosaicing processing (also called demosaicing processing or synchronization processing), and RGB / YCrCb conversion processing on the image data input via the AFE 220, and stores the image data after the digital signal processing in the memory 260 again. Note that the demosaicing processing is, for example, a process of calculating all RGB color information for each pixel from a mosaic image made of RGB in the case of the image sensor 210 made of RGB three-color filters, and generates image data of three synchronized RGB planes from the mosaic data (dot-sequential RGB data).

[0053] The RGB / YCrCb conversion process converts the synchronized RGB data into luminance data (Y) and color difference data (Cr, Cb).

[0054] Furthermore, when recording a still image or a moving image, the processor 240 performs a compression process on the uncompressed luminance data Y and color difference data Cb, Cr temporarily stored in the RAM of the memory 260. In the case of a still image, the data is compressed in, for example, a Joint Photographic coding Experts Group (JPEG) format, and in the case of a moving image, the data is compressed in, for example, an H.264 format. The compressed image data is recorded in the flash memory of the memory 260. In addition, the processor 240 reads out the compressed image data from the flash memory of the memory 260 in a playback mode, performs a decompression process on the read image data, generates uncompressed image data, and causes the data to be displayed on the LCD 290 or the like via the display driver 250.

[0055] When displaying a live view image on the EVF 26 or the LCD 290, the processor 240 outputs digital image signals that have been captured at a predetermined frame rate (e.g., 30 fps or 60 fps) and digitally processed to the display driver 250. The display driver 250 converts the input time-series digital image signals into a signal format for display and outputs them sequentially to the EVF 26 or the LCD 290. As a result, the captured image is displayed in real time on the EVF 26 or the LCD 290.

[0056] The shutter release button 22 is a shooting instruction section for inputting instructions to shoot still images and videos, and is configured as a two-stroke switch consisting of a so-called "half press" (S1 press) and a "full press" (S2 press).

[0057] An S1_ON signal is output when the shutter release button 22 is "half-pressed," and an S2_ON signal is output when the shutter release button 22 is further pressed from the "half-press" to the "full press." In the still image shooting mode, when the S1_ON signal is output, the processor 240 executes shooting preparation processes such as AF control (automatic focus adjustment) and AE control (automatic exposure control), and when the S2_ON signal is output, it executes still image shooting and recording processes.

[0058] When performing AF control, the processor 240 calculates values ​​necessary for AF control based on a digital image signal. In the case of so-called contrast AF, for example, the processor 240 calculates an integrated value (focus evaluation value) of high frequency components of G signals within a predetermined AF area. The processor 240 moves a focus lens included in the lens group of the interchangeable lens 100 to a position where the focus evaluation value is maximized during AF control (i.e., a position where the contrast is maximized). Note that AF is not limited to contrast AF, and may be, for example, phase difference AF that detects a defocus amount based on pixel data of a phase difference detection pixel provided in an image sensor and moves the focus lens so that this defocus amount becomes zero.

[0059] When performing AE control, the processor 240 detects the brightness of the subject (subject luminance) and calculates a value (exposure value (EV)) necessary for AE control corresponding to the subject luminance. The processor 240 can determine the F-number, shutter speed, and ISO sensitivity from a predetermined program diagram based on the calculated EV value, and perform AE control.

[0060] Incidentally, the AF control and the AE control are automatically performed when the auto mode is set by the operation unit 280, and it goes without saying that the AF control and the AE control are not performed when the manual mode is set.

[0061] In addition, in the case of video recording mode, when the shutter release button 22 is fully pressed and an S2_ON signal is output, the camera body 200 enters a video recording mode in which video recording begins, and performs image processing and recording processing of the video. After that, when the shutter release button 22 is fully pressed again and an S2_ON signal is output, the camera body 200 enters a standby state and temporarily suspends the video recording processing.

[0062] [Shake check processing] Next, a blur confirmation process for confirming whether or not a captured image (still image) is blurred will be described.

[0063] In the case of an imaging device equipped with an image sensor with a large number of pixels, it is difficult to check whether or not a captured image is blurred even if the captured image is displayed on an LCD on the back of the imaging device.

[0064] The blur of the captured image to be confirmed by the imaging device 10 of this embodiment is, for example, a slight blur (blur) of the captured image caused by the blur of the imaging device 10 (tripod blur) when capturing an image with the camera body 200 (or the interchangeable lens 100) fixed to a tripod.

[0065] Tripod shake refers to a case where camera body 200 vibrates due to the operation of a mechanical drive unit inside camera body 200. The mechanical drive unit is a focal plane shutter (not shown), and camera body 200 vibrates due to mechanical shock during shooting (shutter shock caused by the collision of the front curtain of the focal plane shutter).

[0066] This tripod shake varies depending on the performance of the tripod, the environment in which the tripod is installed, the type of interchangeable lens, and the like.

[0067] FIG. 4 is an example of an image captured by the imaging device.

[0068] First, the processor 240 performs a process of detecting two directions, that is, a first direction which is the maximum blur direction and a second direction which intersects with the first direction, based on the output of the blur detection sensor 270 detected during the capture of the captured image.

[0069] That is, processor 240 acquires angular velocity signals in two intersecting directions (tilt direction T and pan direction P shown in FIG. 2) that accompany the shake of the imaging device body (camera body 200) during shooting from shake detection sensor 270. By integrating the acquired angular velocity signals in tilt direction T and pan direction P over time, it is possible to detect the shake angles in tilt direction T and pan direction P during shooting (exposure). Then, from each shake angle in tilt direction T and pan direction P, a first direction, which is the maximum shake direction of camera body 200, and a second direction that intersects (is perpendicular to) this first direction are detected.

[0070] Next, the processor 240 sets an evaluation target area in the captured image where blur evaluation is performed.

[0071] FIG. 5 is a diagram showing a focused area in a captured image and an evaluation target area set in the captured image.

[0072] When setting the evaluation target area 310, the processor 240 first detects the in-focus area 300 in the captured image. The in-focus area 300 can be extracted as an area that contains a lot of edge information in the captured image (an area with high contrast) using an edge extraction filter or the like.

[0073] Next, processor 240 extracts an area in which the texture is substantially uniform in two different directions (for example, the up-down direction and the left-right direction in the captured image) from within detected focused area 300, and sets the extracted area as evaluation target area 310. That is, processor 240 performs an automatic extraction process that automatically extracts an area that is in focused area 300 of the captured image and has a substantially uniform texture in two different directions as evaluation target area 310.

[0074] A region with a substantially uniform texture in two directions can be automatically extracted as an evaluation target region 310 by acquiring the frequency components in each of the two analysis directions for each local region in the focused region 300 and extracting the local region in which the frequency components in the two directions are closest to each other, or the local region in which the frequency components in the two directions are within a threshold value. In this example, the evaluation target region 310 is a rectangular region of a fixed size. The fixed size can be set as appropriate.

[0075] FIG. 6 is a diagram showing an image of the evaluation target area extracted from a captured image (extracted image), a first direction which is the maximum shake direction of the camera body, and a second direction which intersects with the first direction.

[0076] FIG. 6A shows an image (extracted image) of the evaluation target region, and FIG. 6B shows the extracted image with a first direction 320 and a second direction 330 superimposed thereon.

[0077] The first direction 320 is the direction in which the shaking of the camera body 200 during exposure is maximum, and the second direction 330 is a direction perpendicular to the first direction 320.

[0078] The first direction 320 and the second direction 330 can be detected based on the output of the shake detection sensor 270 as described above.

[0079] The processor 240 performs a frequency analysis in two directions (first direction 320, second direction 330) based on the extracted image, which is an image within the evaluation target area 310, and performs a blur evaluation value calculation process to calculate a first blur evaluation value indicating the result of the frequency analysis in the first direction 320 and a second blur evaluation value indicating the result of the frequency analysis in the second direction 330. The frequency analysis in this example performs an FFT (Fast Fourier Transform) on the extracted image in each of the first direction 320 and the second direction to calculate a spatial frequency-power spectrum in the first direction 320 (first blur evaluation value) and a spatial frequency-power spectrum in the second direction 330 (second blur evaluation value).

[0080] FIG. 7 shows an extracted image, a graph (first graph) showing an example of the spatial frequency-power spectrum in a first direction of the extracted image, and a graph (second graph) showing an example of the spatial frequency-power spectrum in a second direction.

[0081] FIG. 7A shows an extracted image similar to FIG. 6A, and FIG. 7B shows a first graph 340 showing an example of a spatial frequency-power spectrum in a first direction 320, and a second graph 350 showing an example of a spatial frequency-power spectrum in a second direction 330.

[0082] However, the actual spatial frequency-power spectrum in the first direction and the spatial frequency-power spectrum in the second direction are jagged (the intensity of each frequency component varies more significantly) than the first graph 340 and the second graph 350 shown in 7A. Therefore, it is preferable that the processor 240 performs a process to remove high-frequency components from the spatial frequency-power spectrum in the first direction, which is the first blur evaluation value, and the spatial frequency-power spectrum in the second direction, which is the second blur evaluation value, to create the first graph 340 and the second graph 350 shown in 7B. That is, it is preferable that the processor 240 performs a filtering process using an averaging filter or obtains a moving average on the spatial frequency-power spectrum in the first direction 320 and the spatial frequency-power spectrum in the second direction 330 to remove high-frequency components that would cause jaggedness, and to create the smooth first graph 340 and the second graph 350 shown in 7B.

[0083] Comparing first graph 340 and second graph 350 shown in FIG. 7B, in the mid-range of spatial frequencies, the power spectrum (power per bandwidth obtained by FFT analysis) shown by first graph 340 is lower than the power spectrum shown by second graph 350.

[0084] This is thought to be because the blur in the first direction 320, which is the direction of maximum blur, is greater than the blur in the second direction 330, which is perpendicular to the first direction 320. As a result, the high-frequency components in the first direction 320 in the extracted image are fewer than the high-frequency components in the second direction 330, resulting in a difference in the power spectra of the two.

[0085] The present invention performs frequency analysis on the extracted image within the evaluation target area for each of the first direction 320 and the second direction 330, and checks whether the captured image is blurred (whether the camera body 200 shook during shooting) by comparing a first blur evaluation value (power spectrum in the first direction 320) indicating the result of the frequency analysis with a second blur evaluation value (power spectrum in the second direction 330).

[0086] <First embodiment of shake confirmation> The processor 240 performs a display process to display the calculated first and second blurring evaluation values ​​on a display.

[0087] That is, processor 240 performs a blur evaluation value calculation process to calculate a first blur evaluation value (power spectrum) for the spatial frequency in first direction 320 of the extracted image, and a second blur evaluation value (power spectrum) for the spatial frequency in second direction 330, creates a first graph 340 indicating the first blur evaluation value and a second graph 350 indicating the second blur evaluation value, and displays the created first graph 340 and second graph 350 on LCD 290.

[0088] FIG. 8 is a diagram showing an example of an LCD screen, and in particular shows a display example when shaking occurs in the camera body during shooting.

[0089] On the LCD screen shown in FIG. 8, a captured image, a first graph 340, a second graph 350, and the like are displayed.

[0090] 8, first graph 340 and second graph 350 are displayed for comparison in a graph display area having spatial frequency on the horizontal axis and power spectrum (having the dimension of squared amplitude) on the vertical axis. In this graph display area, the vertical axis is longer than the horizontal axis so that the difference between first graph 340 and second graph 350 (difference in power spectrum) can be easily seen.

[0091] By visually checking the difference between the first graph 340 and the second graph 350 displayed in the graph display area of ​​the LCD screen shown in FIG. 8, the user can confirm that the captured image is blurred (that blur has occurred in the camera body 200 during shooting).

[0092] If the user notices that the captured image is blurred, the user can change the shooting position (location of the tripod) so that the imaging device 10 does not shake during shooting, or replace the tripod itself, or change the shooting conditions and take the image again.

[0093] FIG. 9 is a diagram showing another example of the LCD screen, and in particular shows a display example when no shaking occurs in the camera body during shooting.

[0094] In FIG. 9, a first graph 340 and a second graph 350 displayed in the graph display area of ​​the LCD screen show power spectra that are similar across the entire range of spatial frequencies.

[0095] By visually confirming that the difference between first graph 340 and second graph 350 is small, the user can confirm that the captured image is not blurred (that there is almost no blurring of camera body 200 during shooting).

[0096] <Second embodiment of shake confirmation> The processor 240 performs a first blur determination process in which the calculated first blur evaluation value and second blur evaluation value are compared to perform a first blur determination for the captured image.

[0097] That is, the processor 240 calculates the difference or ratio between the first and second blur evaluation values ​​(power spectra) for each spatial frequency, and performs the first blur determination based on the calculated difference or ratio and a threshold value. In this example, the maximum gap at which the difference (gap) between the two power spectra is maximum is obtained, and if this maximum gap is equal to or greater than the threshold value, it is determined that the captured image is blurred (a blur occurred in the camera body 200 during shooting).

[0098] It is preferable that processor 240 performs a notification process to notify the result of the first blur determination on the display. In this example, the result of the first blur determination is displayed so as to be visible by changing the color of the frame of the graph display area in which first graph 340 and second graph 350 shown in Fig. 8 and Fig. 9 are displayed. For example, when the captured image is blurred as shown in Fig. 8, a red frame 360 ​​is displayed in the graph display area, and when the captured image is not blurred as shown in Fig. 9, a white frame 362 is displayed in the graph display area.

[0099] In addition, the LCD screen shown in Figures 8 and 9 displays text information such as the angle (90° or 60) indicating the direction of shaking of the camera body 200 (maximum shaking direction), the maximum gap value, image size, etc., and the evaluation target area 310, first direction 320, and second direction 330 shown in Figure 6B, etc. are displayed in the captured image.

[0100] <Manual setting of evaluation target area> In the above embodiment, the processor 240 automatically extracts, based on the captured image, an area to be evaluated within the captured image that is a focused area of ​​the captured image and has uniform texture in two different directions (up-down and left-right), but the present invention is not limited to this and the area to be evaluated may be manually set by a user operation using the operation unit 280, etc.

[0101] That is, the processor 240 displays the captured image on the display (LCD 290), and when an evaluation target area in the captured image is designated by a user operation, performs a reception process for receiving the setting of the evaluation target area.

[0102] The processor 240, for example, performs peaking processing on the captured image, and displays the captured image subjected to peaking processing on the LCD 290, thereby displaying the in-focus area of ​​the captured image visibly on the LCD 290. The peaking processing is a process of adding a specific color (e.g., red) to the edge portion of the captured image, and adds the specific color to the portion where the amplitude of the edge portion extracted from the edge exceeds a threshold value. The peaking processing is known as a technique for assisting manual focus.

[0103] The user identifies the in-focus area from the captured image that has been subjected to the peaking process, searches for an area within the in-focus area that has a substantially uniform texture, and selects and instructs that area as an area to be evaluated through the operation unit 280. An area with a substantially uniform texture can be determined from an area where there is no bias in the direction of the peaking-processed edge.

[0104] When the evaluation target area is set by a user operation, the evaluation target area can be changed as appropriate, and blurring can be checked for each of a plurality of evaluation target areas.

[0105] <Second blur detection> In the second embodiment of blur confirmation, the first blur evaluation value and the second blur evaluation value are compared to perform a first blur determination of the captured image. In addition to this first blur determination, the following second blur determination may also be performed.

[0106] The processor 240 performs a process of acquiring a first shake angle in a first direction and a second shake angle in a second direction based on the output of the shake detection sensor 270 detected during shooting of the captured image, and a second shake determination process of comparing the first shake angle with the second shake angle to perform a second shake determination of the captured image.

[0107] That is, the processor 240 obtains the shake angles in the tilt direction T and pan direction P based on the output of the shake detection sensor 270 during shooting, detects a first direction 320 which is the maximum shake direction of the camera body 200, and a second direction 330 which is perpendicular to the first direction 320, and further obtains the shake angle in the first direction 320 (first shake angle) and the shake angle in the second direction (second shake angle) based on the output of the shake detection sensor 270. Then, the processor 240 compares the first shake angle with the second shake angle, and performs a second shake judgment of the captured image based on the comparison result.

[0108] Now, assuming that the first shake angle is A and the second shake angle is B, the processor 240 calculates the ratio (A / B) of the first shake angle A to the second shake angle B, and if the ratio (A / B) is equal to or greater than a threshold value, it can determine that the captured image is blurred (that blur occurred in the camera body 200 during shooting).

[0109] The second shake determination may be performed using the difference (AB) instead of the ratio (A / B) between the first shake angle A and the second shake angle B. In addition, it is preferable that the processor 240 notifies the result of the second shake determination on a display (for example, the LCD 290).

[0110] In addition, it is not necessary to perform both the first judgment process for performing the first blur judgment and the second judgment process for performing the second blur judgment. For example, if the second judgment process first determines that the captured image is blurred, the first judgment process can be omitted, and conversely, if the second judgment process first determines that the captured image is blurred, the second judgment process can be omitted.

[0111] FIG. 10 is a diagram showing an embodiment in which an area for performing frequency analysis is further selected from an image (extracted image) within the evaluation target area.

[0112] FIG. 10 shows a case where the up-down direction of a rectangular extracted image is the maximum blur direction (first direction), and the left-right direction is a direction (second direction) perpendicular to the maximum blur direction.

[0113] In this case, possible methods include performing FFT analysis on the entire area of ​​the extracted image for each of the first and second directions, as in the two extracted images on the left side of Figure 10, and performing FFT analysis on a portion of the extracted image (a strip-shaped area in the center of the extracted image) for each of the first and second directions, as in the two extracted images on the right side of Figure 10.

[0114] The former has a slow analysis speed but high analysis accuracy because the FFT analysis area is wide, while the latter has a fast analysis speed but low analysis accuracy because the FFT analysis area is narrow and strip-shaped. Since there is a trade-off between analysis speed and analysis accuracy, it is preferable to allow the user to select according to their needs. For example, if the user wants to shorten the time until the live view image display is restored after shooting, the user can select a method with a fast analysis speed, and if the live view image display is not performed, such as in the case of processing in playback mode, the user can select a method with a high analysis accuracy.

[0115] [How to check for blur] <First embodiment of blur confirmation method> 11 and 12 are flow charts showing a first embodiment of a blur checking method according to the present invention, and FIGS. 11 and 12 are flow charts showing processing during shooting and blur checking processing during playback, respectively.

[0116] <During shooting> 11, the imaging device 10 switches between modes such as still image shooting mode, video shooting mode, and playback mode by the operation unit 280, and when the imaging device 10 is set to the still image shooting mode, the processor 240 performs shooting preparation processes such as AF control and AE control when the shutter release button 22 is "half-pressed" to input an S1_ON signal (step S10). After that, when the shutter release button 22 is "fully pressed" to input an S2_ON signal (step S12), still image shooting (exposure) begins (step S14).

[0117] Next, processor 240 determines whether or not the "blur confirmation function", which can be set ON / OFF by operation unit 280, is set ON (step S16), and if the "blur confirmation function" is set ON, transitions to step S18, and if it is set OFF, transitions to step S26.

[0118] In step S18, processor 240 acquires in-focus area information at the timing when shutter release button 22 is "fully pressed" (S2_ON), ​​and stores the acquired in-focus area information in memory 260. In addition, processor 240 stores the output of blur detection sensor 270 during shooting (from the start of exposure to the end of exposure) in memory 260 (step S20).

[0119] Next, the processor 240 extracts an area of ​​a certain size with uniform texture in all directions from within the focused area of ​​the captured image as an evaluation target area (step S22). The processor 240 also detects the first direction 320, which is the maximum blur direction, and the second direction 330, which is perpendicular to the first direction 320, based on the output of the shake detection sensor 270 during shooting (see 7A in FIG. 7).

[0120] Processor 240 performs FFT analysis on the image of the evaluation area (extracted image) separately in first direction 320 and second direction 330, and obtains a spatial frequency-power spectrum in first direction 320 (first blur evaluation value) and a spatial frequency-power spectrum in second direction 330 (second blur evaluation value) (step S24).

[0121] If it is determined in step S16 that the "blur check function" is ON, the process proceeds to step S26 after processing in steps S18 to S24, where the display state in which the live view image is displayed is restored; if it is determined that the "blur check function" is OFF, the process skips the processing in steps S18 to S24 and proceeds to step S26, where the live view image continues to be displayed.

[0122] 11, the processor 240 stores the captured image (image file) captured by "fully pressing" (S2_ON) the shutter release button 22 in the memory 260. In addition, it is preferable that the in-focus area information stored in step S18, the output of the shake detection sensor 270 stored in step S20, and the spatial frequency-power spectrum in two directions, the first direction 320 and the second direction, acquired in step S24, are stored in association with each other as auxiliary information of the image file.

[0123] <First embodiment when checking blur> In the first embodiment, blur confirmation is performed when playing back an image captured when the "blur confirmation function" is ON.

[0124] 12, when playback button 29 is pressed to switch from shooting mode to playback mode, processor 240 then determines whether the playback mode is one in which the "blur confirmation function" is set to ON and whether the captured image being played back is an image captured when the "blur confirmation function" was ON (step S50). If the determination in step S52 is "Yes," processor 240 transitions to step S54, and if the determination is "No," transitions to step S66.

[0125] In step S66, the captured image is displayed on the entire screen of the LCD 290, and blur confirmation information and the like are not displayed.

[0126] Meanwhile, in step S54, it is determined whether the captured image was "panned". "Panning" is a shooting method in which shooting is performed while moving the camera in accordance with the movement of the subject. Processor 240 determines whether "panning" is performed from the change in angular velocity during shooting (during exposure) based on the output of shake detection sensor 270 saved at the time of shooting. If "panning" is not performed (if "No"), the process transitions to step S56, and if "panning" is performed (if "Yes"), the process transitions to step S66. Therefore, for a captured image that has been "panned", the captured image is displayed on the entire screen of LCD 290, and no blur confirmation information is displayed.

[0127] In step S56, processor 240 detects first direction 320, which is the maximum blur direction, and second direction 330 perpendicular to first direction 320, based on the output of shake detection sensor 270 saved at the time of shooting, and obtains first shake angle A in the first direction and second shake angle B in the second direction. Then, processor 240 obtains the ratio (A / B) of first shake angle A to second shake angle B, and determines whether or not the ratio (A / B) is equal to or greater than a threshold value (performs second shake determination processing).

[0128] If the ratio (A / B) is equal to or greater than the threshold (if "Yes"), the process proceeds to step S60, and if the ratio (A / B) is less than the threshold (if "No"), the process proceeds to step S58.

[0129] In step S58, processor 240 calculates the maximum gap between the spatial frequency-power spectrum in first direction 320 and the spatial frequency-power spectrum in second direction 330 acquired during shooting (acquired in step S24 of FIG. 11), and determines whether the maximum gap is greater than or equal to a threshold (performs a first blur determination process).

[0130] If the maximum gap is equal to or greater than the threshold (if "Yes"), the process proceeds to step S60, and if the maximum gap is less than the threshold (if "No"), the process proceeds to step S62.

[0131] In step S60, processor 240 displays on the screen of LCD 290 the captured image, first graph 340 and second graph 350 showing the spatial frequency-power spectrum in first direction 320 and the spatial frequency-power spectrum in second direction 330 acquired at the time of shooting, as shown in Fig. 8, and also displays information showing the blur confirmation result. Since it was determined in step S56 or step S58 that there is blur, in this example, the periphery of the graph display area is displayed in red (a red frame 360 ​​is displayed).

[0132] On the other hand, in step S62, processor 240 displays on the screen of LCD 290 the captured image, first graph 340 and second graph 350 showing the spatial frequency-power spectrum in first direction 320 and the spatial frequency-power spectrum in second direction 330 acquired at the time of shooting, as shown in Fig. 9, and further displays information showing the blur confirmation result. Since it was determined in step S58 that there was no blur, in this example, the periphery of the graph display area is displayed in white (a white frame 362 is displayed).

[0133] The user can check for shaking by the first graph 340 and the second graph 350 as shown in FIG. 7 or FIG. 8, and the frame color of the graph display area.

[0134] Thereafter, when a button operation indicating the end of blur confirmation is performed on operation unit 280 (step S64), the process transitions to step S66, where the display of the graph display area and the like is terminated, and the captured image is displayed on the entire screen of LCD 290.

[0135] 11 and 12, the in-focus area information and the output of the shake detection sensor 270 at the time of shooting are stored. However, the present invention is not limited to this. Information indicating the first direction 320, which is the maximum shake direction calculated based on the output of the shake detection sensor 270, and the second direction 330 perpendicular to the first direction 320, and the first shake angle A in the first direction 320 and the second shake angle B in the second direction 330 (or the ratio (A / B) of the first shake angle A and the second shake angle B) may be stored. In this case, if the latter information is stored, it is possible to calculate the spatial frequency-power spectrum in the first direction 320 and the spatial frequency-power spectrum in the second direction 330, and the first and second graphs indicating the spatial frequency-power spectrum in the first direction 320 and the spatial frequency-power spectrum in the second direction 330 based on the captured image at the time of playback, and it is also possible to perform the first judgment process, the second judgment process, and display of the judgment results.

[0136] <Second embodiment when checking blur> Fig. 13 is a flowchart showing a modification of the flowchart showing the blur confirmation process during playback shown in Fig. 12. In Fig. 13, steps common to those shown in the flowchart in Fig. 12 are given the same step numbers, and detailed descriptions thereof will be omitted.

[0137] The flowchart in FIG. 13 differs from the flowchart in FIG. 12 in that the order of the process in step S56 and the process in step S58 is reversed.

[0138] In the second embodiment of the blur confirmation shown in FIG. 13, the maximum gap between the spatial frequency-power spectrum in the first direction 320 and the spatial frequency-power spectrum in the second direction 330 acquired during shooting is calculated based on both power spectra, and a process of determining whether or not the maximum gap is equal to or greater than a threshold is performed first (first blur determination process in step S58). If it is determined in the first blur determination process in step S58 that the maximum gap is less than the threshold (in the case of “No”), then a process of determining whether or not the ratio (A / B) of the first blur angle A in the first direction to the second blur angle B in the second direction is equal to or greater than a threshold is performed (second blur determination process).

[0139] That is, the second embodiment when checking blur shown in FIG. 13 is the same as the first embodiment when checking blur shown in FIG. 12 in that the order of the first blur determination process and the second blur determination process is reversed, except for the other points.

[0140] <Second embodiment of blur confirmation method> Fig. 14 is a flow chart showing a second embodiment of the blur checking method according to the present invention. In Fig. 14, the same steps as those in the first embodiment of the blur checking method shown in Figs. 11 and 12 are given the same step numbers, and detailed descriptions thereof will be omitted.

[0141] The second embodiment of the blur confirmation method shown in FIG. 14 is processing performed during shooting and does not include processing during playback, and differs from the first embodiment of the blur confirmation method in that the first blur determination process or the second blur determination process is performed on the captured image, and even if the captured image is determined to be blurred by these determination processes, no notification is given to the user.

[0142] In FIG. 14, if the second blur determination process in step S56 determines that the ratio (A / B) of the first blur angle A in the first direction to the second blur angle B in the second direction is equal to or greater than a threshold value (if "Yes"), or if the first blur determination process in step S58 determines that the maximum gap between the spatial frequency-power spectrum in the first direction 320 and the spatial frequency-power spectrum in the second direction 330 is equal to or greater than a threshold value (if "Yes"), the captured image is saved in folder 2 (a second folder separate from the usual folder) in memory 260 as a blurred image (step S70).

[0143] On the other hand, if the captured image is taken with the blur confirmation function OFF, or if the captured image is taken with the blur confirmation function ON and is determined to be not blurred by the first blur determination process in step S56 and the second blur determination process in step S58 (if both steps S56 and S58 return "No"), the captured image is saved in folder 1 (normally the first folder) in memory 260 (step S72).

[0144] As a modified example of the second embodiment of the blur checking method, the blur check result may be displayed on the LCD 290 during shooting, similar to the first embodiment, and the display state of the live view image may be restored after the user inputs an instruction to record or not record. This allows reshooting if blur is found in the captured image.

[0145] In addition, in both the first and second embodiments of the blur confirmation method, the first blur determination process and the second blur determination process are performed, but any method may be used as long as it includes at least the first blur determination process.

[0146] [others] The imaging device according to the present invention is not limited to a mirrorless digital single-lens camera, but includes various cameras such as digital single-lens reflex cameras, compact cameras with an integrated lens, and cameras built into smartphones and the like.

[0147] In this embodiment, the hardware structure of a processing unit that executes various processes, such as the processor 240 of the camera body 200, is various processors as shown below. The various processors include a CPU (Central Processing Unit), which is a general-purpose processor that executes software (programs) and functions as various processing units, a programmable logic device (PLD), such as an FPGA (Field Programmable Gate Array), whose circuit configuration can be changed after manufacture, and a dedicated electric circuit, such as an ASIC (Application Specific Integrated Circuit), which is a processor having a circuit configuration designed specifically for executing specific processes.

[0148] One processing unit may be configured with one of these various processors, or may be configured with two or more processors of the same or different types (for example, multiple FPGAs, or a combination of a CPU and an FPGA). Also, multiple processing units may be configured with one processor. As an example of configuring multiple processing units with one processor, first, as represented by a computer such as a client or server, there is a form in which one processor is configured with a combination of one or more CPUs and software, and this processor functions as multiple processing units. Second, as represented by a system on chip (SoC), there is a form in which a processor is used that realizes the functions of the entire system including multiple processing units with one IC (Integrated Circuit) chip. In this way, the various processing units are configured using one or more of the above various processors as a hardware structure.

[0149] Furthermore, the hardware structure of these various processors is, more specifically, an electric circuit that combines circuit elements such as semiconductor elements.

[0150] The present invention also includes a blur confirmation program that, when installed in a computer, causes the computer to function as the image processing device according to the present invention, and a non-volatile storage medium on which this blur confirmation program is recorded.

[0151] Furthermore, the present invention is not limited to the above-described embodiment, and various modifications are possible without departing from the spirit of the present invention. [Explanation of symbols]

[0152] 10. Imaging device 22 Shutter release button 23 Shutter Speed ​​Dial 24 Exposure compensation dial 25 Power Lever 27 MENU / OK key 28 Cross key 29 Play button 100 Interchangeable Lenses 200 Camera body 210 Image Sensor 220 Camera body 230 Sensor Driver 240 processors 250 Display Driver 260 Memory 262 Body Mount 270 Shake detection sensor 280 Operation section 290 LCD 300 focus area 310 Evaluation Area 320 1st direction 330 Second direction 340 Graph 1 350 Graph 2 360 Red Frame 362 White frame S10~S26, S50~S66, S70~S72 steps

Claims

1. An image processing device including a shake detection sensor and a processor, The processor, A process of detecting two directions, a first direction being a maximum blur direction and a second direction intersecting the first direction, based on an output of the shake detection sensor detected during shooting of a photographed image; a blur evaluation value calculation process for performing a frequency analysis in the two directions based on an image within an evaluation target area set within the captured image, and calculating a first blur evaluation value indicating a result of the frequency analysis in the first direction and a second blur evaluation value indicating a result of the frequency analysis in the second direction; performing a display process for displaying the first blurring evaluation value and the second blurring evaluation value on a display; Image processing device.

2. An image processing device including a shake detection sensor and a processor, The processor, A process of detecting two directions, a first direction being a maximum blur direction and a second direction intersecting the first direction, based on an output of the shake detection sensor detected during shooting of a photographed image; a blur evaluation value calculation process for performing a frequency analysis in the two directions based on an image within an evaluation target area set within the captured image, and calculating a first blur evaluation value indicating a result of the frequency analysis in the first direction and a second blur evaluation value indicating a result of the frequency analysis in the second direction; performing a first blur determination process by comparing the first blur evaluation value with the second blur evaluation value to perform a first blur determination of the captured image; Image processing device.

3. The second direction is a direction perpendicular to the first direction.

3. The image processing device according to claim 1 or 2.

4. The processor, acquiring a first shake angle in the first direction and a second shake angle in the second direction based on the output of the shake detection sensor detected during shooting of the captured image; a second blur determination process for performing a second blur determination of the captured image by comparing the first blur angle with the second blur angle; The image processing device according to claim 2 .

5. the processor performs a notification process for notifying a result of at least one of the first shake determination and the second shake determination. The image processing device according to claim 4.

6. the blur evaluation value calculation process includes creating a first graph showing the first blur evaluation value versus a spatial frequency in the first direction, and a second graph showing the second blur evaluation value versus a spatial frequency in the second direction. The image processing device according to claim 1 .

7. the blur evaluation value calculation process includes a process of removing high frequency components from the first blur evaluation value and the second blur evaluation value, and creating the first graph and the second graph. The image processing device according to claim 6.

8. the first blur determination process calculates a difference or a ratio between the first blur evaluation value and the second blur evaluation value, and performs the first blur determination based on the difference or the ratio and a threshold value. The image processing device according to claim 2 .

9. the display process displays the first graph and the second graph on the display; 8. The image processing device according to claim 6 or 7.

10. The processor performs an automatic extraction process to automatically extract, as the evaluation target region, a region that is a focused region of the photographed image and has a uniform texture in two different directions. The image processing device according to claim 1 .

11. The processor displays the captured image on the display, and performs a reception process of receiving a setting of the evaluation target area within the captured image through a user operation.

8. The image processing device according to claim 1, 6 or 7.

12. the processor causes the display to visibly display a focus area of ​​the captured image, and performs a reception process of receiving a selection instruction for the evaluation target area by a user operation within the focus area.

8. The image processing device according to claim 1, 6 or 7.

13. the processor performs a storage process of storing the captured image in a memory in association with the output of the shake detection sensor detected during capture of the captured image or the detected maximum shake direction; The image processing device according to claim 1 .

14. the first blurring evaluation value and the second blurring evaluation value are power spectra.

13. The image processing device according to claim 1, 6, 7, 9, 11 or 12.

15. An image processing device comprising: Imaging device.

16. An image processing device comprising: an image processing apparatus according to claim 2 or 8; and a memory; the processor performs a storage process of storing the captured image determined to be not blurred by the first blur determination process in a first folder in the memory, and storing the captured image determined to be blurred in a second folder in the memory. Imaging device.

17. acquiring a captured image; acquiring an output of a shake detection sensor during capture of the captured image; detecting two directions, a first direction being a maximum shake direction and a second direction intersecting the first direction, based on the output of the shake detection sensor; performing a frequency analysis in the two directions based on an image within an evaluation target area set within the captured image, and calculating a first blur evaluation value indicating a result of the frequency analysis in the first direction and a second blur evaluation value indicating a result of the frequency analysis in the second direction; displaying the first blur evaluation value and the second blur evaluation value on a display; A blur checking method in which a processor performs processing for each step.

18. acquiring a captured image; acquiring an output of a shake detection sensor during capture of the captured image; detecting two directions, a first direction being a maximum shake direction and a second direction intersecting the first direction, based on the output of the shake detection sensor; performing a frequency analysis in the two directions based on an image within an evaluation target area set within the captured image, and calculating a first blur evaluation value indicating a result of the frequency analysis in the first direction and a second blur evaluation value indicating a result of the frequency analysis in the second direction; a step of comparing the first blur evaluation value with the second blur evaluation value to perform a first blur determination of the captured image, A blur checking method in which a processor performs processing for each step.

19. acquiring a first shake angle in the first direction and a second shake angle in the second direction based on the output of the shake detection sensor detected during shooting of the captured image; and comparing the first blur angle with the second blur angle to perform a second blur determination of the captured image. The method for checking blur according to claim 18.

20. when the second blur determination does not confirm the blur of the photographed image, the first blur determination is used to determine the blur of the photographed image. The method for checking blur according to claim 19.

21. A function for acquiring photographed images; A function of acquiring an output of a shake detection sensor while capturing the captured image; a function of detecting two directions, a first direction being a maximum shake direction and a second direction intersecting the first direction, based on the output of the shake detection sensor; a function of performing a frequency analysis in the two directions based on an image within an evaluation target area set within the captured image, and calculating a first blur evaluation value indicating a result of the frequency analysis in the first direction and a second blur evaluation value indicating a result of the frequency analysis in the second direction, a function of displaying the first blur evaluation value and the second blur evaluation value on a display; A computer-based blur checking program.

22. A function for acquiring photographed images; A function of acquiring an output of a shake detection sensor while capturing the captured image; a function of detecting two directions, a first direction being a maximum shake direction and a second direction intersecting the first direction, based on the output of the shake detection sensor; a function of performing a frequency analysis in the two directions based on an image within an evaluation target area set within the captured image, and calculating a first blur evaluation value indicating a result of the frequency analysis in the first direction and a second blur evaluation value indicating a result of the frequency analysis in the second direction, a function of performing a first blur determination of the captured image by comparing the first blur evaluation value with the second blur evaluation value; A computer-based blur checking program.

Images