Image processing apparatus, image processing method, and program

Abstract

To provide an image processing apparatus, an image processing method, and a program capable of improving accuracy of estimating a PSF and properly adding a blurring effect or a deblurring effect to a specific region of an image.SOLUTION: An apparatus includes identifying means configured to identify a specific region in an image, and PSF estimation means configured to perform PSF estimation for the specific region. The PSF estimation means is capable of performing at least one of edge extraction and mask processing on the image, and is configured to estimate the PSF for the processed image.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present invention relates to a technique for imparting a blur effect or a deblur effect to an image. [Background technology]

[0002] Conventionally, there has been known a technique for estimating a point spread function (PSF) that represents the characteristics of the motion or blur of a subject included in an image, and applying a blur effect or deblur effect to the image based on the estimated PSF. For example, Patent Document 1 discloses a method for reducing (deblurring) the image motion included in a captured image by identifying a specific region in the captured image, calculating a PSF by referring to the image of the specific region, and deconvolving the obtained PSF. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2007-306548 A [Patent Document 2] JP 2002-77711 A [Non-patent literature]

[0004] [Non-Patent Document 1] Aki Shibuya, Yoshiaki Sugiyama, Yasuo Ariki, "Automatic Classification of Sports Articles and Search for Similar Scenes", Proceedings of the 55th National Conference of the Information Processing Society of Japan, pp.65-66 (September 1997) [Non-Patent Document 2] Ryosuke Araki, Kosuke Mano, Takeshi Onishi, Masanori Hirano, Tsubasa Hirakawa, Takayoshi Yamashita, Hironobu Fujiyoshi, “Object Grasping Using Object Pose Estimation by Iterative Updates Based on Backpropagation of Image Generative Networks”, 2020 Annual Conference of the Robotics Society of Japan, 2020. Summary of the Invention [Problem to be solved by the invention]

[0005] A feature of PSF estimation is that the estimated PSF result may be degraded depending on the pattern of the subject being referenced. For example, a fine lattice pattern, such as that of a tennis string, appears as a shape in the estimated characteristics. In other words, the blur characteristics of the subject cannot be estimated correctly. In the method of Patent Document 1, correction processing is performed using such an incorrect PSF, which may cause image quality problems.

[0006] The present invention has been made in consideration of the above problems, and has an object to provide an image processing device, an image processing method, and a program that can improve the estimation accuracy of the PSF and preferably apply a blur effect or deblur effect to a specific area of ​​an image. [Means for solving the problem]

[0007] In order to achieve the above object, an image processing device according to the present invention comprises: identification means for identifying a specific region within an image; and PSF estimation means for estimating a PSF in the specific region, wherein the PSF estimation means is capable of performing at least one of edge extraction and mask processing on the image, and estimates a PSF for the processed image. Effect of the Invention

[0008] According to the present invention, it is possible to provide an image processing device, an image processing method, and a program that improve the estimation accuracy of the PSF and enable a blur effect or deblur effect to be preferably applied to a specific area of ​​an image. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram for explaining the configuration of a digital camera according to a first embodiment. [Diagram 2] FIG. 2 is a diagram for explaining the configuration of an image processing unit in the first embodiment. [Diagram 3] 5 is a flowchart for explaining the operation of an image processing unit in the first embodiment. [Figure 4]FIG. 4 is a diagram for explaining a captured image in the first embodiment. [Diagram 5] FIG. 4 is a diagram for explaining a target region of PSF estimation in the first embodiment. [Figure 6] 5A to 5C are diagrams for explaining the characteristics of a filter applied to a target region for PSF estimation in the first embodiment. [Figure 7] 5A to 5C are diagrams for explaining a PSF estimation result in the first embodiment. [Figure 8] 5A to 5C are diagrams for explaining the effect of the blurring process in the first embodiment. [Figure 9] FIG. 11 is a diagram for explaining the configuration of an image processing unit in the second embodiment. [Figure 10] 10 is a flowchart for explaining the operation of an image processing unit in the second embodiment. [Figure 11] FIG. 11 is a diagram for explaining a table for conversion into weights in the second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] [Example 1] A preferred embodiment of the present invention will be described in detail below with reference to the drawings. The embodiment described below is an imaging device, and an example in which the present invention is applied to a digital camera as an example of an imaging device will be described. The present invention relates to a correction process applied to a captured image, more specifically, a method of correcting an image signal for estimation, estimating a PSF, and applying a blur effect or deblur effect to a specific area of ​​the image based on the estimated PSF. Therefore, a configuration related to shooting and recording in the imaging device is not essential. In this embodiment, "blur" includes out-of-focus due to focus deviation of an image, and "shake" includes, but is not limited to, camera shake and subject shake.

[0011] 1 is a block diagram showing the functional configuration of a digital camera according to an embodiment of the present invention. A control unit 101, such as a CPU, reads out an operation program for each block of the digital camera 100 from a ROM 102, loads it into a RAM 103, and executes it to control the operation of each block of the digital camera 100.

[0012] ROM 102 is a rewritable non-volatile memory, and stores parameters and the like necessary for the operation of each block in addition to the operating programs of each block of digital camera 100. RAM 103 is a rewritable volatile memory, and is used as a temporary storage area for data output during the operation of each block of digital camera 100.

[0013] The optical system 104 forms a subject image on the imaging unit 105. The optical system 104 includes, for example, a fixed lens, a variable magnification lens that changes the focal length, a focus lens that adjusts the focus, etc. The optical system 104 also includes an aperture, and the aperture adjusts the aperture diameter of the optical system to adjust the amount of light during shooting.

[0014] The imaging unit 105 is an imaging element such as a CCD or CMOS sensor, which photoelectrically converts an optical image formed on the imaging element by the optical system 104, and outputs the obtained analog image signal to the A / D conversion unit 106. The A / D conversion unit 106 applies A / D conversion processing to the input analog image signal, and outputs the obtained digital image data to the RAM 103 for storage.

[0015] The image processing unit 107 applies various image processing such as white balance adjustment, color interpolation, and gamma processing to the image data stored in the RAM 103, and outputs the image data to the RAM 103. The image processing unit 107 also performs processing to impart a blur effect or a deblur effect to the captured image and the image data stored in the RAM 103.

[0016] The recording medium 108 is a removable memory card or the like, and images processed by the image processing unit 107 stored in the RAM 103 and images A / D converted by the A / D conversion unit 106 are recorded as recorded images.

[0017] The display unit 109 is a display device such as an LCD. The display unit 109 provides various types of information for the digital camera 100, such as providing an electronic viewfinder function by displaying a through-display of the subject image captured by the imaging unit 105, and reproducing and displaying images recorded on the recording medium 108.

[0018] Operation input unit 110 includes a user input interface such as a release switch, a setting button, a mode setting dial, etc., and when it detects an operation input made by a user, it outputs a control signal corresponding to the operation input to control unit 101. In addition, in an embodiment in which display unit 109 is equipped with a touch panel sensor, operation input unit 110 also functions as an interface that detects a touch operation made to display unit 109.

[0019] The configuration and basic operation of digital camera 100 have been described above.

[0020] Next, the operation of the image processing unit 107, which is a feature of the first embodiment of the present invention, will be described in detail. In this embodiment, it is assumed that a subject performing a tennis swing is photographed. In general, in photographing sports involving the action of swinging a racket, club, bat, etc. held in the hand, such as tennis, golf, and baseball batting (hereinafter referred to as swing sports), it is preferable that the face of the subject playing the game is not blurred or blurred. On the other hand, if the swinging area, such as the racket or arm, is blurred in the swinging direction, it gives a sense of dynamism, and the intensity of the play can be conveyed more impressively to the viewer, resulting in an image that leaves a good impression. In this embodiment, a method of estimating the PSF in the swinging area with high accuracy and applying a blur effect based on the estimated PSF will be described.

[0021] The blur effect applying process by the image processing unit 107 will be described with reference to Figures 2 and 3. Figure 2 is a block diagram of the image processing unit 107. Figure 3 is a flowchart of the process performed by the image processing unit 107. The blur effect applying process can be performed on an image that has been captured while an instruction to execute the process has been given, or an image that has been recorded on the recording medium 108, for example, and for which an instruction to execute the process has been given from a menu screen or the like.

[0022] 2, image processing unit 107 is made up of a main subject detection unit 201, a PSF estimation region extraction unit 202, a PSF estimation region correction unit 203, a PSF estimation unit 204, and an image correction unit 205. Each unit of image processing unit 107 executes its respective function under the command of control unit 101.

[0023] Next, in S301 of FIG. 3, the imaging unit 105 captures an image of the subject under the control of the control unit 101. When an analog image signal is obtained by the capture, the A / D conversion unit 106 A / D converts it and stores it in the RAM 103. The control unit 101 sends it to the image processing unit 107 as image data 206. An example of the captured image data is shown in FIG. 4. FIG. 4 captures a person subject 401 performing a tennis swing. The exposure time is 1 / 500 seconds, and there is no blurring of the face. Also, the swinging racket is subject to slight motion blurring. Also, the image size is 1024×680 pixels, and the size of one pixel is 5 μm. In this embodiment, tennis is used as an example of a sport that involves a swinging motion, but this embodiment can be applied to other sports. Specific examples include golf, badminton, table tennis, baseball, lacrosse, hockey, fencing, kendo, canoeing, and boating. In each sport, a dynamic sports image can be obtained by performing processing to impart a blur effect that creates a motion blur in the area of ​​the swinging object (sports equipment).

[0024] In S302, under the control of the control unit 101, the main subject detection unit 201 detects the main subject in the image data 208 captured in S301. Here, the main subject is detected as a human subject 401. A known technique may be used to detect the human subject 401 as the main subject, and for example, the technique disclosed in Patent Document 2 may be used.

[0025] In S303, under the control of the control unit 101, the PSF estimation region extraction unit 202 extracts a rectangular region including the swinging object (tennis racket) from the human subject 401 detected by the main subject detection unit 201 (extraction of a specific region). A known technique may be used to extract the tennis racket region, and the region may be extracted using a technique such as that disclosed in Patent Document 2, as with the main subject in S302. Alternatively, the user may be allowed to specify the PSF estimation region via the operation input unit 110. In this embodiment, the size of the PSF estimation region is a square region of 256×256 pixels. The extracted region is shown in FIG. 5(a). In FIG. 5(a), a region including the swinging object (tennis racket) is extracted from the image in FIG. 4 as a target region for PSF estimation.

[0026] In S304, under the control of the control unit 101, the PSF estimation region correction unit 203 applies a filter to extract edges to the region extracted by the PSF estimation region extraction unit 202. This processing step removes gut parts that degrade the PSF estimation result. Specifically, a band-pass filter is applied that can extract edges while removing high-frequency components contained in the PSF estimation region.

[0027] An example of the characteristics of the bandpass filter is shown in FIG. 6. In FIG. 6, the horizontal axis is the frequency, and "1" is the Nyquist frequency. The cutoff frequency is set with reference to the size of the PSF estimation region. In this embodiment, a fine object that repeats at a 5-pixel period, which is about 2% of the size of the PSF estimation region, is cut off when it passes through the bandpass filter. The PSF estimation region after edge extraction is shown in FIG. 5(b). Note that in this embodiment, the extraction band during edge extraction is controlled to remove the object pattern that deteriorates the accuracy of the PSF estimation, but this method is not limited to this method. For example, the edge distance may be calculated for a signal that has been subjected to an edge extraction filter such as a Sobel filter, and a masking process may be performed so that the signal value is low for an area with a short edge distance such as a gut area. Alternatively, a masking process may be performed so that the signal value is 0 (luminance signal is 0) for the gut area in the signal before the edge extraction process, and the edge may be extracted from the signal after the masking process. Whichever method is used, the object pattern that deteriorates the PSF estimation result can be removed in advance, and the accuracy of the subsequent PSF estimation can be improved.

[0028] In S305, the PSF estimation unit 204 estimates the PSF using the signal output by the PSF estimation region correction unit 203 under the control of the control unit 101. A known technique can be used for the PSF estimation. Below, an overview of PSF estimation using an iterative method will be described as an example.

[0029] Let B be the image signal of the PSF estimation region, K be the PSF to be estimated, and L be an image (latent image) in the PSF estimation region that is not blurred or blurred. If we ignore noise in the image, the relationship expressed by the following equation (1) holds.

[0030]

number

[0031] In equation (1), Conv represents a convolution operation. Since both K and L are unknowns, equation (1) cannot be solved analytically. Therefore, the following estimation is performed.

[0032] First, determine an appropriate initial value for the latent image L. As the initial value for the latent image L, an image obtained by performing edge extraction processing on a captured image can be used.

[0033] Next, in the energy function E(K) expressed by the following formula (2), L is fixed and K is an unknown, and K that minimizes the energy function E(K) is calculated.

[0034]

number

[0035] In equation (2), σ represents a regularization term, and here, the L2 norm, which is the sum of the squares of each element of the PSF, is used.

[0036] Next, using the calculated K, L that minimizes the energy function E(L) expressed by the following equation (3) is calculated with K fixed and L as an unknown.

[0037]

number

[0038] Then, the PSF is estimated by repeatedly calculating K that minimizes the energy function E(K) using the calculated L, and calculating L that minimizes the energy function E(L) using the calculated K until the calculated K converges to a certain value. This concludes the outline of PSF estimation using the iterative method.

[0039] Fig. 7(a) shows the result (kernel) of PSF estimation using the signal output by PSF estimation area correction unit 203 in Fig. 5(b). In Fig. 7(a), the direction of the tail represents the direction of the shake, the length of the tail represents the amount of shake, and the black and white shading represents the intensity of the shake. On the other hand, Fig. 7(b) shows the result of PSF estimation using a signal without conventional correction. In Fig. 7(b), it can be seen that the lattice pattern on the edge of the string area affects the PSF estimation result. The pattern of the subject is not visible in Fig. 7(a), and it can be seen that the direction and amount of racket shake were correctly estimated.

[0040] In S306, under the control of the control unit 101, the image correction unit 205 refers to the signal output by the PSF estimation unit 204 and applies a blur effect to the image data 206. First, a kernel that emphasizes the blur is generated in the direction of the blur with reference to the kernel in Fig. 7(a).

[0041] The generated kernel is shown in Figure 8(a). Next, in the image data 206, a convolution operation is performed with the kernel that emphasizes blur on the corresponding area extracted by the PSF estimation area extraction unit 202 in S303, to impart a blur effect. The result of imparting the blur effect is shown in Figure 8(b). As can be seen from Figure 8(b), there is no blur or out-of-focus on the face of the person subject playing the game. On the other hand, there is strong motion blur in the area of ​​the swinging racket, and a dynamic image has been generated.

[0042] For example, when the exposure time is used as a reference for the amount of blur to be emphasized, the exposure time in this embodiment is 1 / 500 seconds, and when this is to be made equivalent to 1 / 250 seconds, the amount (length) and strength of blur in the PSF estimation result should be doubled. Also, the size of the sports equipment in the screen or the value obtained by dividing the distance to the subject by the focal length at the time of shooting may be used as a reference, and the amount of blur emphasis corresponding to the reference value is determined in advance. Alternatively, presets such as the degree of emphasis (weak, medium, strong) may be prepared, and the degree may be determined according to the user's selection. By appropriately determining the characteristics of the blur effect to be applied in this way, it is possible to generate a suitable image.

[0043] In this embodiment, the blur effect is applied to the swinging racket area, but the deblur effect can also be applied to an area where a blur (including out of focus) occurs (for example, a face area). In this case, in S303, the PSF estimation area extraction unit 202 extracts the face area of ​​the human subject 401 detected by the main subject detection unit 201 as a specific area. Then, in S304, the PSF estimation area correction unit 203 detects a skin-colored area in the extracted area so that only the edges of the skin-colored part of the area extracted by the PSF estimation area extraction unit 202 are valid, and performs a mask process of the skin-colored area on the edge extraction result. In this way, it is possible to prevent deterioration of the PSF estimation due to the influence of an object with a fine pattern that exists around the face, such as a mesh-like hat. Next, the PSF is estimated using the signal of the corrected PSF estimation area. Finally, the image correction unit 205 applies the deblur effect by performing a deconvolution operation between the face area in the image data 206 and the estimated PSF. The deconvolution operation is expressed by the following equation (4).

[0044]

number

[0045] In equation (4), F represents the Fourier transform and F^-1 represents the inverse Fourier transform.

[0046] In this embodiment, the size of the PSF estimation region is a square region of 256×256 pixels, but the size is not limited to this, and regions other than a square may be targeted. For example, when the object being swung is a long and thin object such as a baseball bat, it is preferable to make the PSF estimation region rectangular. Such a configuration can prevent other subjects existing in the background from being mixed in, and improve the accuracy of the PSF estimation for the swinging object. In this case, the corresponding rectangular region is divided into multiple square rectangles, and the PSF is estimated for each divided rectangle, and the PSF estimation result for the original rectangle can be calculated by adding them in frequency space. Specifically, when the rectangle is divided into n regions, the value (F(K_all)) of the PSF estimation result for the rectangle in frequency space is expressed by the following formula (5).

[0047]

number

[0048] Furthermore, if the PSF is not estimated for all divided regions due to computational load or other reasons, the priority of the divided regions to be used for PSF estimation can be determined with reference to the edge extraction results. Specifically, the more widely the edges present within a region are dispersed, the more stable the PSF estimation results will be, so such regions are given priority for reference in the estimation. For example, it is advisable to give priority to regions that include parts of sporting equipment where the manufacturer's logo or text is printed.

[0049] In addition, using a circular object rather than a linear object results in less edge bias and allows for more stable PSF estimation results with respect to the direction of blur. For example, in the case of a golf club, it is better to give priority to the area including the head rather than the area including the shaft for estimation.

[0050] [Example 2] A second embodiment of the present invention will be described below. Note that the embodiment described below is an image capture device similar to the first embodiment, and the present invention will be described by using an example in which the present invention is applied to a digital camera as an example of an image capture device.

[0051] In the first embodiment, it was possible to obtain dynamic images by improving the accuracy of PSF estimation, but in the present embodiment, scene recognition (scene estimation) using machine learning and deep learning techniques is added to stably generate high-quality dynamic images.

[0052] The configuration of the digital camera according to the embodiment of the present invention is the same as that shown in the block diagram of Fig. 1 described in the first embodiment, and therefore a description thereof will be omitted. In this embodiment, unlike the first embodiment, the image processing unit 107 has a configuration as shown in Fig. 9. Details will be described below.

[0053] 9, a main subject detection unit 901, a PSF estimation region extraction unit 902, a PSF estimation region correction unit 903, and a PSF estimation unit 904 are the same as those in the first embodiment. An image correction unit 905 receives image data 906 and outputs from the PSF estimation unit 904 and a scene recognition unit 907. The scene recognition unit 907 is a process that is newly added in the second embodiment.

[0054] Fig. 10 is a flowchart of the process performed by the image processing unit 107 in the embodiment 2. Note that in the embodiment 2, it is assumed that a subject making a tennis swing is photographed as in Fig. 4. The process from S1001 to S1003 in Fig. 10 is the same as the process from S301 to S303 in the embodiment 1, and therefore the description thereof will be omitted.

[0055] In S1004, under the control of the control unit 101, the scene recognition unit 907 recognizes (estimates) the scene captured in the image data 906. Here, the type of sport is recognized. A known technique may be used to recognize what sport is being played in the input scene, and for example, a technique such as that disclosed in Non-Patent Document 1 may be used. It is possible to recognize that not only tennis, but also sports such as golf and baseball are being played.

[0056] Furthermore, the scene recognition unit 907 estimates the direction of movement of the racket during shooting from the result of the recognition of the type of sport, the facing direction of the digital camera 100 and the human subject 401, and the result of estimating the position and orientation of the racket in the image data of the racket area extracted in S1003. Note that the method of estimating the position and orientation of the racket may be, for example, a method disclosed in Non-Patent Document 2.

[0057] The process of S1005 is the same as that of S304 in the first embodiment, and therefore the description thereof will be omitted.

[0058] In S1006, under the control of the control unit 101, the PSF estimation unit 904 estimates the PSF using the signal output by the PSF estimation area correction unit 903. In the PSF estimation in this embodiment, in addition to the processing in the first embodiment, an evaluation value of the edge bias degree in the input signal is calculated. Specifically, the angle of each edge included in the area is calculated, and the variance value of the angle in the entire area is set as the evaluation value of the bias degree. The higher the evaluation value, the more edges with various angles are present in the area. In the PSF estimation, the more edges with various angles are present in the estimation area, the more stable the estimation result of the blur direction, so the higher the evaluation value of the edge bias degree, the more accurate the PSF estimation is generated. The PSF estimation unit 904 outputs the PSF estimation result and the evaluation value of the edge bias degree to the image correction unit 905.

[0059] In S1007, under the control of the control unit 101, the image correction unit 905 refers to the signal output by the PSF estimation unit 904 and the output signal of the scene recognition unit 907, and applies a blur effect to the image data 906. The direction (angle) of the blur to be applied is determined by integrating the direction estimated by the PSF estimation and the direction estimated by the scene recognition unit using a weight generated from the evaluation value of the edge bias degree. FIG. 11 shows a table for converting the evaluation value of the edge bias degree into a weight α. As can be seen from FIG. 11, the weight α is designed to be larger as the evaluation value of the edge bias degree becomes higher (the accuracy of the PSF estimation becomes higher). Next, the direction of the blur to be finally applied (DIR_BLUR_F) is expressed by the following formula (6), where the direction estimated by the PSF estimation is DIR_BLUR_PSF and the direction estimated by the scene recognition unit is DIR_BLUR_SCENE. DIR_BLUR_F = DIR_BLUR_PSF*α + DIR_BLUR_SCENE(1.0-α) (6) A kernel that emphasizes blur is generated along the calculated direction of DIR_BLUR_F. The method of emphasizing blur and the method of adding blur are the same as those in the first embodiment.

[0060] With the configuration of this embodiment, even if the edges in the region are biased toward a certain angle, the direction in which the swinging object moves can be estimated by using machine learning and deep learning techniques such as position and orientation estimation. As a result, even for sports equipment with many straight line components such as a baseball bat, the direction of the shake can be calculated with high accuracy, and high-quality, dynamic images can be stably obtained.

[0061] Although the preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and various modifications and changes are possible within the scope of the gist of the present invention.

[0062] (Other embodiments) The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions. [Explanation of symbols]

[0063] 100 Digital Camera 101 Control section 102 ROM 103 RAM 104 Optical system 105 Imaging unit 106 A / D conversion section 107 Image Processing Unit 108 Recording media 109 Display section 110 Operation input section

Claims

1. A means for identifying a specific region within an image; a PSF estimation means for estimating a PSF in the specific region, The image processing device is characterized in that the PSF estimation means executes edge extraction on the image and performs PSF estimation on the image after the edge processing.

2. The image processing device further comprises a calculation means for applying a blur effect or a deblur effect to the specific region, 2. The image processing apparatus according to claim 1, wherein the calculation means determines the characteristics of the blur effect or the deblur effect based on the result of the PSF estimation.

3. The method further comprises: a scene estimation means for estimating a scene in the image, including a direction in which the specific region moves; 3. The image processing apparatus according to claim 2, wherein the calculation means determines the characteristics of the blur effect or the deblur effect using the result of the scene estimation.

4. The image processing device described in Claim 3, characterized in that the scene estimation means performs at least one of recognizing the type of sport shown in the image and estimating the position and orientation of the specific area.

5. The image processing device described in Claim 4, characterized in that the scene estimation means estimates the direction in which the specific area moves based on the recognition result of the type of sport shown in the image and the estimation result of the position and posture of the specific area.

6. 6. The image processing device according to claim 1, wherein the specific area is a face area of ​​a main subject or an object of a swing in sports involving a swing motion.

7. 7. The image processing device according to claim 6, wherein the object of the swing in the sports involving the swing motion is a sporting implement in the sports.

8. 4. The image processing device according to claim 2, wherein the characteristics of the blur effect include at least one of the amount, direction, and strength of the blur to be applied to the specific region.

9. An image processing device as described in claim 2 or 3, characterized in that the characteristics of the deblur effect include at least one of the amount, direction, and intensity of blur reduced from the specific area.

10. The image processing device according to claim 1, characterized in that the PSF estimation means performs mask processing on the image.

11. 11. The image processing apparatus according to claim 10, wherein, when the specific region is a face region of a main subject, the PSF estimation means performs edge extraction on the image and then performs masking on a skin-colored region.

12. identifying a specific region within the image; a PSF estimating step of estimating a PSF in the specific region, In the PSF estimation step, edge extraction is performed on the image; an image processing method comprising: estimating a PSF for the image after the edge processing;

13. A program for causing a computer to function as each of the means of the image processing apparatus according to any one of claims 1 to 3.

Images