Image processing device and method, imaging device, program, and storage medium

Abstract

To obtain a skin-beautifying effect desired by a user, in a person in an out-of-focus state.SOLUTION: The presence or absence of execution of predetermined correction processing and a first correction amount when executing the correction processing are set for a correction object area having a predetermined feature in an inputted image. It is determined whether a predetermined condition is satisfied. When the condition is satisfied, it is determined whether or not the first correction amount is equal to or greater than a predetermined threshold. When the first correction amount is equal to or greater than the predetermined threshold, a change to a second correction amount smaller than the first correction amount is made. When the execution of the correction processing is set, the correction processing is executed to the correction object area of the image on the basis of the second correction amount or the first correction amount when the change to the second correction amount is not made by change means.SELECTED DRAWING: Figure 4

Description

Technical Field

[0001] The present invention relates to an image processing apparatus and method, an imaging apparatus, a program, and a storage medium, and particularly to control of a correction amount in skin beautification processing.

Background Art

[0002] Conventionally, an imaging apparatus that performs skin beautification processing on a person's face during shooting and records it has been known. Skin beautification processing is a process of smoothing the skin, such as reducing roughness, removing dullness and stains. Since the resolution of the processed part decreases, it is necessary to apply the effect at an appropriate effect level.

[0003] For example, Patent Document 1 discloses a technique of correcting with a correction intensity associated with a preset correction level when a specific region is detected in a captured image.

[0004] Further, Patent Document 2 discloses a technique of invalidating or reducing contour correction for a specific hue when the focus mode is manual and during focus adjustment.

[0005] On the other hand, in recent years, a shooting mode suitable for product reviews and an autofocus operation have been demanded by users. In product reviews, since the product to be introduced is the main subject, instead of face-priority AF that tracks a person's face or fixed focus on a single point on the screen, AF that quickly focuses on a subject closer (hereinafter referred to as "review AF") is demanded.

Prior Art Documents

Patent Documents

[0006]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0007] However, in review AF, the focus may not be on the face of a person. On the other hand, in the prior arts disclosed in Patent Documents 1 and 2 described above, the intensity of the beauty effect by beauty processing is determined depending only on the face detection state and the set value of the focus mode. Therefore, when beauty processing is further applied to a face that is out of focus by review AF, there is a problem that the expected amount of beauty effect cannot be obtained.

[0008] FIG. 9 is an explanatory diagram of the autofocus operation and the distance to a person. In FIG. 9(a), the imaging device 100 performs an operation of focusing on the position on the plane 903 where the face is located if it is face-priority AF, or the position on the plane 902 where the product is located if it is review AF. In this case, since the distance between the person and the product is short, even if the focus is on the position on the plane 902, a suitable beauty effect can be obtained for the person by beauty processing.

[0009] On the other hand, in FIG. 9(b), the product is closer to the imaging device 100 than in FIG. 9(a), and the position to be focused during review AF is the position on the plane 901. In this case, the face at the position on the plane 903 is out of focus. However, when a face is detected in the captured image, if beauty processing is further applied to the out-of-focus face, the resolution feeling further decreases and the person becomes too blurry.

[0010] The present invention has been made in view of the above problems, and an object thereof is to obtain a beauty effect desired by a user for a person who is not in a focused state.

Means for Solving the Problems

[0011] In order to achieve the above object, an image processing apparatus according to the present invention includes an input means for inputting an image, and for a correction target region having predetermined features in the image, determining whether to perform a predetermined correction process, and setting means for setting a first correction amount when performing the correction process, a determination means for determining whether a predetermined condition is satisfied, and when the condition is satisfied, determining whether the first correction amount is equal to or greater than a predetermined threshold value, and when it is determined by the determination means that the value is equal to or greater than the threshold value, a changing means for changing to a second correction amount smaller than the first correction amount, and when the execution of the correction process is set, based on the second correction amount or the first correction amount if not changed to the second correction amount by the changing means, correction means for performing the correction process on the correction target region of the image.

Effect of the Invention

[0012] According to the present invention, it is possible to obtain a skin beautifying effect desired by the user for a person who is not in a group photo state.

Brief Description of the Drawings

[0013]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Embodiments for Carrying Out the Invention

[0014] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the invention according to the claims. Although a plurality of features are described in the embodiments, not all of these plurality of features are essential to the invention, and the plurality of features may be arbitrarily combined. Further, in the accompanying drawings, the same or similar configurations are denoted by the same reference numerals, and redundant descriptions are omitted.

[0015] <The First Embodiment> FIG. 1 is a block diagram showing the configuration of an imaging device 100 having an image processing function according to an embodiment of the present invention. In the present embodiment, a digital camera will be described as an example of the imaging device 100, but any electronic device having a camera function may be used. For example, other cameras such as digital video cameras may be used, or a mobile phone with a camera, a computer with a camera, a game machine, etc. may also be used.

[0016] The display unit 28 is provided on the back surface of the imaging device 100 and displays images and various information. The operation members include a shutter button 61 for giving a shooting instruction, a mode switch 60 for switching various modes, a power switch 72 for switching the power on / off, a rotatable controller wheel 73, other buttons, and operation members such as a touch panel. Instructions from the user are received via various operations on the operation members.

[0017] The connector 112 is a connector between the connection cable 111 and the imaging device 100. The recording medium 200 is a recording medium such as a memory card or a hard disk, and is stored in the recording medium slot 201. The recording medium 200 stored in the recording medium slot 201 can communicate with the imaging device 100. The lid 203 is the lid of the recording medium slot 201.

[0018] FIG. 2 is a block diagram showing the functional configuration of the imaging device 100 in the present embodiment. In FIG. 2, the photographing lens 103 is a lens group including a zoom lens and a focus lens. The shutter 101 is a shutter having an aperture function. Here, the photographing lens 103 is described as being integrally configured with the imaging device 100, but the present invention is not limited to this, and it may have a configuration detachable from the imaging device 100.

[0019] The imaging unit 22 is composed of a CCD, a CMOS element, etc., and has an A / D conversion processing function. It converts the optical image formed on the imaging surface into an electrical signal via the photographing lens 103 and the shutter 101, and performs A / D conversion on the obtained electrical signal to output a digital image signal (image data).

[0020] The AF evaluation value detection unit 23 calculates an AF evaluation value from contrast information, phase difference information, etc. obtained from the digital image signal output from the imaging unit 22, and outputs the obtained AF evaluation value from the imaging unit 22 to the system control unit 50. When outputting phase difference information, at least a part of the pixels constituting the imaging unit 22 is configured to be able to acquire phase difference information such as the light receiving region being biased or divided into a plurality of light receiving regions.

[0021] The barrier 102 is a protective member that covers the imaging system of the imaging device 100 including the photographing lens 103, the shutter 101, and the imaging unit 22 to prevent dirt and damage to the imaging system. The flash light 90 can emit light during photographing to supplement the illuminance during photographing in a low illuminance scene or a backlight scene.

[0022] The image processing unit 24 performs various image processes such as predetermined pixel interpolation processing, resizing processes such as reduction / enlargement, color conversion processing, and beauty skin processing on the image data output from the imaging unit 22 or the image data from the memory control unit 15. Further, the image processing unit 24 performs predetermined image processing and arithmetic processing using the image data obtained by imaging, and based on the obtained arithmetic result, the system control unit 50 performs exposure control and focus control. Thereby, AE (Automatic Exposure) processing and EF (Flash Automatic Light Control Emission) processing in the TTL (Through-the-Lens) method are performed. Note that AF (Auto Focus) processing may be performed in the image processing unit 24 instead of the AF evaluation value detection unit 23.

[0023] Furthermore, in the image processing unit 24, predetermined arithmetic processing is performed using the image data obtained by imaging by the imaging unit 22, and based on the obtained arithmetic result, AWB (Auto White Balance) processing in the TTL method is also performed.

[0024] The output data output from the imaging unit 22 is written into the memory 32 via the image processing unit 24 and the memory control unit 15, or directly via the memory control unit 15. The memory 32 also serves as an image display memory (video memory), has a storage capacity sufficient to store a predetermined number of still images, a moving image for a predetermined time, and audio, and stores the image data output from the imaging unit 22 and the image data for display on the display unit 28.

[0025] The image data for display written into the memory 32 is converted into an analog signal by the D / A converter 13 and displayed on a display device such as an LCD of the display unit 28. By converting the digital image signal A / D-converted by the imaging unit 22 and stored in the memory 32 into an analog signal by the D / A converter 13 and sequentially transferring it to the display unit 28 for display, it can function as an electronic viewfinder and perform through-image display.

[0026] The non-volatile memory 56 is an electrically erasable and recordable memory, such as a flash memory, and stores constants, programs, etc. for the operation of the system control unit 50. The programs stored in the non-volatile memory 56 include programs for executing various flowcharts described later in this embodiment.

[0027] The system control unit 50 is a control unit composed of at least one processor or circuit, and controls the entire imaging device 100. By executing the programs recorded in the non-volatile memory 56, the various processes described later are realized. For the system memory 52, for example, RAM is used, and constants, variables, programs read from the non-volatile memory 56, etc. for the operation of the system control unit 50 are expanded. Also, the system control unit 50 performs display control by controlling the memory 32, D / A converter 13, display unit 28, etc. The system timer 53 is a timing unit that measures the time used for various controls and the time of the built-in clock.

[0028] The mode switch 60, shutter button 61, and operation unit 70 are operation members for inputting various operation instructions to the system control unit 50. Also, the operation unit 70 includes the controller wheel 73 shown in FIG. 1, other buttons, a touch panel, etc. The power switch 72 can switch and set the power-on and power-off modes of the imaging device 100.

[0029] The mode switch 60 switches the operation mode of the system control unit 50 to any one of the still image recording mode, video recording mode, playback mode, etc. The still image recording mode includes the auto shooting mode, the auto scene discrimination mode, the manual mode, various scene modes for shooting settings according to shooting scenes, the program AE mode, the custom mode, etc. The mode switching switch 60 can directly switch to any of these modes included in the still image recording mode. Alternatively, after once switching to the still image shooting mode with the mode switching switch 60, it may be possible to switch to any of these modes included in the still image shooting mode using other operation members.

[0030] Similarly, in the present embodiment, the moving image shooting mode also includes a plurality of modes. In addition to the shooting modes similar to the still image recording mode, it includes the beauty effect moving image mode and the product review moving image mode.

[0031] The beauty effect moving image mode is a mode that performs beauty effect processing on the face detected in the image as image processing. As the AF mode (mode related to focus adjustment), by default, among the subjects existing within the shooting angle of view, the face priority AF mode that preferentially focuses on the face is set. However, in the beauty effect moving image mode, it is possible to change to other AF modes as the AF mode.

[0032] The product review moving image mode is a mode in which the review AF mode aimed at preferentially focusing on the product among the subjects existing within the shooting angle of view is set as the AF mode. Instead of tracking a person or a specific subject, it focuses on an object existing closer. On the other hand, as image processing, it is possible to change whether to perform beauty effect processing or not. In addition, in the moving image modes other than the above, it may be configured so that it is possible to set whether to perform beauty effect processing or not.

[0033] The first shutter switch 62 turns on during the operation of the shutter button 61, that is, a so-called half-press (shooting preparation instruction), and generates the first shutter switch signal SW1. The first shutter switch signal SW1 starts shooting preparation operations such as AF processing, AE processing, AWB processing, and EF processing.

[0034] The second shutter switch 63 turns ON when the operation of the shutter button 61 is completed, i.e., when it is fully pressed (shooting instruction), and generates a second shutter switch signal SW2. The system control unit 50 starts a series of shooting process operations from reading the signal from the imaging unit 22 to writing the obtained image data as an image file to the recording medium 200 based on the second shutter switch signal SW2.

[0035] Each operation member of the operation unit 70 has a function appropriately assigned according to the display content by selecting and operating various function icons displayed on the display unit 28, and acts as various function buttons. Examples of function buttons include an end button, a return button, an image advance button, a jump button, a narrowing-down button, an attribute change button, etc. For example, when the menu button is pressed, various configurable menu screens are displayed on the display unit 28. The user can intuitively perform various settings using the menu screen displayed on the display unit 28 and the four-direction buttons (up, down, left, and right) and the SET button.

[0036] The controller wheel 73 is a rotatable operation member included in the operation unit 70 and is used when indicating a selection item together with the direction buttons. When the controller wheel 73 is rotated, an electrical pulse signal is generated according to the operation amount, and based on this pulse signal, the system control unit 50 controls each part of the imaging device 100. Based on this pulse signal, it is possible to determine the angle by which the controller wheel 73 has been rotated and how many rotations have been made. Note that the controller wheel 73 may be any type as long as it is an operation member capable of detecting a rotational operation. For example, it may be a dial operation member in which the controller wheel 73 itself rotates in response to the user's rotational operation to generate a pulse signal. Also, it may be an operation member composed of a touch sensor, where the controller wheel 73 itself does not rotate, and it detects the rotational movement of the user's finger on the controller wheel 73 (so-called touch wheel).

[0037] In this embodiment, it is possible to set whether or not to perform skin beautification processing performed by the image processing unit 24 by the operation of the operation unit 70 described above. In the skin beautification processing, for the face portion (correction target area) of a person in the captured image, processing such as removing or reducing skin wrinkles and dullness, or making the area around the eyes blend with the surrounding skin color is performed. For example, it includes a smoothing process using a smoothing filter. Further, a process of reducing eye congestion or a process of correcting teeth to be white may be performed. The intensity of the skin beautification processing is divided into a plurality of levels, and the user can set it using the operation unit 70 as a skin beautification effect setting value. The setting information regarding whether or not to perform the set skin beautification processing and the skin beautification effect setting value are held in the memory 32. Note that since various processes related to the skin beautification processing are all known techniques in the technical field of image processing, detailed descriptions thereof are omitted here.

[0038] The power control unit 80 is composed of a battery detection circuit, a DC-DC converter, a switch circuit for switching the energized block, etc., and detects the presence or absence of battery installation, the type of battery, and the remaining battery level. Further, the power control unit 80 controls the DC-DC converter based on the detection result and the instruction of the system control unit 50, and supplies a necessary voltage to each unit including the recording medium 200 for a necessary period. The power supply unit 40 includes a primary battery such as an alkaline battery or a lithium battery, a secondary battery such as a NiCd battery, a NiMH battery, or a Li battery, and an AC adapter.

[0039] The recording medium I / F 18 is an interface with a recording medium 200 such as a memory card or a hard disk. The recording medium 200 is a recording medium such as a memory card for recording the captured image, and is composed of a semiconductor memory, a magnetic disk, or the like.

[0040] Next, with reference to FIGS. 3 and 4, the correction process for the skin beautification effect amount in the first embodiment will be described. FIG. 3 is a flowchart showing the beauty effect amount setting process of the imaging device 100 in the first embodiment. Each process in this flowchart is realized by the system control unit 50 expanding and executing a program stored in the nonvolatile memory 56 in the system memory 52.

[0041] When starting the beauty effect amount setting process, the system control unit 50 acquires the set shooting mode in S301. Then, in S302, it determines the acquired shooting mode. If it is the beauty video mode, it proceeds to S303. If it is the product review video mode, it proceeds to S304. Otherwise, it proceeds to S305. In S305, it determines whether to perform the beauty process. If performing the beauty process, it proceeds to S306; if not, the process ends.

[0042] In S303, as an AF mode, it determines whether the review AF mode is set (condition). If the review AF mode is not selected, for example, assuming the default face priority AF mode is selected, it proceeds to S306. If the review AF mode is set, it proceeds to S307.

[0043] In S306, the system control unit 50 reads out the set beauty effect setting value, applies the beauty effect amount (correction amount) associated with the read beauty effect setting value, and ends the process.

[0044] On the other hand, in S304, it determines whether beauty processing is set as the image processing. If not performing the beauty process, the process ends. If performing the beauty process, it proceeds to S307. In S307, beauty effect amount correction processing is performed. Details of the beauty effect amount correction processing performed in S307 will be described later with reference to FIG. 4.

[0045] FIG. 4 is a flowchart showing the beauty effect amount correction processing of S307 in FIG. 3. When the skin beautifying effect amount correction process is started, in S401, the system control unit 50 acquires the set skin beautifying effect setting value. Next, in S402, the system control unit 50 compares the skin beautifying effect setting value with a predetermined reference value. If the skin beautifying effect setting value is set to a strength stronger than the reference value, in S403, the system control unit 50 changes the skin beautifying effect setting value to the reference value. Note that the skin beautifying effect setting value does not necessarily have to be changed to the reference value, and any skin beautifying effect amount that can reduce the skin beautifying effect compared to the currently set skin beautifying effect setting value may be used.

[0046] On the other hand, in S402, if it is determined that the skin beautifying effect setting value is less than or equal to the reference value, in S404, the set skin beautifying effect setting value is retained. After that, in S405, the system control unit 50 calculates the skin beautifying effect amount (correction amount) actually applied to the image based on the skin beautifying effect setting value determined in S403 or S404, and ends the process.

[0047] Through the above process, even when the user uses the product review mode and the skin beautifying process in combination, by correcting the skin beautifying effect amount for the product review mode as needed based on the AF mode and the skin beautifying effect setting value, an image with a suitable skin beautifying effect can be obtained.

[0048] In this embodiment, two cases are described: when the shooting mode is the review video mode and the skin beautifying process is set, and when the shooting mode is the skin beautifying video mode and the AF mode is set to the review video mode. However, the hierarchy of modes and settings is not limited to this, and it is possible to correct the skin beautifying effect amount when using in combination an AF setting where the face is not necessarily prioritized and the skin beautifying process.

[0049] In this embodiment, the skin beautifying effect amount is corrected by correcting the skin beautifying effect setting value. However, instead of correcting the skin beautifying effect setting value, the skin beautifying effect amount may be corrected by directly attenuating the processing parameters of each process in the skin beautifying process.

[0050] Also, when performing skin beautification correction in S306, the skin beautification effect amount may be varied according to the shooting mode information determined in S301. For example, when it is determined in S301 that it is the skin beautification video mode, the skin beautification effect amount may be suppressed more than when it is determined to be the review video mode. This is because in the case of the skin beautification video mode, it is presumed that the priority of the photographer is the image quality of the face area, so there is a high need to suppress excessive correction of the face area due to skin beautification correction.

[0051] On the other hand, when it is determined in S301 that it is the review video mode, a form may be adopted in which the skin beautification effect amount is suppressed more than when it is determined to be the skin beautification video mode. This is because in the case of the review video mode, it is presumed that the introduction of the product is prioritized over the beauty of the face area, so there is a high need to prevent the entire image from becoming a failed image due to excessive correction of the face area.

[0052] <Second Embodiment> Next, a second embodiment of the present invention will be described. In the second embodiment, a case where the skin beautification effect amount is corrected based on the information of the captured image will be described. Note that also in the second embodiment, the description will be omitted on the assumption that the imaging device 100 described in the first embodiment is used.

[0053] FIG. 5 is a flowchart showing the skin beautification effect amount setting process by the imaging device 100 in the second embodiment. Each process in this flowchart is realized by the system control unit 50 expanding and executing a program stored in the non-volatile memory 56 in the system memory 52.

[0054] When starting the skin beautification effect amount correction process, in S501, the system control unit 50 acquires the setting information of the skin beautification process, and in S502, determines whether the skin beautification process is set based on the acquired setting information of the skin beautification process. If the skin beautification process is not set, the process ends as it is. On the other hand, if it is determined in S502 that the skin beautification process is set, the system control unit 50 advances the process to S503.

[0055] In S503, the system control unit 50 performs face detection processing on the captured image and stores the coordinates of the detected face position in the memory 32. Next, in S504, the system control unit 50 acquires focus position information indicating at which position of the image the focus was set during shooting. Note that the order of the processes in S503 and S504 may be reversed, or they may be performed in parallel. Then, in S505, the distance on the image between the coordinates of the face stored in the memory 32 and the coordinates of the focus position is calculated.

[0056] In S506, the system control unit 50 determines whether the distance between the calculated face position and the focus position is equal to or less than a predetermined distance (condition). If it is equal to or less than the predetermined distance, in S507, the amount of beauty effect associated with the beauty effect setting value is applied, and the process ends. On the other hand, if it is determined in S506 that the distance exceeds the predetermined distance, in S508, beauty effect amount correction processing is performed. Note that as the beauty effect amount correction processing, for example, the same processing as the processing described in FIG. 4 is performed.

[0057] FIG. 6 is a diagram for explaining the relationship between the distance between the face position and the focus position and the amount of beauty effect in the second embodiment. 601 and 604 are the center positions of the face, 602 and 605 are the focus positions at the time of shooting, and 603 and 606 indicate the distance on the image between the center position of the face and the focus position.

[0058] In FIG. 6(a), the distance on the image between the center position 601 of the face and the focus position 602 is short. On the other hand, in FIG. 6(b), the distance on the image between the center position 604 of the face and the focus position 605 is long, and compared with the case where the distance is short as in FIG. 6(a), the face is not within the depth of field of the subject and is likely to be blurry. However, since face detection is successful, beauty processing is applied. In the second embodiment, in such a scene, in order to fix the amount of beauty effect, it is possible to prevent the beauty effect from being applied excessively.

[0059] <The Third Embodiment> Next, a second embodiment of the present invention will be described. In the third embodiment, a case where the amount of beautifying effect is corrected based on the depth of field will be described. Note that also in the third embodiment, the description will be omitted on the assumption that the imaging device 100 described in the first embodiment is used.

[0060] FIG. 7 is a flowchart showing the beautifying effect amount setting process by the imaging device 100 in the third embodiment. Each process in this flowchart is realized by the system control unit 50 expanding and executing a program stored in the non-volatile memory 56 in the system memory 52. Also, in the following description, it will be described on the assumption that the photographing lens 103 has a configuration detachable from the imaging device 100.

[0061] When starting the beautifying effect amount correction process, in S701, the system control unit 50 acquires the setting information of the beautifying process, and in S702, determines whether the beautifying process is set based on the acquired setting information of the beautifying process. If the beautifying process is not set, the process ends as it is. On the other hand, if it is determined in S702 that the beautifying process is set, the system control unit 50 advances the process to S703.

[0062] In S703, the system control unit 50 acquires the lens information of the photographing lens 103 attached to the imaging device 100. Note that when the photographing lens 103 is an integrated lens with the imaging device 100, the lens information stored in advance in the memory 32 or the like is used. Next, in S704, the system control unit 50 acquires photographing information including the F value, the distance to the focus, and the depth of focus.

[0063] In S705, the system control unit 50 calculates the depth of field at the time of shooting from the lens information and the shooting information, and in S706, determines whether or not the depth of field is equal to or greater than a predetermined threshold value (condition). When it is determined that the depth of field is equal to or greater than the threshold value, the system control unit 50 applies the amount of skin beautifying effect associated with the skin beautifying effect setting value in S707 and ends the process. On the other hand, when it is determined in S706 that the depth of field is less than the threshold value, the system control unit 50 performs a skin beautifying effect amount correction process in S708. Note that as the skin beautifying effect amount correction process, for example, the same process as the process described in FIG. 4 is performed.

[0064] When the depth of field at the time of shooting is deep, there is a high possibility that the person is within the subject depth, but when the depth of field is shallow, there is a low possibility that the person is outside the subject depth, so it is likely to be blurry. However, since face detection is successful, the skin beautifying effect is applied. In the third embodiment, in such a scene, in order to reduce the amount of skin beautifying effect, it is possible to prevent the skin beautifying effect from being applied excessively.

[0065] <Fourth Embodiment> Next, a fourth embodiment of the present invention will be described. In the fourth embodiment, a case where the amount of skin beautifying effect is corrected based on the depth of field and the distance to the face will be described. Note that also in the fourth embodiment, description will be omitted on the assumption that the imaging device 100 described in the first embodiment is used.

[0066] FIG. 8 is a flowchart showing the skin beautifying effect amount setting process by the imaging device 100 in the fourth embodiment. Each process in this flowchart is realized by the system control unit 50 expanding and executing a program stored in the nonvolatile memory 56 in the system memory 52. Further, in the following description, it is assumed that the photographing lens 103 has a configuration detachable from the imaging device 100.

[0067] Since the processes from S701 to S705 in FIG. 8 are the same as the processes shown in FIG. 7, description thereof will be omitted.

[0068] In S806, the system control unit 50 performs face detection processing on the captured image, and obtains the distance to the detected face (distance measurement) based on the phase difference information detected by the AF evaluation value detection unit 23. Next, in S807, the system control unit 50 determines whether the detected face is within the depth of field (condition) based on the distance to the focus acquired in S703, the depth of field acquired in S705, and the distance to the face acquired in S806.

[0069] In S807, when the detected face is within the depth of field, in S808, the amount of beauty effect associated with the beauty effect setting value is applied and the process ends. On the other hand, when it is determined in S807 that the face is not within the depth of field, the system control unit 50 performs beauty effect amount correction processing in S809. Note that as the beauty effect amount correction processing, for example, the same processing as the processing described with reference to FIG. 4 is performed.

[0070] In the above-described processing, the distance to the focus and the distance to the face have been described as being detected respectively. However, by generating a so-called defocus map indicating the distance to each subject, it may be determined whether each face is within the depth of field.

[0071] According to the fourth embodiment, in a scene where a person exists outside the subject depth and the image is blurry, but face detection is successful, the amount of beauty effect can be reduced to prevent the beauty effect from being overly applied.

[0072] Note that in the above-described first to fourth embodiments, the beauty processing has been described as being performed in the imaging device. However, the present invention is not limited to this. For example, a computer, a tablet terminal, or the like may be used as an image processing device to acquire image data captured by the imaging device and perform beauty processing on the acquired image data.

[0073] <Other Embodiments> Also, the present invention can also be realized by supplying a program that implements one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and causing one or more processors in a computer of the system or device to read and execute the program. It can also be realized by a circuit (for example, ASIC) that implements one or more functions.

[0074] <Summary> The disclosure of the present embodiment includes the following configurations.

[0075] (Item 1) Input means for inputting an image, Setting means for setting whether to perform a predetermined correction process on a correction target area having predetermined characteristics in the image, and a first correction amount when performing the correction process, Determination means for determining whether a predetermined condition is satisfied, and when the condition is satisfied, determining whether the first correction amount is equal to or greater than a predetermined threshold value, Changing means for changing to a second correction amount smaller than the first correction amount when it is determined by the determination means that the value is equal to or greater than the threshold value, Correction means for performing the correction process on the correction target area of the image based on the second correction amount, or the first correction amount if it has not been changed to the second correction amount by the changing means when the execution of the correction process is set, An image processing apparatus characterized by comprising: (Item 2) Further comprising selection means for selecting any one of a plurality of modes related to focus adjustment, including a review AF mode for selecting a subject close to the imaging device among the subjects existing within the shooting angle of view and focusing on it, The image processing apparatus according to Item 1, wherein the determination means determines that the condition is satisfied when the review AF mode is selected. (Item 3) The determination means determines that the condition is satisfied when the distance between the position of the correction target area and the focus position in the image is longer than a predetermined distance. The image processing apparatus according to item 1 or 2, characterized in that. (Item 4) An acquisition means for acquiring lens information of a photographing lens used for photographing the image and photographing information used for photographing the image; A calculation means for calculating a depth of field based on the lens information and the photographing information, further comprising: The determination means determines that the condition is satisfied when the depth of field is shallower than a predetermined depth. The image processing apparatus according to any one of items 1 to 3, characterized in that. (Item 5) An acquisition means for acquiring lens information of a photographing lens used for photographing the image and photographing information used for photographing the image; A calculation means for calculating a depth of field based on the lens information and the photographing information; A distance measuring means for obtaining a distance to a subject in the correction target area, further comprising: The determination means determines that the condition is satisfied when the distance to the subject is not included in the depth of field centered on the distance to the focus included in the photographing information. The image processing apparatus according to any one of items 1 to 4, characterized in that. (Item 6) The correction target area further includes a detection means for detecting a face area from the image; The correction means performs correction processing on the face area. The image processing apparatus according to any one of items 1 to 5, characterized in that. (Item 7) The correction processing includes a smoothing process. The image processing apparatus according to any one of items 1 to 6, characterized in that. (Item 8) An imaging means for imaging and outputting an image; The image processing apparatus according to any one of items 1 to 7 An imaging apparatus, characterized in that it has. (Item 9) An input step of inputting an image, A setting step of setting, for a correction target region having predetermined features in the image, whether to perform predetermined correction processing and, when performing the correction processing, a first correction amount, A determination step of determining whether predetermined conditions are satisfied, and when the conditions are satisfied, determining whether the first correction amount is equal to or greater than a predetermined threshold value, A changing step of changing to a second correction amount smaller than the first correction amount when it is determined in the determination step that the amount is equal to or greater than the threshold value, A correction step of performing the correction processing on the correction target region of the image based on the second correction amount or, if not changed to the second correction amount in the changing step, the first correction amount when the execution of the correction processing is set, An image processing method characterized by comprising the above. (Item 10) A program for causing a computer to function as each means of the image processing apparatus according to any one of Items 1 to 7. (Item 11) A computer-readable storage medium storing the program according to Item 10.

[0076] The invention is not limited to the above embodiments, and various changes and modifications are possible without departing from the spirit and scope of the invention. Therefore, claims are attached to disclose the scope of the invention.

Explanation of Signs

[0077] 22: Imaging unit, 24: Image processing unit, 28: Display unit, 32: Memory, 50: System control unit, 52: System memory, 56: Non-volatile memory, 60: Mode changeover switch, 70: Operation unit, 73: Controller wheel, 100: Imaging device, 103: Photographing lens

Claims

1. input means for inputting an image; setting means for setting whether to perform predetermined correction processing on a correction target area having predetermined characteristics in the image, and a first correction amount when performing the correction processing; determination means for determining whether a predetermined condition is satisfied, and when the condition is satisfied, determining whether the first correction amount is equal to or greater than a predetermined threshold; changing means for changing to a second correction amount smaller than the first correction amount when determined to be equal to or greater than the threshold by the determination means; correction means for performing the correction processing on the correction target area of the image based on the second correction amount, or the first correction amount if not changed to the second correction amount by the changing means, when the execution of the correction processing is set; An image processing apparatus, characterized by comprising:

2. further comprising selection means for selecting any one of a plurality of modes related to focus adjustment, including a review AF mode for selecting a subject close to the imaging device among the subjects existing within the shooting angle of view and focusing thereon; The image processing apparatus according to claim 1, wherein the determination means determines that the condition is satisfied when the review AF mode is selected.

3. The image processing apparatus according to claim 1, wherein the determination means determines that the condition is satisfied when the distance between the position of the correction target area and the focus position in the image is longer than a predetermined distance.

4. acquisition means for acquiring lens information of a shooting lens used for shooting the image and shooting information used for shooting the image; further comprising calculation means for calculating a depth of field based on the lens information and the shooting information; The determination means determines that the condition is satisfied when the depth of field is shallower than a predetermined depth, according to the image processing apparatus described in claim 1.

5. An acquisition means for acquiring lens information of a photographing lens used for photographing the image and photographing information used for photographing the image; A calculation means for calculating the depth of field based on the lens information and the photographing information; A distance measuring means for obtaining the distance to a subject in the correction target area, further comprising: The determination means determines that the condition is satisfied when the distance to the subject is not included in the depth of field centered on the distance to the focus included in the photographing information, according to the image processing apparatus described in claim 1.

6. The correction target area further includes a detection means for detecting a face area from the image; The correction means performs correction processing on the face area, according to the image processing apparatus described in claim 1.

7. The correction processing includes a smoothing process, according to the image processing apparatus described in claim 1.

8. An imaging means for imaging and outputting an image; The image processing apparatus according to any one of claims 1 to 7 And an imaging apparatus characterized by comprising:

9. An input step of inputting an image; A setting step of setting, for a correction target area having a predetermined feature in the image, whether to perform a predetermined correction process and a first correction amount when performing the correction process; A determination step of determining whether a predetermined condition is satisfied, and when the condition is satisfied, determining whether the first correction amount is equal to or greater than a predetermined threshold; When it is determined in the determination step that the value is equal to or greater than the threshold value, a change step of changing to a second correction amount smaller than the first correction amount; When the execution of the correction process is set, based on the second correction amount, or the first correction amount if the second correction amount has not been changed to the second correction amount in the change step, a correction step of performing the correction process on the correction target area of the image; An image processing method, characterized by comprising the above.

10. A program for causing a computer to function as each means of the image processing apparatus according to any one of Claims 1 to 7.

11. A computer-readable storage medium storing the program according to Claim 10.

Images