Image processing apparatus and method, capturing apparatus, program and storage medium
Patent Information
- Application Number
- JP2022099801
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2022-06-21
- Publication Date
- 2025-05-12
- Estimated Expiration
- 2042-06-21
AI Technical Summary
Conventional skin beautification correction methods result in unnatural transitions at the boundaries of face areas due to changes in correction methods, leading to a loss of three-dimensional effect in facial images.
An image processing apparatus that extracts AC signals in multiple frequency bands, adjusts their amplitudes using specific adjustment values, and corrects skin regions to maintain a natural three-dimensional effect by attenuating AC signals differently based on frequency, face size, age, and facial parts.
Achieves appropriate skin beautification while preserving the natural three-dimensional appearance of faces in images, reducing the visibility of pores and wrinkles.
Smart Images
Figure 00000000_0000_ABST
Abstract
Description
[Technical field]
[0001] The present invention relates to an image processing device and method, an imaging device, a program, and a storage medium. [Background technology]
[0002] Conventionally, there are imaging devices such as digital cameras that have a shooting function called skin beautification, which provides an image of beautiful skin by correcting the skin area of the person in the captured image when photographing a person. Conventional skin beautification has been achieved by applying a simple average value filter process or the like to the image, which has the problem that the skin becomes flat and the three-dimensional effect of the face is lost.
[0003] To address this issue, Patent Document 1 discloses the following technology for acquiring a three-dimensional image. First, a reference value that serves as a luminance standard is acquired from within the person's face area, taking into account the facial features of the subject. Then, using a luminance threshold value set based on the acquired reference value, the person's face area is divided into multiple exclusive parts, and the luminance of each of the multiple parts is corrected using a different correction method. [Prior art documents] [Patent documents]
[0004] [Patent Document 1] JP 2018-117288 A Summary of the Invention [Problem to be solved by the invention]
[0005] However, in the conventional technology disclosed in Patent Document 1, the correction method is changed for each part of the face, and therefore there is a problem that unnaturalness due to the change in correction method remains at the boundary parts of the areas, etc.
[0006] The present invention has been made in consideration of the above problems, and aims to achieve appropriate skin correction in a natural state while maintaining the three-dimensional effect of a person's face in an image obtained by an imaging device. [Means for solving the problem]
[0007] In order to achieve the above-mentioned object, the image processing device of the present invention comprises an input means for inputting image data, an extraction means for extracting a plurality of AC signals of a plurality of different frequency bands from the image data, a generation means for generating a first adjustment value for adjusting a positive component of the amplitude of the AC signal and a second adjustment value for adjusting a negative component for each of the plurality of frequency bands, an adjustment means for adjusting each of the plurality of AC signals using the first adjustment value and the second adjustment value, a detection means for detecting a skin area from the image data, and a correction means for correcting the image data of the skin area using the plurality of AC signals adjusted by the adjustment means, wherein the generation means generates the first adjustment value so that the positive component of the amplitude of the AC signal in a first frequency band among the plurality of frequency bands is less attenuated than the positive component of the amplitude of the AC signal in a second frequency band lower than the first frequency band. Effect of the Invention
[0008] According to the present invention, it is possible to realize appropriate skin correction in a natural state for an image obtained by an imaging device while maintaining the three-dimensional effect of a person's face. [Brief description of the drawings]
[0009] [Figure 1] 1 is a block diagram showing a schematic functional configuration of an image capturing apparatus according to an embodiment of the present invention. [Diagram 2] FIG. 2 is a block diagram showing the functional configuration of an image processing unit in the embodiment. [Diagram 3] FIG. 4 is a block diagram showing the functional configuration of a skin correction processing unit in the embodiment. [Figure 4] 5 is a flowchart showing the flow of skin cosmetic correction processing in the embodiment. [Diagram 5] 5 is a flowchart showing the flow of skin correction processing in the embodiment. [Figure 6] 6A to 6C are conceptual diagrams illustrating an extraction process of AC components in the skin correction process according to the embodiment. [Figure 7] 5A to 5C are diagrams illustrating the concept of a method for adjusting AC components in the skin correction process according to the embodiment. [Figure 8] 5A to 5C are diagrams for explaining adjustment parameters of AC components in the embodiment. [Figure 9] 6A and 6B are diagrams showing examples of adjustment gains of adjustment parameters for AC components in the embodiment. [Figure 10] 11A and 11B are diagrams showing other examples of adjustment gains of adjustment parameters for AC components in the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0010] Hereinafter, the embodiments will be described in detail with reference to the attached drawings. Note that the following embodiments do not limit the invention according to the claims. Although the embodiments describe a number of features, not all of these features are essential to the invention, and the features may be combined in any manner. Furthermore, in the attached drawings, the same reference numbers are used for the same or similar configurations, and duplicated descriptions are omitted.
[0011] First, an image capturing device to which the image processing of the present invention can be applied will be described. Note that, in this embodiment, an image capturing device will be described as an example, but the present invention is not limited to the image capturing device, and may be an image processing device that can input image data obtained by the image capturing device and perform image processing.
[0012] FIG. 1 is a block diagram showing a schematic functional configuration of an image capturing apparatus according to this embodiment. The imaging unit 101 includes a lens, an image sensor, an A / D conversion processing unit, and a development processing unit, and captures an image of a subject and generates image data based on a control signal output from the system control unit 103 in response to an instruction from the operation unit 107. The image processing unit 102 performs image processing including skin beautification processing on the image data input from the imaging unit 101, the recording unit 105, and the network processing unit 106, based on a control signal output from the system control unit 103. The content of the skin beautification processing performed by the image processing unit 102 will be described in detail later.
[0013] The system control unit 103 is a control function unit that controls and manages the operation of the entire apparatus of this embodiment. Based on instructions transmitted from a network processing unit 106 and an operation unit 107, the system control unit 103 performs drive control of the imaging unit 101 and the like.
[0014] The display unit 104 receives an output signal from the image processing unit 102 and performs processing to display an image on a display device configured as a liquid crystal display, an organic EL (Electro Luminescence) display, etc. The recording unit 105 has a function of recording data such as image data, and may include an information recording medium using, for example, a memory card equipped with a semiconductor memory or a package containing a rotating recording medium such as a magneto-optical disk, or the like, and may also be configured so that the information recording medium is removable.
[0015] The network processing unit 106 acquires image data from an external input device via a network. Furthermore, the network processing unit 106 receives an output signal from the image processing unit 102 and performs processing to transmit the image data to an external display device or an external image processing device such as a PC via the network. The bus 108 is used to exchange image data, control signals, and the like between the imaging unit 101, the image processing unit 102, the system control unit 103, the display unit 104, the recording unit 105, and the network processing unit 106.
[0016] Next, the functional configuration for performing the skin correction process in the image processing unit 102 in this embodiment will be described with reference to FIG.
[0017] Image processing unit 102 in this embodiment has, as functional components for performing skin beautification processing, an image input unit 201, a skin correction processing unit 202, a skin area detection unit 203, a synthesis processing unit 204, and an image output unit 205. Image data input to image input unit 201 is image data to be processed for skin beautification, and image data output from image output unit 205 is an image that has been skin beautified.
[0018] Next, the skin cosmetic correction process performed by the image processing unit 102 in this embodiment will be described with reference to the flowchart in FIG.
[0019] First, in S401, the image input unit 201 inputs image data to be subjected to skin correction. Then, in S402, the skin correction processing unit 202 performs skin correction processing on the image data input in S401. The skin correction processing performed in S402 will be described in detail later.
[0020] In S403, the skin region detection unit 203 performs a process of detecting a skin region for the image data input in S401. The method of detecting the skin region is not particularly limited, and a general method of extracting a region corresponding to skin color based on the hue, saturation, and brightness of the image may be used, or a detection process based on learning data such as deep learning may be performed.
[0021] In S404, the synthesis processing unit 204 weights and synthesizes the image data input in S401 and the image data that has been subjected to the skin correction processing in S402 according to the detection result of the skin area detected in S403. Here, for pixel coordinates (x, y), the signal value of the output image data after processing by the synthesis processing unit 204 is out_pix(x, y), the signal value of the image data input in S401 is in_pix(x, y), and the signal value of the image data that has been subjected to the skin correction processing in S402 is cor_pix(x, y), which is expressed by equation (1). Note that α(x, y) indicates the detection result of the skin area detected in S403, and indicates that the possibility of it being a skin area increases as the value approaches 1.0 from 0.0.
[0022] out_pix(x,y)=(1.0-α(x,y))×in_pix(x,y)+α(x,y)×cor_pix(x,y) …(1) In this way, the higher the possibility (reliability) that the area is a skin area, the higher the ratio of signal values that have been subjected to skin correction processing, and the lower the possibility (reliability) that the area is a skin area, the higher the ratio of signal values of the input image data that have not been subjected to skin correction processing.
[0023] In S405, the image data combined in S404 is output as output image data.
[0024] Next, the configuration of the skin correction processing unit 202 in this embodiment will be described with reference to FIG.
[0025] The skin correction processing unit 202 is composed of a first LPF (low pass filter) processing unit 301, a second LPF processing unit 302, a third LPF processing unit 303, a first subtraction unit 304, a second subtraction unit 305, a third subtraction unit 306, a face detection unit 307, a parameter generation unit 308, a first amplitude adjustment unit 309, a second amplitude adjustment unit 310, a third amplitude adjustment unit 311, a third adder unit 312, a second adder unit 313, and a first adder unit 314.
[0026] The first LPF processing unit 301, the second LPF processing unit 302, and the third LPF processing unit 303 each perform LPF processing using a general LPF such as a Gaussian filter, an average filter, etc. The cutoff frequencies of the first LPF processing unit 301, the second LPF processing unit 302, and the third LPF processing unit 303 are arranged in ascending order as the first LPF processing unit 301, the second LPF processing unit 302, and the third LPF processing unit 303.
[0027] Next, the skin correction process in S402 of FIG. 3, which is performed by the skin correction processing unit 202 having the above configuration, will be described with reference to the flowchart of FIG.
[0028] In S501, the first LPF processing unit 301 performs the first LPF processing on the input image data. In S502, the second LPF processing unit 302 performs a second LPF processing on the image data that has been subjected to the first LPF processing in S501. In S503, the third LPF processing unit 303 performs a third LPF processing on the image data that has been subjected to the second LPF processing in S502.
[0029] In S504, the first subtraction unit 304 extracts a first AC component by subtracting the image data after the first LPF processing in S501 from the input image data. For coordinates (x, y), if the input signal is in_pix(x, y) and the image signal after the first LPF processing is lpf1_pix(x, y), the first AC component ac1_pix(x, y) is expressed by equation (2). ac1_pix(x,y)=in_pix(x,y)-lpf1_pix(x,y) …(2)
[0030] Next, in S505, the second subtraction unit 305 subtracts the image data after the second LPF processing in S503 from the image data after the first LPF processing in S502 to extract a second AC component. For coordinates (x, y), if the image signal after the first LPF processing is lpf1_pix(x, y) and the image signal after the second LPF processing is lpf2_pix(x, y), the second AC component ac2_pix(x, y) is expressed by equation (3). ac2_pix(x,y)=lpf1_pix(x,y)-lpf2_pix(x,y) …(3)
[0031] Furthermore, in S506, the third subtraction unit 306 subtracts the image data after the third LPF processing in S504 from the image data after the second LPF processing in S503 to extract a third AC component. For coordinates (x, y), if the image signal after the second LPF processing is lpf2_pix(x, y) and the image signal after the third LPF processing is lpf3_pix(x, y), the third AC component ac3_pix(x, y) is expressed by equation (4). ac3_pix(x,y)=lpf2_pix(x,y)-lpf3_pix(x,y) …(4)
[0032] FIG. 6 is a conceptual diagram of the processes from S501 to S506. 6(a) is a one-dimensional diagram of an image signal, showing the processes from S501 to S503. An image signal 601 of an input image shown as a square wave gradually becomes a smoother signal, with an image signal 602 obtained by the first LPF process, an image signal 603 obtained by the second LPF process, and an image signal 604 obtained by the third LPF process.
[0033] 6(b) is a conceptual diagram showing an AC signal obtained by the process of generating an AC signal from S504 to S506 for the signal shown in Fig. 6(a). By the process in each step, a first AC component 605 in a high frequency band is extracted from the first subtraction unit 304, a second AC component 606 in a mid frequency band is extracted from the second subtraction unit 305, and a third AC component 607 in a low frequency band is extracted from the third subtraction unit 306.
[0034] Returning to Fig. 5, in S507, the face detection unit 307 performs face detection processing on the input image data. Note that the method of face detection processing in this embodiment is not particularly determined, but here, a known method using learning data such as deep learning is used. In addition, in this embodiment, not only coordinate information indicating the position of the face as a face detection result, but also coordinate information indicating organs such as the eyes and mouth, and identification information such as age and gender are output.
[0035] In S508, based on the face detection result obtained in S507, the parameter generating unit 308 generates adjustment parameters (adjustment values) for the first to third AC components to be used in the processes from S508 to S510.
[0036] Then, in S509 to S511, first amplitude adjustment section 309, second amplitude adjustment section 310, and third amplitude adjustment section 311 adjust the first to third AC components extracted in S504 to S506, respectively, using the adjustment parameters generated in S508. In this embodiment, nonlinear amplitude adjustment is performed using the adjustment parameters, and the amplitude adjustment is performed using a table expressed by broken lines as an adjustment method. Note that the method of generating and adjusting the adjustment parameters for the first to third AC components performed here will be described in detail later with reference to Figs. 7 and 8.
[0037] After the adjustment of the first to third AC components is completed, in S512, the third adder 312 adds the third AC component adjusted in S511 to the image data after the third LPF process in S503. This process is expressed by equation (5), where the image signal after addition of the coordinates (x, y) is add3_pix(x, y), the adjusted third AC component is ac3_o_pix(x, y), and the image signal after the third LPF process is lpf3_pix(x, y). add3_pix(x,y)=ac3_o_pix(x,y)+lpf3_pix(x,y) …(5)
[0038] In S513, the second adder 313 adds the second AC components adjusted in S510 to the image data obtained by performing the addition process in S512. This process is expressed by equation (6), where the image signal after addition of the coordinates (x, y) is add2_pix(x, y), the adjusted second AC components are ac2_o_pix(x, y), and the image signal after the addition process in S512 is add3_pix(x, y). add2_pix(x,y)=ac2_o_pix(x,y)+add3_pix(x,y) …(6)
[0039] In S514, the first addition unit 314 adds the first AC component adjusted in S509 to the image data obtained by performing the addition process in S513. This process is expressed by equation (7), where the image signal after addition of the coordinates (x, y) is out_pix(x, y), the adjusted first AC component is ac1_o_pix(x, y), and the image signal after the addition process in S513 is add2_pix(x, y).
[0040] out_pix(x,y)=ac1_o_pix(x,y)+add2_pix(x,y) …(7) The image data after the process of S514 is performed as described above is output as corrected output image data, and the process ends.
[0041] Next, a method for adjusting an AC component in this embodiment will be described. First, the concept of the method for adjusting an AC component will be described with reference to FIG. Fig. 7(a) shows the input / output relationship when no adjustment is made to the AC component, with the horizontal axis showing the AC component input signal and the vertical axis showing the AC component output signal. In this case, if the output signal of the AC component at coordinates (x, y) is ac_out(x, y) and the input signal is ac_in(x, y), it is expressed by equation (8), and as shown in Fig. 7(b), the AC signal is output without adjustment. ac_out(x,y)=ac_in(x,y) …(8)
[0042] Fig. 7(c) shows the input / output relationship when weakening the amplitude of the AC component as in Fig. 7(d). In this case, if the output signal of the AC component at coordinates (x, y) is ac_out(x, y), the input signal is ac_in(x, y), and the adjustment parameter is grad, it is expressed by equation (9). ac_out(x,y)=grad×ac_in(x,y),(0≦grad<1) …(9)
[0043] Fig. 7(e) shows the input / output relationship when only the negative AC component is weakened as in Fig. 7(f). In this case, the output signal of the AC component at coordinates (x, y) is ac_out(x, y), the input signal is ac_in(x, y), and the adjustment parameter is grad, which is expressed by equation (10).
[0044] When ac_in(x,y)≧0:ac_out(x,y)=ac_in(x,y) When ac_in(x,y)<0: ac_out(x,y)=grad×ac_in(x,y),(0≦grad<1) …(10)
[0045] Next, a method for generating adjustment parameters for each AC component, which is performed in S508, will be described.
[0046] FIG. 8 is a diagram for explaining the domain of the input signal (AC component) to which the adjustment parameters in this embodiment are applied. In this embodiment, a gradient p_grad(n) for adjusting the positive AC component and a gradient m_grad(n) for adjusting the negative AC component are generated as adjustment parameters for each of the first AC component, the second AC component, and the third AC component. If the output signal of the AC component is ac(n)_out and the input signal is ac(n)_in, the adjustment process of the AC component can be expressed by equation (11). Note that (n) here takes values from 1 to 3, and indicates that it corresponds to the first AC component, the second AC component, and the third AC component.
[0047] When ac(n)_in≧0: ac(n)_out=p_grad(n)×ac(n)_in When ac(n)_in<0: ac(n)_out=m_grad(n)×ac(n)_in …(11)
[0048] In this embodiment, for the high-frequency AC signal (first AC component), adjustment parameters are generated to attenuate the amplitude of the negative AC component as shown in Fig. 7(e) and (f). For the low-frequency AC signal (second AC component and third AC component), adjustment parameters are generated to attenuate the amplitude of both positive and negative AC components as shown in Fig. 7(c) and (d). Therefore, different values are generated for the gradients p_grad(n) and m_grad(n) in equation (11) for the first AC component, the second AC component, and the third AC component.
[0049] The reasons for generating different adjustment parameters in this way are as follows. First, attenuating the positive side of the high-frequency AC signal gives the impression that the face is less three-dimensional overall, but this effect can be prevented. Also, attenuating the negative side of the high-frequency AC signal makes pores and wrinkles on the person's skin less noticeable, which contributes greatly to the skin beautification effect. Another reason is that attenuating both the positive and negative sides of the low-frequency AC signal in order to reduce unevenness in blemishes on the person's skin results in a greater skin beautification effect.
[0050] For the reasons described above, only the negative side of the high-frequency AC signal is attenuated, and both positive and negative AC signals of the low-frequency AC signal are attenuated. However, as shown in FIG. 9(a), when the size of the face changes depending on the scene, the target frequency changes. Therefore, adjustment parameters may be generated according to the size of the face. If the predetermined gradient value is def_grad, and the positive side of the gradient adjustment gain determined by the size of the face is p_adj(n) and the negative side is m_adj(n), the gradient p_grad(n) for adjusting the positive side AC component and the gradient m_grad(n) for adjusting the negative side AC component are expressed by Equation (12).
[0051] p_grad(n)=p_adj(n)×def_grad m_grad(n)=m_adj(n)×def_grad …(12) In this embodiment, def_grad is 1.0, but a fixed value that attenuates it may be given.
[0052] The tilt adjustment gains p_adj(n) and m_adj(n) are realized by having a table for each AC component that determines the value according to the face size, as shown in Figures 9(b) to (d). The vertical axis of the table indicates the value of the tilt adjustment gain, and the horizontal axis indicates the face size. Note that in the examples of Figures 9(b) to (d), the face size is shown as the ratio of the face area to the entire image, but the number of pixels in the face area may be used as is.
[0053] As shown in FIG. 9(b), when the face size is equal to or smaller than a predetermined size, the first AC component corresponding to the high frequency side corresponds to the frequency of uneven spots, etc., so the value of the slope adjustment gain is set to attenuate both the positive and negative AC components. On the other hand, as the face becomes larger, the frequency affects the three-dimensional appearance and pores of the face, so when the face size is larger than a predetermined size, the table does not attenuate the positive AC components, but attenuates only the negative AC components. In addition, for the second AC component shown in FIG. 9(c) and the third AC component shown in FIG. 9(d), as the AC components become lower in frequency, the change in frequency that affects the three-dimensional appearance and pores of the face due to the size of the face decreases, so the range of the slope adjustment gain that attenuates both the positive and negative AC components becomes wider. In this way, by determining the slope adjustment gain, it is possible to adaptively change the frequency to perform skin beautification even if the size of the face changes.
[0054] As another element, in addition to the adjustment gain for the face size, an age-dependent inclination adjustment gain may be added as in equation (13). The difference from equation (12) is that the age-dependent inclination adjustment gain age_adj is further multiplied as a coefficient.
[0055] p_grad(n)=p_adj(n)×age_adj×def_grad m_grad(n)=m_adj(n)×age_adj×def_grad …(13) As for the age-dependent gradient adjustment gain age_adj, it is better to generate a gradient adjustment gain that attenuates the AC components more as the age increases, as shown in the table in Fig. 10(a). This is because as the age increases, wrinkles and age spots become more uneven, and a stronger skin-beautifying effect is required.
[0056] Also, as shown in FIG. 10(b), a tilt adjustment gain may be applied according to the part of the face. Specifically, when it is desired to weaken the skin beautifying effect for the skin region around the eyes to make eyelashes and makeup stand out, the tilt adjustment gain is weakened, and when it is desired to strengthen the skin beautifying effect for the skin region around the mouth to make unshaved areas less noticeable, the tilt adjustment gain is weakened. If the adjustment gain according to the part is part_adj(x, y), it is expressed as in Equation (14). In this case, the gain is set according to the coordinates (x, y) in order to change the gain according to the part of the face.
[0057] p_grad(n)(x,y)=p_adj(n)×part_adj(x,y)×def_grad m_grad(n)(x,y)=m_adj(n)×part_adj(x,y)×def_grad …(14)
[0058] Furthermore, different adjustment gains may be applied depending on the gender.
[0059] As described above, according to this embodiment, the AC components extracted from an image for each frequency are adjusted for each frequency to match the characteristics of a person's skin and face, thereby maintaining the three-dimensionality of the person's face and achieving appropriate skin beautification in a natural state.
[0060] <Other embodiments> The present invention may be applied to a system made up of a plurality of devices, or to an apparatus made up of a single device.
[0061] The present invention can also be realized by supplying a program for implementing one or more of the functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions.
[0062] <Summary> The disclosure of this embodiment includes the following configuration.
[0063] (Configuration 1) An input means for inputting image data; An extraction means for extracting a plurality of AC signals having a plurality of different frequency bands from the image data; a generating means for generating a first adjustment value for adjusting a positive component of an amplitude of the AC signal and a second adjustment value for adjusting a negative component of the amplitude of the AC signal for each of the plurality of frequency bands; an adjusting means for adjusting each of the plurality of AC signals using the first adjustment value and the second adjustment value; A detection means for detecting a skin region from the image data; a correction unit that corrects the image data of the skin region by using the plurality of AC signals adjusted by the adjustment unit, the generating means generates the first adjustment value so that a positive component of the amplitude of the AC signal in a first frequency band among the plurality of frequency bands is less attenuated than a positive component of the amplitude of the AC signal in a second frequency band lower than the first frequency band.
[0064] (Configuration 2) The detection means further detects a reliability of the skin region, The image processing device according to configuration 1, wherein the correction means corrects the AC signal of the skin region by replacing the AC signal with an AC signal obtained by adding together the AC signals adjusted by the adjustment means and the AC signal of the skin region and weighting the AC signal according to the reliability.
[0065] (Configuration 3) The generating means includes: the first adjustment value of the first frequency band does not attenuate positive components of the amplitude of the AC signal, and the second adjustment value attenuates negative components of the amplitude of the AC signal; The first adjustment value and the second adjustment value of the second frequency band attenuate positive and negative components of the amplitude of the AC signal. 3. The image processing device according to configuration 1 or 2, wherein the first adjustment value and the second adjustment value are generated as described above.
[0066] (Configuration 4) A face detection unit detects a face area from the image data, 3. The image processing device according to configuration 1 or 2, wherein the generating means generates the first adjustment value and the second adjustment value based on information about the face region.
[0067] (Configuration 5) 5. The image processing device according to configuration 4, wherein the generating means generates the first adjustment value and the second adjustment value based on the size of the face region.
[0068] (Configuration 6) The generating means includes: generating the first adjustment value that does not attenuate a positive component of the amplitude of the AC signal in the first frequency band and the second adjustment value that attenuates a negative component of the amplitude of the AC signal in the first frequency band when the size of the face region is greater than a predetermined first size, and generating the first adjustment value and the second adjustment value that attenuate the positive component and the negative component of the amplitude of the AC signal in the first frequency band when the size of the face region is equal to or less than the first size; generating the first adjustment value that does not attenuate a positive component of the amplitude of the AC signal in the second frequency band and the second adjustment value that attenuates a negative component of the amplitude of the AC signal in the second frequency band when the size of the face region is greater than a predetermined second size, and generating the first adjustment value and the second adjustment value that attenuate the positive component and the negative component of the amplitude of the AC signal in the second frequency band when the size of the face region is equal to or less than the second size; 6. The image processing device according to configuration 5, wherein the first size is smaller than the second size.
[0069] (Configuration 7) The face region information includes an age of the person, 7. The image processing device according to any one of configurations 4 to 6, wherein the generating means further generates the first adjustment value and the second adjustment value based on the age.
[0070] (Configuration 8) The image processing device of configuration 7, wherein the generation means adjusts the first adjustment value and the second adjustment value so that the amplitude of the AC signal is attenuated more when the age is higher than a first age than when the age is a second age lower than the first age.
[0071] (Configuration 9) The face region information includes a person's gender, 9. The image processing device according to any one of configurations 4 to 8, wherein the generating means further generates the first adjustment value and the second adjustment value based on the gender.
[0072] (Configuration 10) The face region information includes a face part, 10. The image processing device according to any one of configurations 4 to 9, wherein the generating means further generates the first adjustment value and the second adjustment value based on the part.
[0073] (Configuration 11) The image processing device according to configuration 10, wherein the generating means adjusts the first adjustment value so as to reduce attenuation of the amplitude of the AC signal in the first frequency band in an area of the image data that indicates the periphery of the eye when the part is an eye.
[0074] (Configuration 12) The image processing device according to configuration 10 or 11, characterized in that, when the part is a mouth, the generating means adjusts the first adjustment value so as to increase attenuation of the amplitude of the AC signal in the first frequency band in an area of the image data indicating the periphery of the mouth.
[0075] (Configuration 13) An image processing device according to any one of configurations 1 to 12, An imaging means for taking a photograph and outputting the image data; An imaging device comprising:
[0076] (Configuration 14) an input step in which an input means inputs image data; An extraction step in which an extraction means extracts a plurality of AC signals having a plurality of different frequency bands from the image data; a generating step of generating, for each of the plurality of frequency bands, a first adjustment value for adjusting a positive component of the amplitude of the AC signal and a second adjustment value for adjusting a negative component of the amplitude of the AC signal; an adjusting step in which an adjusting unit adjusts amplitudes of the plurality of AC signals by using the first adjustment value and the second adjustment value; A detection step in which a detection means detects a skin region from the image data; a correction step of correcting the image data of the skin region by using the plurality of AC signals adjusted in the adjustment step, an image processing method, characterized in that in the generating step, the first adjustment value is generated so that a positive component of the amplitude of the AC signal in a first frequency band among the plurality of frequency bands is less attenuated than a positive component of the amplitude of the AC signal in a second frequency band lower than the first frequency band.
[0077] (Configuration 15) A program for causing a computer to function as each of the means of the image processing device according to any one of configurations 1 to 12.
[0078] (Configuration 16) A computer-readable storage medium storing the program according to configuration 15.
[0079] The invention is not limited to the above-described embodiments, and various modifications and variations are possible without departing from the spirit and scope of the invention. Accordingly, the following claims are appended to apprise the public of the scope of the invention. [Explanation of symbols]
[0080] 101: imaging unit, 102: image processing unit, 103: system control unit, 104: display unit, 105: recording unit, 106: network processing unit, 107: operation unit, 108: bus, 201: image input unit, 202: skin correction processing unit, 203: skin region detection unit, 204: synthesis processing unit, 205: image output unit, 301: first LPF processing unit, 302: second LPF processing unit, 303: third LPF processing unit, 304: first subtraction unit, 305: second subtraction unit, 306: third subtraction unit, 307: face detection unit, 308: parameter generation unit, 309: first amplitude adjustment unit, 310: second amplitude adjustment unit, 311: third amplitude adjustment unit, 312: third addition unit, 313: second addition unit, 314: first addition unit
Claims
1. An input means for inputting image data; An extracting means for extracting a plurality of AC signals having a plurality of different frequency bands from the image data; generating means for generating a first adjustment value for adjusting a positive component of the amplitude of the AC signal and a second adjustment value for attenuating a negative component; an adjusting means for adjusting each of the plurality of AC signals using the first adjustment value and the second adjustment value; A detection means for detecting a skin region from the image data; a correction unit that corrects the image data of the skin region using the plurality of AC signals adjusted by the adjustment unit, The image processing device is characterized in that the generation means generates the second adjustment value so that a positive component of the amplitude of the AC signal in a second frequency band among the multiple frequency bands is more easily attenuated than a positive component of the amplitude of the AC signal in a first frequency band higher than the second frequency band.
2. The detection means further detects a reliability of the skin region, 2. The image processing device according to claim 1, wherein the correction means corrects the AC signal of the skin region by replacing the AC signal of the skin region with an AC signal obtained by adding together the plurality of AC signals adjusted by the adjustment means and weighting and synthesizing the AC signal of the skin region according to the reliability.
3. The generating means includes: the first adjustment value of the first frequency band does not attenuate positive components of the amplitude of the AC signal, and the second adjustment value attenuates negative components of the amplitude of the AC signal; The first and second adjustments of the second frequency band attenuate positive and negative components of the amplitude of the AC signal.
2. The image processing apparatus according to claim 1, wherein the first adjustment value and the second adjustment value are generated in such a manner that:
4. A face detection unit detects a face area from the image data, The image processing apparatus according to claim 1 , wherein the generating means generates the first adjustment value and the second adjustment value based on the face area.
5. 5. The image processing apparatus according to claim 4, wherein the generating means generates the first adjustment value and the second adjustment value based on a size of the face area.
6. The generating means includes: generating the first adjustment value that does not attenuate a positive component of the amplitude of the AC signal in the first frequency band and the second adjustment value that attenuates a negative component of the amplitude of the AC signal in the first frequency band when the size of the face region is greater than a predetermined first size, and generating the first adjustment value and the second adjustment value that attenuate the positive component and the negative component of the amplitude of the AC signal in the first frequency band when the size of the face region is equal to or less than the first size; when the size of the face region is greater than a predetermined second size, generate the first adjustment value that does not attenuate a positive component of the amplitude of the AC signal in the second frequency band and the second adjustment value that attenuates a negative component of the amplitude of the AC signal, and when the size of the face region is equal to or less than the second size, generate the first adjustment value and the second adjustment value that attenuate the positive and negative components of the amplitude of the AC signal in the second frequency band; 6. The image processing apparatus according to claim 5, wherein the first size is smaller than the second size.
7. The face detection means further detects an age of a person corresponding to the face area, 5. The image processing apparatus according to claim 4, wherein the generating means further generates the first adjustment value and the second adjustment value based on the age.
8. 8. The image processing device according to claim 7, wherein the generation means adjusts the first adjustment value and the second adjustment value so that the amplitude of the AC signal is attenuated more when the age is higher than a first age than when the age is a second age lower than the first age.
9. The face detection means further detects the gender of the person corresponding to the face area, 5. The image processing apparatus according to claim 4, wherein the generating means further generates the first adjustment value and the second adjustment value based on the gender.
10. The face detection means further detects a part of a face corresponding to the face area, The image processing apparatus according to claim 4 , wherein the generating means further generates the first adjustment value and the second adjustment value based on the part.
11. 11. The image processing device according to claim 10, wherein the generating means generates the first adjustment value so that, when the part is an eye, the attenuated amplitude of the AC signal in the first frequency band in an area of the image data representing the periphery of the eye is greater than a predetermined amplitude.
12. The image processing device according to claim 10, characterized in that the generating means generates the first adjustment value so that, when the part is a mouth, the attenuated amplitude of the AC signal in the first frequency band in a region of the image data indicating the periphery of the mouth is smaller than a predetermined amplitude.
13. An image processing device according to any one of claims 1 to 12, An imaging means for taking a photograph and outputting the image data; An imaging device comprising:
14. an input step in which an input means inputs image data; An extraction step in which an extraction means extracts a plurality of AC signals having a plurality of different frequency bands from the image data; A generating step in which a generating means generates a first adjustment value for adjusting a positive component of the amplitude of the AC signal and a second adjustment value for attenuating a negative component; an adjusting step in which an adjusting unit adjusts amplitudes of the plurality of AC signals by using the first adjustment value and the second adjustment value; A detection step in which a detection means detects a skin region from the image data; a correction step of correcting the image data of the skin region by using the plurality of AC signals adjusted in the adjustment step, an adjustment value generation step of generating the second adjustment value such that a positive component of the amplitude of the AC signal in a second frequency band among the plurality of frequency bands is more easily attenuated than a positive component of the amplitude of the AC signal in a first frequency band higher than the second frequency band, the second adjustment value being generated in the generation step of the image processing method.
15. A program for causing a computer to function as each of the means of the image processing apparatus according to any one of claims 1 to 12.
16. A computer-readable storage medium storing the program according to claim 15.