Image processing apparatus and method, electronic apparatus, program, and storage medium

Abstract

To perform skin enhancement correction to achieve a natural impression, while preserving the texture of the skin of a person.SOLUTION: An image processing apparatus extracts a first signal in a predetermined first frequency range from an image signal input by input means, and adjusts the amplitude of the extracted first signal to be an amplitude within a predetermined range.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an image processing device and method, an electronic device, a program, and a storage medium, and in particular to a skin smoothing technique. [Background technology]

[0002] Conventionally, there is known a skin beautification technique that provides a skin beautification image by correcting the skin region of an image containing a person. For example, a method has been proposed that applies an ε-filter that separates and removes small-amplitude high-frequency noise components superimposed on a signal waveform, thereby smoothing small-amplitude gradients such as wrinkles and blemishes while preserving edges with large gradients such as contours (Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2001-118064 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the method disclosed in Patent Document 1 has the problem that, along with wrinkles and blemishes, the fine irregularities on the surface of a person's skin, such as texture, are lost, resulting in an unnatural impression of skin texture.

[0005] The present invention has been made in consideration of the above problems, and has as its object to perform skin beautification that gives a natural impression while retaining the texture of a person's skin. [Means for solving the problem]

[0006] In order to achieve the above object, the image processing device of the present invention has an input means for inputting an image signal, an extraction means for extracting a first signal of a predetermined first frequency band from the image signal, and an adjustment means for adjusting the amplitude of the first signal so that it is within a predetermined range of amplitude. [Effects of the Invention]

[0007] According to the present invention, it is possible to perform skin correction that gives a natural impression while preserving the texture of a person's skin. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a block diagram showing a schematic functional configuration of an imaging apparatus according to an embodiment of the present invention. [Figure 2] FIG. 2 is a block diagram showing the functional configuration of an image processing unit in the embodiment. [Figure 3] FIG. 2 is a block diagram showing the functional configuration of a development processing unit in the embodiment. [Figure 4] FIG. 2 is a block diagram showing the functional configuration of a skin correction processing unit in the embodiment. [Figure 5] 10 is a flowchart showing the flow of processing for generating a cosmetically corrected image in an embodiment. [Figure 6] 4 is a flowchart showing a development process in an embodiment. [Figure 7] 10 is a flowchart showing the flow of skin correction processing in the embodiment. [Figure 8] 5A and 5B are conceptual diagrams of AC component extraction processing in the skin correction processing according to the embodiment. [Figure 9] 5A to 5C are diagrams illustrating the concept of a method for adjusting AC components in the skin correction processing according to the embodiment. [Figure 10] 5A and 5B are diagrams for explaining adjustment parameters of AC components in the embodiment. [Figure 11] 10A to 10C are diagrams illustrating results after amplitude adjustment in the embodiment. [Figure 12] 5A to 5C are diagrams for explaining a method of selecting a frequency based on a face detection result in the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the scope of the invention claimed. Although multiple features are described in the embodiments, not all of these multiple features are necessarily essential to the invention, and multiple features may be combined arbitrarily. Furthermore, in the accompanying drawings, the same reference numerals are used to designate the same or similar components, and redundant explanations will be omitted.

[0010] In the drawings, configurations represented as blocks may be realized by integrated circuits (ICs) such as ASICs or FPGAs, by discrete circuits, or by a combination of memory and a processor that executes a program stored in the memory. Also, one block may be realized by multiple integrated circuit packages, or multiple blocks may be realized by a single integrated circuit package. Furthermore, the same block may be implemented in different configurations depending on the operating environment, required capabilities, etc.

[0011] In the following embodiments, the present invention will be described as being implemented in an imaging device such as a digital camera, but the present invention can also be implemented in any electronic device having an imaging function. Such electronic devices include not only imaging devices but also computer devices (personal computers, tablet computers, media players, PDAs, etc.), mobile phones, smartphones, game consoles, robots, drones, drive recorders, etc. These are merely examples, and the present invention can also be implemented in other electronic devices.

[0012] ●Equipment configuration 1 is a block diagram showing a schematic functional configuration of an image capturing apparatus 100 according to an embodiment of the present invention. In FIG. 1, only a representative configuration of the image capturing apparatus 100 is shown. The optical system 101 includes multiple lenses, an aperture that also serves as a shutter, an aperture drive mechanism, etc., and an optical image of a subject (subject image) is formed on the light receiving surface of the imaging unit 102 via the optical system 101. The multiple lenses include movable lenses such as a focus lens that adjusts the focal length of the optical system 101 and a zoom lens that changes the angle of view, and the optical system 101 includes a drive mechanism that drives these movable lenses.

[0013] The imaging unit 102 is a color image sensor, such as a known CCD or CMOS sensor, having a primary-color Bayer array of color filters. It includes a pixel array in which a plurality of pixels are arranged two-dimensionally, and peripheral circuits for driving each pixel and reading out signals from each pixel. Each pixel accumulates charge according to the amount of incident light through photoelectric conversion. Then, a pixel signal (analog image signal) representing the subject image formed by the optical system 101 is obtained by reading out from each pixel a voltage signal according to the amount of charge accumulated during the exposure period.

[0014] The A / D conversion unit 103 converts the analog image signal read out from the imaging unit 102 into a digital image signal (image data) by A / D conversion. Note that if the imaging unit 102 has an A / D conversion function, the A / D conversion unit 103 is not necessary.

[0015] The image processing unit 104 applies predetermined image processing to image data output by the A / D conversion unit 103 and image data read from the recording unit 109, to generate signals and image data according to the intended use, and to acquire and / or generate various types of information. The image processing unit 104 may be a dedicated hardware circuit such as an ASIC (Application Specific Integrated Circuit) designed to realize a specific function. Alternatively, the image processing unit 104 may be configured to realize a specific function by a processor executing software, such as a DSP (Digital Signal Processor) or GPU (Graphics Processing Unit).

[0016] The image processing that the image processing unit 104 can apply to image data can include, for example, preprocessing, color interpolation processing, correction processing, detection processing, data processing, evaluation value calculation processing, special effect processing, and the like.

[0017] Preprocessing may include signal amplification, reference level adjustment, defective pixel correction, etc. Color interpolation processing is performed when a color filter is provided in the imaging unit 102, and is processing to interpolate values ​​of color components not included in the individual pixel data that make up the image data. Color interpolation processing is also called demosaicing processing. Correction processing may include white balance adjustment, gradation correction, correction of image degradation caused by optical aberration of the optical system 101 (image restoration), correction of the effects of peripheral light falloff of the optical system 101, color correction, etc. Detection processing may include detection of feature regions (e.g., face regions and human body regions) and their movements, person recognition processing, etc.

[0018] Data processing may include processes such as area extraction (trimming), compositing, scaling, encoding and decoding, and header information generation (data file generation). Data processing also includes the generation of image data for display and image data for recording. Evaluation value calculation may include processes such as the generation of signals and evaluation values ​​used for autofocus detection (AF) and the generation of evaluation values ​​used for automatic exposure control (AE). Special effect processing may include processes such as adding a blur effect, changing color tones, and relighting.

[0019] These are examples of processes that the image processing unit 104 can apply to image data, and do not limit the processes that the image processing unit 104 can apply. For example, the skin beautification process of this embodiment can be performed by combining the above-mentioned color interpolation process, correction process, detection process, and special effect process. FIG. 2 is a block diagram showing the functional configuration of the image processing unit 104 for performing the skin beautification process, and includes a development processing unit 201, a skin correction processing unit 202, a skin region detection unit 203, and a synthesis processing unit 204. The skin beautification process will be described in detail later.

[0020] 1, the exposure control unit 105 determines the shooting conditions (aperture value, shutter speed, ISO sensitivity) based on the evaluation value calculated by the image processing unit 104 and a predetermined program diagram. Then, at the time of shooting, the exposure control unit 105 controls the operation of the optical system 101 (aperture) and the imaging unit 102 based on the determined shooting conditions. In addition, the exposure control unit 105 drives the optical system 101 (focus lens) based on the evaluation value calculated by the image processing unit 104, and adjusts the focal distance of the optical system 101.

[0021] The system control unit 106 includes, for example, a processor (CPU, MPU, etc.) capable of executing a program, a ROM, and a RAM. The system control unit 106 loads a program stored in the ROM into the RAM and executes the program, thereby controlling the operation of each unit of the imaging device 100 and realizing the functions of the imaging device 100. The ROM stores programs executed by the processor, various setting values ​​of the imaging device 100, GUI data, etc. The RAM is a main memory used by the system control unit 106 when executing programs.

[0022] The operation unit 107 is a general term for input devices (buttons, switches, dials, etc.) provided for the user to input various instructions to the imaging device 100. Functions are assigned to the input devices that make up the operation unit 107, and include, for example, a release switch, a video recording switch, a shooting mode selection dial for selecting a shooting mode, a menu button, direction keys, an enter key, etc.

[0023] The release switch is a switch for recording still images, and the system control unit 106 recognizes a half-pressed state of the release switch as a shooting preparation instruction and a full-pressed state as a shooting start instruction. Furthermore, the system control unit 106 recognizes a video recording switch pressed in shooting standby mode as a video recording start instruction, and a video recording stop instruction when pressed during video recording. The operation unit 107 can also be used to set whether or not to perform the skin smoothing process in this embodiment. The functions assigned to the same input device may be variable. The input device may be software buttons or keys using a touch display. The operation unit 107 may also include an input device compatible with non-contact input methods, such as voice input or eye-gaze input.

[0024] The display unit 108 displays image data obtained by shooting and images based on image data read from the recording unit 109. The display unit 108 is, for example, a liquid crystal display or an organic EL display. By displaying the moving image obtained by shooting on the display unit 108 while shooting a moving image, the display unit 108 can function as an electronic viewfinder (EVF).

[0025] The recording unit 109 is a storage device that stores image data for recording generated by the image processing unit 104. The recording unit 109 may be a storage device that uses, for example, a nonvolatile memory or a magnetic disk. The recording unit 109 may also be a storage device that uses a removable recording medium. The memory 110 is, for example, a RAM, and is used to temporarily store various data such as intermediate data by the image processing unit 104 and the system control unit 106. A part of the memory 110 may also be used as a video memory. Bus 111 is used for communicating data and control signals between the connected blocks.

[0026] Skin beautification image generation process Next, a series of operations performed by the imaging device 100 having the above configuration when generating a skin-beautifying image of this embodiment will be described with reference to the flowchart in Fig. 5. This processing is executed when skin-beautifying processing is selected by operating the operation unit 107.

[0027] First, in S501, when the system control unit 106 detects that a shooting start instruction has been input from the operation unit 107, it instructs the exposure control unit 105 to start shooting a still image. When shooting is performed, image data is supplied to the image processing unit 104 via the imaging unit 102 and the A / D conversion unit 103.

[0028] Next, in S502, the development processing unit 201 of the image processing unit 104 shown in FIG. 2 applies development processing, including demosaic processing, to the input image data to generate full-color image data. Data for each pixel constituting the image data supplied from the A / D conversion unit 103 to the image processing unit 104 contains information on one color component (R (red), G (green), or B (blue) component) corresponding to the primary-color Bayer-array color filter of the imaging unit 102. Therefore, the development processing unit 201 generates full-color image data from the input image data according to the intended use. Full-color image data is image data in which each pixel data contains color components necessary to represent a color image, such as YUV or RGB. The functional configuration of the development processing unit 201 and details of the development processing performed in S502 will be described later with reference to FIGS. 3 and 6.

[0029] Next, in S503, the skin correction processing unit 202 of the image processing unit 104 applies skin correction processing to the full-color image data generated by the development processing unit 201. In the skin correction processing, AC components are extracted for each frequency from the input full-color image data, the AC components are adjusted for each frequency in accordance with the skin and facial features of a person, and the adjusted AC components are reconstructed to generate skin-corrected image data. Note that the functional configuration of the skin correction processing unit 202 and details of the skin correction processing performed in S503 will be described later with reference to FIGS. 4 and 7.

[0030] Then, in S504, the skin region detection unit 203 of the image processing unit 104 applies a process for detecting a skin region to the full-color image data generated by the development processing unit 201. In this embodiment, a general method is used for extracting a region corresponding to skin color based on the hue, saturation, and brightness of the image, but detection processing based on learning data such as deep learning may also be performed. The skin region detection result is output to the synthesis processing unit 204.

[0031] In S505, the synthesis processing unit 204 of the image processing unit 104 synthesizes the full-color image data generated by the development processing unit 201 and the skin-corrected image data generated by the skin correction processing unit 202 according to the detection result of the skin area detected by the skin area detection unit 203.

[0032] More specifically, if the signal value of the synthesized image at coordinates (x, y) is out_pix(x, y), the image signal generated in S502 is in_pix(x, y), and the image signal subjected to skin correction processing by skin area detection unit 203 in S503 is cor_pix(x, y), the synthesis processing is expressed by equation (1). α(x, y) indicates the detection result of the skin area detected in S504, and takes a value between 0.0 and 1.0, with the probability of it being a skin area increasing as the value approaches 1.0 from 0.0.

[0033] out_pix(x,y)=(1.0-α(x,y))×in_pix(x,y)+α(x,y)×cor_pix(x,y) …(1)

[0034] In S506, the system control unit 106 stores the composite image data, to which the skin smoothing process has been applied and which has been output from the image processing unit 104, in a data file of a predetermined format and records the data in the recording unit 109. Note that the system control unit 106 can apply necessary processes, such as encoding, to the composite image data before recording.

[0035] Furthermore, the image processing unit 104 may generate image data for display from the generated composite image data and write the generated image data into a video memory area of ​​the memory 110, thereby causing the display unit 108 to display the composite image.

[0036] ●Developing process Next, a description will be given of the details of the development processing performed in S502 by the development processing unit 201 of the image processing unit 104. Fig. 3 is a block diagram showing the functional configuration of the development processing unit 201, and Fig. 6 is a flowchart showing the development processing.

[0037] First, in S601, the white balance processing unit 301 applies white balance processing to the image data input from the A / D conversion unit 103, adjusting the color balance in accordance with the color temperature of the ambient light at the time of shooting. The white balance processing is a process of adjusting the color balance in the image by applying individual gains to the R, G, and B components, as shown in the following equation (2).

[0038] In equation (2), Rin, Gin, and Bin are the values ​​of the R, G, and B components before white balance processing, Rout, Gout, and Bout are the values ​​of the R, G, and B components after white balance processing, and GainR, GainG, and GainB are the gains (white balance coefficients) to be applied to the R, G, and B components, respectively.

[0039] Rout = GainR × Rin Gout = GainG × Gin …(2) Bout = GainB × Bin

[0040] The white balance coefficients can be calculated from input image data using a known method, for example, by estimating the color temperature of the ambient light, extracting a white area corresponding to the color temperature from the image, and determining a coefficient value that will make the pixels in the white area achromatic. The white balance processing unit 301 outputs the processed image data to the noise reduction processing unit 302.

[0041] In S602, the noise reduction processing unit 302 applies noise reduction processing to the input image data to reduce dark current noise and optical shot noise. The noise reduction processing can be performed by a known method using, for example, a low-pass filter or a bilateral filter. The noise reduction processing unit 302 outputs the processed image data to the demosaic processing unit 303.

[0042] In S603, the demosaic processing unit 303 applies demosaic processing to the input image data. Demosaic processing is also called color interpolation processing, and can be achieved using a known method. The demosaic processing unit 303 interpolates missing color component values ​​in individual pixel data using, for example, values ​​of surrounding pixels, so that each pixel has values ​​of the three components R, G, and B. The demosaic processing unit 303 outputs the processed image data to the color matrix processing unit 304.

[0043] In S604, the color matrix processing unit 304 applies color matrix processing to the input image data to match the color gamut of the image data to the color gamut of the output device. The color matrix processing can be implemented by a known method based on the spectral characteristics of the imaging unit 102 (image sensor) and the color gamut of the output device of the image data. Specifically, the color matrix processing unit 304 applies a matrix formed of 3×3 coefficients k11 to k33 to the color component values ​​Rin, Gin, and Bin for each pixel data of the input image data, as shown in the following equation (3), for example. This converts the color component values ​​of each pixel data into color component values ​​Rout, Gout, and Bout that are suitable for the color gamut handled by the output device. TIFF2025132820000002.tif1786 The color matrix processing unit 304 outputs the processed image data to the gamma processing unit 305.

[0044] In S605, the gamma processing unit 305 applies an optical-electrical transfer function (OETF) corresponding to an electro-optical transfer function (EOTF) that represents the input / output characteristics of the destination display device to the input image data. The EOTF and OETF are also called gamma curves or gamma characteristics, and gamma processing converts the values ​​of the image data into values ​​that can be appropriately displayed on the destination device.

[0045] The full-color image data obtained by the development processing as described above is output to a skin correction processing unit 202, a skin region detection unit 203, and a synthesis processing unit 204, respectively.

[0046] Skin correction processing Next, details of the skin correction processing performed by the skin correction processing unit 202 of the image processing unit 104 in S503 will be described. Fig. 4 is a block diagram showing the functional configuration of the skin correction processing unit 202, and Fig. 7 is a flowchart showing the skin correction processing. The skin correction processing unit 202 is made up of a first LPF processing unit 401, a second LPF processing unit 402, a third LPF processing unit 403, a first subtraction unit 404, a second subtraction unit 405, a third subtraction unit 406, a face detection unit 407, a parameter generation unit 408, a first amplitude adjustment unit 409, a second amplitude adjustment unit 410, a third amplitude adjustment unit 411, a third adder 412, a second adder 413, and a first adder 414.

[0047] First, in S701, the first LPF processing unit 401 performs first LPF processing on input full-color image data. Here, a general filter such as a Gaussian filter or a mean filter is used as the LPF. Next, in S702, the second LPF processing unit 402 performs second LPF processing on the image data that has been subjected to the first LPF processing in S701. As for the low-pass filter, a general filter is used, as in S701. Furthermore, in S703, the third LPF processing unit 403 performs the third LPF processing on the image data that has been subjected to the second LPF processing in S702. As for the low-pass filter, a general filter is used, as in S701.

[0048] In S704, the first subtraction unit 404 subtracts the image data that has been subjected to the first LPF processing in S701 from the input full-color image data to generate a first AC component. For coordinates (x, y), if the input signal is in_pix(x, y) and the image signal after the first LPF processing is lpf1_pix(x, y), the first AC component ac1_pix(x, y) is expressed by equation (4).

[0049] ac1_pix(x,y)=in_pix(x,y)-lpf1_pix(x,y) …(4)

[0050] Next, in S705, the second subtraction unit 405 subtracts the image data that has been subjected to the second LPF processing in S702 from the image data that has been subjected to the first LPF processing in S701 to generate a second AC component. For coordinates (x, y), if the image signal that has been subjected to the first LPF processing is lpf1_pix(x, y) and the image signal that has been subjected to the second LPF processing is lpf2_pix(x, y), the second AC component ac2_pix(x, y) is expressed by equation (5).

[0051] ac2_pix(x,y)=lpf1_pix(x,y)-lpf2_pix(x,y) …(5)

[0052] In S706, the third subtraction unit 406 subtracts the image data that has been subjected to the third LPF processing in S703 from the image data that has been subjected to the second LPF processing in S702 to generate a third AC component. For coordinates (x, y), if the image signal that has been subjected to the second LPF processing is lpf2_pix(x, y) and the image signal that has been subjected to the third LPF processing is lpf3_pix(x, y), the third AC component ac3_pix(x, y) is expressed by equation (6).

[0053] ac3_pix(x,y)=lpf2_pix(x,y)-lpf3_pix(x,y) …(6)

[0054] 8(a) is a one-dimensional diagram of an image signal, illustrating the processing from S701 to S703. Image signal 801 of the input image, shown as a square wave, gradually becomes a smoother signal, with image signal 802 obtained by the first LPF processing, image signal 803 obtained by the second LPF processing, and image signal 804 obtained by the third LPF processing.

[0055] 8(b) is a conceptual diagram showing the AC signal obtained by the process of generating the AC signal from S704 to S706 for the signal shown in Fig. 8(a). Through the process at each step, first subtraction unit 404 extracts a first AC component 805 in the high frequency band, second subtraction unit 405 extracts a second AC component 806 in the mid frequency band, and third subtraction unit 406 extracts a third AC component 807 in the low frequency band.

[0056] 7, in S707, the face detection unit 407 performs face detection processing on the input image data. Note that, although the method of face detection processing in this embodiment is not particularly defined, it is assumed here that a known method using learning data such as deep learning is used.

[0057] In S708, the parameter generation unit 408 generates adjustment parameters for the first to third AC components to be used in the processes of S709 to S711 based on the face detection result obtained in S707. Note that a method for generating the adjustment parameters for the first to third AC components will be described in detail later with reference to FIG.

[0058] In S709, first amplitude adjustment unit 409 adjusts the first AC component generated in S704 using the adjustment parameter generated in S708. If the output signal of the AC component at coordinates (x, y) is ac_out(x, y), the input signal is ac_in(x, y), and the adjustment parameter is grad, the adjustment method for the AC component is expressed by the following equation (7).

[0059] ac_out(x,y)=grad×ac_in(x,y) …(7)

[0060] Next, in S710, second amplitude adjustment unit 410 adjusts the second AC component generated in S705 using the adjustment parameter generated in S708. Note that the adjustment method is the same as in S709, and therefore description thereof will be omitted.

[0061] In addition, in S711, the third amplitude adjustment unit 411 adjusts the third AC component generated in S706 using the adjustment parameter generated in S708. Note that the adjustment method is the same as in S709, and therefore description thereof will be omitted.

[0062] After completing adjustment of the first to third AC components, in S712, the third adder 412 adds the third AC component adjusted in S711 to the image data after the third LPF process in S703. This process is expressed by equation (8), where the image signal after addition of the coordinates (x, y) is add3_pix(x, y), the adjusted third AC component is ac3_o_pix(x, y), and the image signal after the third LPF process is lpf3_pix(x, y).

[0063] add3_pix(x,y)=ac3_o_pix(x,y)+lpf3_pix(x,y) …(8)

[0064] In S713, the second addition unit 413 adds the second AC component adjusted in S710 to the image data obtained by performing the addition process in S712. This process is expressed by equation (9), where the image signal after addition of the coordinates (x, y) is add2_pix(x, y), the adjusted second AC component is ac2_o_pix(x, y), and the image signal after the addition process in S712 is add3_pix(x, y).

[0065] add2_pix(x,y)=ac2_o_pix(x,y)+add3_pix(x,y) …(9)

[0066] In S714, the first adder 414 adds the first AC component adjusted in S709 to the image data obtained by performing the addition process in S713. This process is expressed by equation (10), where the image signal after addition of the coordinates (x, y) is cor_pix(x, y), the adjusted first AC component is ac1_o_pix(x, y), and the image signal after the addition process in S713 is add2_pix(x, y).

[0067] cor_pix(x,y)=ac1_o_pix(x,y)+add2_pix(x,y) …(10)

[0068] The image data after the processing of S714 is performed as described above is output as skin-corrected image data, and the skin correction processing ends.

[0069] ●Generating adjustment parameters This embodiment is characterized by generating adjustment parameters that keep the amplitude of AC components in the high frequency band within a certain level range based on a predetermined value, and generating adjustment parameters that attenuate the amplitude of AC components other than high frequencies.

[0070] The reason for generating such adjustment parameters is that high-frequency AC components are easily extracted from the fine irregularities on a person's skin surface, and if the high-frequency AC components are attenuated, the overall texture of the skin will disappear, resulting in an unnatural impression.

[0071] Another reason is that keeping the amplitude of high-frequency AC components within a certain range of levels enhances the skin smoothing effect. This is because skin with even fine irregularities on the surface of human skin, known as texture, is generally recognized as beautiful skin. Another reason is that attenuating the amplitude of AC components other than high frequencies in order to reduce wrinkles and blemishes on human skin enhances the skin smoothing effect.

[0072] First, referring to Fig. 9(a), the input-output relationship when no adjustment is made to the AC component will be described. The horizontal axis represents the input signal of the AC component, and the vertical axis represents the output signal of the AC component. In this case, if the output signal of the AC component at the coordinates (x, y) is denoted as ac_out(x, y) and the input signal as ac_in(x, y), the input-output relationship shown in Fig. 9(a) can be expressed by Equation (11), and the adjustment parameter grad in Equation (7) described above is 1. Then, as shown in Fig. 9(c), the AC signal is output without being adjusted.

[0073] ac_out(x,y)=ac_in(x,y) …(11)

[0074] Next, referring to Fig. 9(b), the input-output relationship when the amplitude of the AC component is weakened will be described. In this case, the input-output relationship shown in Fig. 9(b) can be expressed by Equation (7) described above, and the adjustment parameter grad is greater than 0 and less than 1 (0 < grad < 1).

[0075] ac_out(x,y)=grad×ac_in(x,y) …(7)

[0076] In this embodiment, for the reasons described above, with respect to the second and third AC components, an adjustment parameter that takes a value of 0 < grad < 1 and has an input-output relationship as shown in Fig. 9(b) is used to adjust the amplitude. With such an adjustment parameter, as shown in Fig. 9(d), an adjustment is made such that the amplitudes of the AC components of the input signals on both the positive and negative sides of the second and third AC components are weakened. Note that the adjustment parameters for the second and third AC components may be the same value or different values. [[ID=一十七]] [[ID=一十八]]

[0077] [[ID=一十九]] [[ID=二十]]Also, for the reasons described above, in this embodiment, with respect to the first AC component in the high-frequency band, an adjustment parameter with input-output characteristics as shown in Fig. 10(a) is used to adjust the amplitude. [[ID=二十一]] As shown in Figure 10(a), for a positive input signal, adjustment is performed using adjustment parameters so that the output signal falls within a range from signal value rh to signal value rl, with a predetermined signal value at the center. When the signal value of the input signal is equal to or greater than b, the output signal falls to the upper limit signal value rh. Similarly, for negative input signals, adjustment is performed using adjustment parameters so that the output signal falls within the range of signal values ​​-rh to -rl, with a predetermined signal value at the center. When the input signal value is -b or less, the output signal falls to the lower limit signal value -rh.

[0078] In addition, it is possible that minute noise components present in image data may be mistakenly detected as skin texture, but as a countermeasure to this, adjustment parameters may be used to remove the noise components in advance. In the example shown in Figure 10(a), for AC components whose signal value of the input signal is less than the threshold a (from -a to a), the output signal is set to 0, thereby removing the minute noise components.

[0079] Fig. 10(b) is a diagram showing adjustment parameters for achieving the input / output characteristics shown in Fig. 10(a), and shows the adjustment parameters relative to the absolute value of the signal value of the input signal. As shown in Fig. 10(b), by determining the adjustment parameter grad according to the absolute value of the signal value of the input signal, the amplitude can be adjusted using Equation (7).

[0080] Fig. 11 is a graph showing an example of the first AC component whose amplitude has been adjusted using the adjustment parameters shown in Fig. 10(b). Note that the graph in Fig. 11 does not show the results of removing the minute noise components described above for convenience, but only shows the results of adjusting the amplitude. The horizontal axis represents the pixel position of the image data, and the vertical axis represents the pixel value, which is color information such as RGB. The dotted line represents the pixel value of the person's skin surface in the original image data, and positive and negative image components exist around a predetermined pixel value.

[0081] The solid line in Fig. 11(a) shows an example of the results when amplitude adjustment is performed using the adjustment parameters in Fig. 10. The amplitude of each image component is adjusted to fall within a certain level range (rh to rl, -rh to -rl) based on a predetermined value, and image components that originally have large amplitudes are attenuated, while image components that originally have small amplitudes are amplified.

[0082] 11(b) is a diagram showing an example of an adjustment result by a conventional adjustment method, as a comparison with the adjustment result shown in FIG. 11(a) of this embodiment, and shows a case where the same input signal as in FIG. 11(a) is subjected to amplitude adjustment using the adjustment parameters of FIG. 9(b). As shown by the solid line, the amplitude is attenuated for all image components. As such, when the amplitude of high-frequency AC components is adjusted using these adjustment parameters, as with AC components other than high frequencies, image components that are originally small in amplitude are further attenuated, and the texture of the skin is likely to be lost.

[0083] In addition, in this embodiment, the aim is to keep the amplitude of the AC components in the high frequency band within a certain level range based on a predetermined value, but if the size of the face changes depending on the input image as shown in Figure 12(a), the frequency band to be adjusted to the above-mentioned certain level range changes. Therefore, the frequency band to be adjusted to the certain level range may be changed depending on the size of the face.

[0084] Selection of the target frequency band can be achieved by having a table that determines the target frequency band according to the size of the face, as shown in Fig. 12. The vertical axis of the table indicates the threshold for frequency band selection, and the horizontal axis indicates the size of the face. In this embodiment, the size of the face indicates the ratio of the face area to the entire image, but the number of pixels in the face area, etc., may also be used as is.

[0085] When the face size is relatively small and below the threshold Th_mid, AC components in the high frequency band affect the skin texture, so AC components in the high frequency band are selected as the target frequency band. Conversely, when the face size is relatively large and the value on the vertical axis of the table is equal to or greater than the threshold Th_low, AC components in the low frequency band also affect the skin texture, so AC components in the low frequency band are selected as the target frequency band. Furthermore, when the value on the vertical axis of the table is equal to or greater than the threshold Th_mid but less than the threshold Th_low, AC components in the mid frequency band are selected as the target frequency band. In this way, the target frequency band to be adjusted is adaptively changed within a certain level range according to changes in face size. Then, for AC components in frequency bands that were not selected, the amplitude is adjusted using the adjustment parameters generated as shown in Figure 9(b).

[0086] In the above example, the correction method using the adjustment parameter according to Equation (7) is described, but the present invention is not limited to this. For example, a table of the amplitude of the output signal relative to the amplitude of the input signal may be stored, and an output signal with an amplitude corresponding to the amplitude of the input signal may be output.

[0087] 5, the above-described skin correction process is performed on the entire image in S503, and then the signal of the skin region detected in S504 is combined with the signal of the other regions in S505. However, the present invention is not limited to this. The skin region may be detected before the skin correction process, and the skin correction process may be performed only on the detected skin region. In this case, the combining process in S505 is unnecessary.

[0088] As described above, according to this embodiment, it is possible to perform skin correction that leaves a natural impression while retaining the texture of the person's skin.

[0089] <Other embodiments> The present invention may be applied to a system made up of a plurality of devices, or to an apparatus made up of a single device.

[0090] The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program.The present invention can also be realized by a circuit (e.g., ASIC) that realizes one or more functions.

[0091] <Summary> The disclosure of this embodiment includes the following configuration.

[0092] (Item 1) an input means for inputting an image signal; extraction means for extracting a first signal in a predetermined first frequency band from the image signal; an adjusting means for adjusting the amplitude of the first signal so that the amplitude falls within a predetermined range; An image processing device comprising: (Item 2) 2. The image processing device according to item 1, wherein the adjustment means adjusts the amplitude of the first signal that is greater than a predetermined first threshold value so that the amplitude falls within a predetermined range. (Item 3) 3. The image processing device according to item 1 or 2, characterized in that the adjustment means adjusts the amplitude of the first signal to be within the range by attenuating the amplitude of the first signal when the amplitude of the first signal is larger than the range of amplitude, and by amplifying the amplitude of the first signal when the amplitude of the first signal is smaller than the range of amplitude. (Item 4) the extraction means further extracts a second signal in a second frequency band different from the first frequency band from the image signal; 4. The image processing device according to any one of items 1 to 3, wherein the adjustment means further attenuates the amplitude of the second signal. (Item 5) a first detection means for detecting a skin region in an image represented by the image signal; a combining means for combining the image signal and the image signal adjusted by the adjusting means in accordance with the detection result by the first detecting means; 5. The image processing device according to any one of items 1 to 4, further comprising: (Item 6) the first detection means detects a probability that the image represented by the image signal is a skin region; 6. The image processing device according to item 5, wherein the synthesis means performs the synthesis based on the probability. (Item 7) 7. The image processing device according to any one of items 1 to 6, wherein the first frequency band is a high-frequency frequency band. (Item 8) a second detection means for detecting a face area in the image represented by the image signal; a selection means for selecting the first frequency band to be extracted by the extraction means in accordance with the size of the face area detected by the second detection means; 7. The image processing device according to any one of items 1 to 6, further comprising: (Item 9) a first detection means for detecting a skin region in an image represented by the image signal; 5. The image processing device according to any one of items 1 to 4, wherein the extraction means and the adjustment means perform processing on the image signal of the detected skin area. (Item 10) an imaging means for capturing an image and outputting an image signal; An image processing device according to any one of items 1 to 9, An electronic device comprising: (Item 11) an input step of inputting an image signal; an extraction step of extracting a first signal in a predetermined first frequency band from the image signal; an adjusting step of adjusting the amplitude of the first signal so that the amplitude falls within a predetermined range; An image processing method comprising: (Item 12) A program for causing a computer to function as each of the means of the image processing device according to any one of items 1 to 9. (Item 13) Item 13. A computer-readable storage medium storing the program described in item 12.

[0093] The invention is not limited to the above-described embodiments, and various changes and modifications can be made without departing from the spirit and scope of the invention. Accordingly, the following claims are appended to apprise the public of the scope of the invention. [Explanation of symbols]

[0094] 100: imaging device, 101: optical system, 102: imaging section, 103: A / D conversion section, 104: image processing section, 105: exposure control section, 106: system control section, 107: operation section, 108: display section, 109: recording section, 110: memory, 201: development processing section, 202: skin correction processing section, 203: skin region detection section, 204: synthesis processing section

Claims

1. an input means for inputting an image signal; extraction means for extracting a first signal in a predetermined first frequency band from the image signal; an adjusting means for adjusting the amplitude of the first signal so that the amplitude falls within a predetermined range; An image processing device comprising:

2. 2. The image processing device according to claim 1, wherein the adjusting means adjusts the amplitude of the first signal that is greater than a predetermined first threshold value so that the amplitude falls within a predetermined range.

3. 2. The image processing device according to claim 1, wherein the adjustment means adjusts the amplitude of the first signal to be within the range by attenuating the amplitude of the first signal when the amplitude of the first signal is larger than the range of amplitude, and by amplifying the amplitude of the first signal when the amplitude of the first signal is smaller than the range of amplitude.

4. the extraction means further extracts a second signal in a second frequency band different from the first frequency band from the image signal; 2. The image processing apparatus according to claim 1, wherein said adjusting means further attenuates the amplitude of said second signal.

5. a first detection means for detecting a skin region in an image represented by the image signal; a combining means for combining the image signal and the image signal adjusted by the adjusting means in accordance with the detection result by the first detecting means; 2. The image processing apparatus according to claim 1, further comprising:

6. the first detection means detects a probability that the image represented by the image signal is a skin region; 6. The image processing apparatus according to claim 5, wherein the combining means performs the combining based on the probability.

7. 2. The image processing apparatus according to claim 1, wherein the first frequency band is a high frequency band.

8. a second detection means for detecting a face area in the image represented by the image signal; a selection means for selecting the first frequency band to be extracted by the extraction means in accordance with the size of the face area detected by the second detection means; 2. The image processing apparatus according to claim 1, further comprising:

9. a first detection means for detecting a skin region in an image represented by the image signal; 2. The image processing device according to claim 1, wherein the extraction means and the adjustment means perform processing on the image signal of the detected skin area.

10. an imaging means for capturing an image and outputting an image signal; An image processing device according to any one of claims 1 to 9; An electronic device comprising:

11. an input step of inputting an image signal; an extraction step of extracting a first signal in a predetermined first frequency band from the image signal; an adjusting step of adjusting the amplitude of the first signal so that the amplitude is within a predetermined range; An image processing method comprising:

12. A program for causing a computer to function as each of the means of the image processing apparatus according to any one of claims 1 to 9.

13. A computer-readable storage medium storing the program according to claim 12.

Images