Camera system, method for skin property analysis, and computer readable medium

Abstract

A camera system for skin property analysis is provided, which has a skin camera and a color camera. The color camera is configured to receive incident light and output a color image signal. The skin camera includes a first image sensor configured to receive incident light and output a color signal, a first color filter including red color filter pixels and blue color filter pixels and disposed on the first image sensor, a band pass filter configured to transmit a specific waveband and disposed on the first color filter, an image processing unit configured to receive the color signal from the first image sensor and generate a first image signal of light filtered through the red color filter pixels and a second image signal of light filtered through the blue color filter pixels, and a combiner configured to receive the first image signal, the second image signal, and the color image signal from the color camera and generate and output a combined image signal by combining the color image signal, the first image signal, and the second image signal.

Description

Technical Field

[0001] Various example embodiments relate to a camera system, a method for skin characteristic analysis, and a non-transitory computer-readable medium storing program instructions for implementing the method. More specifically, they relate to a camera system, method, and medium storing program instructions that provide images illustrating different characteristics of facial skin. Background Technology

[0002] In recent years, color imaging technology has been developed and widely used in applications such as mobile phones. With the development of high-speed / high-capacity data communication and high-speed microprocessors, there is a demand for various functions, such as higher color resolution.

[0003] However, conventional color imaging technology cannot provide small, low-cost systems for skin characteristic analysis and / or skin care.

[0004] In conventional camera systems, analyzing facial skin characteristics requires the use of multiple devices to perform numerous procedures, which leads to an increase in the size and cost of the equipment.

[0005] For example, it is necessary to prepare a camera with a red color filter, take photos of facial skin to obtain a first image, prepare a camera with a blue color filter, take photos of the skin to obtain a second image for analysis, and combine the first image and the second image to acquire a combined image. Summary of the Invention

[0006] This disclosure was made in view of the above circumstances, and therefore, this disclosure provides a small-sized, low-cost camera system for skin characteristic analysis.

[0007] According to one aspect of this disclosure, a camera system for skin characteristic analysis is provided, comprising a skin camera and a color camera, wherein:

[0008] The color camera is configured to receive incident light and output a color image signal; and

[0009] The skin camera includes:

[0010] The first image sensor is configured to receive incident light and output a color signal.

[0011] The first color filter includes red and blue filter pixels and is disposed on the first image sensor.

[0012] A bandpass filter, configured to transmit a specific wavelength band and located on the first color filter.

[0013] An image processing unit is configured to receive the color signal from a first image sensor and generate a first image signal of light filtered through the red filter pixel and a second image signal of light filtered through the blue filter pixel.

[0014] The combiner is configured to receive the first image signal, the second image signal, and the color image signal from the color camera, and to generate a combined image signal and output the combined image signal by combining the color image signal, the first image signal, and the second image signal.

[0015] According to one aspect of this disclosure, a method for skin property analysis is provided, the method comprising:

[0016] Color image signals are acquired using a color camera;

[0017] A first image signal and a second image signal are acquired using a skin camera that includes a first color filter and a bandpass filter. The first color filter has red and blue color filter pixels and is disposed on a first image sensor. The bandpass filter transmits a certain wavelength and is disposed on the first color filter. The first image signal is an image signal of light filtered by the red color filter pixel, and the second image signal is an image signal of light filtered by the blue color filter pixel. A combined image signal is generated by combining the color image signal, the first image signal, and the second image signal.

[0018] According to one aspect of this disclosure, a non-transitory computer-readable medium is provided for storing program instructions for implementing the above-described methods. Attached Figure Description

[0019] These aspects and advantages, and / or other aspects and advantages, of the embodiments of this disclosure will become apparent and more readily understood from the following description taken with reference to the accompanying drawings. In the drawings:

[0020] Figure 1 A photograph showing the characteristics of facial skin is shown;

[0021] Figure 2 It shows Figure 1 A photograph of a cross-sectional view of the skin shown;

[0022] Figure 3 It is a graph showing the relative reflectance ratios of melanin and hemoglobin contained in the skin;

[0023] Figure 4 This is a block diagram schematically illustrating the overall arrangement of a camera system including a skin camera and a color camera according to an embodiment of the present disclosure;

[0024] Figure 5This is a perspective view illustrating the configuration of a green cutoff filter for a skin camera and an image sensor with red / blue color filters included in a camera system according to an embodiment of this disclosure;

[0025] Figure 6 It is a plan view showing the pixel arrangement pattern of the red / blue color filter;

[0026] Figure 7 It is a plan view showing a color filter in which the pixel arrangement pattern included in a color camera is arranged in a Bayer pattern;

[0027] Figure 8 It is a graph showing the transmittance of red and blue light;

[0028] Figure 9 It is a graph showing the transmittance of red, green, and blue light;

[0029] Figure 10A It is a graph showing the transmittance of light filtered through a bandpass filter, such as a green cutoff filter.

[0030] Figure 10B It is a graph showing the sensitivity to red and blue light;

[0031] Figure 10C It is a graph showing the sensitivity of red and blue light filtered through a bandpass filter, such as a green cutoff filter;

[0032] Figure 11 It is a graph showing the sensitivity to red, green, and blue light;

[0033] Figure 12 This is a flowchart illustrating a skin camera and a color camera according to an embodiment of the present disclosure;

[0034] Figure 13 The image shows a photograph of facial skin taken with a color camera;

[0035] Figure 14 This shows photographs of facial skin captured by a skin camera, in which the weight of the color image under red light is greater than the weight of the color image under blue light; and

[0036] Figure 15 This shows photographs of facial skin taken in a skin camera, where the weight of the color image under blue light is greater than the weight of the color image under red light. Detailed Implementation

[0037] Embodiments of this disclosure will be described in detail, and examples of embodiments will be shown in the accompanying drawings. Throughout the description, the same or similar elements and components having the same or similar functions are indicated by the same reference numerals. The embodiments described herein with reference to the accompanying drawings are illustrative and intended to illustrate this disclosure, and should not be construed as limiting this disclosure.

[0038] like Figure 1 As shown, facial skin has specific reflective properties depending on the wavelength of light.

[0039] Color image 1 shows facial skin under normal lighting conditions.

[0040] Color image 2 shows the same skin under blue light (e.g., 450 nm). It clearly shows wrinkles, freckles, and / or age spots on the skin.

[0041] Color image 3 shows skin under green light (e.g., 550 nm). Compared to color image 2, it does not clearly show wrinkles, freckles, and / or age spots.

[0042] Color image 4 shows skin under red light (e.g., 650 nm). It shows a flat, smooth, and clean skin image where wrinkles, freckles, and / or age spots are not noticeable compared to color images 1 and 2.

[0043] Figure 2 Cross-sectional views of the same skin under the three lighting conditions described above are shown.

[0044] Color image 11 shows a cross-sectional view of skin under blue light (e.g., 450 nm). It shows the reflection of blue light at the skin surface. Wrinkles, freckles, and / or age spots on the skin surface are very noticeable.

[0045] Color image 12 shows a cross-sectional view of the skin under green light (e.g., 550 nm). Compared to color image 11, this color image shows green light reflecting from the surface to intermediate depths at the middle layer. Wrinkles, freckles, and / or age spots are less noticeable compared to color image 11.

[0046] Color image 13 shows a cross-sectional view of the skin under red light (e.g., 650 nm). It shows the reflection of red light from the surface to the deeper layers. Wrinkles, freckles, and / or age spots are less noticeable compared to color images 11 and 12.

[0047] Figure 3 The characteristics of the skin, including the relative reflectance ratios of hemoglobin and melanin, are shown depending on the wavelength of light.

[0048] Hemoglobin has a higher reflectivity than melanin. Within a certain bandwidth, such as 500 to 600 nm, the reflectivity of hemoglobin is lower. Therefore, melanin can stand out within this bandwidth.

[0049] Therefore, within this bandwidth, skin images with prominent wrinkles, freckles, and / or pigmentation are obtained.

[0050] In contrast, hemoglobin has a higher reflectivity than melanin at wavelengths above 600 nm, thus clean skin images with less noticeable wrinkles, freckles, and / or age spots can be obtained within this bandwidth.

[0051] Reference Figure 4 According to one embodiment of the present disclosure, a camera system 100 includes a skin camera 101 and a color camera 201.

[0052] The skin camera 101 includes: an optical system 111; a sensor unit 112 including an image sensor, a color filter and a bandpass filter; an image processing unit 113; a combiner including a weight setter and a first adder 114; and a second adder 115.

[0053] The optical system 111 includes: at least one optical lens, preferably a lens group having multiple optical lenses; an aperture adjustment mechanism; a zoom mechanism; and an autofocus mechanism. The optical system 111 outputs incident light 121 through the optical lens.

[0054] The sensor unit 112 includes: an image sensor, such as a charge-coupled device (CCD) or complementary metal-oxide-semiconductor (CMOS) image sensor having multiple photoelectric conversion elements; a color filter including red and blue color filter pixels disposed on the image sensor; and a bandpass filter disposed on the color filter.

[0055] The sensor unit 112 receives incident light 121 from the optical system 111 and outputs a color signal 122 including red and blue signals.

[0056] The image processing unit 113 receives red and blue color signals 122, performs specific processing such as interpolation as described below, performs predetermined processing such as white balance and color correction, and outputs a color image signal 123 including red and blue image signals.

[0057] The weight setter and the first adder 114 receive the red image signal and the blue image signal, set the first weight coefficient of the red image signal and the second weight coefficient of the blue image signal, output the weighted red image signal and the weighted blue image signal, add the weighted red image signal and the weighted blue image signal, and output the image signal after addition.

[0058] The color camera 201 includes: an optical system 211; a sensor unit 212 including an image sensor and a color filter; and an image processing unit 213.

[0059] The optical system 211 includes: at least one optical lens, preferably a lens group having multiple optical lenses; an aperture adjustment mechanism; a zoom mechanism; and an autofocus mechanism. The optical system 211 outputs incident light 221 through the optical lens.

[0060] The sensor unit 212 includes: an image sensor, such as a CCD or CMOS image sensor having multiple photoelectric conversion elements; and a color filter including red, green and blue filter pixels, which is disposed on the image sensor.

[0061] The sensor unit 212 receives incident light 221 from the optical system 211 and outputs a color signal 222 including red, green and blue signals.

[0062] The image processing unit 213 receives red, green and blue color signals 222, performs predetermined processing such as white balance and color correction, and outputs a color image signal 223 including red image signal, green image signal and blue image signal.

[0063] The second adder 115 of the skin camera 101 receives the summed image signal from the weight setter and the first adder 114 and the color image signal 223 from the image processing unit 213 of the color camera 201. It adds the summed image signal and the color image signal and outputs the combined image signal skin characteristics for skin characteristic analysis.

[0064] Camera system 100 is used for both still and moving images.

[0065] exist Figure 5 The perspective view shows the color filter 21 and bandpass filter 22 of the sensor unit 112.

[0066] A color filter 21 is disposed on an image sensor (not shown), and a bandpass filter 22 is disposed on the color filter 21. The bandpass filter 22 may, for example, be disposed outside the color filter 21.

[0067] like Figure 6As shown, color filter 21 includes red and blue color filter pixels, and they can be arranged in a checkerboard pattern. Color filter 21 may include only red and blue color filter pixels, excluding other color filter pixels such as green color filter pixels.

[0068] like Figure 7 As shown, the color filters 23 included in the sensor unit 212 of the color camera 201 are arranged in a Bayer configuration and include red, green, and blue color filter pixels. As described above, red and blue color filter pixels are required when analyzing skin characteristics.

[0069] In contrast, the green filter pixels are not used for skin characteristic analysis. Therefore, since color filter 21 only includes red and blue filter pixels, it can achieve a higher image resolution for skin characteristic analysis compared to color filter 23 of color camera 201, which includes red, green, and blue filter pixels.

[0070] like Figure 8 As shown, blue light has higher transmittance (sensitivity) in low wavelengths, such as below 490 nm, while red light has higher transmittance in high wavelengths, such as above 590 nm.

[0071] Figure 9 The sensitivity of red light obtained by a conventional color filter 23 arranged in a Bayer configuration with red, green and blue color filters is shown.

[0072] As mentioned above, when acquiring images of smooth, clean skin without wrinkles, freckles, and / or blemishes, light with a wavelength greater than 590 nm is required.

[0073] However, as Figure 9 As shown, red light exhibits a certain level of sensitivity in the wavelength range of 570 (nm) to 590 (nm). Therefore, it is necessary to cut off this red light wavelength range between 570 (nm) and 590 (nm).

[0074] Because of the use of color filter 21, the band that hinders skin characteristic analysis, especially the green light region, can be cut off by using, for example, a green cutoff filter that cuts off green light as a bandpass filter 22.

[0075] like Figure 10A As shown, bandpass filter 22 cuts off the band between 490 (nm) and 590 (nm). It is also desirable to cut off wavelengths greater than 690 (nm) to obtain images unaffected by infrared radiation, which can hinder skin characterization.

[0076] It can be obtained through color filter 21 Figure 10BThe sensitivity of blue and red light is shown.

[0077] like Figure 10C As shown, by using a bandpass filter 22 set on the color filter 21, the sensitivity of blue light and red light, in which the band between 490 (nm) and 590 (nm) is cut off, is obtained.

[0078] Manufacturing with Figure 10C Achieving the required sensitivity with a color filter is quite difficult. According to one embodiment of this disclosure, such sensitivities can be easily obtained at low cost by combining color filter 21 and bandpass filter 22.

[0079] In addition, such as Figure 11 As shown, images can be easily obtained under blue light (390 nm–450 nm) clearly showing wrinkles, freckles, and / or age spots, and under red light (590 nm–650 nm) showing clean skin that is essentially free of wrinkles, freckles, and / or age spots, as shown. Figure 11 As shown, these two images exhibit two different specific skin characteristics.

[0080] exist Figure 12 The diagram shows the process executed by the camera system 100.

[0081] The sensor unit 112 of the skin camera 101 includes a color filter 21, which has red and blue color filter pixels, and as described above, they can be arranged in a checkerboard pattern. The red color filter pixels, which constitute half of the color pixels, form one unit of the color filter 21, and the other half is composed of blue color filter pixels. Therefore, a red signal is generated for one half of the unit, and a blue signal is generated for the other half.

[0082] Interpolate the red and blue signals separately to obtain a red signal for one unit and a blue signal for one unit.

[0083] Image 301 with red signal shows clean skin without wrinkles, freckles, and / or age spots, while image 302 with blue signal clearly shows wrinkles, freckles, and / or age spots.

[0084] Each of the red and blue signals is filtered, for example, by a high-pass filter or a band-pass filter. The filtered image 303 of the red signal has a fine or smooth texture, while the filtered image 304 of the blue signal has a coarse texture.

[0085] The weight setter and the first adder 114 include a first weight setter 114a, a second weight setter 114b, and a first adder 114c.

[0086] The first weight setter 114a is, for example, an amplifier, which sets a first weight coefficient W_y (Weight_young, representing "young skin"), receives the filtered image of the red signal, and outputs the weighted image of the red signal.

[0087] The second weight setter 114b is, for example, an amplifier that sets a second weight coefficient W_o (Weight_old, representing "older skin"), receives the filtered image of the blue signal, and outputs the weighted image of the blue signal.

[0088] The first adder 114c receives the weighted image of the red signal and the weighted image of the blue signal, adds them together, and outputs the image resulting from the addition of the red and blue signals.

[0089] The sensor unit 212 of the color camera 201 includes a color filter 23 arranged in a Bayer configuration, having red, green and blue color filter pixels.

[0090] Image processing unit 213 receives red, green, and blue color signals, performs predetermined processing such as interpolation, white balance, and color correction, and outputs red, green, and blue color signals. Image 311 of the red, green, and blue color signals shows the original color image of the skin. If necessary, the image is filtered, for example, by a low-pass filter. Filtered image 312 shows that some skin texture has been removed. Filtered image of red, green, and blue color signals is output from color camera 201.

[0091] The second adder 115 receives the image resulting from the addition of the red and blue signals from the first adder 114c, as well as the image of the red, green, and blue color signals from the color camera 201, adds them together, and outputs the combined image of the red, green, and blue color signals.

[0092] When the first weighting coefficient W_y (Weight_young) is greater than the second weighting coefficient W_o (Weight_old), the color image 321 shows clean skin (similar to young skin) without wrinkles, freckles and / or age spots.

[0093] When the second weighting coefficient W_o (Weight_old) is greater than the first weighting coefficient W_y (Weight_young), the color image 322 clearly shows wrinkles, freckles, and / or age spots (similar to aged skin).

[0094] It should be noted that, Figure 12The process shown can be implemented by the hardware of the camera system 100 or by program instructions installed in the camera system 100.

[0095] Figure 13 The original color image output from the color camera 201 is shown.

[0096] Figure 14 The image shown is a color image output from camera system 100 when the first weighting coefficient W_y (Weight_young) is greater than the second weighting coefficient W_o (Weight_old). This image demonstrates a reduction in wrinkles, freckles, and / or age spots that were clearly present in the original color image.

[0097] Figure 15 The image shown is a color image output from camera system 100 when the second weighting coefficient W_o (Weight_old) is greater than the first weighting coefficient W_y (Weight_young). Wrinkles, freckles, and / or blemishes that are not clearly visible in the original color image are enhanced.

[0098] For example, when the second weighting coefficient W_o (Weight_old) is greater than the first weighting coefficient W_y (Weight_young), the image clearly shows wrinkles, freckles, and / or age spots. The image can be used to analyze the poor condition of a person's facial skin, and images including obvious wrinkles, freckles, and / or age spots can be used to illustrate the effects of improper skin care.

[0099] Furthermore, when the first weighting coefficient W_y (Weight_young) is greater than the second weighting coefficient W_o (Weight_old), wrinkles, freckles, and / or age spots will not be clearly shown in the image. This image can be used to analyze the poor condition of a person's facial skin and to teach people how to take good skin care steps to achieve clean skin similar to that shown in the image.

[0100] Furthermore, the camera system and method for skin characteristic analysis according to the above embodiments of this disclosure can be used for shooting in beauty mode.

[0101] In the embodiments described above in this disclosure, the analysis of facial skin characteristics was described. However, the analysis described above is not limited to facial skin. To obtain skin that can be properly cared for, skin in other areas can also be analyzed.

[0102] In the description of embodiments of this disclosure, it should be understood that terms such as “center,” “longitudinal,” “lateral,” “length,” “width,” “thickness,” “upper,” “lower,” “front,” “rear,” “left,” “right,” “vertical,” “horizontal,” “top,” “bottom,” “inner,” “outer,” “clockwise,” and “counterclockwise” should be interpreted as referring to the direction or position as described or shown in the discussed drawings. These relative terms are used only to simplify the description of this disclosure and do not indicate or imply that the mentioned devices or elements must have a particular orientation, or be constructed or operated in a particular orientation. Therefore, these terms should not be construed as limiting this disclosure.

[0103] Furthermore, terms such as “first” and “second” are used herein for descriptive purposes and are not intended to indicate or imply relative importance or significance or to imply the number of technical features indicated. Therefore, a feature defined by “first” and “second” may include one or more of that feature. In the description of this disclosure, unless otherwise stated, “a plurality” means two or more.

[0104] In the description of the embodiments of this disclosure, unless otherwise stated or limited, the terms "mounted," "connected," "coupled," etc. are used extensively and can be, for example, a fixed connection, a detachable connection, or an integral connection between two elements; it can also be a mechanical connection or an electrical connection; it can also be a direct connection or an indirect connection through an intermediate structure; it can also be internal communication, as those skilled in the art will understand according to the specific circumstances.

[0105] In embodiments of this disclosure, unless otherwise stated or limited, a structure in which the first feature is located "above" or "below" the second feature can include embodiments in which the first feature and the second feature are in direct contact, or embodiments in which the first feature and the second feature are not in direct contact with each other, but are in contact through other features formed between them. Furthermore, "above," "on top of," or "on the second feature" can include embodiments in which the first feature is exactly or obliquely above, above, or "on top of" the second feature, or simply means that the height of the first feature is greater than the height of the second feature; while "below," "below," or "at the bottom of," the first feature can include embodiments in which the first feature is exactly or obliquely below, below, or "at the bottom of," the second feature, or simply means that the height of the first feature is less than the height of the second feature.

[0106] Various embodiments and examples have been provided in the foregoing description to implement different structures of this disclosure. Certain elements and arrangements have been described above to simplify this disclosure. However, these elements and arrangements are merely examples and are not intended to limit this disclosure. Additionally, reference numerals and / or letters may be repeated in different examples of this disclosure. Such repetition is for the purpose of simplicity and clarity and does not indicate a relationship between different embodiments and / or arrangements. Furthermore, examples of different processes and materials are provided in this disclosure. However, those skilled in the art will understand that other processes and / or materials may also be applied.

[0107] In this specification, references to "embodiment," "some embodiments," "exemplary embodiments," "example," "specific example," or "some examples" mean that a particular feature, structure, material, or characteristic described in connection with an embodiment or example is included in at least one embodiment or example of this disclosure. Therefore, the appearance of these phrases in this specification does not necessarily refer to the same embodiment or example of this disclosure. Furthermore, in one or more embodiments or examples, a particular feature, structure, material, or characteristic may be combined in any suitable manner.

[0108] Although embodiments of the present disclosure have been shown and described, those skilled in the art will understand that these embodiments are illustrative and should not be construed as limiting the present disclosure, and that changes, modifications, substitutions and variations may be made in the embodiments without departing from the scope of the present disclosure.

Claims

1. A camera system for skin characteristic analysis, comprising a skin camera and a color camera, wherein: The color camera is configured to receive incident light and output a color image signal; as well as The skin camera includes: A first image sensor, configured to receive incident light and output a color signal; A first color filter, comprising a red color filter pixel and a blue color filter pixel and disposed on the first image sensor; A bandpass filter, configured to transmit a specific wavelength band and disposed on the first color filter; the bandpass filter is a green cutoff filter for blocking green light; An image processing unit, configured to receive the color signal from the first image sensor and generate a first image signal of light filtered through the red filter pixels and a second image signal of light filtered through the blue filter pixels; and A combiner configured to receive a first image signal, a second image signal, and a color image signal from the color camera, and to generate a combined image signal by combining the color image signal, the first image signal, and the second image signal, and to output the combined image signal.

2. The camera system according to claim 1, wherein, The combiner includes: A weight setter is configured to: set a first weight coefficient for the first image signal and a second weight coefficient for the second image signal, receive the first image signal and the second image signal, and output a weighted first image signal and a weighted second image signal; A first adder, configured to: receive the weighted first image signal and the weighted second image signal, and output a third image signal by adding the weighted first image signal and the weighted second image signal; and A second adder is configured to receive the color image signal from the color camera and the third image signal from the first adder, and to generate the combined image signal by combining the color image signal and the third image signal.

3. The camera system according to claim 1 or 2, wherein, The color camera includes a second color filter, which comprises red, blue, and green filter pixels and is disposed on a second image sensor.

4. The camera system according to claim 1 or 2, wherein, The red and blue filter pixels of the first color filter are arranged in a checkerboard pattern.

5. A method for skin characteristic analysis, applied to a camera system as described in any one of claims 1-4; the method comprising: Color image signals are acquired using a color camera; A skin camera, comprising a first image sensor, a first color filter, a bandpass filter, and an image processing unit, acquires a first image signal and a second image signal. The first image sensor acquires a color signal. The first color filter has red and blue filter pixels and is disposed on the first image sensor. The bandpass filter transmits a certain wavelength and is disposed on the first color filter. The bandpass filter is a green cutoff filter used to block green light. The image processing unit receives the color signal from the first image sensor and generates a first image signal of light filtered through the red filter pixels and a second image signal of light filtered through the blue filter pixels. A combined image signal is generated by combining the color image signal, the first image signal, and the second image signal using a combiner, which is included in the skin camera.

6. The method according to claim 5, further comprising: A first weighting coefficient for the first image signal and a second weighting coefficient for the second image signal are set using a skin camera. A weighted first image signal is generated using the first weighting coefficient, and a weighted second image signal is generated using the second weighting coefficient; and the color image signal, the weighted first image signal, and the weighted second image signal are combined to obtain the combined image signal.

7. The method according to claim 5 or 6, wherein, The color camera includes a second color filter, which comprises red, blue, and green filter pixels and is disposed on a second image sensor.

8. The method according to claim 5 or 6, wherein, The red and blue filter pixels of the first color filter are arranged in a checkerboard pattern.

9. A non-transitory computer-readable medium having a program stored thereon that, when executed by a processor, implements the method according to any one of claims 6 to 8.

Images