Image processing device, image processing method, image processing program, and recording medium

Abstract

To provide an image processing device which can visualize a living body surface such as a skin surface of a human with high accuracy in comparison with a conventional technique.SOLUTION: An image processing device according to the present invention comprises: an image sensor which images a surface of a processing object to generate polarization image data and non-polarization image data; a first calculation unit which calculates and outputs a ratio of luminance or a difference of luminance for each pixel between the polarization image data and the non-polarization image data; a background extraction processing unit which extracts and outputs background image data related to a background image from image data being any of (1) image data including the ratio of luminance for each pixel from the first calculation unit, (2) image data including the difference of luminance for each pixel from the first calculation unit, (3) the polarization image data and (4) the non-polarization image data; and a second calculation unit which subtracts the background image data from the background extraction processing unit from the image data including the ratio of luminance or the difference of luminance for each pixel from the first calculation unit to generate image data including the difference for each pixel.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to an image processing apparatus, an image processing method, an image processing program, and a recording medium storing the image processing program for visualizing, for example, the uneven state of a living body surface.

Background Art

[0002] For example, the skin of a human (human body) is an organ that medically protects the body from external factors and plays a role in maintaining homeostasis, but the evaluation of its properties is also highly regarded from a cosmetic perspective. The properties of the skin are expressed in various terms such as makeup texture, dullness, or transparency. However, different from general dermatological diagnosis for diagnosing pathological skin findings, it is difficult to quantitatively capture subtle changes in the skin that are included in the normal category from a cosmetic perspective, and currently, no practical method has been established.

[0003] When observing the human skin magnified, a pattern of uneven shapes of fine patterns, so-called "wrinkles" or "texture," can be seen on the surface.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Non-Patent Documents

[0005]

Non-Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0006] The surface of human skin contains numerous wrinkles and textures, which are grid-like grooves approximately 10-20 μm deep. While visual inspection has traditionally been used to easily obtain the distribution of these wrinkles and textures, issues of accuracy and cost necessitate a method for automatically acquiring this information with high accuracy and at low cost. Although it is common to photograph the skin surface with an RGB camera to easily acquire skin information, accurately visualizing wrinkles and textures using only RGB images (e.g., unpolarized images) is difficult because their colors closely resemble the surrounding areas.

[0007] For example, a conventional imaging device has been disclosed for accurately visualizing the skin surface (see, for example, Patent Document 1). This non-invasive imaging device system and method includes an illumination source including an incident light source that guides light onto the skin, and a detector for detecting the degree of polarization of the light reflected from the skin. Furthermore, the system and method for determining the skin condition is based on one aspect of the polarization of the reflected light. That is, the conventional imaging device discloses determining the skin condition based on the polarization of the reflected light.

[0008] However, even with the conventional imaging device in question, there was a problem in that it was not possible to visualize the surface irregularities of the object being photographed, such as the condition of human skin, with high precision.

[0009] The first object of the present invention is to solve the above problems and to provide an image processing apparatus, an image processing method, an image processing program, and a recording medium storing the image processing program that can visualize the surface irregularities of an object to be photographed, such as a biological surface like the surface of human skin, with higher accuracy than the prior art.

[0010] A second object of the present invention is to provide an image processing device, an image processing method, an image processing program, and a recording medium storing the image processing program that can analyze the surface irregularities of an object to be photographed, such as a biological surface like the surface of human skin, with higher accuracy than the prior art, using predetermined feature quantities. [Means for solving the problem]

[0011] An image processing apparatus according to a first aspect of the present invention is: An image sensor that captures images of the surface of an object to be processed and generates polarized image data and unpolarized image data, A first calculation unit that calculates and outputs the ratio of brightness or the difference in brightness for each pixel between the polarized image data and the unpolarized image data, (1) Image data from the first calculation unit including the ratio of brightness for each pixel, (2) Image data from the first calculation unit including the difference in brightness for each pixel, (3) Polarization image data and, (4) The non-polarized image data and, A background extraction processing unit that extracts and outputs background image data related to the background image from any of the image data, The system includes a second calculation unit that generates image data including the difference for each pixel by subtracting background image data from the background extraction processing unit from image data including the ratio or difference in brightness for each pixel from the image data from the first calculation unit.

[0012] An image processing apparatus according to a second aspect of the present invention is an image processing apparatus according to the first aspect of the present invention, The system further includes a feature analysis unit that calculates Gapore features by changing at least one of angle and period on image data from the post-processing unit using a Gapore filter, thereby generating an angular distribution of luminance values, a periodic distribution of luminance values, and a luminance value distribution in a two-dimensional plane of angle and period. [Effects of the Invention]

[0013] Therefore, according to the first aspect of the present invention, the image processing device, etc., can visualize the surface irregularities of an object to be photographed, such as a biological surface like the surface of human skin, with higher accuracy than the conventional technology.

[0014] Also, according to the image processing apparatus and the like according to the second aspect of the present invention, the uneven state of the surface of a photographing object such as a living body surface such as a human skin surface can be analyzed with high accuracy using predetermined feature amounts as compared with the prior art.

Brief Description of the Drawings

[0015] [Figure 1] It is a block diagram showing a configuration example of an image processing system according to Embodiment 1. [Figure 2] It is a flowchart showing the image processing executed by the image processing unit 12 in FIG. 1. [Figure 3A] It is a captured image showing the skin of a human (human body) that is an image processing target and has a smooth skin surface. [Figure 3B] It is a captured image showing the skin of a human (human body) that is an image processing target and has a disrupted skin surface. [Figure 4A] It is a captured image showing a polarized image captured by the camera 1 in FIG. 1. [Figure 4B] It is a captured image showing a non-polarized image captured by the camera 1 in FIG. 1. [Figure 4C] It is a captured image showing a UV image captured by another ultraviolet (UV) camera. [Figure 5A] It is a captured image showing the polarized image in FIG. 4A. [Figure 5B] It is a captured image showing the image of the R (red) channel of the polarized image in FIG. 4A. [Figure 5C] It is a captured image showing the image of the G (green) channel of the polarized image in FIG. 4A. [Figure 5D] It is a captured image showing the image of the B (blue) channel of the polarized image in FIG. 4A. [Figure 6A] It is a captured image showing the non-polarized image in FIG. 4B. [Figure 6B] It is a captured image showing the image of the R (red) channel of the non-polarized image in FIG. 4B. [Figure 6C] It is a captured image showing the image of the G (green) channel of the non-polarized image in FIG. 4B. [Figure 6D]Figure 4B is an image showing the B (blue) channel of the unpolarized image. [Figure 7A] This is an image captured for the purpose of visualizing wrinkles or skin texture, and it shows the B channel image of a polarized image. [Figure 7B] This is an image captured for the purpose of visualizing wrinkles or skin texture, and it shows the B channel image of an unpolarized image. [Figure 8] Figure 2 is a conceptual diagram of image processing, illustrating the image processing using the captured image and the processed image. [Figure 9A] This is an image related to Visualization Example 1, which is an captured image showing the polarization image to be processed. [Figure 9B] This is an image related to Visualization Example 1, and is an captured image showing the unpolarized image to be processed. [Figure 9C] This image relates to Visualization Example 1 and is the visualization image after image processing for Figures 2 and 8. [Figure 10A] This is an image related to Visualization Example 2, which is an captured image showing the polarization image to be processed. [Figure 10B] This is an image related to Visualization Example 2, and is an captured image showing the unpolarized image to be processed. [Figure 10C] This image relates to Visualization Example 2 and is the visualization image after image processing for Figures 2 and 8. [Figure 11A] This is an image related to Visualization Example 3, and is an captured image showing the polarization image to be processed. [Figure 11B] This is an image related to Visualization Example 3, and is an captured image showing the unpolarized image to be processed. [Figure 11C] This image relates to Visualization Example 3 and is the visualization image after image processing for Figures 2 and 8. [Figure 12A] This is an image related to Visualization Example 4, which is an captured image showing the polarization image to be processed. [Figure 12B] This is an image related to Visualization Example 4, and is an captured image showing the unpolarized image to be processed. [Figure 12C]This image relates to Visualization Example 4 and is the visualization image after image processing for Figures 2 and 8. [Figure 13A] This is an image related to Visualization Example 5, and is an captured image showing the polarization image to be processed. [Figure 13B] This is an image related to Visualization Example 5, and is an captured image showing the unpolarized image to be processed. [Figure 13C] This image relates to Visualization Example 5 and is the visualization image after image processing for Figures 2 and 8. [Figure 14A] This is an image related to Visualization Example 6, and is an captured image showing the polarization image to be processed. [Figure 14B] This is an image related to Visualization Example 6, and is an captured image showing the unpolarized image to be processed. [Figure 14C] This image relates to Visualization Example 6 and is the visualization image after image processing for Figures 2 and 8. [Figure 15A] This is an image related to Visualization Example 7, and is an captured image showing the polarization image to be processed. [Figure 15B] This is an image related to Visualization Example 7, and is an captured image showing the unpolarized image to be processed. [Figure 15C] This image relates to Visualization Example 7 and is the visualization image after image processing for Figures 2 and 8. [Figure 16] This is a block diagram showing an example configuration of an image processing system according to Embodiment 2. [Figure 17] Figure 16 is a flowchart showing the image processing performed by the image processing unit 12A. [Figure 18A] Figure 16 is a diagram illustrating the concept of the one-dimensional Gapole function used in the feature analysis unit 27 of the image processing unit 12A. [Figure 18B] Figure 16 is a diagram illustrating the concept of the two-dimensional Gapole function used in the feature analysis unit 27 of the image processing unit 12A. [Figure 19A] This image shows analysis example 1 (directionality) of the feature analysis unit 27, and is a visualization image (DWE (Dermo Wrinkle Emphasis) image) before linear filtering. [Figure 19B] This image shows an example of analysis 1 (directionality) by the feature analysis unit 27, and is an analysis processed image after linear filtering. [Figure 20A] This graph shows an example of analysis 2 (directionality) by the feature analysis unit 27, and is a graph showing the angular distribution of the average brightness after linear filtering. [Figure 20B] This graph shows analysis example 2 (directionality) of the feature analysis unit 27, and is the analyzed image at the time of maximum brightness Lmax after linear filtering. [Figure 20C] This graph shows an example of analysis 2 (directionality) by the feature analysis unit 27, and is the analyzed image at the minimum brightness Lmin after linear filtering. [Figure 21A] This image shows analysis example 3 (periodicity) of the feature analysis unit 27, and is a visualization image (DWE image) before linear filtering. [Figure 21B] This image shows analysis example 3 (periodicity) of the feature analysis unit 27, and is an analysis processed image after linear filtering. [Figure 22A] This graph shows analysis example 4 (periodicity) of the feature analysis unit 27, and is a graph showing the periodic distribution of the average brightness after linear filtering. [Figure 22B] This graph shows analysis example 4 (periodicity) of the feature analysis unit 27, and is the analyzed image at the time of maximum brightness Lmax after linear filtering. [Figure 23] This graph shows analysis example 5 (directivity and periodicity) of the feature analysis unit 27, and is a graph image in which brightness values ​​are represented in contour heatmap format on a two-dimensional plane of period with respect to angle of direction. [Figure 24A] This image shows an example of analysis 5 (directivity and periodicity) by the feature analysis unit 27, and is an RGB image before processing (shown as a black and white image for patent purposes). [Figure 24B] This image shows analysis example 5 (directivity and periodicity) of the feature analysis unit 27, and is a visualized image (DWE image) after visualization processing. [Figure 24C]This graph shows analysis example 5 (directivity and periodicity) of the feature analysis unit 27, and is a graph image in which brightness values ​​are represented in contour heatmap format on a two-dimensional plane of period with respect to angle of direction. [Figure 24D] This graph shows analysis example 5 (directivity and periodicity) of the feature analysis unit 27, and is a graph showing the angular distribution of the average brightness after linear filtering. [Figure 24E] This graph shows analysis example 5 (directivity and periodicity) of the feature analysis unit 27, and is a graph showing the periodic distribution of the average brightness after linear filtering. [Figure 25A] This image shows an example of analysis 6 (directivity and periodicity) by the feature analysis unit 27, and is an RGB image before processing. [Figure 25B] This image shows analysis example 6 (directivity and periodicity) of the feature analysis unit 27, and is a visualized image (DWE image) after visualization processing. [Figure 25C] This graph shows analysis example 6 (directivity and periodicity) of the feature analysis unit 27, and is a graph image in which brightness values ​​are represented in contour heatmap format on a two-dimensional plane of period with respect to angle of direction. [Figure 25D] This graph shows analysis example 6 (directivity and periodicity) of the feature analysis unit 27, and is a graph showing the angular distribution of the average brightness after linear filtering. [Figure 25E] This graph shows analysis example 6 (directivity and periodicity) of the feature analysis unit 27, and is a graph showing the periodic distribution of the average brightness after linear filtering. [Figure 26A] This image shows an example of analysis 7 (directivity and periodicity) by the feature analysis unit 27, and is an RGB image before processing. [Figure 26B] This image shows an example of analysis 7 (directivity and periodicity) by the feature analysis unit 27, and is a visualized image (DWE image) after visualization processing. [Figure 26C] This graph shows an example of analysis 7 (directivity and periodicity) of the feature analysis unit 27, and is a graph image in which brightness values ​​are represented in contour heatmap format on a two-dimensional plane of period with respect to angle of direction. [Figure 26D]This graph shows analysis example 7 (directivity and periodicity) of the feature analysis unit 27, and is a graph showing the angular distribution of the average brightness after linear filtering. [Figure 26E] This graph shows an example of analysis 7 (directivity and periodicity) of the feature analysis unit 27, and is a graph showing the periodic distribution of the average brightness after linear filtering. [Figure 27A] This image shows an example of analysis 8 (directivity and periodicity) by the feature analysis unit 27, and is an RGB image before processing. [Figure 27B] This image shows analysis example 8 (directivity and periodicity) of the feature analysis unit 27, and is a visualized image (DWE image) after visualization processing. [Figure 27C] This graph shows an example of analysis 8 (directivity and periodicity) of the feature analysis unit 27, and is a graph image in which the brightness values ​​are represented in contour heatmap format on a two-dimensional plane of period with respect to angle of direction. [Figure 27D] This graph shows analysis example 8 (directivity and periodicity) of the feature analysis unit 27, and is a graph showing the angular distribution of the average brightness after linear filtering. [Figure 27E] This graph shows analysis example 8 (directivity and periodicity) of the feature analysis unit 27, and is a graph showing the periodic distribution of the average brightness after linear filtering. [Figure 28A] This image shows an example of analysis 9 (directivity and periodicity) by the feature analysis unit 27, and is an RGB image before processing. [Figure 28B] This image shows an example of analysis 9 (directivity and periodicity) by the feature analysis unit 27, and is a visualized image (DWE image) after visualization processing. [Figure 28C] This graph shows an example of analysis 9 (directivity and periodicity) of the feature analysis unit 27, and is a graph image in which brightness values ​​are represented in contour heatmap format on a two-dimensional plane of period with respect to angle of direction. [Figure 28D] This graph shows an analysis example 9 (directivity and periodicity) of the feature analysis unit 27, and is a graph showing the angular distribution of the average brightness after linear filtering. [Figure 28E]This graph shows an analysis example 9 (directivity and periodicity) of the feature analysis unit 27, and is a graph showing the periodic distribution of the average brightness after linear filtering. [Figure 29] This is a block diagram showing an example configuration of an image processing system according to Embodiment 3. [Modes for carrying out the invention]

[0016] Embodiments and modified examples of the present invention will be described below with reference to the drawings. The same or similar components are denoted by the same reference numerals.

[0017] (Inventor's insights) For example, to easily obtain skin information such as human skin, it is common to photograph the skin surface with an RGB camera. However, because wrinkles and texture areas are very similar in color to the surrounding areas, it has been difficult to accurately visualize them using only RGB images (e.g., unpolarized images).

[0018] Therefore, in this invention, we attempted to visualize wrinkles and skin texture by processing close-up images of skin taken using a dermatological camera (registered trademark) capable of simultaneously acquiring polarized and unpolarized images, rather than using general RGB camera images. The unpolarized image is a general RGB image that does not take polarization into consideration, while the polarized image is taken by shining polarized light on the target and capturing the light that has passed through a polarizing plate. This image suppresses direct reflection from the skin surface and makes it easier to obtain information about the inner surface of the skin.

[0019] In wrinkles and skin texture, which are grid-like grooves, light reflection occurs in a complex manner, and skin tones differ subtly. Therefore, in unpolarized images, there is a minute difference in brightness between wrinkles / texture and other areas. On the other hand, polarized images mainly capture light scattered from below the surface, so information about wrinkles and texture is less likely to be included in the image. Therefore, the difference between these two images is mainly the presence or absence of direct reflected light from the skin surface, which corresponds to the brightness information due to reflected light in wrinkles and texture. In addition, both polarized and unpolarized images contain RGB information, but due to the optical properties caused by the shape of the minute grooves in wrinkles and texture, and the skin color, the brightness difference in wrinkles and texture is greater in the B channel image compared to the G and R channels. Therefore, only the B channel information is used for visualization processing.

[0020] In the embodiments described below, an image processing device was devised that can accurately visualize wrinkles and texture of the skin using predetermined image processing, with the aim of more accurately diagnosing the health condition of the skin, and then analyze the patterns of wrinkles and texture.

[0021] (Embodiment 1) Figure 1 is a block diagram showing an example configuration of an image processing system according to Embodiment 1. The image signal processing system in Figure 1 comprises a camera 1, an image processing device 10, and a display 2. Here, the image processing device 10 comprises an image interface 11 having an image memory 11m, an image processing unit 12, and an image interface 13 having an image memory 13m. The image processing unit 12 comprises a luminance ratio calculation unit 21, a background extraction processing unit 22, an image memory 23, a luminance difference calculation unit 24, a post-processing unit 25, and a control unit 20 that controls the operation of the processing units 21 to 25.

[0022] In Figure 1, camera 1 is, for example, a DZ-D100 type skin observation and imaging camera manufactured by Casio Computer Co., Ltd., which is a "dermatograph camera" (registered trademark). For example, it captures polarized and unpolarized images of the surface of human skin and outputs these image data to the brightness ratio calculation unit 21 via the image interface 11. Here, the "dermatograph camera" (registered trademark) can capture ultraviolet (UV) images in addition to polarized and unpolarized images, but capturing UV images is not essential in the embodiment of the present invention. That is, camera 1 only needs to be capable of capturing polarized and unpolarized images of the object to be photographed, and the polarized and unpolarized images may be captured by separate image sensors or the like.

[0023] Here, polarized images are generally characterized by their ability to suppress light reflection and capture the color and structure of the skin just beneath the thin layer of skin; unpolarized images are generally characterized by their ability to capture lesions on the surface of the skin; and UV images are generally characterized by their ability to capture the edges of hidden "spots" or blurred "moles" that do not appear in polarized images. The "dermocamera" can capture all three types of images with the same field of view.

[0024] The image interface 11 stores the input image data in its built-in image memory 11m, then performs a predetermined image format conversion on the stored image data and outputs it to the next stage, the luminance ratio calculation unit 21. The image interface 13 stores the input image data in its built-in image memory 13m, then performs a predetermined image format conversion on the stored image data and outputs it to the next stage, the display 2.

[0025] Figure 2 is a flowchart showing the image processing performed by the image processing unit 12 in Figure 1.

[0026] In step S1 of Figure 2, the luminance ratio calculation unit 21 receives polarized and unpolarized image data captured by the camera 1 via the image interface 11, calculates the luminance ratio (ratio of luminance values) of each corresponding pixel in the polarized and unpolarized image data, and generates and outputs image data having a luminance ratio value for each pixel. Next, in step S2, the background extraction processing unit 22 performs background extraction processing on the image data output from the luminance ratio calculation unit 21 using a smoothing filter such as a Gaussian filter, to extract and retain the background color, and generates and outputs image data.

[0027] Furthermore, in step S3, the luminance difference calculation unit 24 calculates the luminance difference (difference in luminance values) of each corresponding pixel between the image data output from the background extraction processing unit 22 and the image data output from the luminance ratio calculation unit 21 via the image memory 23, thereby generating and outputting image data in which the background color is removed and wrinkle or texture information is emphasized, and which has a luminance difference at each pixel. In step S4, the post-processing unit 25 performs post-processing, including binarization and closing processing, on the image data output from the luminance difference calculation unit 24, thereby generating image data in which wrinkle or texture information is visualized, outputting it to the display 2 via the image interface 13 for display, and ending the image processing.

[0028] The luminance ratio calculation unit 21 in Figure 1 calculates the luminance ratio (ratio of luminance values) of each corresponding pixel in the polarized image data and the unpolarized image data. However, the present invention is not limited to this, and may also calculate the luminance difference (difference in luminance values) of each corresponding pixel in the polarized image data and the unpolarized image data. This is because the luminance ratio of each corresponding pixel in the polarized image data and the unpolarized image data may relatively correspond to their difference.

[0029] Next, the image processing described above will be explained in detail below.

[0030] Figure 3A is an image of human skin, the subject of image processing, showing a smooth skin surface. Figure 3B is an image of human skin, the subject of image processing, showing a rough skin surface.

[0031] Factors that can cause loss of skin texture in humans include photoaging due to ultraviolet radiation, dry skin, poor blood circulation, and slowed metabolism. Therefore, information on skin wrinkles and texture is useful for evaluating the health of the skin. Conventional methods for measuring skin wrinkles and texture include the following. (1) Skin wrinkles and texture are measured based on skin images captured with an RGB camera. In this case, the skin has the characteristic of having uneven skin tone, or the color information of the wrinkled and textured areas is similar to that of other areas. (2) Skin wrinkles and texture are measured using 3D data of the skin measured with a 3D scanner. In this case, the measuring instrument is expensive, the measurement time is long, and the amount of data is enormous.

[0032] Therefore, conventional measurement methods have the challenge of being unable to obtain information on skin wrinkles and texture inexpensively and easily.

[0033] Figure 4A shows a polarized image captured by camera 1 in Figure 1. Figure 4B shows a non-polarized image captured by camera 1 in Figure 1. Furthermore, Figure 4C shows a UV image captured by a different ultraviolet (UV) camera. From Figure 4A, it can be seen that the color and structure inside the skin are visible, but wrinkles and texture are not. From Figure 4B, information on the skin surface is visible, and wrinkles and texture are somewhat visible. From Figure 4C, "spots" and "moles" are visible, and "wrinkles" and "texture" are also visible.

[0034] Figure 5A is an image showing the polarized image of Figure 4A, Figure 5B is an image showing the R channel image of the polarized image of Figure 4A, Figure 5C is an image showing the G channel image of the polarized image of Figure 4A, and Figure 5D is an image showing the B channel image of the polarized image of Figure 4A. Figures 5B to 5D are images obtained by separating the polarized image of Figure 4A into images for each of the RGB channels. This shows that even in polarized images where reflected light from the skin surface is suppressed, the B channel image on the short wavelength side contains some information about the skin surface. In other words, in this embodiment, it is preferable to detect the unevenness of the skin surface of the human body using the B channel image, and it is preferable to use polarized image data and unpolarized image data of the B channel when inputting image data to the luminance ratio calculation unit 21 in Figure 1.

[0035] Figure 6A is an image showing the unpolarized image of Figure 4B, Figure 6B is an image showing the R channel of the unpolarized image of Figure 4B, Figure 6C is an image showing the G channel of the unpolarized image of Figure 4B, and Figure 6D is an image showing the B channel of the unpolarized image of Figure 4B. Figures 6B to 6D are images obtained by separating the unpolarized image of Figure 4B into images for each of the RGB channels. From this, it was found that the unpolarized image contains more information about the skin surface than the polarized image. In other words, in this embodiment, it is preferable to detect the unevenness of the skin surface of the human body using the unpolarized image, and it is preferable that the image processing unit 12 in Figure 1 is configured to extract information from the unpolarized image data.

[0036] Figure 7A is an image captured for the purpose of considering the visualization of wrinkles or skin texture, and shows the B channel image of a polarized image. Figure 7B is also an image captured for the purpose of considering the visualization of wrinkles or skin texture, and shows the B channel image of an unpolarized image. In the B channel image of the polarized image in Figure 7A, there is little information about wrinkles and skin texture, but it can be said that there is sufficient color information about the inside of the skin. On the other hand, the B channel image of the unpolarized image in Figure 7B contains information about wrinkles and skin texture, and also contains sufficient color information about the inside of the skin. Therefore, in this embodiment, wrinkles and skin texture are visualized from the B channel information of the polarized image and the unpolarized image. Note that UV images are unsuitable for visualizing wrinkles and skin texture due to the presence of blemish information other than wrinkles and skin texture.

[0037] Figure 8 is a conceptual diagram of the image processing shown in Figure 2, using the captured image and the processed image.

[0038] In Figure 8, first, the luminance ratio image of each pixel in the B channel of the polarized and unpolarized images is calculated. Areas other than wrinkles and texture are very similar in the polarized and unpolarized images, so the luminance ratio is small. However, the luminance ratio is large in the wrinkle and texture areas, so wrinkles and texture are emphasized in the luminance ratio image. Next, in order to remove skin color unevenness and shadows caused by the way incident light hits the luminance ratio image, a background extraction process using a smoothing filter such as a Gaussian filter is applied to the luminance ratio image. Then, the difference between the luminance ratio images before and after the application is taken to generate an image in which only the wrinkle and texture areas are significantly emphasized. Finally, by applying post-processing including binarization and closing processes, a visualization image of wrinkles and texture (DWE (Dermo Wrinkle Emphasis) image) can be generated.

[0039] Furthermore, the images of reflected and scattered light used in the image processing device 10 described above will be explained in more detail below.

[0040] An unpolarized image is an image of general light, captured when the image sensor of camera 1 receives reflected light reflected from the surface of the object and scattered light scattered from the surface layer of the object, generating an image signal containing image data. A polarized image, on the other hand, is an image of light scattered from the surface layer of the object. Linearly polarized light is shone onto the object, and the image sensor of camera 1 receives the light that has passed through a polarizing plate positioned to cut out light with the same polarization state, capturing an image signal containing image data.

[0041] Reflected light from the surface of an object does not pass through a polarizer because its polarization direction does not change, but scattered light does pass through a polarizer because its polarization direction changes. As a result, a polarized image can capture only the light scattered from the surface of the object. Furthermore, short-wavelength light (the B channel of an RGB image) tends to be less likely to penetrate into the interior of the object and more likely to be reflected from the surface of the object compared to long-wavelength light (the R channel of an RGB image), and the B channel image contains a lot of information about minute irregularities on the surface of the object. For this reason, in this embodiment, it is preferable to visualize wrinkles and texture of skin using the B channel image. When visualizing objects other than skin, it is considered necessary to change the wavelength information (RGB information) used according to the scattering characteristics of the surface of the object.

[0042] Furthermore, an example of visualization of image processing using the image processing device 10 according to Embodiment 1 will be described below.

[0043] (Visualization example 1) Figure 9A is an image relating to Visualization Example 1, showing the polarized image to be processed, and Figure 9B is an image relating to Visualization Example 1, showing the unpolarized image to be processed. Furthermore, Figure 9C is an image relating to Visualization Example 1, showing the visualization processed image after image processing of Figures 2 and 8. As is clear from Figure 9C, it can be seen that an image can be obtained in which wrinkles and texture are emphasized and visualized.

[0044] (Visualization example 2) Figure 10A is an image relating to Visualization Example 2, showing the polarized image to be processed, and Figure 10B is an image relating to Visualization Example 2, showing the unpolarized image to be processed. Furthermore, Figure 10C is an image relating to Visualization Example 2, showing the visualization processed image after image processing of Figures 2 and 8. As is clear from Figure 10C, it can be seen that an image can be obtained in which wrinkles and texture are emphasized and visualized.

[0045] (Visualization example 3) Figure 11A is an image relating to Visualization Example 3, showing the polarized image to be processed, and Figure 11B is an image relating to Visualization Example 3, showing the unpolarized image to be processed. Furthermore, Figure 11C is an image relating to Visualization Example 3, showing the visualization processed image after image processing of Figures 2 and 8. As is clear from Figure 11C, it can be seen that an image can be obtained in which wrinkles and texture are emphasized and visualized.

[0046] (Visualization example 4) Figure 12A is an image relating to Visualization Example 4, showing the polarized image to be processed, and Figure 12B is an image relating to Visualization Example 4, showing the unpolarized image to be processed. Furthermore, Figure 12C is an image relating to Visualization Example 4, showing the visualization processed image after image processing of Figures 2 and 8. As is clear from Figure 12C, it can be seen that an image can be obtained in which wrinkles and texture are emphasized and visualized.

[0047] (Visualization example 5) Figure 13A is an image relating to Visualization Example 5, showing the polarized image to be processed, and Figure 13B is an image relating to Visualization Example 5, showing the unpolarized image to be processed. Furthermore, Figure 13C is an image relating to Visualization Example 5, showing the visualization processed image after image processing of Figures 2 and 8. As is clear from Figure 13C, it can be seen that an image can be obtained in which wrinkles and texture are emphasized and visualized.

[0048] (Visualization example 6) Figure 14A is an image relating to Visualization Example 6, showing the polarized image to be processed, and Figure 14B is an image relating to Visualization Example 6, showing the unpolarized image to be processed. Furthermore, Figure 14C is an image relating to Visualization Example 6, showing the visualization processed image after image processing of Figures 2 and 8. As is clear from Figure 14C, it can be seen that an image can be obtained in which wrinkles and texture are emphasized and visualized.

[0049] (Visualization example 7) Figure 15A is an image relating to Visualization Example 7, showing the polarized image to be processed, and Figure 15B is an image relating to Visualization Example 7, showing the unpolarized image to be processed. Furthermore, Figure 15C is an image relating to Visualization Example 7, showing the visualization processed image after image processing of Figures 2 and 8. As is clear from Figure 15C, it can be seen that an image can be obtained in which wrinkles and texture are emphasized and visualized.

[0050] (Summary of Embodiment 1) As described above, wrinkles and skin texture can be visualized using the method according to this embodiment, and in particular, it can be seen that wrinkles and skin texture can be visualized by utilizing feature quantities based on the difference in brightness information of the B channel of polarized and unpolarized images. In other words, the image processing device 10 according to Embodiment 1 can visualize the unevenness of the surface of an object to be photographed, such as the surface of human skin, with higher accuracy than the conventional technology.

[0051] (modified version) In the above embodiment 1, post-processing is performed in the post-processing unit 25, but the present invention is not limited to this, and it may be omitted. Furthermore, the post-processing may consist of at least binarization.

[0052] In the above embodiment 1, the image processing unit 12 can be configured, for example, by a digital computer having an image memory. The image processing in Figure 2 may be configured as an image processing program and executed by a hardware CPU. Alternatively, the image processing program may be recorded on a computer-readable recording medium (program product) such as an optical disc, and the storage medium may be inserted into an optical disc drive to read and execute the program on the computer.

[0053] In the above embodiment 1, preferably, the B channel polarized and unpolarized images are used to visualize skin wrinkles and texture by using feature quantities based on brightness difference information. However, the present invention is not limited to this, and feature quantities based on brightness difference information may be used with images of other wavelengths in addition to the R channel or G channel. Even in this case, skin wrinkles and texture can be visualized with higher accuracy compared to the prior art.

[0054] (Embodiment 2) Figure 16 is a block diagram showing an example configuration of an image processing system according to Embodiment 2. The image processing system in Figure 16 differs from the image processing system in Figure 1 in the following ways. (1) Instead of the image processing unit 12, an image processing unit 12A is provided between the post-processing unit 25 and the image interface 13, which includes an image memory 26 and a feature analysis unit 27. (2) The image processing unit 12A is equipped with a control unit 20A instead of the control unit 20. (3) Instead of the image processing device 10, an image processing device 10A equipped with an image processing unit 12A is provided. The differences are explained below.

[0055] Figure 17 is a flowchart showing the image processing performed by the image processing unit 12A in Figure 16. The image processing in Figure 17 differs from the image processing in Figure 2 in the following ways. (1) Instead of the process in step S4, the process in step S4A is executed. (2) After the processing in step S4A, the processing in step S5 is executed. The differences are explained below.

[0056] In step S4A of Figure 17, the post-processing unit 25 generates image data that visualizes wrinkle or texture information by performing post-processing, including binarization and closing processing, on the image data output from the brightness difference calculation unit 24, and outputs this image data to the feature analysis unit 27 via the image memory 26, and also outputs it to the display 2 via the image memory 26 and the image interface 13. Next, in step S5, the feature analysis unit 27 performs feature analysis using a Gapole filter on the image data output from the post-processing unit 25 via the image memory 26, and generates (1) a graph image showing the angular distribution of the average brightness value, (2) a graph image showing the periodic distribution of the average brightness value, and (3) a grayscale (or color) image (for example, a contour heatmap) showing the average brightness value (distribution of average brightness value) in a two-dimensional plane of angle and period, and outputs these to the display 2 via the image interface 13. Here, at least one of these three images specified by the user may be generated and output.

[0057] Figure 18A is a diagram illustrating the concept of a one-dimensional Gapole function used in the feature analysis unit 27 of the image processing unit 12A in Figure 16, and Figure 18B is a diagram illustrating the concept of a two-dimensional Gapole function used in the feature analysis unit 27 of the image processing unit 12A in Figure 16.

[0058] Embodiment 2 is characterized by detecting the spatial distribution (interval (period) and directionality) of skin wrinkles and texture using Gabor features. Specifically, a linear filter used in image processing texture analysis, such as the Gabor filter disclosed in Non-Patent Document 1, is used to extract specific frequency components for each direction in the local region around each point in the image. As shown in Figure 18A, the Gabor filter uses a Gabor function generated by multiplying a Gaussian function of a normal distribution with a trigonometric function of a cosine wave. In this embodiment, it is particularly used for biometric recognition such as face, iris, and fingerprint authentication. Here, the Gabor filter is said to be a model of initial visual processing in the human brain.

[0059] Next, the parameters of the Gabor filter, including its period and angle, will be explained below.

[0060] (1) Gabor filter parameters In a Gabor filter applied to a visualization image of skin wrinkles and texture, the period and angle parameters can be set. In this embodiment, the period and angle of a black and white pattern (a kernel obtained by localizing a sine / cosine function with a Gaussian function) are changed for the visualization image, and the direction and angle of the black and white pattern contained in the visualization image are detected from the strength of the response value (average brightness) when the Gabor filter is applied.

[0061] (2) Period of the Gabor filter The response when applying the Gabor filter is strongest when the spacing between wrinkles and textures (the spacing between consecutive white lines) on the visualized image is most similar to the spacing of the black and white pattern of the Gabor filter. In other words, when applying Gabor filters of various periods to a visualized image of wrinkles and textures, the period parameter of the Gabor filter that shows the greatest response corresponds to the spacing between wrinkles and textures.

[0062] (3) Gabor filter angle The response when applying the Gabor filter is strongest when the direction of wrinkles and texture on the visualized image (the direction of continuous white lines) is most similar to the direction of the black and white pattern of the Gabor filter. The difference between the maximum response value (average brightness after applying the filter) and the minimum response value (average brightness after applying the filter) corresponds to the strength of the directionality of the wrinkles and texture. If the difference between the maximum response value (average brightness after applying the filter) and the minimum response value (average brightness after applying the filter) is small, it means that there are equal amounts of wrinkles and texture in all directions. In other words, the wrinkles and texture are not distorted. If the difference between the maximum response value (average brightness after applying the filter) and the minimum response value (average brightness after applying the filter) is large, it means that there are strong wrinkles and texture in some directions and weak wrinkles and texture in other directions. In other words, the wrinkles and texture are distorted.

[0063] (Analysis example 1) Figure 19A is an image showing analysis example 1 (direction) of the feature analysis unit 27, which is a visualization processed image (DWE image) before linear filtering, and Figure 19B is an image showing analysis example 1 (direction) of the feature analysis unit 27, which is an analysis processed image after linear filtering. Figure 19B shows the DWE images (period: 20 pixels / cycle) obtained when Gabor filters in various directions are applied to the visualization processed image (DWE image) of Figure 19A by fixing the period of the Gabor filter and changing the direction. This shows that the strength of the directionality of wrinkles and texture can be detected.

[0064] (Analysis example 2) Figure 20A is a graph showing analysis example 2 (directionality) of the feature analysis unit 27, and is a graph showing the angular distribution of the average brightness after linear filtering. Figure 20B is a graph showing analysis example 2 (directionality) of the feature analysis unit 27, and is the analyzed image at the time of maximum brightness Lmax after linear filtering. Furthermore, Figure 20C is a graph showing analysis example 2 (directionality) of the feature analysis unit 27, and is the analyzed image at the time of minimum brightness Lmin after linear filtering. Here, the brightness difference ΔL = Lmax - Lmin.

[0065] As is clear from Figures 20A to 20C, the maximum luminance value (maximum response value) Lmax is observed at an angle of 56 degrees, and the minimum luminance value (minimum response value) Lmin is observed at an angle of 157 degrees. In other words, a strong response is detected when a Gabor filter is applied at an angle of 56 degrees, indicating that the strength of wrinkles and texture directionality can be detected.

[0066] (Analysis example 3) Figure 21A is an image showing analysis example 3 (periodicity) of the feature analysis unit 27, and is a visualization image (DWE image) before linear filtering. Figure 21B is an image showing analysis example 3 (periodicity) of the feature analysis unit 27, and is an analysis image after linear filtering. Figure 21B shows the DWE images (angle: 55 degrees) obtained when Gabor filters of various periods are applied to the visualization image (DWE image) of Figure 21A by fixing the direction (angle) of the Gabor filter and changing the period. This shows that the strength of the appearance interval (appearance period) of wrinkles and texture can be detected.

[0067] (Analysis example 4) Figure 22A is a graph showing analysis example 4 (periodicity) of the feature analysis unit 27, and is a graph showing the periodic distribution of the average brightness after linear filtering. Figure 22B is also a graph showing analysis example 4 (periodicity) of the feature analysis unit 27, and is the analyzed processed image at the time of maximum brightness Lmax after linear filtering.

[0068] As is clear from Figures 22A and 22B, a strong response is detected when a Gabor filter with a period of 86 pixels / cycle and a maximum brightness of 87.4 is applied. This indicates that the strength of the interval (= appearance period) between the appearance of wrinkles and skin texture can be detected.

[0069] (Analysis example 5) Figure 23 is a graph showing analysis example 5 (directivity and periodicity) of the feature analysis unit 27, and is a graph image in which the brightness values ​​are represented in contour heatmap format on a two-dimensional plane of period with respect to direction angle. Figure 23 shows the brightness change of a DWE image to which a Gabor filter has been applied, and visualizes the features of the DWE image by displaying the response values ​​as a heatmap (contour display) when both the direction and period of the Gabor filter are changed. Here, in order to show the patent drawing as a black and white image, the response values ​​are changed by cross-hatching or the hatching interval, but Figure 23 is a graph image with a heatmap display where yellow is strong and blue is weak, which has been redrawn for the patent drawing, and the same applies to Figures 24 and onward.

[0070] Figures 24A to 24E are images showing analysis example 5 (directivity and periodicity) of the feature analysis unit 27. Figure 24A is the RGB image before processing (shown as a black and white image for patent purposes), Figure 24B is the visualization image after visualization processing (DWE image), and Figure 24C is a graph image showing the luminance values ​​in contour heatmap format on a two-dimensional plane of period with respect to angle of directionality. Figure 24D is a graph showing the angular distribution of average luminance after linear filtering, and Figure 24E is a graph showing the periodic distribution of average luminance after linear filtering. In Figures 24A and 24B, rectangles in the images indicate the analysis target area.

[0071] As is clear from Figures 24C and 24D, it is possible to detect angles with large brightness differences in the angular distribution, and strong wrinkles and texture can be observed only in certain directions. Furthermore, as is clear from Figures 24C and 24E, it is possible to detect periods with strong responses, and from Figures 24C and 24E, it can be seen that the spacing between wrinkles and texture is wide.

[0072] (Analysis example 6) Figures 25A to 25E are images showing analysis example 6 (directivity and periodicity) of the feature analysis unit 27. Figure 25A is the RGB image before processing, Figure 25B is the visualization image (DWE image) after visualization processing, and Figure 25C is a graph image showing the luminance values ​​in contour heatmap format on a two-dimensional plane of period with respect to angle for directionality. Figure 25D is a graph showing the angular distribution of average luminance after linear filtering, and Figure 25E is a graph showing the periodic distribution of average luminance after linear filtering. In Figures 25A and 25B, the rectangles in the images indicate the analysis target area.

[0073] As is clear from Figures 25C and 25D, the brightness difference in the angular distribution is small, and the peak positions are scattered. This indicates that the wrinkles and texture directionality are weak. Furthermore, as is clear from Figures 25C and 25E, the peak position of the periodic distribution is only one (Lmax), and the spacing of the wrinkles and texture is uniform.

[0074] (Analysis example 7) Figures 26A to 26E are images showing analysis example 7 (directivity and periodicity) of the feature analysis unit 27. Figure 26A is the RGB image before processing, Figure 26B is the visualization image (DWE image) after visualization processing, and Figure 26C is a graph image showing the luminance values ​​in contour heatmap format on a two-dimensional plane of period with respect to angle for directionality. Figure 26D is a graph showing the angular distribution of average luminance after linear filtering, and Figure 26E is a graph showing the periodic distribution of average luminance after linear filtering. In Figures 26A and 26B, the rectangles in the images indicate the analysis target area.

[0075] As is clear from Figures 26C and 26D, there is some difference in brightness in the angular distribution, indicating a slight directional pattern of wrinkles and texture. Furthermore, as is clear from Figures 26C and 26E, there is only one peak position in the periodic distribution (Lmax), indicating that the spacing of wrinkles and texture is uniform.

[0076] (Analysis example 8) Figures 27A to 27E show examples of feature analysis 8 (directivity and periodicity) performed by the feature analysis unit 27 on the left scapula of a human (scar after suturing). Figure 27A is the RGB image before processing, Figure 27B is the visualized image (DWE image) after visualization processing, and Figure 27C is a graph image representing the luminance values ​​in a contour heatmap format on a two-dimensional plane of period with respect to angle of directionality. Figure 27D is a graph showing the angular distribution of average luminance after linear filtering, and Figure 27E is a graph showing the periodic distribution of average luminance after linear filtering. In Figures 27A and 27B, the rectangles in the images indicate the analysis target area.

[0077] As is clear from Figures 27C and 27D, the brightness difference in the angular distribution is small, indicating that the directionality (angular change) of wrinkles or texture is weak (small angular change). Also, as is clear from Figures 27C and 27E, the peak position of the periodic distribution is small, indicating that the spacing between wrinkles and texture is narrow (small period).

[0078] (Analysis example 9) Figures 28A to 28E show images illustrating analysis example 9 (directivity and periodicity) of the feature analysis unit 27 on the left scapula of a human (scar after suturing). Figure 28A is the RGB image before processing, Figure 28B is the visualization image (DWE image) after visualization processing, and Figure 28C is a graph image representing the luminance values ​​in contour heatmap format on a two-dimensional plane of period with respect to angle of direction. Figure 28D is a graph showing analysis example 9 (directivity and periodicity) of the feature analysis unit 27, showing the angular distribution of average luminance after linear filtering, and Figure 28E is a graph showing analysis example 9 (directivity and periodicity) of the feature analysis unit 27, showing the periodic distribution of average luminance after linear filtering. In Figures 28A and 28B, the rectangles in the images indicate the analysis target area.

[0079] As is clear from Figures 28C and 28D, the difference in brightness in the angular distribution is small, and there are two peaks. This indicates that while the directionality of wrinkles and texture is weak, there is a tendency for strong brightness in two directions. Furthermore, as is clear from Figures 28C and 28E, the surrounding peak positions are slightly smaller, which indicates that the spacing between wrinkles and texture is slightly narrower (the period is slightly smaller).

[0080] (Summary of Embodiment 2) As described above, in Embodiment 2, Gabor feature analysis was performed using a Gabor filter to determine the spacing and directionality of wrinkles and textures. Here, a Gabor filter is a linear filter used in image processing for texture analysis, etc., which extracts specific frequency components for each direction. The response value (average brightness value) was calculated when a Gabor filter of arbitrary direction and period was applied to a visualization image of wrinkles and textures, and the distortion and spacing of the grid pattern of wrinkles and textures were quantitatively evaluated. Regarding the distortion of the pattern, attention was paid to the strength of the response when a Gabor filter of arbitrary direction was applied, and the difference between the maximum and minimum responses corresponds to the amount of pattern distortion. Regarding the spacing of wrinkles and textures, attention was paid to the strength of the response when a Gabor filter of arbitrary period was applied, and the period parameter (pixels / cycle) of the Gabor filter at the time of maximum response is considered to correspond to the spacing.

[0081] Here, we will explain in particular the cases of (1) Figures 25A to 25E (Analysis Example 6), (2) Figures 27A to 27E (Analysis Example 8), and (3) Figures 28A to 28E (Analysis Example 9).

[0082] Table 1 shows the angle, period, and response differences at the time of maximum response. However, the angle depends on the orientation of the camera at the time of shooting. Here, analysis examples 8 and 9 are examples of post-suturing scars, and it can be confirmed from the visualized images and Gabor features that the spacing and distortion of wrinkles and texture differ between the area around the suture site in analysis example 8 and the suture site in analysis example 9. From these results, it can be seen that quantitative evaluation is possible from parameters corresponding to the distortion and spacing of the grid pattern of wrinkles and texture, and that feature analysis of wrinkles and texture using a Gabor filter is useful.

[0083] [Table 1]

[0084] In Embodiment 2, a method for visualizing wrinkles and skin texture using B-channel information from polarized and unpolarized images of skin captured by a dermatoscope was proposed. Gabor features were then used to evaluate the spacing and directionality of wrinkles and skin texture. Furthermore, the usefulness of this proposed method for evaluation in diagnosing skin disease areas was suggested.

[0085] As described above, the image processing device 10A according to Embodiment 2 can analyze the surface irregularities of an object to be photographed, such as a biological surface like the surface of human skin, with higher accuracy than the conventional technology, using predetermined feature quantities.

[0086] In particular, the following describes what kind of medical judgments or diagnoses can be made using Embodiments 1 and 2. Generally, it is difficult to visualize the condition of wrinkles and texture of the skin, so visualized wrinkle and texture information is useful for diagnosing the health of the skin and for cosmetic judgments. The appearance of wrinkles and texture varies depending on the area of ​​the skin, and there is a healthy appearance of wrinkles and texture in each area, so it is thought that it is possible to diagnose diseased areas by examining the appearance of abnormal wrinkles and texture. In particular, visualized information of wrinkles and texture is useful for diagnosing the healing process of scars, as scars may or may not be noticeable after healing, and it is thought that image diagnosis and quantitative evaluation (feature analysis using Gabor filters) of the condition of wrinkles and texture during the healing process (skin tension, presence or absence of unevenness in wrinkles and texture, etc.) are useful for monitoring the progress.

[0087] In the above embodiment 2, the post-processing unit 25 performs binarization and closing processes, but the present invention is not limited to this, and the state of unevenness can be made to appear with high precision by performing at least binarization.

[0088] (Embodiment 3) Figure 29 is a block diagram showing an example configuration of an image processing system according to Embodiment 3. The image processing system in Figure 29 differs from the image processing system in Figure 1 in the following ways. (1) Instead of the image processing unit 12, an image processing unit 12B is provided. (2) The image processing unit 12B is equipped with a control unit 20B instead of the control unit 20. (3) Instead of the image processing device 10, an image processing device 10B is provided, which includes an image processing unit 12B and an external interface 14. (4) The system further includes an optical disc drive 30 connected to the external interface 14 into which an optical disc 31 is inserted. The differences are explained below.

[0089] In Embodiment 3, the image processing program shown in Figure 2 or Figure 17 may be recorded on a computer-readable recording medium (program product), such as an optical disc like a CD or DVD, or an SSD (Solid State Drive). The storage medium may then be inserted into a drive device, such as an optical disc drive 30, to read the program, load it into the memory of the control unit 20B via the external interface 14, and have the control unit 20B (computer) execute it.

[0090] In Embodiment 2 described above, the image processing system of Embodiment 1 is provided with an external interface 14 and an optical disc drive 30. However, the present invention is not limited to this, and the image processing system of Embodiment 2 may also be provided with an external interface 14 and an optical disc drive 30.

[0091] (Other variations) In the embodiments described above, the background extraction processing unit 22 extracts background image data from image data from the luminance ratio calculation unit 21 (or image data including the difference in luminance for each pixel (modified version)). However, the present invention is not limited to this, and background image data may be extracted from polarized image data or unpolarized image data input to the luminance ratio calculation unit 21. That is, the background extraction processing unit 22, (1) Image data including the ratio of brightness for each pixel from the brightness ratio calculation unit 21, (2) Image data including the difference in brightness for each of the pixels, (3) Polarization image data input to the luminance ratio calculation unit 21, (4) The non-polarized image data input to the luminance ratio calculation unit 21, Background image data related to the background image may be extracted and output from any of the image data.

[0092] In the embodiments described above, post-processing is performed in the post-processing unit 25, but the present invention is not limited to this, and it may be omitted. Furthermore, the post-processing may consist of at least binarization.

[0093] In the embodiments described above, the image processing units 12, 12A, and 12B can be configured, for example, by a digital computer having an image memory. The image processing shown in Figures 2 and 17 may be configured as an image processing program and executed by a hardware CPU. [Industrial applicability]

[0094] As described in detail above, the present invention aims to provide an image processing apparatus and an image processing method that can visualize biological surfaces, such as the surface of human skin, with higher accuracy than the prior art. Furthermore, the present invention can provide an image processing apparatus and an image processing method that can analyze biological surfaces, such as the surface of human skin, using predetermined parameters with higher accuracy than the prior art.

[0095] In this invention, the following are applicable as subjects for image processing. Basically, any object that has a certain degree of fine irregularities on its surface and possesses a predetermined high transparency, such as the skin or surface of living organisms including the human body, animals, and other living things, can be used as a subject for image processing. Specifically, it is possible to visualize the state of irregularities on the surface of human skin, animal skin, leaves of living organisms, mushrooms, etc. [Explanation of Symbols]

[0096] 1 Camera 2 displays 10, 10A, 10B Image Processing Device 11 Image Interface 11m image memory 12, 12A, 12B Image Processing Unit 13 Image Interface 13MB image memory 14 External Interfaces 20,20A Control Unit 21 Brightness Ratio Calculation Unit 22 Background extraction processing unit 23 Image memory 24 Brightness Difference Calculation Unit 25 Post-processing unit 26 Image memory 27 Feature Analysis Unit 30 Optical disc drives 31 Optical Discs

Claims

1. An image sensor that captures images of the surface of an object to be processed and generates polarized image data and unpolarized image data, A first calculation unit that calculates and outputs the ratio of brightness or the brightness difference for each pixel between the polarized image data and the unpolarized image data, (1) Image data from the first calculation unit including the ratio of brightness for each pixel, (2) Image data from the first calculation unit including the difference in brightness for each pixel, (3) The polarization image data and, (4) The non-polarized image data and, A background extraction processing unit that extracts and outputs background image data related to the background image from any of the image data, A second calculation unit generates image data including the difference for each pixel by subtracting background image data from the background extraction processing unit from image data including the ratio or difference in brightness for each pixel from the image data from the first calculation unit, A post-processing unit that performs binarization on the image data from the second calculation unit and outputs the result, A feature analysis unit calculates Gapore features by changing at least one of angle and period on the image data from the post-processing unit using a Gapore filter, thereby generating an angular distribution of luminance values, a periodic distribution of luminance values, and a luminance value distribution in a two-dimensional plane of angle and period. An image processing device equipped with the following features.

2. The image processing apparatus according to claim 1, wherein the first calculation unit calculates the ratio or difference in brightness of each pixel between the B (blue) channel image data of the polarized image data and the B (blue) channel image data of the unpolarized image data.

3. The image processing apparatus according to claim 1, wherein the background extraction processing unit extracts background image data relating to the background image from the image data using a Gaussian filter.

4. The image processing apparatus according to claim 1, wherein the post-processing unit performs binarization and closing processing on the image data from the second calculation unit and outputs it.

5. The image sensor captures an image of the surface of the object to be processed and generates polarized image data and unpolarized image data. The first calculation unit calculates and outputs the ratio or difference in brightness of each pixel between the polarized image data and the unpolarized image data. The background extraction processing unit, (1) Image data from the first calculation unit including the ratio of brightness for each pixel, (2) Image data from the first calculation unit including the difference in brightness for each pixel, (3) The polarization image data and, (4) The non-polarized image data and, The steps include: extracting and outputting background image data related to the background image from any of the image data; The second calculation unit subtracts the background image data from the background extraction processing unit from the image data from the first calculation unit, which includes the ratio or difference in brightness of each pixel, to generate image data that includes the difference for each pixel. The post-processing unit performs a binarization process on the image data from the second calculation unit and outputs it; The feature analysis unit calculates Gapole features by changing at least one of angle and period on the image data from the post-processing unit using a Gapole filter, thereby generating an angular distribution of luminance values, a periodic distribution of luminance values, and a luminance value distribution in a two-dimensional plane of angle and period. Image processing methods, including those mentioned above.

6. The step of calculating the ratio or difference in brightness includes calculating the ratio or difference in brightness for each pixel between the B (blue) channel image data of the polarized image data and the B (blue) channel image data of the unpolarized image data, among the polarized image data and the unpolarized image data. The image processing method according to claim 5.

7. The step of extracting the background image data includes using a Gaussian filter to extract background image data related to the background image from the image data. The image processing method according to claim 5.

8. The step of performing the binarization process and outputting the result includes performing the binarization process and closing process on the image data from the second calculation unit and outputting the result. The image processing method according to claim 5.

9. An image processing program executed by a computer, The image sensor captures an image of the surface of the object to be processed and generates polarized image data and unpolarized image data. The first calculation unit calculates and outputs the ratio or difference in brightness of each pixel between the polarized image data and the unpolarized image data. The background extraction processing unit, (1) Image data from the first calculation unit including the ratio of brightness for each pixel, (2) Image data from the first calculation unit including the difference in brightness for each pixel, (3) The polarization image data and, (4) The non-polarized image data and, The steps include: extracting and outputting background image data related to the background image from any of the image data; The second calculation unit subtracts the background image data from the background extraction processing unit from the image data from the first calculation unit, which includes the ratio or difference in brightness of each pixel, to generate image data that includes the difference for each pixel. The post-processing unit performs a binarization process on the image data from the second calculation unit and outputs it; The feature analysis unit calculates Gapole features by changing at least one of angle and period on the image data from the post-processing unit using a Gapole filter, thereby generating an angular distribution of luminance values, a periodic distribution of luminance values, and a luminance value distribution in a two-dimensional plane of angle and period. An image processing program that includes this feature.

10. The step of calculating the ratio or difference in brightness includes calculating the ratio or difference in brightness for each pixel between the B (blue) channel image data of the polarized image data and the B (blue) channel image data of the unpolarized image data, among the polarized image data and the unpolarized image data. The image processing program according to claim 9.

11. The step of extracting the background image data includes using a Gaussian filter to extract background image data related to the background image from the image data. The image processing program according to claim 9.

12. The step of performing the binarization process and outputting the result includes performing the binarization process and closing process on the image data from the second calculation unit and outputting the result. The image processing program according to claim 9.

13. A recording medium for storing an image processing program that can be read by a computer, The aforementioned image processing program is: The image sensor captures an image of the surface of the object to be processed and generates polarized image data and unpolarized image data. The first calculation unit calculates and outputs the ratio or difference in brightness of each pixel between the polarized image data and the unpolarized image data. The background extraction processing unit, (1) Image data from the first calculation unit including the ratio of brightness for each pixel, (2) Image data from the first calculation unit including the difference in brightness for each pixel, (3) The polarization image data and, (4) The non-polarized image data and, The steps include: extracting and outputting background image data related to the background image from any of the image data; The second calculation unit subtracts the background image data from the background extraction processing unit from the image data from the first calculation unit, which includes the ratio or difference in brightness of each pixel, to generate image data that includes the difference for each pixel. The post-processing unit performs a binarization process on the image data from the second calculation unit and outputs it; The feature analysis unit calculates Gapole features by changing at least one of angle and period on the image data from the post-processing unit using a Gapole filter, thereby generating an angular distribution of luminance values, a periodic distribution of luminance values, and a luminance value distribution in a two-dimensional plane of angle and period. A recording medium that includes this.

14. The step of calculating the ratio or difference in brightness includes calculating the ratio or difference in brightness for each pixel between the B (blue) channel image data of the polarized image data and the B (blue) channel image data of the unpolarized image data, among the polarized image data and the unpolarized image data. The recording medium according to claim 13.

15. The step of extracting the background image data includes using a Gaussian filter to extract background image data related to the background image from the image data. The recording medium according to claim 13.

16. The step of performing the binarization process and outputting the result includes performing the binarization process and closing process on the image data from the second calculation unit and outputting the result. The recording medium according to claim 13.

Images