Skin surface analysis device and skin surface analysis method

Abstract

A local image enhancement process is carried out on an image in which a transfer material is captured. The enhanced image is divided into a plurality of patch images, which are input to a machine learning identification device. Segmented patch images which are output from the machine learning identification device are combined, and a likelihood map image of cristae cutis is generated from the overall image on the basis of the segmentation result. The likelihood map is subjected to a binarization process, thereby generating a binary image. A cristae cutis region is extracted on the basis of the binary image, and the surface area of the cristae cutis region is calculated.

Description

[Technical field]

[0001] The present disclosure relates to a skin surface analysis device and a skin surface analysis method for analyzing a human skin surface. [Background technology]

[0002] The surface of human skin (skin surface) has groove-like parts called skin grooves and protuberant parts called skin mounds separated by the skin grooves. Even when a person is at rest, he sweats, albeit in a small amount, and this sweating at rest is called basal sweating. Sweat during basal sweating is mainly secreted into the skin grooves and is related to the moisture content of the stratum corneum, and is known to play an important role in maintaining the barrier function of the skin. For example, inflammatory skin diseases such as atopic dermatitis, cholinergic urticaria, prurigo, and amyloid lichen may occur due to a decrease in the barrier function of the skin, i.e., basal sweating disorder, or the symptoms may worsen due to basal sweating disorder. If the basal sweating of a patient can be detected, it is possible to determine a treatment plan, alleviate symptoms, and determine the degree of healing, which is effective for diagnosis and treatment.

[0003] As a method for detecting basal sweating, for example, the impression mold technique (IMT or IM method) is known. IMT is a sweating function quantitative measurement method that obtains the sweating state as well as the skin surface structure by applying a dental silicone impression material to the skin surface in the form of a film and leaving it for a specified time, and then peeling the silicone impression material off the skin. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Re-tabled publication 2018 / 230733 Summary of the Invention [Problem to be solved by the invention]

[0005] By using IMT, the skin surface structure is precisely transferred to the film-like silicone material, so that it is possible to identify the skin ridges and measure the area of ​​the skin ridges, and furthermore, the sweat droplets are precisely transferred to the silicone material, so that it is possible to measure the number, diameter, and area of ​​the sweat droplets. This allows the condition of the skin surface to be analyzed. The advantage of using the results of this analysis is that, for example, in the case of atopic dermatitis, it is possible to quantitatively obtain the tendency that the area of ​​the skin ridges is larger and the number of sweat droplets is smaller than that of healthy individuals.

[0006] In IMT, the discrimination between skin ridges and sweat drops can be performed based on an enlarged image of the transfer surface of the silicone material. Specifically, an image of the transfer surface of the silicone material is obtained by enlarging it with an optical microscope and projected onto a monitor. Then, while viewing the image on the monitor, an inspector distinguishes between skin ridges and skin grooves, surrounds and colors the areas corresponding to the skin ridges, calculates the area of ​​the colored areas, finds sweat drops, colors the areas corresponding to the sweat drops, and calculates the area of ​​the colored areas. In this way, the condition of the skin surface can be quantitatively obtained, but there are problems as described below.

[0007] That is, the structure of the skin surface is complex and also varies greatly depending on the skin disease that has developed, so when an examiner judges which parts of the image are skin grooves or which parts are skin ridges, it takes time, and there is a limit to the number of samples that can be processed in a certain time. In addition, silicone may contain air bubbles, which are difficult to distinguish from sweat droplets, and distinguishing sweat droplets is a time-consuming and labor-intensive task. In addition, the time required to distinguish skin grooves and skin ridges and sweat droplets increases, and since the judgment is made by looking at the image, there is also the problem of individual differences, such as the result of differentiating depending on the ability of the examiner. As the work takes longer, there is a possibility that oversights may occur.

[0008] Furthermore, the number of sweat droplets varies depending on the location on the skin surface even for the same person, and unless the location with the average number of sweat droplets is selected as the measurement target, the analysis results may be inappropriate. In order to obtain this average location, it is necessary to distinguish the above-mentioned skin grooves and skin ridges and sweat droplets over a wide area of ​​the skin surface, which is a factor that makes the time required for analysis even longer.

[0009] The present disclosure has been made in consideration of the above points, and has as its object to improve the accuracy of analyzing the condition of the skin surface and to shorten the time required for the analysis. [Means for solving the problem]

[0010] In order to achieve the above object, the first disclosure provides a skin surface analysis device that analyzes a skin surface using a transfer material to which a human skin surface structure is transferred, the device comprising: an image input unit to which an image of the transfer material is input; a local image enhancement processing unit that executes local image enhancement processing to enhance the contrast of a local region of the image input to the image input unit to generate an enhancement-processed image; a patch image generation unit that divides the enhancement-processed image generated by the local image enhancement processing unit into a plurality of patch images; and a mechanical simulation unit that receives each of the patch images generated by the patch image generation unit and executes segmentation of each of the input patch images. the machine learning classifier; an overall image generation unit that generates an overall image by synthesizing the patch images after segmentation output from the machine learning classifier; a likelihood map generation unit that generates a likelihood map image of the skin tumour from the overall image generated by the overall image generation unit based on the segmentation result; a binarization processing unit that performs a binarization process on the likelihood map image generated by the likelihood map generation unit to generate a binarized image; a region extraction unit that extracts a skin tumour region based on the binarized image generated by the binarization processing unit; and a skin tumour analysis unit that calculates the area of ​​the skin tumour region extracted by the region extraction unit.

[0011] According to this configuration, when an image of a transfer material to which a human skin surface structure is transferred is input, a local image enhancement process is performed on the image to generate an enhancement-processed image. This increases the visibility of details of the image. The image before the local image enhancement process may be a color image or a grayscale image. The enhancement-processed image is divided into a plurality of patch images, and then each patch image is input to a machine learning classifier, where each patch image is segmented. The method of segmenting each patch image is a deep learning method that has been used conventionally, and this segmentation determines which category each pixel belongs to, for example, and classifies the pixel into a skin ridge, skin groove, sweat drop, and other. When the patch images after segmentation output from the machine learning classifier are synthesized to generate an overall image, a likelihood map image of the skin ridge is generated from the overall image based on the segmentation result. When a binary image is generated from the likelihood map image, for example, if white is the skin ridge region, it is possible to distinguish the skin ridge region by extracting the white region. By calculating the area of ​​the extracted skin ridge region, it is possible to analyze the skin surface.

[0012] In the second and third disclosures, the device includes a likelihood map generation unit that generates a likelihood map image of sweat droplets based on the segmentation result from the overall image generated by the overall image generation unit, a sweat droplet extraction unit that extracts sweat droplets based on the likelihood map image generated by the likelihood map generation unit, and a sweat droplet analysis unit that calculates the distribution of the sweat droplets extracted by the sweat droplet extraction unit.

[0013] According to this configuration, when the patch images after segmentation output from the machine learning classifier are synthesized to generate an overall image, a likelihood map image of sweat droplets is generated from the overall image based on the segmentation result. If, for example, white in the likelihood map image represents sweat droplets, it is possible to identify sweat droplets by extracting the white regions. It is possible to analyze the skin surface by calculating the distribution of the extracted sweat droplets.

[0014] In the fourth disclosure, the transfer material is obtained using the impression mold technique, and the configuration includes a grayscale processing unit that converts an image of the transfer material into a grayscale image.

[0015] That is, IMT enables precise transfer of the skin surface using silicone, which further improves the analysis accuracy. This silicone may be colored, for example, pink, but with this configuration, the image captured of the transfer material is converted to grayscale by the grayscale processing unit, so it can be handled as a grayscale image suitable for analysis. This allows the processing speed to be increased.

[0016] In the fifth disclosure, the patch image generating section can generate patch images such that adjacent patch images partially overlap each other.

[0017] In other words, when dividing into a plurality of patch images, if adjacent patch images are not overlapped, it is conceivable that the edge of a skin ridge or a sweat drop happens to overlap the boundary between adjacent patch images, and there is a risk that the accuracy of distinguishing the skin ridge or sweat drop overlapping the boundary will decrease. In contrast, according to the present configuration, adjacent patch images partially overlap each other, so that even skin ridges and sweat drops in the above-mentioned positions can be distinguished with high accuracy.

[0018] In the sixth disclosure, the machine learning classifier can make the resolution of the input image and the resolution of the output image the same. With this configuration, for example, the shape of a fine skin ridge or the size of a sweat drop can be accurately output.

[0019] In the seventh disclosure, the skin ridge analysis unit can set a plurality of grids of a predetermined size on the image, and calculate the ratio of the skin ridge region to the skin sulcus region within each of the grids.

[0020] According to this configuration, for example, when it is desired to evaluate the fineness of the skin surface, it can be evaluated based on the ratio of the skin ridge area to the skin groove area within the grid set on the binary image. When the ratio of the skin ridge area is equal to or greater than a predetermined value, it can be used as one of the guidelines for judging that the skin is coarse, and when the ratio of the skin ridge area is less than the predetermined value, it can be used as one of the guidelines for judging that the skin is fine.

[0021] In the eighth disclosure, the skin ridge analysis unit can convert the ratio of skin ridge regions and skin groove regions in each grid into numerical values ​​and calculate a frequency distribution (histogram).

[0022] In the ninth disclosure, after extracting the skin ridge region, the region extraction unit can determine whether each part of the extracted skin ridge region is convex or not, and divide the skin ridge region by parts that are determined to be not convex.

[0023] In other words, depending on the pathology, a groove may be formed in part of the skin ridge, in which case the extracted skin ridge region will have a non-convex part, i.e., a concave part. By dividing the skin ridge region by this concave part, it is expected that it can be used for pathological and clinical evaluation.

[0024] In the tenth disclosure, an information output unit is provided which generates and outputs information regarding the shape of the skin ridge region extracted by the region extraction unit, so that each piece of information can be presented to medical professionals, etc. for use in diagnosis, etc. Effect of the Invention

[0025] As described above, according to the present disclosure, a machine learning classifier is utilized to generate a likelihood map image of the skin surface, and the likelihood map image can be utilized to identify skin ridge regions and sweat droplets, thereby eliminating individual differences during analysis, improving the accuracy of analysis of the skin surface condition, and reducing the time required for analysis. [Brief description of the drawings]

[0026] [Figure 1] 1 is a schematic diagram showing a configuration of a skin surface analysis device according to an embodiment of the present invention. [Diagram 2] FIG. 2 is a block diagram of the skin surface analysis device. [Diagram 3] 11 is a flowchart illustrating the first half of the flow of a skin surface analysis method. [Figure 4] 13 is a flowchart illustrating the latter half of the flow of the skin surface analysis method. [Figure 5A] FIG. 1 is a diagram for explaining the prior art, showing how to identify a skin ridge and measure its area using IMT. [Figure 5B] FIG. 1 is a diagram illustrating the prior art, showing how IMT identifies sweat droplets and measures their number, diameter, and area. [Figure 6] FIG. 1 is a diagram illustrating an example of a grayscale image. [Figure 7] FIG. 13 is a diagram illustrating an example of a local image enhancement processed image. [Figure 8] FIG. 13 is a diagram showing a state in which a local image enhancement processed image is divided into a plurality of patch images. [Figure 9] FIG. 13 is a diagram illustrating an example of segmentation by a machine learning classifier. [Figure 10] FIG. 13 is a diagram showing an example of an overall image of a skin ridge and a skin sulcus. [Figure 11] FIG. 1 is a diagram showing an example of an overall image of a sweat droplet. [Figure 12] FIG. 13 is a diagram showing an example of a likelihood map image of skin ridges and skin sulci. [Figure 13] FIG. 13 is a diagram showing an example of a sweat droplet likelihood map image. [Figure 14] FIG. 13 is a diagram showing an example of an image in which a likelihood map of skin ridges and skin sulci has been binarized. [Figure 15] FIG. 13 is a diagram showing an example of an image in which skin ridges and skin grooves have been extracted. [Figure 16] FIG. 13 is a diagram showing an example of an image in which sweat droplets are extracted. [Figure 17] FIG. 13 is a diagram showing an example of an image comparing the positions of sweat droplets with skin ridges and skin grooves. [Figure 18] FIG. 13 shows an example of an image in which sweat droplets located in the skin ridges and skin sulci are identified. [Figure 19]13 is a histogram of skin ridge information. [Figure 20] FIG. 13 is a diagram illustrating an example of a heat map image of sweat droplets. [Figure 21] FIG. 13 is a diagram showing an example of a skin tumour region image. [Figure 22] 1 is a table showing the specifications of the skin ridge area. [Figure 23] 1 is a graph showing the two-dimensional distribution of skin ridges and skin grooves per grid. [Figure 24] FIG. 13 is a diagram showing an example of an image when an analysis is performed by setting a plurality of grids. [Diagram 25] FIG. 13 is a diagram showing an example of an image obtained by combining imaging regions of nine fields of view. [Figure 26] This is a graph showing the ratio of skin ridges and sulci on the forearm of a healthy subject, where a grid of 100 x 100 pixels is set. [Figure 27] This is a graph showing the ratio of skin ridges and sulci on the forearm of a healthy subject, where a grid of 150 x 150 pixels is set. [Figure 28] This is a graph showing the ratio of skin ridges and sulci on the forearm of a healthy subject, where a grid of 200 x 200 pixels is set. [Figure 29] 13 is a graph showing the ratio of skin ridges and sulci on the thighs of a patient with atopic dermatitis, where a grid of 250 x 250 pixels is set. [Diagram 30] 13 is a graph showing the ratio of skin ridges and sulci on the thighs of a patient with atopic dermatitis, where a grid of 100 x 100 pixels is set. [Diagram 31] 13 is a graph showing the ratio of skin ridges and sulci on the thighs of a patient with atopic dermatitis, where a grid of 150 x 150 pixels is set. [Diagram 32] 13 is a graph showing the ratio of skin ridges and sulci on the thighs of a patient with atopic dermatitis, where a grid of 200 x 200 pixels is set. [Diagram 33] 13 is a graph showing the ratio of skin ridges and sulci on the thighs of a patient with atopic dermatitis, where a grid of 250 x 250 pixels is set. [Diagram 34] 13 is a graph showing the ratio of skin ridges and sulci on the forehead of a patient with atopic dermatitis, where a grid of 100 x 100 pixels is set. [Diagram 35] 13 is a graph showing the ratio of skin ridges and sulci on the forehead of a patient with atopic dermatitis, where a grid of 150 x 150 pixels is set. [Diagram 36] 13 is a graph showing the ratio of skin ridges and sulci on the forehead of a patient with atopic dermatitis, where a grid of 200 x 200 pixels is set. [Figure 37] 13 is a graph showing the ratio of skin ridges and sulci on the forehead of a patient with atopic dermatitis, where a grid of 250 x 250 pixels is set. [Figure 38] 13 is a graph showing the ratio of skin ridges and sulci on the elbows of patients with atopic dermatitis, where a grid of 100 x 100 pixels is set. [Figure 39] 13 is a graph showing the ratio of skin ridges and sulci on the elbows of patients with atopic dermatitis, where a grid of 150 x 150 pixels is set. [Diagram 40] 13 is a graph showing the ratio of skin ridges and sulci on the elbows of patients with atopic dermatitis, where a grid of 200 x 200 pixels is set. [Diagram 41] 13 is a graph showing the ratio of skin ridges and sulci on the elbows of patients with atopic dermatitis, where a grid of 250 x 250 pixels is set. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0027] Hereinafter, the preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. Note that the following description of the preferred embodiments is merely illustrative in nature and is not intended to limit the present invention, its applications, or its uses.

[0028] 1 is a schematic diagram showing the configuration of a skin surface analysis device 1 according to an embodiment of the present invention. The skin surface analysis device 1 is a device that analyzes the skin surface using a transfer material 100 onto which a human skin surface structure is transferred, and the skin surface analysis method according to the present invention can be executed by using this skin surface analysis device 1.

[0029] In the description of this embodiment, a case will be described in which the skin surface is analyzed using transfer material 100 obtained by IMT, but transfer material 100 may also be one onto which the structure of a human skin surface has been transferred by a method other than IMT.

[0030] IMT is a quantitative measurement method of sweat function in which a dental silicone impression material is applied to the skin surface in a film form, left for a specified time, and then peeled off from the skin to obtain the skin surface structure and sweating state. This IMT has been used as a method for detecting basal sweating, so a detailed description is omitted. Dental silicone impression materials are sometimes colored, for example, pink.

[0031] 1 illustrates a case where the silicone is spread on the forearm, left for a predetermined time, hardened, and then peeled off from the skin to obtain transfer material 100, but the present invention is not limited to this and may be a transfer of the skin surface structure of any part of the body, such as the legs, chest, back, forehead, etc. By using IMT, the skin surface structure is precisely transferred to the film-like silicone material, making it possible to identify the skin ridges and measure their area, and furthermore, since sweat droplets are precisely transferred to the silicone material, the number, diameter, and area of ​​sweat droplets can also be measured.

[0032] Fig. 5A is a diagram explaining the prior art, showing how to identify a skin ridge by IMT and measure the area of ​​the skin ridge. This diagram is based on an image captured by enlarging the transfer surface of transfer material 100 with a reflective stereomicroscope 101 (shown in Fig. 1). An inspector projects this image onto a monitor and distinguishes between skin ridge regions and skin groove regions using the shading and light and dark as clues. The area of ​​the skin ridge can be obtained by drawing a shape to surround the region identified as a skin ridge region and measuring the area of ​​this shape.

[0033] On the other hand, FIG. 5B is a diagram for explaining the prior art, showing how to distinguish sweat droplets by IMT and measure the number, diameter, and area of ​​sweat droplets. In this diagram, too, an inspector uses an image of the transfer surface of the transfer material 100 magnified with a stereomicroscope 101, projects this image on a monitor, and distinguishes sweat droplets based on the shading, brightness, and shape. Sweat droplets are marked with circles. The marks are colored differently to distinguish sweat droplets on the skin mound from those on the skin groove. This makes it possible to measure the number, diameter, and area of ​​sweat droplets. Note that silicone may contain air bubbles, so parts that are close to a circle with a diameter of, for example, 40 μm or less are determined to be air bubbles.

[0034] The above is the method by which inspectors distinguish between skin ridges and skin grooves, and between sweat droplets, but as shown in Figures 5A and 5B, the structure of the skin surface is complex and differs greatly depending on the skin disease that has developed, so when inspectors distinguish between skin grooves and skin ridges in the image, it takes time, and there is a limit to the number of samples that can be processed within a certain time. Also, silicone may contain air bubbles, which are difficult to distinguish from sweat droplets, and distinguishing between sweat droplets is a time-consuming and laborious task.

[0035] The skin surface analysis device 1 of this embodiment can generate a likelihood map image of the skin surface using the machine learning classifier 24 described later, even for images such as those shown in Figures 5A and 5B, and can identify skin ridge regions and sweat droplets using the likelihood map image, thereby improving the accuracy of analyzing the condition of the skin surface and reducing the time required for analysis.

[0036] The configuration of the skin surface analysis device 1 will be specifically described below. As shown in Fig. 1, the skin surface analysis device 1 can be configured, for example, by a personal computer or the like, and includes a main body 10, a monitor 11, a keyboard 12, and a mouse 13. For example, the skin surface analysis device 1 can be configured by installing a program for executing the control contents, image processing, arithmetic processing, and statistical processing described below in a general-purpose personal computer. The skin surface analysis device 1 may also be configured by dedicated hardware in which the program is implemented.

[0037] The monitor 11 displays various images, user interface images for settings, etc., and can be configured, for example, with a liquid crystal display. The keyboard 12 and mouse 13 are conventionally used as operating means for personal computers, etc. Instead of the keyboard 12 and mouse 13, or in addition to the keyboard 12 and mouse 13, a touch operation panel or the like may be provided. The main body 10, monitor 11, and operating means may be integrated.

[0038] As shown in FIG. 2, the main body 10 includes a communication unit 10a, a control unit 10b, and a storage unit 10c. The communication unit 10a is a unit that exchanges data with the outside, and is composed of various communication modules. By connecting to a network line such as the Internet via the communication unit 10a, it becomes possible to read data from the outside and to send data from the main body 10. The storage unit 10c is composed of, for example, a hard disk or SSD (Solid State Drive), and is capable of storing various images, setting information, analysis results, statistical processing results, and the like. The storage unit 10c may be composed of an external storage device, or may be composed of a so-called cloud server, or the like.

[0039] Although not shown, the control unit 10b can be configured with, for example, a system LSI, an MPU, a GPU, a DSP, or dedicated hardware, and performs numerical calculations and information processing based on various programs, and controls each part of the hardware. Each piece of hardware is connected to be able to communicate bidirectionally or one-way via an electrical communication path (wiring) such as a bus. The control unit 10b is configured to be able to perform various processes, as described below, and these processes may be realized by a logic circuit or may be realized by executing software. The processes that the control unit 10b can perform include various general image processes. The control unit 10b can also be configured with a combination of hardware and software.

[0040] First, the configuration of the control unit 10b will be described, and then a skin surface analysis method performed by the control unit 10b will be described with specific image examples.

[0041] (Configuration of control unit 10b) The control unit 10b is capable of capturing an image from the outside via the communication unit 10a or directly. The captured image can be stored in the storage unit 10c. The captured image is an image captured by enlarging the transfer surface of the transfer material 100 with a stereomicroscope 101, and is, for example, the image that is the basis of Figures 5A and 5B. The captured image may be a color image or a grayscale image that has been converted into a grayscale image.

[0042] The control unit 10b includes an image input unit 20 to which a color image or a grayscaled image is input. An image that has been grayscaled by a grayscale processing unit 21 described later may be input to the image input unit 20, or an image that has been grayscaled in advance outside the skin surface analysis device 1 may be input. When inputting an image to the image input unit 20, the user of the skin surface analysis device 1 can input the image, similar to the above-mentioned reading of an image into the grayscale processing unit 21. A color image can also be input to the image input unit 20.

[0043] The control unit 10b includes a grayscale processing unit 21 that converts the captured image into a grayscale image when the image is a color image. It is not necessary to convert the color image into a grayscale image, and the local image enhancement process and subsequent processes described later may be performed on the color image as is.

[0044] For example, the image can be captured by a user of the skin surface analysis device 1. For example, the image enlarged by the stereo microscope 101 is captured by an imaging element (not shown), and the image data obtained thereby can then be read into the grayscale processing unit 21. In this example, an image in which the image data output from the imaging element is saved in JPEG format is used, but this is not limiting, and image data compressed in another compression format or a RAW image may also be used. Also, in this example, the image size is 1600×1200 pixels, but this can also be set arbitrarily.

[0045] The grayscale processing unit 21 converts a color image into a grayscale image, for example, with 8-bit gradation. Specifically, the grayscale processing unit 21 converts the image into an image in which the sample values ​​of each pixel constituting the image do not contain information other than luminance. This grayscale differs from a binary image, and expresses the image with light and dark grays between white, which has the strongest luminance, and black, which has the weakest luminance. This gradation is not limited to 8 bits, and can be set to any gradation.

[0046] The control unit 10b includes a local image enhancement processing unit 22. The local image enhancement processing unit 22 executes local image enhancement processing for enhancing the contrast of a local region of the grayscaled image input to the image input unit 20 to generate an enhancement-processed image. This enhances the visibility of details of the image. Examples of the local image enhancement processing include processing such as histogram equalization that enhances the visibility of details by enhancing the contrast of a local region of the image.

[0047] The control unit 10b includes a patch image generating unit 23. The patch image generating unit 23 is a unit that divides the enhancement-processed image generated by the local image enhancement processing unit 22 into a plurality of patch images. Specifically, assuming that an enhancement-processed image has a size of, for example, 1600×1200 pixels, the patch image generating unit 23 divides the image into images (patch images) each having a size of 256×256 pixels. The patch image generating unit 23 can also generate patch images such that adjacent patch images partially overlap each other. In other words, the patch images generated by the patch image generating unit 23 overlap with a portion of the adjacent patch images, and this overlap range can be set to, for example, about 64 pixels. The setting of this overlap range can be called, for example, a "64 pixel stride." Note that the above-mentioned pixel value is an example, and other values ​​can also be used.

[0048] If adjacent patch images are not overlapped when divided into a plurality of patch images, it is conceivable that the edge of a skin ridge or a sweat drop happens to overlap the boundary between adjacent patch images, and there is a risk that the accuracy of discrimination by the machine learning classifier 24 (described later) of skin ridges and sweat drops overlapping the boundary will decrease. In contrast, by partially overlapping adjacent patch images as in this example, it becomes possible to accurately discriminate even skin ridges and sweat drops located in the above-mentioned positions.

[0049] The control unit 10b includes a machine learning classifier 24. The machine learning classifier 24 is a part that receives each patch image generated by the patch image generating unit 23 and executes segmentation of each input patch image. The machine learning classifier 24 itself segments the input image according to a well-known deep learning method, and by this segmentation, for example, determines which category each pixel belongs to and outputs it as an output image. The machine learning classifier 24 has an input layer to which the input image is input and an output layer to which the output image is output, and has multiple hidden layers between the input layer and the output layer. The machine learning classifier 24 automatically extracts common features by learning a large amount of teacher data, enabling flexible judgment, and has completed learning.

[0050] In this example, the machine learning classifier 24 sets the resolution of the input image and the resolution of the output image to be the same. In the case of a general machine learning classifier, the resolution of the input image is high and the resolution of the output image is reduced before being output, but in this example, since it is necessary to accurately determine the shape of fine skin mounds and the size of sweat droplets, the resolution of the output image is not reduced. For example, when a patch image of a size of 256×256 pixels is input to the input layer of the machine learning classifier 24, an output image of a size of 256×256 pixels is output from the output layer.

[0051] Moreover, the machine learning classifier 24 of this example is configured to be able to simultaneously detect skin ridges and skin grooves and detect sweat droplets. That is, the machine learning classifier 24 has a skin ridge / skin groove detector 24a that detects skin ridges and skin grooves, and a sweat droplet detector 24b that detects sweat droplets. The skin ridge / skin groove detector 24a and the sweat droplet detector 24b can each be constructed using, for example, a Unet network.

[0052] The control unit 10b includes an entire image generating unit 25. The entire image generating unit 25 is a part that generates an entire image by synthesizing the patch images after segmentation output from the machine learning classifier 24. Specifically, the entire image generating unit 25 synthesizes the patch images output from the skin ridge / skin groove detector 24a in the same manner as the image before division to generate an entire image for skin ridge / skin groove discrimination, and also synthesizes the patch images output from the sweat drop detector 24b in the same manner to generate an entire image for sweat drop discrimination. The entire image has the same size as the image before division.

[0053] The control unit 10b includes a likelihood map generating unit 26. The likelihood map generating unit 26 is a part that generates a likelihood map image of the skin ridge based on the segmentation result by the machine learning classifier 24 from the whole image for determining the skin ridge and skin groove generated by the whole image generating unit 25. The likelihood map image is an image in which the likelihood of each pixel is displayed in a different color according to the likelihood, and indicates which pixels have a high or low likelihood relatively. For example, the pixel with the highest likelihood is red, the pixel with the lowest likelihood is blue, and a color map image in which the pixels between them are expressed in 8-bit gradation can be used as the likelihood map image of the skin ridge and skin groove. Note that this display form is one example, and the image may be displayed in grayscale, may be displayed in a form with different brightness, or may have a gradation other than 8 bits.

[0054] Furthermore, the likelihood map generating unit 26 generates a likelihood map image of sweat droplets based on the segmentation result by the machine learning classifier 24 from the entire image for discriminating sweat droplets generated by the entire image generating unit 25. The likelihood map image of sweat droplets can be a color map image in which pixels having the highest likelihood of being sweat droplets are colored red and pixels having the lowest likelihood of being sweat droplets are colored blue, and the range between these is expressed in 8-bit gradation. The likelihood map image of sweat droplets may be displayed in grayscale, as in the case of skin ridges and skin grooves, or in a display form with different brightness, and the gradation may be other than 8-bit.

[0055] The control unit 10b has a binarization processing unit 27. The binarization processing unit 27 is a part that performs binarization processing on the likelihood map image generated by the likelihood map generating unit 26 to generate a binarized image (black and white image). The threshold value Th used in the binarization processing may be set to any value, and for example, when the gradation is 8 bits, it can be set to a value such as Th=150. By using a likelihood map image based on the entire image for distinguishing skin ridges and skin grooves, it is possible to distinguish, for example, that black represents skin grooves and white represents skin ridges. Also, by using the entire image for distinguishing sweat droplets, it is possible to distinguish, for example, that white represents sweat droplets and black represents areas other than sweat droplets.

[0056] The control unit 10b includes a region extraction unit 28. The region extraction unit 28 is a part that extracts a skin ridge region based on the binarized image generated by the binarization processing unit 27. Specifically, when the black in the binarized image is a skin ridge, a group of white pixels in the binarized image is extracted as a skin ridge region. The region extraction unit 28 may also extract a skin groove region based on the binarized image generated by the binarization processing unit 27. In this case, when the black in the binarized image is a skin groove, a group of white pixels in the binarized image is extracted as a skin groove region. After extracting the skin groove, the region extraction unit 28 may extract other regions as skin ridge regions. After extracting the skin ridge, the region extraction unit 28 may also extract other regions as skin groove regions.

[0057] The control unit 10b includes a sweat drop extraction unit 29. The sweat drop extraction unit 29 is a unit that extracts sweat drops based on the likelihood map image of sweat drops. Specifically, when the white (or red) of the likelihood map image of sweat drops is sweat drop, a group of white (or red) pixels of the likelihood map image of sweat drops is extracted as sweat drops. The sweat drop extraction unit 29 may also extract a portion other than the sweat drop based on the likelihood map image of sweat drops. In this case, when the black (or blue) of the likelihood map image of sweat drops is the portion other than sweat drops, a group of black (or red) pixels of the likelihood map image of sweat drops is extracted as the portion other than sweat drops. After extracting the portion other than sweat drops of the likelihood map image of sweat drops, the sweat drop extraction unit 29 may extract the other area as sweat drops.

[0058] The transfer material 100 may contain air bubbles, which may be judged as sweat droplets. In this case, a discrimination method using dimensions is also applied. For example, by setting "40 μm" as an example of a discrimination threshold, small areas with a diameter of 40 μm or less are judged to be air bubbles, and only areas with a diameter exceeding 40 μm are judged to be sweat droplets. Another example of the discrimination threshold is area, and for example, the area of ​​a circle with a diameter of 40 μm is calculated, and small areas less than this area are judged to be air bubbles, and only areas exceeding this area are judged to be sweat droplets. The above "diameter" may be, for example, the major axis in the case of an ellipse approximation.

[0059] The control unit 10b includes a skin ridge analysis unit 30. The skin ridge analysis unit 30 is a part that calculates the area of ​​the skin ridge region extracted by the region extraction unit 28. The skin ridge analysis unit 30 can obtain the shape of the skin ridge by, for example, generating a contour line surrounding the skin ridge region extracted by the region extraction unit 28. The skin ridge analysis unit 30 can calculate the area of ​​the skin ridge by determining the area of ​​the region surrounded by the contour line of the skin ridge. The skin ridge analysis unit 30 can also obtain the shape of the skin groove by, for example, generating a contour line surrounding the skin groove region extracted by the region extraction unit 28. The skin ridge analysis unit 30 can also calculate the area of ​​the skin groove by determining the area of ​​the region surrounded by the contour line of the skin groove.

[0060] The skin ridge analysis unit 30 sets a plurality of grids of a predetermined size on the binary image or grayscale image, and calculates the ratio of the skin ridge area and the skin groove area in each grid. Specifically, when the grid is set to equally divide the binary image into nine parts and the first to ninth divided images are assumed, the skin ridge analysis unit 30 calculates the area of ​​the skin ridge area and the skin groove area included in each divided image, and calculates the ratio of the area of ​​the skin ridge area and the area of ​​the skin groove area. For example, when it is desired to evaluate the fineness of the skin surface, it can be evaluated based on the ratio of the skin ridge area and the skin groove area in the grid set on the binary image or grayscale image. When the ratio of the skin ridge area is a predetermined value or more, it can be a criterion for judging that the texture is coarse. Also, when the ratio of the skin ridge area is less than a predetermined value, it can be a criterion for judging that the texture is fine.

[0061] In the following description of the embodiment, the results of analyzing the skin ridges and skin grooves (skin ridges are close to white and skin grooves are close to black) in a grayscale image by the skin ridge analysis unit 30 are used. In the case of a healthy person, the boundary between the skin ridges and skin grooves is clear, and it is possible to measure the area of ​​the skin ridges, but in the case of a patient with atopic dermatitis, the boundary between the skin ridges and skin grooves may not be clear. In this case, the grayscale image is used directly for analysis, the ratio of the skin ridges and skin grooves is divided into multiple grid sizes, and the ratio of the skin ridges and skin grooves is analyzed using the grayscale values ​​of the pixels in the grid, and by displaying it as a histogram, it can be used as a criterion for judging the fineness of the skin texture, etc. (described later).

[0062] The skin ridge analysis unit 30 also quantifies the ratio of the skin ridge area to the skin groove area in each grid to calculate a frequency distribution. Specifically, after calculating the ratio of the area of ​​the skin ridge area to the area of ​​the skin groove area, it quantifies it and collects the data in the form of a frequency distribution table. The skin ridge analysis unit 31 can also calculate the position of the center of gravity of each skin ridge area, the perimeter of the skin ridge area, the rectangular approximation, the elliptical approximation, the circularity, the aspect ratio, the density, etc.

[0063] In addition, depending on the pathology, a groove may be formed in a part of the skin ridge, and in this case, a non-convex part, i.e., a concave part, will be present in the extracted skin ridge region. Dividing the skin ridge region by this concave part can be a criterion for appropriately evaluating the pathology and clinical evaluation. In response to this, after extracting the skin ridge region, the skin ridge analysis unit 30 judges whether each part of the extracted skin ridge region is convex or not, and divides the skin ridge region by the part that is judged to be non-convex. For example, a groove-shaped part may be present in the skin ridge region, and in this case, the entire skin ridge region is not convex, but a part (groove-shaped part) is concave. Since the part that is judged to be non-convex, i.e., the part that is judged to be concave, is a groove-shaped part, the skin ridge region is divided by this groove-shaped part, and one skin ridge region becomes multiple skin ridge regions.

[0064] The control unit 10b includes a sweat droplet analysis unit 31. The sweat droplet analysis unit 31 calculates the distribution of sweat droplets extracted by the sweat droplet extraction unit 29. The sweat droplet analysis unit 31 calculates the distribution of sweat droplets per unit area (1 mm 2 , 1cm 2 The number of sweat droplets present on the skin surface (e.g., the size (diameter) of each sweat droplet, the area of ​​the sweat droplet, etc.) can be calculated. The sweat droplet analysis unit 31 can also calculate the total area of ​​sweat droplets present per unit area of ​​the skin surface.

[0065] The control unit 10b includes an information output unit 32. The information output unit 32 generates and outputs information on the shape of the skin ridge region extracted by the region extraction unit 28 and information on sweat droplets extracted by the sweat droplet extraction unit 29. The information on the shape of the skin ridge region includes the results calculated by the skin ridge analysis unit 30, such as the area of ​​the skin ridge region, the center of gravity of the skin ridge region, the perimeter of the skin ridge region, rectangular approximation, elliptical approximation, circularity, aspect ratio, and density. The information on sweat droplets includes the results calculated by the sweat droplet analysis unit 31, such as the number of sweat droplets per unit area and the total area of ​​sweat droplets per unit area.

[0066] (Skin surface analysis method) Next, a skin surface analysis method performed using the skin surface analysis device 1 configured as above will be described with reference to specific image examples. The flow of the skin surface analysis method is as shown in the flowcharts of Figures 3 and 4. In step S1 of the flowchart shown in Figure 3, IMT is performed. In this step, as shown in Figure 1, a dental silicone impression material is applied in the form of a film to the skin surface and left for a predetermined period of time, and then the silicone impression material is peeled off from the skin to obtain a transfer material 100 to which a human skin surface structure has been transferred.

[0067] Then, the process proceeds to step S2. In step S2, the transfer material 100 is set on the stereo microscope 101 and observed at a predetermined magnification, and the observation field is imaged by the imaging element. As a result, a color image (1600 x 1200 pixels) in JPEG format is acquired. Next, the process proceeds to step S3, where the color image captured by the imaging element is read into the control unit 10b of the skin surface analysis device 1. Then, the process proceeds to step S4, where the color image read in step S3 is converted to 8-bit grayscale by the grayscale processing unit 21 (shown in FIG. 2) to generate a grayscale image. An example of the generated grayscale image is shown in FIG. 6. The light-colored parts in the grayscale image are skin ridges and the dark-colored parts are skin grooves, but the boundaries between them are not clear, and when an inspector judges which parts in the image are skin grooves or which parts are skin ridges, it takes time, and there is a limit to the number of samples that can be processed within a certain time. If the image read into the control unit 10b is a grayscale image, grayscale processing is not necessary.

[0068] In the following step S5, the grayscale image is input to the image input unit 20. This step is the image input step. After that, in step S6, the local image enhancement processing unit 22 executes local image enhancement processing on the grayscale image input in step S5. This step is the local image enhancement processing step. An image after local image enhancement processing is performed is shown in FIG. 7. It can be seen that the image shown in FIG. 7 has enhanced contrast in local regions and improved visibility of details compared to the image shown in FIG. 6.

[0069] Then, the process proceeds to step S7. In step S7, the patch image generating unit 23 divides the enhancement-processed image generated in step S6 into a plurality of patch images. FIG. 8 shows the division into patch images, with grid lines corresponding to the boundaries of the patch images. At this time, adjacent patch images are overlapped with each other in the vertical and horizontal directions of the figure by a "64 pixel stride." This step is the patch image generating step.

[0070] After generating the patch images, the process proceeds to step S8. In step S8, each patch image generated in step S7 is input to the machine learning classifier 24, and segmentation of each input patch image is performed by the machine learning classifier 24. At this time, the same patch image is input to both the skin ridge / skin groove detector 24a and the sweat drop detector 24b (steps S9 and S10). This step is the segmentation step.

[0071] Specifically, as shown in Fig. 9, when eight patch images exist as an input image, the eight patch images are input to the skin ridge / skin groove detector 24a and also to the sweat drop detector 24b. The skin ridge / skin groove detector 24a generates and outputs an output image in which the color of each pixel is set to whiter for all input images as the likelihood of a skin ridge increases, and to blacker for all input images as the likelihood of a skin ridge decreases (the likelihood of a skin groove increases). The sweat drop detector 24b generates and outputs an output image in which the color of each pixel is set to whiter for all input images as the likelihood of a sweat drop increases, and to blacker for all input images as the likelihood of a sweat drop decreases.

[0072] 9 shows an example of a skin ridge / skin groove output image output from the skin ridge / skin groove detector 24a and a sweat drop output image output from the sweat drop detector 24b. The white parts in the skin ridge / skin groove output image are skin ridge regions, and the black parts are skin groove regions. The white parts in the sweat drop output image are sweat drops.

[0073] In this example, as described above, when dividing into a plurality of patch images in step S7, adjacent patch images are overlapped. If the patch images were not overlapped, it is possible that the edge of a skin ridge or a sweat drop happens to overlap the boundary between adjacent patch images, and the accuracy of identifying the skin ridge or sweat drop overlapping the boundary may decrease. In contrast, in this example, adjacent patch images are partially overlapped with each other, so that even skin ridges and sweat drops at the above-mentioned positions can be identified with high accuracy.

[0074] Then, the process proceeds to step S11, where the skin ridge and skin groove output images (patch images) after step S9 are synthesized to generate an entire image as shown in Fig. 10. Also, in this step S11, the sweat drop output images (patch images) after step S9 are synthesized to generate an entire image as shown in Fig. 11. The number of pixels in each entire image is the same as the number of pixels in the image input in step S5. This step is the entire image generation step.

[0075] Next, proceeding to step S12 shown in Fig. 4, the likelihood map generating unit 26 generates a likelihood map image of the skin ridge and a likelihood map image of the sweat droplet based on the segmentation result from the overall image generated in step S11. This step is the likelihood map generating step. Fig. 12 shows an example of the likelihood map image of the skin ridge. For convenience, this figure shows a grayscale image, but in this example, the pixel with the highest likelihood of being a skin ridge is red, the pixel with the lowest likelihood of being a skin ridge is blue, and a color image is displayed with 8-bit gradation between them. This makes it easier to distinguish between the skin ridge region and the skin groove region.

[0076] Fig. 13 shows an example of a likelihood map image of sweat droplets. This image is also originally in color, with pixels with the highest likelihood of being sweat droplets in red and pixels with the lowest likelihood of being sweat droplets in blue, and the intermediate pixels are expressed in 8-bit gradations. This makes it easy to distinguish sweat droplets.

[0077] After generating the likelihood map image of the skin tumulus and the likelihood map image of the sweat droplet, the process proceeds to step S13. In step S13, a binarization process is performed on the likelihood map image of the skin tumulus generated in step S12 to generate a binary image. This step is performed by the binarization processor 27, and is a binary processing step. Fig. 14 shows the binary image generated by performing a binarization process on the likelihood map image of the skin tumulus.

[0078] Then, the process proceeds to step S14, where the region extraction unit 28 extracts a skin ridge region based on the binarized image generated in step S13. At this time, a skin groove region may be extracted. FIG. 15 shows an image in which the skin ridge and skin groove are extracted, and the skin ridge region is displayed by surrounding it with a black line. This step is the region extraction step.

[0079] In step S15, the sweat droplet extraction unit 29 extracts sweat droplets based on the sweat droplet likelihood map image generated in step S12. This step is a sweat droplet extraction step. Fig. 16 shows an image in which sweat droplets have been extracted, and the sweat droplets are displayed by being surrounded by black lines.

[0080] Next, the process proceeds to step S16. In step S16, the position of the sweat droplet is compared with the skin ridges and skin grooves. The position and range of the sweat droplet can be specified by the XY coordinates on the image. The position and range of the skin ridges and skin grooves can also be specified by the XY coordinates on the image. Since the image specifying the position and range of the sweat droplet and the image specifying the position and range of the skin ridges and skin grooves are originally the same, the sweat droplet can be placed on the image showing the skin ridges and skin grooves, as shown in FIG. 17. This makes it possible to obtain the relative positional relationship between the sweat droplet and the skin ridges and skin grooves. At this time, the area of ​​the skin ridge and the coordinates of the center of gravity of the sweat droplet can be used.

[0081] Then, the process proceeds to step S17. In step S17, sweat droplets located on the skin ridges and skin grooves are identified. FIG. 18 shows an image in which sweat droplets located on the skin ridges and skin grooves have been identified, which makes it possible to distinguish between sweat droplets located on the skin ridges and sweat droplets located in the skin grooves. In FIG. 18, the droplets that are nearly round are sweat droplets.

[0082] After the identification, the process proceeds to steps S18 and S19. Steps S18 and S19 can be performed in any order. In step S18, a histogram of skin ridge information is created and displayed on the monitor 11. First, the skin ridge analysis unit 30 calculates the area of ​​each skin ridge region extracted in step S14. Then, as shown in FIG. 19, a histogram is created with the area on the horizontal axis and the frequency on the vertical axis. This step is the skin ridge analysis step. This makes it possible to grasp the distribution of the area of ​​the skin ridge regions. For example, in the case of atopic dermatitis, the area of ​​one skin ridge tends to be large, and if the frequency of a large area is high, it can be seen that there is a strong tendency for sweating disorders in atopic dermatitis.

[0083] In step S19, a heat map image of sweat droplets is created and displayed on the monitor 11. First, the sweat droplet analysis unit 31 calculates the distribution of sweat droplets extracted in step S15. For example, as shown in FIG. 20, a grid is formed on an image of the transfer material 100, and the number of sweat droplets present in each grid is counted. This is possible by determining in which grid the coordinates of the center of gravity of the sweat droplets extracted in step S15 are located. For example, a grid with no sweat droplets, a grid with one sweat droplet, a grid with two sweat droplets, a grid with three sweat droplets, ... are colored, and the distribution of sweat droplets can be grasped by coloring each grid, and an image displayed in such a color-coded manner can be called a heat map image. This step is a sweat droplet analysis step. If the distribution of sweat droplets is sparse, it can be seen that there is a strong tendency for sweating disorders due to atopic dermatitis.

[0084] Another advantage of creating a heat map image is that it allows the state of sweating and skin ridges in a small area to be judged as a pattern, which cannot be obtained from individual analysis areas, or cannot be judged by averaging the entire area even in a wide area. Also, the heat map images can be arranged in chronological order and displayed on the monitor 11. For example, by generating heat map images one week, two weeks, and three weeks after a patient with atopic dermatitis starts treatment and displaying them in a list format, it is possible to judge whether the symptoms have improved and to quantitatively judge the progress.

[0085] Fig. 21 is an example of a skin ridge region image showing lines surrounding each skin ridge region extracted in step S14. The image shown in this figure can be generated by the skin ridge analysis unit 30 and displayed on the monitor 11. In the figure, when the 15th skin ridge region indicated by "15" and the 16th skin ridge region indicated by "16" exist, the skin ridge analysis unit 30 creates the measurement results of the specifications in a table format as shown in Fig. 22, and displays it on the monitor 11.

[0086] In the table shown in FIG. 22, "Label" is provided to distinguish the 15th skin ridge region from the 16th skin ridge region. "Area" is the area of ​​the skin ridge region, "XM" and "YM" are the center of gravity of the skin ridge region, "Perimeter" is the perimeter of the skin ridge region, "BX", "BY", "Width" and "Height" are rectangular approximations, "Major", "Minor" and "Angle" are elliptical approximations, "Circularity" is circularity, "Aspect Ratio" is aspect ratio, and "Solidity" is a property indicating density. These property values ​​can be calculated by the skin ridge analysis unit 30 using, for example, image analysis software. In this way, by using not only one index but multiple indexes, it is possible to make a judgment in correspondence with clinical information. In addition, since this index can also contribute to the determination of the fineness of the skin surface, it is also possible to determine the fineness of the skin surface using the machine learning classifier 24. In addition, as shown in FIG. 22, it is also possible to process it statistically (total, maximum, minimum, deviation, etc.).

[0087] FIG. 23 is a graph showing the two-dimensional distribution of skin ridges and skin grooves per 128×128 pixel grid, and such a graph can also be generated by the skin ridge analysis unit 30 and displayed on the monitor 11. For example, it can be displayed as an 8-bit color image with the skin ridge area in red and the skin groove area in blue. For example, it can be used as an example of a method for expressing the fineness of the skin surface and the improvement of symptoms, and can be made into a heat map. When the ratio of the area of ​​the skin ridges and skin grooves is quantified and displayed as a histogram, if the skin surface is fine, the frequency around the median will be high, while in the case of atopic dermatitis, the distribution will be wider overall and the tail will be wider. This makes it possible to quantify two-dimensional information and use it as diagnostic information.

[0088] FIG. 24 shows a case where the skin ridge analysis unit 30 sets a plurality of grids (18 in this example) of a predetermined size on the image and calculates the ratio of the skin ridge area to the skin groove area in the grid. In this case, the ratio of the skin ridge area to the skin groove area in each grid can be quantified to calculate a frequency distribution, which can be displayed on the monitor 11 in the form of a histogram. For example, a method of using only the area of ​​the skin ridge to evaluate the fineness of the skin surface is conceivable, but in that case, when two adjacent skin ridges are very close and are judged as one skin ridge, the skin ridge becomes about twice as large, which may cause the analysis result to be inaccurate. By calculating the ratio of the skin ridge and the skin groove in each grid as in this example, the fineness of the skin surface can be quantitatively calculated.

[0089] FIG. 25 is an image obtained by synthesizing imaging areas of 3×3=9 fields of view. This allows a wide range to be observed, and the above-mentioned analyses are performed on the image of an average field of view of sweating from the wide range. For example, if only one field of view is focused on, it is not possible to determine whether the field of view has little sweating, a lot of sweating, or average sweating. However, by analyzing sweat droplets in all of the wide fields of view of about 9 fields of view, it is possible to exclude the fields of view with little sweating and the fields of view with a lot of sweating and select an average field of view, that is, a field of view suitable for skin surface analysis. This makes the analysis results more accurate. In the case of analysis by an inspector, due to time constraints, the number of fields of view to be processed was about three, but the present invention makes it possible to analyze a large number of fields of view that far exceed three, and more accurate analysis of the skin surface is possible.

[0090] The skin ridge analysis unit 30 can also display images such as those shown in Fig. 25 in chronological order on the monitor 11. For example, by generating images as shown in Fig. 25 one week, two weeks, and three weeks after a patient with atopic dermatitis starts treatment and displaying them in a list format on the monitor 11, it is possible to determine whether the symptoms have improved and to quantitatively determine the progress of the condition.

[0091] (Quantification of skin texture based on the ratio of skin ridges and sulci) FIG. 26 is a graph (histogram) showing the ratio of skin ridges and skin grooves of the forearm of a healthy subject, in which a 100×100 pixel grid is set in a grayscale image. The horizontal axis is the ratio of skin ridges and skin grooves, and the vertical axis is the number. The graph on the right side of FIG. 26 also shows a graph of kernel density estimation. Similarly, FIG. 27 shows the case where a 150×150 pixel grid is set, FIG. 28 shows the case where a 200×200 pixel grid is set, and FIG. 29 shows the case where a 250×250 pixel grid is set.

[0092] When the skin is finely textured, such as the forearm of a healthy subject, the distribution has a peak in the center for all grid sizes of 100 x 100, 150 x 150, 200 x 200, and 250 x 250 pixels. In addition, because the ratio of skin ridges to skin grooves can be determined depending on the grid size, it is possible to quantify not only the size of the skin ridges but also the size of the skin grooves.

[0093] Next, the case of a patient with atopic dermatitis will be described. Figures 30 to 33 are graphs showing the ratio of skin ridges and skin sulci on the thighs of a patient with atopic dermatitis, and correspond to Figures 26 to 29, respectively. Compared to the graphs of Figures 26 to 29 showing healthy subjects, the peaks are shifted from the center and multiple peaks are formed, so by looking at these graphs, the differences between healthy subjects and patients with atopic dermatitis can be understood.

[0094] Moreover, Figures 34 to 37 are graphs showing the ratio of skin ridges and skin grooves on the forehead of patients with atopic dermatitis, and correspond to Figures 26 to 29, respectively. Compared to the graphs of Figures 26 to 29 showing healthy subjects, the peaks are generally shifted to the right (the side with a larger ratio of skin ridges and skin grooves) and there are multiple peaks, so by looking at these graphs, it is possible to understand the differences between healthy subjects and patients with atopic dermatitis as well as to understand the fineness of the skin texture and skin condition of patients with atopic dermatitis, making it possible to present the effectiveness of treatment as an objective indicator in follow-up observations.

[0095] Figures 38 to 41 are graphs showing the ratio of skin ridges and skin sulci on the elbows of patients with atopic dermatitis, and correspond to Figures 26 to 29, respectively. Compared to the graphs of Figures 26 to 29 showing healthy subjects, the peaks are shifted from the center and there are multiple peaks, so by looking at these graphs it is possible to understand the differences between healthy subjects and patients with atopic dermatitis, as well as to understand the fineness of the skin texture and skin condition of patients with atopic dermatitis, making it possible to present the effectiveness of treatment as an objective indicator during follow-up observation.

[0096] (Effects of the embodiment) As described above, according to this embodiment, a likelihood map image of the skin surface is generated using the machine learning classifier 24, and the likelihood map image can be used to identify skin ridge regions and sweat droplets, thereby eliminating individual differences during analysis, improving the accuracy of analysis of the skin surface condition, and shortening the time required for analysis.

[0097] The above-described embodiment is merely illustrative in all respects and should not be construed as limiting. Furthermore, all modifications and variations within the scope of the claims are within the scope of the present invention. [Industrial Applicability]

[0098] As described above, the skin surface analysis device and the skin surface analysis method according to the present invention can be used, for example, when analyzing the human skin surface. [Explanation of symbols]

[0099] 1 Skin surface analysis device 20 Image input section 21 Grayscale processing section 22 Image enhancement processing section 23 Patch image generation unit 24 Machine Learning Classifier 24a Skin ridge and skin groove detector 24b Sweat Droplet Detector 25 Whole image generation unit 26 Likelihood map generator 27 Binarization processing section 28 Region extraction part 29 Sweat Droplet Extraction Unit 30 Skin hill analysis department 31 Sweat Droplet Analysis Unit 100 Transfer material

Claims

1. A skin surface analysis device for analyzing a skin surface using a transfer material onto which a human skin surface structure is transferred, an image input unit to which an image of the transfer material is input; a local image enhancement processing unit that performs local image enhancement processing to enhance the contrast of a local region of the image input to the image input unit and generates an enhancement-processed image; a patch image generating unit that divides the enhancement-processed image generated by the local image enhancement processing unit into a plurality of patch images; a machine learning classifier that receives the patch images generated by the patch image generation unit and executes segmentation of the input patch images; an overall image generating unit that generates an overall image by synthesizing the segmented patch images output from the machine learning classifier; a likelihood map generating unit that generates a likelihood map image of a skin ridge based on the segmentation result from the entire image generated by the entire image generating unit; a binarization processing unit that performs a binarization process on the likelihood map image generated by the likelihood map generating unit to generate a binarized image; a region extraction unit that extracts a skin ridge region based on the binarized image generated by the binarization processing unit; A skin surface analysis device comprising a skin tumour analysis unit that calculates an area of ​​the skin tumour region extracted by the region extraction unit.

2. A skin surface analysis device for analyzing a skin surface using a transfer material onto which a human skin surface structure is transferred, an image input unit to which an image of the transfer material is input; a local image enhancement processing unit that performs local image enhancement processing to enhance the contrast of a local region of the image input to the image input unit and generates an enhancement-processed image; a patch image generating unit that divides the enhancement-processed image generated by the local image enhancement processing unit into a plurality of patch images; a machine learning classifier that receives the patch images generated by the patch image generation unit and executes segmentation of the input patch images; an overall image generating unit that generates an overall image by synthesizing the segmented patch images output from the machine learning classifier; a likelihood map generating unit that generates a likelihood map image of sweat droplets based on the segmentation result from the entire image generated by the entire image generating unit; a sweat droplet extraction unit that extracts sweat droplets based on the likelihood map image generated by the likelihood map generation unit; A skin surface analysis device comprising: a sweat droplet analysis unit that calculates a distribution of the sweat droplets extracted by the sweat droplet extraction unit.

3. The skin surface analysis device according to claim 1, a likelihood map generating unit that generates a likelihood map image of sweat droplets based on the segmentation result from the entire image generated by the entire image generating unit; a sweat droplet extraction unit that extracts sweat droplets based on the likelihood map image generated by the likelihood map generation unit; A skin surface analysis device comprising: a sweat droplet analysis unit that calculates a distribution of the sweat droplets extracted by the sweat droplet extraction unit.

4. The skin surface analysis device according to any one of claims 1 to 3, The transfer material is obtained by the Impression mold technique; A skin surface analysis device comprising a grayscale processing unit that converts the image of the transfer material into a grayscale image.

5. The skin surface analysis device according to any one of claims 1 to 4, The skin surface analysis device, wherein the patch image generating unit generates the patch images such that adjacent patch images partially overlap each other.

6. The skin surface analysis device according to any one of claims 1 to 5, A skin surface analysis device, characterized in that the machine learning classifier has the same resolution as the input image and the output image.

7. The skin surface analysis device according to claim 1, The skin surface analysis device is characterized in that the skin ridge analysis unit sets a plurality of grids of a predetermined size on the image and calculates the ratio of skin ridge areas to skin sulcus areas within each of the grids.

8. The skin surface analysis device according to claim 7, The skin surface analysis device is characterized in that the skin ridge analysis unit quantifies the ratio of skin ridge regions and skin groove regions in each grid to calculate a frequency distribution.

9. The skin surface analysis device according to claim 1, The skin surface analysis device is characterized in that, after extracting the skin ridge region, the region extraction unit determines whether each part of the extracted skin ridge region is convex or not, and divides the skin ridge region by parts that are determined to be not convex.

10. The skin surface analysis device according to claim 3, A skin surface analysis device comprising an information output unit that generates and outputs information regarding the shape of the skin tumour region extracted by the region extraction unit.

11. A skin surface analysis method for analyzing a skin surface using a transfer material to which a human skin surface structure is transferred, comprising: an image input step of inputting an image of the transfer material; a local image enhancement processing step of performing local image enhancement processing to enhance contrast in a local region of the image inputted in the image input step and generating an enhancement processed image; a patch image generating step of dividing the enhancement-processed image generated by the local image enhancement processing step into a plurality of patch images; a segmentation step of inputting each patch image generated by the patch image generation step into a machine learning classifier and executing segmentation of each input patch image by the machine learning classifier; an overall image generating step of synthesizing the patch images after the segmentation step to generate an overall image; a likelihood map generating step of generating a likelihood map image of a skin tumour based on the segmentation result from the entire image generated by the entire image generating step; a binarization processing step of performing a binarization process on the likelihood map image generated by the likelihood map generating step to generate a binary image; a region extraction step of extracting a skin ridge region based on the binarized image generated by the binarization processing step; A skin surface analysis method comprising a skin ridge analysis step of calculating an area of ​​the skin ridge region extracted by the region extraction step.

12. A skin surface analysis method for analyzing a skin surface using a transfer material to which a human skin surface structure is transferred, comprising: an image input step of inputting an image of the transfer material; a local image enhancement processing step of performing local image enhancement processing to enhance contrast in a local region of the image inputted in the image input step and generating an enhancement processed image; a patch image generating step of dividing the enhancement-processed image generated by the local image enhancement processing step into a plurality of patch images; a segmentation step of inputting each patch image generated by the patch image generation step into a machine learning classifier and executing segmentation of each input patch image by the machine learning classifier; an overall image generating step of synthesizing the patch images after the segmentation step to generate an overall image; a likelihood map generating step of generating a likelihood map image of sweat droplets based on the segmentation result from the entire image generated by the entire image generating step; a sweat droplet extraction step of extracting sweat droplets based on the likelihood map image generated by the likelihood map generation step; a sweat droplet analysis step of calculating a distribution of the sweat droplets extracted in the sweat droplet extraction step.

Images