Method for detecting surface shape

Abstract

To provide a method for detecting small irregularities by imaging and image analysis by a portable terminal including a smartphone.SOLUTION: The method for detecting a surface shape includes the steps of: forming a thin film on a surface of a detection target object; taking an image of a part where a thin film to detect is formed; analyzing the taken image; and specifying a part which corresponds to a protruding part and / or a recessed part of the detection target object in the taken image on the basis of the result of analysis.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a method for detecting the surface shape by image analysis using a thinning technique.

Background Art

[0002] The development of technology for measuring the surface unevenness of an object has been remarkable. For example, in industrial applications, it is used for measuring the surface roughness of concrete, and in product quality evaluation, for confirming surface smoothness. More recently, it is also used for measuring the skin surface as a part of skin care services. In particular, due to recent technological developments such as the high-precision improvement of image sensing of imaging devices, the high-performance improvement of computers capable of high-speed calculation processing in analysis, and the generalization of tools due to the expansion of applications, image analysis technology has become more generalized.

[0003] So far, in order to measure fine unevenness, dedicated measuring instruments with high-precision measurement capabilities, dedicated software, etc. have been utilized. Therefore, although fine unevenness could be measured, the shape and location of the measurement object were limited. On the other hand, emphasizing mobility and utilizing mobile terminals such as smartphones, it has also become possible to measure surface unevenness by image analysis technology.

[0004] Due to generalization, image analysis technology is used for applications such as skin measurement. For example, in Patent Document 1, a mobile terminal and an image analysis system are utilized to digitize the wrinkle state, fineness of texture, etc. of the skin and use it for predicting the skin state.

[0005] Also, for example, in Patent Document 2, the face is widely photographed, image analysis is performed, and the ratio of the lines on the face is calculated to measure the skin state.

[0006] Also, for example, in Patent Document 3, a thin film is attached to the face, and an attempt is made to detect wrinkles and sagging by detecting the grid lines displayed on the thin film.

Prior Art Documents

Patent Documents

[0007] [Patent Document 1] Japanese Patent Publication No. 2007-175469 [Patent Document 2] Japanese Patent Publication No. 2021-121328 [Patent Document 3] Japanese Patent Publication No. 2019-198597 [Overview of the project] [Problems that the invention aims to solve]

[0008] However, while Patent Document 1 analyzes images captured with a mobile device to detect irregularities such as wrinkles and pores on the skin surface, it fails to accurately analyze the contrast of the irregularities and cannot detect the depth of the recesses and protrusions, thus failing to quantify even the finest irregularities on the skin surface. Furthermore, the range of objects that can be measured is narrow and does not address the need for a wide measurement range, such as the directionality of wrinkles.

[0009] Furthermore, while Patent Document 2 quantifies the proportion of wrinkled areas by measuring a wide area of ​​the skin surface, it fails to capture the directionality of the fine wrinkle irregularities and the contrast between recessed and raised areas, which are necessary for measuring the future condition of the skin.

[0010] Furthermore, while Patent Document 3 performs measurements over a wide area of ​​the skin surface to measure wrinkles and sagging, it is unable to detect minute wrinkles and other irregularities due to the thickness of the thin film.

[0011] This invention has been made in view of the above circumstances, and provides a method for detecting minute irregularities by imaging and image analysis using a mobile device such as a smartphone. [Means for solving the problem]

[0012] In view of the above, the surface shape detection method of the present invention is A method for detecting the surface shape of the skin, Detected object skin on the surface The material is composed of selected materials from: polyesters and copolymers thereof containing polylactic acid, polyglycolic acid, and polycaprolactone; acrylic resins and silicones and copolymers thereof; cellulose derivatives including cellulose acetate, cellulose propionate acetate, and cellulose butyrate acetate; polycarbonate, cycloolefin copolymer, styrene-butadiene elastomers, and polyimides; proteins including laminin, fibronectin, integrin, tenascin, albumin, keratin, collagen, and gelatin; and polysaccharides including chitin, chitosan, hyaluronic acid, glucomannan, pullulan, dextran, and sacran, with a thickness of 300 nm to 5000 nm. The process of forming a thin film and the light from the light source skin A step of photographing an image of reflected light reflected from a portion where a thin film is formed, a step of analyzing the photographed image, and based on the analysis result, identifying a portion corresponding to a convex portion and / or a concave portion included in the photographed image. skin It includes.

Advantages of the Invention

[0013] According to the present invention, fine irregularities on a detection object can be easily detected optically with a device such as a mobile terminal.

Brief Description of the Drawings

[0014] [Figure 1] Appearance of the thin film attachment and the untreated part on the skin surface [Figure 2] An image obtained by binarizing the image of Fig. 1 and superimposing an image of the detection range of convex portions after setting a threshold value on the appearance image of the thin film attachment and the untreated part on the skin surface [Figure 3] An image of the detection range of convex portions obtained by binarizing the image of Fig. 1 and setting a threshold value (image superimposed on Fig. 2) [[ID=2,6]] [Figure 4] An image obtained by binarizing the image of Fig. 1 and superimposing an image of the detection range of concave portions after setting a threshold value on the appearance image of the thin film attachment and the untreated part on the skin surface [Figure 5] [[ID=3,0]]An image of the detection range of concave portions obtained by binarizing the image of Fig. 1 and setting a threshold value (image superimposed on Fig. 4)

Embodiments for Carrying Out the Invention

[0015] (Detection Object) The detection object may have a surface shape capable of forming a thin film. For example, it is the epidermis of the skin, which is a living body surface, a cloth made of fibers, a plastic surface, a metal surface, a leather surface, a rubber product surface, a ceramic surface, a ceramic product surface, or a concrete surface. It is preferable that the difference between the apex and the bottom of the irregularities on the detection object surface is from 10 μm to 500 μm. Also, the detection object surface may be a horizontal plane or a curved surface.

[0016] (Method for Forming the Thin Film) The thin film is preferably formed to a thickness of 300 nm to 5000 nm in the direction perpendicular to the surface of the concavo-convex portions of the detection object in a manner capable of following the concavo-convex portions of the detection object. If the thin film is too thin, the light reflectance decreases, so it is preferably 300 nm or more. If the thickness of the thin film formed on the surface of the detection object exceeds 5000 nm, the followability of the thin film to the concavo-convex portions decreases, or the fine concavo-convex portions of the detection object are buried by the thin film and cannot be detected, which is not preferable. As a method for forming the thin film, attachment (transfer) of a thin film material sheet, or spraying (spray method) or coating of a coating solution of the thin film material can be used. Also, the thin film itself may be a single layer or a laminate in which a plurality of layers are laminated.

[0017] (Material of the thin film) The material constituting the thin film may be any material capable of constituting the thickness within the above-mentioned appropriate range. For example, polyesters such as polylactic acid, polyglycolic acid, and polycaprolactone and their copolymers can be used. Also, as a material for forming the thin film, resins used as film-forming agents for cosmetics, such as acrylic resins, silicones, and their copolymer resins, and cellulose derivatives such as cellulose acetate, cellulose propionate acetate, and cellulose acetate butyrate can also be used. Alternatively, polycarbonate, cycloolefin copolymer, styrene-butadiene-based elastomer, and polyimide, which are resins with a lot of usage records in medical devices, can be used as the material for forming the thin film. Furthermore, proteins such as laminin, fibronectin, integrin, tenascin, albumin, keratin, collagen, gelatin, and polysaccharides such as chitin, chitosan, hyaluronic acid, glucomannan, pullulan, dextran, and sacran can be used. If the material constituting the thin film is a protein, it is a biocompatible material and can be used relatively safely for living bodies such as humans and animals.

[0018] (Regarding the thin film) The color of the thin film is not particularly limited and can be white, colorless, translucent, or skin-colored. Furthermore, the thin film material may contain the above materials individually or as a mixture of two or more, and may also contain additives. These additives may include powdered inorganic oxides to cause light scattering.

[0019] Furthermore, to improve the accuracy of detecting surface irregularities on the sheet, the thin film may display images such as grid lines, linear patterns, ring patterns, or dot patterns. For example, if the thin film has grid lines or dot patterns, the surface irregularities of the skin can be detected simultaneously based on the displacement and distortion of these images, allowing for depth correction of wrinkles (depressions) and improving detection accuracy.

[0020] (Method for forming a film on a substrate for film formation) The thin film may also be formed on a film-forming substrate. In this case, it can be formed by coating the surface of the film-forming substrate with a thin film material. The materials mentioned above can be used as the thin film material. A resin sheet composed of a thermoplastic resin, a thermosetting resin, a water-soluble resin, etc., can be used as the film-forming substrate.

[0021] In detail, a coating solution, in which a thin film material is dissolved in a solvent, is applied to the surface of a substrate for film formation, and a thin film is formed when the coating solution is dried. Depending on the properties of the thin film material, a non-polar solvent, a protic polar solvent, or an aprotic polar solvent can be used as the solvent for the coating solution. Examples of non-polar solvents include benzene and hexane. Examples of protic polar solvents include water, ethanol, isopropyl alcohol, and acetic acid. Examples of aprotic polar solvents include ethyl acetate, butyl acetate, propyl acetate, ethyl methyl ketone, acetone, and dimethyl sulfoxide.

[0022] The method of applying the above coating liquid is not particularly limited as long as it is a method that can form a coating film of the desired thickness. The application method can be appropriately selected from, for example, gravure coating, microgravure coating, spin coating, spray coating, wire bar coating, and die coating.

[0023] The thin film formed on the substrate for film formation may be peeled off the substrate and supported by another support layer. Alternatively, a protective layer may be provided to protect the thin film transferred to the support layer.

[0024] (Optical detection equipment) Terminal devices with imaging capabilities are devices used by workers to take pictures, and these include digital cameras with imaging capabilities, as well as notebook PCs (Personal Computers), tablet devices, mobile phones, smartphones, and PDAs (Personal Digital Assistants).

[0025] (Method for detecting the surface shape of an object to be detected) The detection of the surface shape of the object to be detected is performed as follows. First, a thin film is formed on the surface of the object to be detected. As described above, the thin film may be formed by directly applying a coating liquid of the thin film material to the object to be detected by a spray method or the like and curing it, or by attaching a thin film that has been formed in advance on a film-forming substrate, or a thin film that has been formed on a film-forming substrate and then transferred to a support layer, to the object to be detected. Next, under natural daylight or an artificial light source, the portion of the object to be detected on which the thin film has been formed is photographed using the camera of the terminal equipment. The format (extension) of the image file is not particularly limited, and JPG, PNG, GIF, TIFF, WebP, SVG, etc. can be used. Next, the captured image is analyzed to identify the parts corresponding to the convex and / or concave portions of the surface of the object to be detected that are included in the captured image.

[0026] Furthermore, in the surface shape detection method according to this embodiment, if a thin film formed on the object to be detected can be detected, then fine irregularities can be detected. Therefore, it is possible to detect changes over time, such as the separation of the thin film from the surface of the object to be detected. For example, it is possible to detect changes in irregularities caused by the peeling of a thin film from the skin surface due to sweating or other factors after it has been applied to the skin.

[0027] Furthermore, it is possible to detect changes in the object after a thin film has been formed on it by applying force, by detecting cracks in the thin film. Cracks occur when the thin film on the object breaks. Here, "wrinkles" means that during image analysis, pixel values ​​in the depth direction around the concave and convex parts of the thin film are detected, while "cracks" means that pixel values ​​in the depth direction are not detected. From the image analysis, it can be seen that the reflection of light from the thin film decreases when the surface of the object is exposed through the crack. For example, by detecting the difference in thinning before and after the crack, it is possible to estimate the mechanical direction in which the crack occurred.

[0028] (Regarding image analysis methods) The image analysis method is not particularly limited, as long as it can identify the convex and concave parts on the surface of the object to be detected. The image analysis algorithm may be performed using a single method selected from image statistics, color space, spatial filtering, frequency filtering, geometric transformation, binary image, and pattern recognition, or a combination of multiple methods may be used. There are no specific requirements for the image analysis software; AT-Image, ImageJ, OpenCV, R / RStudio, Photoshop®, etc., can be used.

[0029] In the surface shape detection method according to this embodiment, a thin film is formed on the surface of the object to be detected. By forming the thin film so as to follow the fine irregularities on the surface of the object to be detected, without filling in the fine irregularities, the thin film provided on the convex parts diffusely reflects light, thus increasing the contrast between the convex parts and the concave parts (the boundary between the areas where light is reflected and the areas where it is not reflected becomes clearer). Therefore, parts corresponding to convex parts and parts corresponding to concave parts can be identified with high accuracy in the captured image. Furthermore, because the convex parts of the object to be detected diffusely reflect light, even parts corresponding to relatively shallow concave parts, such as fine wrinkles in the skin, have a high contrast with parts corresponding to convex parts. Therefore, by providing a thin film on the object to be detected and taking an image, the detection sensitivity of concave parts can also be improved. Moreover, in the detection method according to this embodiment, even if the area on which the thin film is formed is extensive, as long as an image can be taken, it is possible to detect irregularities based on the reflected light of the thin film. [Examples]

[0030] The detection of irregularities by thinning, as described above, will be explained using a specific example.

[0031] (Manufacturing of thin films) A coating solution for forming a thin film was prepared by dissolving DL-polylactic acid (manufactured by Musashino Chemical Research Institute Co., Ltd.) in ethyl acetate. The amount of polylactic acid in the coating solution was adjusted so that the solution had a solid content of 6%. The weight-average molecular weight of the polylactic acid was selected to be 100,000. A PET sheet (manufactured by Toray Industries, Inc.: Lumirror®, S10) was used as the substrate for film formation, and the above coating solution was applied to the substrate using a wire bar to form a coating film. The coating film was dried and solidified by heating it in an oven to form a thin film layer. The thin film layer was formed to have a thickness of 600 nm after drying. The heating temperature during drying was selected to be 120°C.

[0032] Next, a nonwoven fabric (manufactured by Futamura Chemical Co., Ltd.) was laminated onto the thin film as a support layer. The film-forming substrate was then peeled off, and the thin film was transferred from the film-forming substrate to the support layer. The main component of the nonwoven fabric was cellulose, with a basis weight of 20 g / m². 2 That was the case.

[0033] A nonwoven fabric was placed on the side of the thin film opposite the support layer as a protective layer, and it was cut into a 3 cm diameter circle using a trimming cutter, and then further cut into a crescent shape.

[0034] 100 μL of water was dropped onto the back of a human hand as a supply solution, and the water was gently spread with a finger. Then, the crescent-shaped sheet was placed on the skin so that the thin film layer was in contact with the skin, and the nonwoven fabric was removed.

[0035] After applying the thin film, the film on the skin was photographed using a smartphone (Apple: iPhone® SE) with the automatic correction function enabled under fluorescent lighting. The shooting conditions were as follows: the skin was held horizontally, the camera lens was pointed at the skin from the direction of reflection of light incident from the light source, and the reflected light from the light source was photographed at a distance of 10 cm from the skin. The captured images were saved as JPEG files.

[0036] The captured images were binarized using the image analysis software ImageJ, and the detection rate of convex and concave areas in the thinned-film areas and untreated areas of the skin surface was compared.

[0037] (result) The following shows the results of imaging and image analysis after thinning (transferring the thin film described above) was performed on the skin. Figure 1 is an external view of the area where the thin film was applied and the untreated area, captured as a single image. Figure 2 is the image from Figure 1 that has been binarized using ImageJ, with a threshold set and the detection range of convex areas superimposed on the skin image. For the binarization process, the color image was converted to a grayscale image, and the threshold for pixel values ​​from 0 to 255 was set to 148 to detect the light reflection range. Figure 3 is the image from Figure 2 that has been binarized, with the light reflection range set to white. Figure 4 is the image from Figure 1 that has been binarized using ImageJ, with a threshold set and the detection range of concave areas superimposed on the skin image. For the binarization process, the color image was converted to a grayscale image, and the threshold for pixel values ​​from 0 to 255 was set to 105 to detect the area with low light reflectivity. Figure 5 is the image from Figure 4 that has been binarized, with the area with low light reflectivity set to white.

[0038] In the image in Figure 1, the white areas in the image of the thinned skin represent the areas where light from the light source was diffusely reflected. As shown in Figure 1, the thinning process improved the ability to visually recognize raised areas compared to untreated skin due to diffuse reflection. The visibility of the difference between the thinned and recessed areas (relatively darker groove-like areas) also improved.

[0039] In the image in Figure 2, the white areas represent the protrusions extracted from the image of the thinned skin. As shown in Figure 2, the detection range of protrusions improved in the areas where the skin was thinned, and it was also possible to confirm the location and direction of wrinkles based on the orientation of the fine ridges on the skin surface.

[0040] In the image in Figure 3, the white areas represent the areas where the image in Figure 1 was binarized, a threshold was set, and light was reflected. As shown in the image in Figure 3, it was confirmed that the detection area for fine skin ridges on the skin surface was expanded in the areas where the film was thinned, and the detection of protrusions was improved.

[0041] In the image in Figure 4, the black areas represent depressions extracted from the thinned skin image. As shown in Figure 4, the detection range of depressions improved in the thinned areas, and it was also possible to confirm the position and direction of the skin furrows on the skin surface. From the results in Figure 4, it was confirmed that thinning improved the detection not only of diffusely reflective convex areas but also of depressions, and that thinning improved the contrast between convex and concave areas.

[0042] In the image in Figure 5, the white areas represent the regions where the light reflectivity is low after the image in Figure 1 has been binarized and a threshold has been set. As shown in the image in Figure 5, it was confirmed that the detection area for fine depressions on the skin surface increased in the areas where the film was thinned, and the detection of wrinkles (skin furrows) improved.

[0043] Based on the results above, it is possible to improve the detection of fine irregularities by combining thin-film processing and image analysis. [Industrial applicability]

[0044] This invention can be used to detect uneven surface shapes, such as the surface of skin.

Claims

1. A method for detecting the surface shape of skin, A step of forming a thin film with a thickness of 300 nm to 5000 nm on the surface of the skin, composed of a material selected from: polyesters and copolymers thereof containing polylactic acid, polyglycolic acid, and polycaprolactone; acrylic resins, silicones and copolymer resins thereof; cellulose derivatives including cellulose acetate, cellulose propionate acetate, and cellulose butyrate acetate; polycarbonate, cycloolefin copolymer, styrene-butadiene elastomer, polyimide; proteins including laminin, fibronectin, integrin, tenascin, albumin, keratin, collagen, and gelatin; and polysaccharides including chitin, chitosan, hyaluronic acid, glucomannan, pullulan, dextran, and sacran. A step of capturing an image of the reflected light from a light source that is reflected by the portion of the skin on which the thin film is formed, The process of analyzing the captured images, A method for detecting the surface shape of skin, comprising the step of identifying portions corresponding to convex and / or concave parts of the skin included in a captured image based on the analysis results.

2. The method for detecting the surface shape of skin according to claim 1, characterized in that, in the step of forming the thin film, the thin film is formed on which an image of grid lines, linear shapes, annular shapes, dots, or a combination thereof is displayed.

3. A method for detecting the surface shape of skin according to claim 1, characterized in that, in the step of identifying portions corresponding to convex and / or concave parts of the skin, cracks in the thin film caused by deformation of the skin by mechanical action are detected after the step of forming the thin film.

4. The method for detecting the surface shape of skin according to claim 1, characterized in that, in the step of identifying portions corresponding to convex and / or concave parts of the skin, after the step of forming the thin film, a change over time in which the thin film peels off the skin is detected.

5. The method for detecting the surface shape of skin according to claim 1, characterized in that, in the step of identifying portions corresponding to the convex and / or concave parts of the skin, after the step of forming the thin film, the skin color information is combined in the analysis of the captured image.

Images