Skin image imaging system

Abstract

To allow a user of a mobile terminal with a camera function to easily obtain an internal reflected light image of the skin and a surface reflected light image by imaging, using the mobile terminal, a skin image under two kinds of polarization conditions.SOLUTION: A skin image imaging system includes: (i) a mobile terminal 1 with a camera function which is provided with an illumination light source 3, a first imaging lens 4, and a second imaging lens 5; (ii) an illumination polarizer provided on a front surface of the illumination light source 3; (iii) a first polarizer 11 provided on a front surface of the first imaging lens 4; (iv) a second polarizer 12 provided on a front surface of the second imaging lens 5; and (v) an ambient light shielding hood 20. A first image including a surface reflected light is formed by using the illumination light source 3 provided with the illumination polarizer on the front surface and the first imaging lens 4 provided with the first polarizer on the front surface and a second image not including the surface reflected light is formed by using the illumination light source 3 provided with the illumination polarizer on the front surface and the second imaging lens 5 provided with the second polarizer on the front surface. Polarization conditions differ from each other between the first image and the second image.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a skin image capturing system using a portable terminal.

[0002] Skin grooves, fine wrinkles, pores, etc. are observed using light reflected from the skin's surface, and color unevenness such as age spots and freckles is observed using internally reflected light. For example, Patent Document 1 describes that in order to obtain a surface reflected light image and an internally reflected light image of the skin, S-polarized or P-polarized light is made incident on the skin, and the S-polarized and P-polarized components of the incident light are received as reflected light of each, and a surface reflected light image and an internally reflected light image of the skin are calculated from these image signals. However, this imaging device requires a special configuration in which the receiving light path of the reflected light is split by a beam splitter.

[0003] On the other hand, Patent Document 2 proposes that in order to grasp the condition of a wide area of ​​the skin using a general-purpose terminal device such as a smartphone, one of a plurality of cameras of the terminal device is used as a first imaging unit to capture an image of a predetermined part of the body such as the face, and a camera with a higher resolution than the first imaging unit is used as a second imaging unit to capture the skin of the face, etc., and the skin condition of non-overlapping parts is inferred from the skin condition of the overlapping parts. Patent Document 2 also describes that unpolarized and polarized light are selectively used for the illumination light of the second imaging unit to detect skin texture and blemishes.

[0004] However, in the terminal device described in Patent Document 2, in order to perform both texture detection and blemish detection in the second imaging unit, it is necessary to switch the polarization of the illumination light of the second imaging unit, and it is very difficult to keep the positional relationship between the camera and the skin constant before and after the switching. In addition, the switching work is also complicated. A method of switching the polarization by attaching and detaching a polarizing plate arranged in front of the camera is also widely used, but in that case, the switching is still necessary, and the same problem remains. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Publication No. 7-323013 [Patent Document 2] JP 2022-138384 A Summary of the Invention [Problem to be solved by the invention]

[0006] In contrast to the above-mentioned prior art, the object of the present invention is to enable a user of a portable terminal with a camera function, such as a smartphone, to capture skin images using two types of polarization conditions without the need to take the time to replace polarizing plates, and without changing the positional relationship between the camera and the area to be imaged while capturing the skin image, allowing the user to easily obtain internal reflection light images and surface reflection light images of the skin. [Means for solving the problem]

[0007] Generally, a mobile terminal with a camera function such as a smartphone has multiple lenses as an outer camera, and the outer camera side has a relatively strong illumination light source. The inventor of the present invention has come up with the idea that by attaching a polarizing plate to the front of the illumination light source and attaching polarizing plates with different polarization directions to the front of the lenses of the multiple outer cameras, the user can simultaneously or continuously take pictures with different polarization conditions without replacing the polarizing plate, so that the positional relationship between the lens and the image capture location does not change, and even though multiple outer camera lenses are used, these lenses are fixed in a relatively close state, so that parallax between the lenses can be tolerated and the difference in the angle of view between the lenses can be corrected, and therefore, internal reflection light images and surface reflection light images can be easily obtained, and has completed the present invention.

[0008] Here, the polarization condition means a combination of the polarization of the illumination light and the polarization of the light received by the camera.

[0009] That is, the present invention provides: (i) a portable terminal with a camera function, comprising an illumination light source, a first imaging lens, and a second imaging lens; (ii) a lighting polarizing plate provided in front of the lighting light source; (iii) a first polarizing plate provided in front of the first imaging lens; (iv) a second polarizing plate provided in front of the second imaging lens; and (v) comprising an illumination light source, a first imaging lens, a second imaging lens, and an ambient light shielding hood surrounding an imaging location; a first image formed by an illumination light source having an illumination polarizing plate on its front surface and a first imaging lens having a first polarizing plate on its front surface, and a second image formed by an illumination light source having an illumination polarizing plate on its front surface and a second imaging lens having a second polarizing plate on its front surface have different polarization conditions; A skin image capturing system is provided that uses a first imaging lens to form a first image including surface reflected light, and uses a second imaging lens to form a second image not including surface reflected light.

[0010] Further, there is provided a photographing tool for use in the above-mentioned skin image photographing system, (ii) an illumination polarizing plate provided in front of an illumination light source of a portable terminal with a camera function, the portable terminal including an illumination light source, a first imaging lens, and a second imaging lens; (iii) a first polarizing plate provided in front of the first imaging lens; (iv) a second polarizing plate provided in front of the second imaging lens; (v) The first and second polarizing plates and the ambient light shielding hood surrounding the shooting area Equipped with Provided is a photographing tool in which a first image formed by an illumination light source having an illumination polarizing plate on the front side and a first imaging lens having a first polarizing plate on the front side, and a second image formed by an illumination light source having an illumination polarizing plate on the front side and a second imaging lens having a second polarizing plate on the front side have different polarization conditions.

[0011] Furthermore, the present invention provides a method for photographing skin, which uses a portable terminal with a camera function having an illumination light source, a first imaging lens, and a second imaging lens, and the above-mentioned photographing tool, to take a first image including surface reflected light from the part of the skin to be photographed and a second image not including surface reflected light, in which the part of the skin to be photographed is surrounded by an ambient light shielding hood, the first image is taken using an illumination light source with an illumination polarizing plate arranged in front of it and the first imaging lens with the first polarizing plate arranged in front of it, and the second image is taken using the illumination light source with an illumination polarizing plate arranged in front of it and the second imaging lens with the second polarizing plate arranged in front of it, and the polarization conditions of the first image and the second image are different.

[0012] In addition, the present invention provides an image formation method in which a portable terminal forms a surface reflected light image and an internal reflected light image based on a first image and a second image taken by the above-mentioned shooting method using an application installed on the portable terminal; an image formation method in which a portable terminal outputs the first image and the second image taken by the above-mentioned shooting method to a computing device, and the computing device forms a surface reflected light image and an internal reflected light image based on the first image and the second image using an application installed on the computing device; and an image formation method in which a portable terminal transmits the first image and the second image taken by the above-mentioned shooting method to a specific site on the web, and a computing device connected to the site forms a surface reflected light image and an internal reflected light image based on the first image and the second image using an application installed on the computing device. Effect of the Invention

[0013] The skin image capturing system of the present invention can be easily produced by using an off-the-shelf smartphone. With this system, a user of a portable device can obtain a first image including surface reflected light and a second image not including surface reflected light by simply operating a smartphone, and can further obtain an internal reflected light image and a surface reflected light image from the first and second images, thereby enabling an accurate understanding of skin condition.

[0014] In this way, if the user of the portable terminal is a general consumer, the general user can receive appropriate beauty advice by transmitting the internal reflection light image and the surface reflection light image to a beauty advice providing site.

[0015] In addition, when the general user is a cosmetics user, the cosmetics developer can evaluate the effects of the cosmetics by receiving an internal reflection light image and a surface reflection light image from the cosmetics user, or by forming an internal reflection light image and a surface reflection light image from the first image and second image provided by the cosmetics user.

[0016] On the other hand, the cosmetics developer may also be the user of the portable terminal and take pictures of the cosmetics users in stores, etc., and in this case too, the ease of operation and portability of the present system can be enjoyed. [Brief description of the drawings]

[0017] [Figure 1] FIG. 1 is a perspective view of a skin image capturing system according to an embodiment of the present invention in use. [Figure 2A] FIG. 2A is a perspective view of an outer camera portion of a portable terminal with a camera function attached to the skin image capturing system of the embodiment. [Figure 2B] FIG. 2B is a layout diagram of the outer camera portion of another mobile terminal with a camera function. [Diagram 3] FIG. 3 is a diagram showing the arrangement of polarizing plates in the skin image capturing system of the embodiment. [Figure 4A] FIG. 4A is an assembly diagram of a lens cover with a polarizing plate to be attached to the outer camera portion. [Figure 4B] FIG. 4B is a perspective view of the lens cover with a polarizing plate attached to the outer camera portion. [Diagram 5] FIG. 5 is an exploded view of the ambient light shielding hood. [Figure 6A] FIG. 6A is a perspective view of a photographing tool for a skin image capturing system according to an embodiment (hereinafter, also referred to as a photographing tool) as viewed from the top surface side of an environmental light shielding hood. [Figure 6B]FIG. 6B is a cross-sectional view of the photographing jig of FIG. 6A. [Figure 6C] FIG. 6C is a perspective view of the photographing jig in FIG. 6A as viewed from the photographing surface side. [Figure 7] FIG. 7 is a perspective view of the skin image capturing system according to the embodiment in which a mobile terminal is fitted to the photographing jig, as viewed from the photographing surface side. [Figure 8A] FIG. 8A is a schematic diagram of a graph paper image captured under the same conditions as those for capturing the first image. [Figure 8B] FIG. 8B is a schematic diagram of a graph paper image captured under the same conditions as those for capturing the second image. [Figure 9A] FIG. 9A is a schematic diagram of the first image. [Figure 9B] FIG. 9B is a schematic diagram of the second image. [Figure 10] FIG. 10 is a schematic diagram of an enlarged first image. [Figure 11] FIG. 11 is an experimental example of the first image. [Figure 12] FIG. 12 is an experimental example of the second image. [Figure 13] FIG. 13 is an experimental example of the enlarged first image. [Figure 14] FIG. 14 shows an experimental example of a surface reflected light image. [Figure 15] FIG. 15 is an experimental example of a second image for a different subject from that in FIG. [Figure 16] FIG. 16 is an experimental example of a melanin image. [Figure 17] FIG. 17 is an experimental example of hemoglobin imaging. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0018] The present invention will now be described in detail with reference to the drawings, in which the same reference numerals denote the same or equivalent components.

[0019] (overview) FIG. 1 is an oblique view of a skin image capturing system 200 according to one embodiment of the present invention in use, and FIG. 2A is an oblique view of an outer camera portion 2 of a portable terminal 1 with a camera function (hereinafter also referred to as a portable terminal) attached to the skin image capturing system 200.

[0020] The portable terminal 1 used in the present invention is equipped with an illumination light source 3 and multiple lenses in the outer camera section 2. There is no particular limitation on the configuration of the imaging element corresponding to the multiple lenses. The portable terminal 1 can be a camera system of a general portable terminal, such as a smartphone with a camera function or a tablet with a camera function.

[0021] The portable terminal 1 used in this embodiment is provided with an illumination light source 3, an ultra-wide-angle lens 4, a wide-angle lens 5, and a telephoto lens 6 in the out-camera section 2. The illumination light source and lens arrangement in the out-camera section 2 are not particularly limited. For example, as shown in FIG. 2B, the illumination light source 3, the ultra-wide-angle lens 4, the wide-angle lens 5, and the telephoto lens 6 may be arranged in a row in the out-camera section 2. It is preferable that the lens used as the first imaging lens and the lens used as the second imaging lens are close to each other in order to reduce the effect of parallax. In addition, the portable terminal 1 having a function of saving the captured image as RAW data is preferable in that a proportional relationship is maintained with respect to the amount of light incident on the light receiving element of the camera. Note that the name and performance of each lens differ for each portable terminal, so that, for example, the wide-angle lens 5 of the portable terminal 1 in FIG. 2A and the wide-angle lens 5 of the portable terminal 1 in FIG. 2B generally have different performances.

[0022] The skin photographing method of the present invention is, in brief, a method of photographing skin simultaneously or continuously using a portable terminal 1 with a camera function owned by a user or the like (hereinafter also referred to as a portable terminal originating from the user) attached to the photographing tool 100 of the present invention to photograph the skin. An internal reflected light image can be obtained from the second image by image processing described later, and a surface reflected light image can be formed by aligning the sizes of the photographed locations of the first and second images, aligning them, and then taking the difference between the first and second images.

[0023] In the present invention, the photographing jig 100 includes an illumination polarizing plate, a first polarizing plate 11, a second polarizing plate 12, and an ambient light shielding hood 20. In this embodiment, in order to reduce the number of parts, the first polarizing plate 11 is provided in front of the illumination light source 3 and the first imaging lens 4 of the portable terminal 1 with camera function so that the first polarizing plate 11 also serves as an illumination polarizing plate. In addition, the second polarizing plate 12 is provided in front of the second imaging lens 5.

[0024] The portable terminal 1 is usually purchased by the user of the portable terminal 1 from a terminal manufacturer and owned by the user, but the photographing tool 100 is provided by the right holder of the present invention in order to acquire images required for carrying out the image forming method of the present invention. The illumination polarizing plate, the first polarizing plate 11, the second polarizing plate 12, and the environmental light shielding hood 20 constituting the photographing tool 100 do not need to be fully assembled when provided. Possible methods of providing the tool include free distribution, sale, and loan.

[0025] In the present invention, the polarization conditions of a first image formed by an illumination light source having an illumination polarizing plate on its front surface and a first imaging lens having a first polarizing plate on its front surface are made different from those of a second image formed by an illumination light source having an illumination polarizing plate on its front surface and a second imaging lens having a second polarizing plate on its front surface. Here, a specific embodiment of the different polarization conditions is that when the polarization directions of the illumination polarizing plate and the first polarizing plate are made the same, particularly when the first polarizing plate doubles as the illumination polarizing plate as described above, the polarization directions of the first polarizing plate and the second polarizing plate are made different, and preferably the polarization directions of the first polarizing plate and the second polarizing plate are made perpendicular to each other.

[0026] The environmental light shielding hood 20 shields the environmental light by surrounding the first polarizing plate 11, the second polarizing plate 12, and the photographing location, and the skin surrounded by the environmental light shielding hood 20 is the photographing location. The environmental light shielding hood 20 also serves the purpose of keeping the distance between the lens and the photographing location constant or fixing it by abutting the end periphery on the opposite side to the opening 22 as close as possible to the periphery of the photographing location of the photographing subject. There is no particular limit to which part of the skin is the photographing location, and for example, the entire face, cheeks, arms, etc. can be the photographing location. In addition, the skin may be bare skin, or may have cosmetics or the like applied thereto.

[0027] Which of the multiple lenses in the outer camera section 2 of the portable terminal 1 is the first imaging lens and which is the second imaging lens can be appropriately determined according to the purpose of imaging. For example, when it is desired to extract and observe in detail an image of pigments such as melanin and hemoglobin from an internal reflection light image obtained by orthogonally intersecting the polarization direction of the illumination light source and the polarization direction of the observation light, it is preferable to make the image captured by the second imaging lens have a higher resolution than that of the first imaging lens, and conversely, when it is desired to observe in detail a surface reflection light image, it is preferable to make the image captured by the first imaging lens have a higher resolution than that of the second imaging lens. Here, the resolution refers to the number of pixels per actual length of the imaging part in each image.

[0028] In addition, the illumination polarizing plate provided in front of the illumination light source 3 and the first polarizing plate in front of the first imaging lens may be formed as a single polarizing plate or separate polarizing plates, depending on the lens arrangement of the outer camera part of the portable terminal.

[0029] (Photography fixture) 3, in the photographing tool 100 of this embodiment, the super wide-angle lens of the outer camera portion 2 of the portable terminal 1 is the first photographing lens 4, the wide-angle lens is the second photographing lens 5, a first polarizing plate 11 is disposed in front of the illumination light source 3 and the first photographing lens 4, and a second polarizing plate 12, the polarization direction of which is perpendicular to that of the first polarizing plate 11, is disposed in front of the second photographing lens 5. In the photographing tool 100 of this embodiment, in order to enable detailed observation of the dye image, the second polarizing plate 12 in front of the wide-angle lens 5, which can obtain an image with a higher resolution than the super wide-angle lens 4, is perpendicular to the first polarizing plate 11 in front of the illumination light source 3.

[0030] There is no particular limitation on the method of providing the first polarizing plate 11 in front of the illumination light source 3 and the first imaging lens 4, and the second polarizing plate 12 in front of the second imaging lens, and the method can be appropriately determined according to the lens arrangement, shape, etc. of the out-camera part 2 of the mobile terminal 1. In this embodiment, as shown in Fig. 4A, the first polarizing plate 11 and the second polarizing plate 12 are attached to a lens cover 10 that fits the out-camera part 2 to form a lens cover with polarizing plate 14, and the lens cover with polarizing plate 14 is attached to the out-camera part 2 as shown in Fig. 4B. For the purpose of protecting the polarizing plate, etc., two lens covers 10 may be prepared, and the first polarizing plate 11 and the second polarizing plate 12 may be sandwiched between them.

[0031] On the other hand, the ambient light shielding hood 20 of the photographing tool 100 surrounds the first polarizing plate 11, the second polarizing plate 12, and the photographing location. When the portable terminal 1 is attached to the photographing tool 100 as shown in Fig. 1, the ambient light shielding hood 20 surrounds the illumination light source 3, the first imaging lens 4, the second imaging lens 5, and the photographing location.

[0032] The shape and material of the environmental light shielding hood 20 are not particularly limited, and may be, for example, an approximately quadrangular pyramid shape as shown in FIG. 1. More specifically, the approximately quadrangular pyramid shape of the environmental light shielding hood 20 can be made from, for example, cardboard, as shown in the development diagram of FIG. 5. The inner surface of the environmental light shielding hood 20 is preferably black to prevent reflection of illumination light. In addition, it is preferable to attach a color chart 23 to the environmental light shielding hood 20 for adjusting the white color during shooting with RGB. It is preferable that the color chart 23 is placed on the shooting surface of the shooting site when the environmental light shielding hood 20 is used, and is provided in a position that does not interfere with observation of the shooting site.

[0033] As shown in Fig. 6A and Fig. 6B, in the photographing jig 100 of this embodiment, an opening 22 is provided on the top surface 21 of the environmental light shielding hood 20 having a generally quadrangular pyramid shape, and when the portable terminal 1 is fixed to the top surface of the environmental light shielding hood 20, for example, a mobile terminal soft case 15 for the back surface of the portable terminal 1 is attached to the top surface 21 of the environmental light shielding hood 20 so that the lens cover with polarizing plate 14 attached to the out-camera part 2 of the portable terminal 1 is placed in the opening 22. In this way, a photographing jig 100 can be obtained that is composed of the mobile terminal soft case 15 fixed to the top surface 21 of the environmental light shielding hood 20 and the lens cover with polarizing plate 14 attached to the out-camera part 2 of the portable terminal 1. Fig. 6C is a perspective view of this photographing jig 100 as seen from the photographing surface side surrounded by the environmental light shielding hood 20. 6A, 6B, and 6C show the positional relationship when portable terminal 1 is attached, and lens cover 14 with polarizing plate is directly fixed to portable terminal 1. Portable terminal 1 is fitted into this photographing jig 100 as shown in the figures, and the skin image photographing system shown in Fig. 7 can be obtained.

[0034] On the other hand, the first polarizing plate and the second polarizing plate may be integrated, for example by adhering them to the ambient light shielding hood 20, to form a photographing jig, to which a portable terminal may be attached by any method.

[0035] (Shooting method) In the skin image capturing system shown in FIG. 7, a camera system provided in the portable terminal 1 itself is used to capture a first image including surface reflected light by a first imaging lens, and a second image not including surface reflected light is formed by a second imaging lens.

[0036] It is preferable that the first image and the second image are taken simultaneously in order to minimize the deviation in the subject's posture, but some portable terminal imaging systems cannot take images with two lenses at the same time. Therefore, it is preferable to install an imaging application on the portable terminal that allows the first image and the second image to be taken simultaneously or sequentially depending on the imaging system of the portable terminal. When the first image and the second image are taken sequentially, it is preferable to shorten the interval between these images as much as possible within the range where the camera can be switched stably. For example, an imaging application that allows images to be taken continuously in the following imaging flow is installed on the portable terminal 1.

[0037] (Shooting flow) (1) Launching the shooting application (2) Turning on the lighting source (3) Displaying a preview screen of the part to be imaged by the first imaging lens on the display of the portable terminal 1 (4) Tap anywhere on the screen to initiate the following sequence of actions: (5) Displaying a screen to prompt the user to set the shutter speed and exposure when taking a picture with the first imaging lens, or displaying a screen to confirm the setting (6) Autofocus photography using the first imaging lens (7) Displaying a preview screen of the part to be imaged by the second imaging lens (8) Displaying a screen to prompt the user to set the shutter speed and exposure when taking a picture using the second imaging lens, or displaying a screen to confirm the setting; (9) Autofocus photography using the second imaging lens (10)End

[0038] Here, either (6) shooting with the first imaging lens or (9) shooting with the second imaging lens may be performed first. Also, the flow may be temporarily interrupted before the setting of (8) and then resumed by tapping the screen after preparations are complete.

[0039] According to this imaging flow, it is not necessary to replace the polarizing plate between the images (6) and (9), and the images (6) and (9) can be taken continuously. Therefore, there is little risk of the subject moving between the images (6) and (9), and the surface reflected light and internally reflected light can be calculated with high accuracy from these images.

[0040] Which of the shooting locations surrounded by the ambient light shielding hood 20 is focused on is determined by the camera system included in the mobile terminal 1 itself, and is usually focused on the center of the shooting location surrounded by the ambient light shielding hood 20. Some camera systems allow you to specify where in the shooting range you want to focus, in which case you can focus on a region of interest during shooting, such as the cheek. Also, you may set it so that a predetermined position in the shooting location surrounded by the ambient light shielding hood 20 is focused on.

[0041] When a user takes a selfie of his / her face, the user cannot check the preview screens (3) and (7). In that case, as described in JP2021-515640, a mirror may be provided in the ambient light shielding hood 20, and the position of the shooting location may be adjusted while looking at the mirror. In addition, the feeling when the face touches the ambient light shielding hood 20 may be recorded, and the position of the shooting location may be determined based on the feeling. If the shooting location is not as intended, the position of the shooting location may be adjusted again before shooting. Such a re-adjustment flow may be incorporated into the shooting flow.

[0042] The shutter operation may be performed using a remote control shutter or a selfie stick instead of tapping the screen.

[0043] (Image Processing) In the image forming method of the present invention, a surface reflected light image is formed from the difference between a first image including surface reflected light and a second image not including surface reflected light, both of which are obtained by a portable terminal, by performing image processing according to the following outline flow. The second image is treated as an internal reflected light image, and pigment images such as a melanin image and a hemoglobin image are extracted from the internal reflected light image. Therefore, it is preferable that the first and second images are output as RAW data from the portable terminal.

[0044] (Image processing flow) (a) If necessary, the first and second images are corrected so that the pixel values ​​are proportional to the amount of light incident on the camera's light-receiving element, and then the white balance and exposure are corrected using a color chart. (b) Match the size and position of the first and second images. (c) The difference between the first and second images, which have been adjusted in size and position, is taken to obtain a surface reflected light image. (d) Extracting a dye image from the second image.

[0045] In the above image processing flow, step (d) may be carried out before step (b) and step (c).

[0046] Step (b) is a process for making it possible to obtain the difference between the first image and the second image in step (c). In step (b), the size and position of the first image and the second image are aligned by enlarging or reducing the image. The method of converting the size and position can be determined as follows. That is, the graph paper is photographed under the same conditions as the first image to obtain image A shown in FIG. 8A, and the graph paper is photographed under the same conditions as the second image to obtain image B shown in FIG. 8B. Here, as long as the positions of the first image and the second image can be matched, it is also possible to photograph paper with ruled lines of a notebook or a geometric pattern printed thereon instead of the graph paper. Then, in each of images A and B, frames W1 and W2 shown by two-dot chain lines are drawn so that the coordinates of each vertex on the graph paper match within the field of view. The frames W1 and W2 can be drawn accurately by photographing the graph paper. After drawing the frames W1 and W2, a transformation formula is obtained so that the frames W1 and W2 overlap exactly by affine transformation of the image A.

[0047] The method of establishing positional correspondence is not limited to this, and for example, a pattern matching technique in image analysis may be used for the first and second images.

[0048] Alternatively, steps (b) and (c) may be carried out in one process.

[0049] The same size and position conversion formula can be used if the specifications of the cameras and other devices equipped in the mobile terminals are similar, so once a conversion formula is calculated, it can be used repeatedly.

[0050] An enlarged first image of the same size as the second image is formed by enlarging the first image using the above-mentioned conversion formula so that the area of ​​the above-mentioned frame W1 on the first image (Figure 9A) coincides with the area of ​​the above-mentioned frame W2 on the second image (Figure 9B) (Figure 10).

[0051] Although there is a slight parallax between the enlarged first image and the second image, when using multiple lenses on a mobile terminal, the coordinates roughly match in gently shaped areas such as the cheek, and since the internal reflection image changes gradually whereas the surface reflection image changes rapidly, a clear surface reflection image can be obtained by subtracting the second image from the first image after alignment. Note that at this time, for reasons such as image visibility, the entire image may be multiplied by a certain number, or the image may be converted to grayscale using a method such as the weighted average of RGB.

[0052] On the other hand, the second image is an internal reflection image. In this case, too, the entire image may be multiplied by a certain number for reasons such as image visibility.

[0053] FIG. 11 shows a first image of an experimental example obtained by using an ultra-wide-angle lens of the rear camera of a commercially available mobile phone (iPhone (registered trademark) 13pro, Apple Inc.) as the first imaging lens, a wide-angle lens as the second imaging lens, and creating an ambient light shielding hood 20 so that the distance between the center position of the rear camera and the subject is 150 mm, and photographing with autofocus. FIG. 12 shows a second image photographed consecutively to the first image. FIG. 13 shows an enlarged first image enlarged by a transformation formula of an affine transformation obtained in advance, and the coordinates of each vertex of this frame W1 and the frame W2 of FIG. 12 are the same. Next, the second image of FIG. 12 is subtracted from the enlarged first image of FIG. 13. In this way, the surface reflected light image shown in FIG. 14 can be obtained.

[0054] On the other hand, the second image in Fig. 12 represents an internal reflection image. A melanin image representing the concentration of melanin and a hemoglobin image representing the concentration of hemoglobin can be obtained from the internal reflection image by a known calculation method (e.g., Patent No. 6977744, etc.).

[0055] For example, Fig. 15 is a second image of an experimental example formed for a different subject in the same manner as Fig. 12. From the second image (internal reflection light image) of Fig. 15, a melanin image shown in Fig. 16 and a hemoglobin image shown in Fig. 17 could be obtained.

[0056] The surface reflected light images, internal reflected light images, and the melanin and hemoglobin images derived from them obtained in this way are useful for understanding the condition of the skin just by looking at them, but because these images contain detailed information about the skin condition, incorporating an algorithm that can extract various skin conditions from the images can lead to more advanced beauty checks and advice.

[0057] The image forming method of the present invention can be implemented by installing an application for performing the above-mentioned flow in the portable terminal 1. In other words, the portable terminal constituting the skin image capturing system of the present invention may be provided with an application for performing the image forming method of the present invention. This allows the user of the portable terminal to easily observe the surface reflected light image and the internal reflected light image. Also, the application for performing the above-mentioned flow may be installed in a computing device such as a personal computer.

[0058] This allows general users of mobile devices to perform more detailed beauty checks on themselves.

[0059] Alternatively, a general user of the portable terminal 1 may transmit the first and second images to a specific site on the web, and a computing device connected to the site may form a surface reflected light image and an internal reflected light image based on the first and second images using an application installed on the computing device. If the specific site on the web is a beauty advice providing site, it may receive the surface reflected light image and the internal reflected light image and transmit beauty advice based on these to the portable terminal. If the site is a cosmetics development company, it may form a surface reflected light image and an internal reflected light image from the first and second images received from the user to investigate the effectiveness of the cosmetics. Thus, the cosmetics development company may form a network that receives the first and second images from users of portable terminals as a data collection means for investigating the effectiveness of the cosmetics.

[0060] On the other hand, the same can be said if a cosmetic developer becomes a user operating a portable terminal and takes images of the person wearing the cosmetic product. By forming an internal reflection light image and a surface reflection light image from the first and second images, the effects of the cosmetic product can be easily examined.

[0061] In addition, the transmission of the first and second images from a portable terminal can also be utilized for online diagnosis (telemedicine). [Explanation of symbols]

[0062] 1. Mobile devices with camera functions, mobile devices 2 Rear camera part 3 Light source for lighting 4. First imaging lens (ultra-wide-angle lens) 5 Second imaging lens (wide-angle lens) 6. Telephoto Lens 10 Lens cover 11 First polarizing plate 12 Second polarizing plate 14 Lens cover with polarizing plate 15 Soft cases for mobile devices 20 Ambient light shielding hood 21 Top surface 22 Opening 23 Color Chart 100 Skin image capturing system imaging tool, imaging tool 200 Skin Imaging System W1, W2 frames

Claims

1. (i) a portable terminal with a camera function, comprising an illumination light source, a first imaging lens and a second imaging lens; (ii) a lighting polarizing plate provided in front of the lighting light source; (iii) a first polarizing plate provided in front of the first imaging lens; (iv) a second polarizing plate provided in front of the second imaging lens; and (v) comprising an illumination light source, a first imaging lens, a second imaging lens, and an environmental light shielding hood surrounding an imaging location; a first image formed by an illumination light source having an illumination polarizing plate provided on its front surface and a first imaging lens having a first polarizing plate provided on its front surface, and a second image formed by the illumination light source having an illumination polarizing plate provided on its front surface and a second imaging lens having a second polarizing plate provided on its front surface have different polarization conditions; A skin image capturing system that uses a first imaging lens to form a first image including surface reflected light, and uses a second imaging lens to form a second image not including surface reflected light.

2. 2. The skin image capturing system according to claim 1, wherein the illumination polarizing plate and the first polarizing plate have the same polarization direction, and the first polarizing plate and the second polarizing plate have different polarization directions.

3. 3. The skin image capturing system according to claim 1 or 2, wherein the first polarizing plate also serves as a polarizing plate for illumination, a lens cover having the first polarizing plate and the second polarizing plate attached to the lens cover is removably attached to the outer camera portion of the portable terminal, and an environmental light shielding hood is removably attached to the portable terminal.

4. 4. The skin image capturing system according to claim 1, wherein the resolution of an image captured by the second imaging lens is higher than that of an image captured by the first imaging lens.

5. The skin image capturing system according to any one of claims 1 to 4, wherein an application is installed in a portable terminal, which sequentially or simultaneously performs turning on an illumination light source, photographing using one of the first imaging lens and the second imaging lens, and photographing using the other of the first imaging lens and the second imaging lens.

6. A skin image capturing system as described in any one of claims 1 to 5, wherein the system includes a computing device, and the computing device includes an application that sequentially or simultaneously performs image scaling to make the first image and the second image the same size and aligns the first image and the second image, and forms a surface reflected light image from the difference between the first image and the second image.

7. 7. The skin imaging system of claim 6, wherein the computing device comprises an application for forming a melanin image or a hemoglobin image from the second image.

8. 2. A photographing tool for use in the skin image capturing system according to claim 1, (ii) a lighting polarizing plate provided in front of a lighting light source of a portable terminal with a camera function, the portable terminal including a lighting light source, a first imaging lens, and a second imaging lens; (iii) a first polarizing plate provided in front of the first imaging lens; (iv) a second polarizing plate provided in front of the second imaging lens; (v) An ambient light shielding hood surrounding the first and second polarizing plates and the shooting location Equipped with A photographing tool in which a first image is formed by an illumination light source having an illumination polarizing plate on the front side and a first imaging lens having a first polarizing plate on the front side, and a second image is formed by an illumination light source having an illumination polarizing plate on the front side and a second imaging lens having a second polarizing plate on the front side, the polarization conditions of which are different.

9. 9. The photographing tool according to claim 8, wherein the illumination polarizing plate and the first polarizing plate have the same polarization direction, and the first polarizing plate and the second polarizing plate have different polarization directions.

10. 10. The photographing tool according to claim 8, wherein the resolution of an image photographed by the second photographing lens is higher than that of an image photographed by the first photographing lens.

11. A method for photographing skin, using a portable terminal with a camera function having an illumination light source, a first imaging lens, and a second imaging lens, and the photographing tool described in any one of claims 8 to 10, to take a first image including surface reflected light from a photographing location of the skin and a second image not including surface reflected light, wherein the photographing location of the skin is surrounded by an ambient light shielding hood, the first image is taken using an illumination light source with an illumination polarizing plate arranged in front of it and a first imaging lens with a first polarizing plate arranged in front of it, and the second image is taken using the illumination light source with an illumination polarizing plate arranged in front of it and a second imaging lens with a second polarizing plate arranged in front of it, and the polarization conditions of the first image and the second image are made different in this skin photographing method.

12. 12. The method for photographing skin according to claim 11, wherein the polarization directions of the illumination polarizing plate and the first polarizing plate are the same, and the polarization directions of the first polarizing plate and the second polarizing plate are different.

13. 13. An image forming method, comprising: a portable terminal, using an application installed in the portable terminal, forming a surface reflected light image and an internal reflected light image based on a first image and a second image taken by the method according to claim 11 or 12.

14. An image forming method in which a portable terminal outputs a first image and a second image taken by the method described in claim 11 or 12 to a computing device, and the computing device forms a surface reflected light image and an internal reflected light image based on the first image and the second image using an application installed in the computing device.

15. An image forming method in which a portable terminal transmits the first and second images taken by the method described in claim 11 or 12 to a specific site on the web, and a computing device connected to the site forms a surface reflected light image and an internal reflected light image based on the first and second images using an application installed on the computing device.

Images