Skin condition presentation device, skin condition presentation method, and computer program

Abstract

To present a detection result of a skin condition in a form corresponding to the shape and size of the face of an individual user by detection by deforming a range to be detected in accordance with the constitution of the user's face.SOLUTION: A basic range showing a detection range is disposed in a photographed face of a user and the disposed basic range is deformed in accordance with the constitution of the face F of the user. The deformed range is identified as a user's characteristic range 121, and the skin condition of the identified user's characteristic range 121 is detected by step. Based on a detection result corresponding to each step, color is given to the inside of the user's characteristic range 121 to generate a skin condition presentation image 201, and an image screen 41 including the generated skin condition presentation image 201 is displayed on a display screen 12.SELECTED DRAWING: Figure 13

Description

[Technical field]

[0001] The present invention relates to a skin condition display device, a skin condition display method, and a computer program that detects the skin condition of a user's face by stages and displays the results in colors corresponding to the stages, by performing detection within a range tailored to each user's individual face. [Background technology]

[0002] Conventionally, the skin condition of a user's face is detected based on an image of the user's face, and the detection result is expressed in color. For example, the following Patent Document 1 discloses that the skin color is evaluated from the image data of the captured user's face, the image data is divided into a plurality of stages (stages 1-4), and a mark (marking) of a color set for each divided stage is added to the image data of the user's face to generate and display the generated processed data (see paragraphs 0051-0063, Figures 8, 11, etc. of Patent Document 1). Note that Patent Document 1 also describes enlarging and displaying a part of the marked processed data (see paragraph 0063, Figure 8(b), etc. of Patent Document 1), and performing a detection process in a detection area according to an abnormality item to be detected (see paragraph 0045, Figure 5, etc. of Patent Document 1).

[0003] Furthermore, Patent Document 2 listed below also discloses determining analysis areas for wrinkles, dryness, blemishes, pores, etc. from an image of a customer's face, analyzing the areas, and displaying the level of the analysis results on the captured image in a color-coded manner according to color intensity, as well as displaying enlarged images of such captured images (see paragraphs 0030-0035, Figures 3-7, etc. of Patent Document 2).

[0004] Furthermore, the following Patent Document 3 also discloses setting areas on a photographed image of a face, evaluating pores, blemishes, wrinkles, uneven skin tone, etc., and displaying the results on the photographed image in a gradated manner so that they can be distinguished by color, and also discloses enlarging and displaying such images (see paragraphs 0039-0069, Figures 7-12, etc. of Patent Document 3).

[0005] Furthermore, Patent Document 4 listed below also discloses analyzing the degree of items such as texture, pores, blemishes, and color from an image (skin image) taken of a user's face, and displaying the degree of analysis on the face image in different colors (or by changing the density of a specific color) according to the level of the analysis, as well as displaying an enlarged image (see paragraphs 0041-0099, Figures 5-14, etc. of Patent Document 4). [Prior art documents] [Patent documents]

[0006] [Patent Document 1] JP 2021-108061 A [Patent Document 2] JP 2006-142005 A [Patent Document 3] JP 2008-293325 A [Patent Document 4] International Publication No. 2016 / 121518 Summary of the Invention [Problem to be solved by the invention]

[0007] In the above-mentioned Patent Document 1, a different range is set as the face area to be detected for each evaluation item (see paragraph 0045 and FIG. 5 of Patent Document 1), but since the configuration of the face outline and the shape and size of each part of the face are different for each user, even if the detection area is set as described in Patent Document 1, the detection area set in this way may not be an appropriate range that matches the unique face configuration of the user. This causes a problem that detection cannot be performed in an appropriate area according to the evaluation item for each user's face, and further, the detection result cannot be displayed in an appropriate area according to the evaluation item for each user's face.

[0008] In addition, the above-mentioned Patent Document 2 also describes that specific locations such as cheeks and corners of the eyes are set as analysis locations depending on the diagnostic items (see paragraph 0030 of Patent Document 2), but the description is merely a general description of setting specific locations as analysis locations. Therefore, the invention according to Patent Document 2 also has a problem in that it is not possible to narrow down detection to appropriate locations that match the unique facial structure of the user, and the detection results cannot be displayed within the range of appropriate locations that match the facial structure of the user.

[0009] Furthermore, Patent Document 3 describes a case where the face region is set by an operator using a mouse while looking at the screen, and a case where the face region is set automatically, but the region set in Patent Document 3 is intended to exclude parts that are not related to skin evaluation (for example, the background of the image, nostrils, eyebrows, eyes, mouth, etc.) (see paragraphs 0039, 0047, etc. of Patent Document 3). Therefore, even with the invention of Patent Document 3, the region to be detected for each evaluation item cannot be appropriately set according to the configuration of the user's face, and therefore the detection result cannot be appropriately displayed in a region according to the evaluation item that matches the configuration of the user's face.

[0010] Furthermore, Patent Documents 1-4 show that a part of the face image showing the detection result is enlarged and displayed. However, Patent Documents 1, 3, and 4 do not explain what operation is required to enlarge the image. Patent Document 2 explains that when a specific position is selected with a mouse pointer or the like, an enlarged image is displayed, but does not explain the extent to which the image is enlarged in relation to the display screen.

[0011] The present invention has been made in consideration of the above circumstances, and aims to provide a skin condition presentation device, a skin condition presentation method, and a computer program that specify a range to be detected for each different detection item in accordance with the unique configuration of the user's face whose skin condition is to be detected, and that can appropriately detect within a range that suits the user's facial configuration for each item and display the detection results. Another object of the present invention is to provide a skin condition display device, a skin condition display method, and a computer program that enable the range showing the detection results to be easily enlarged and displayed with a one-touch operation while matching the range to the size of the screen displaying the detection results. [Means for solving the problem]

[0012] In order to solve the above problems, the present invention provides a skin condition display device that detects skin condition in stages based on a captured image including a user's face, and performs display processing by applying a color corresponding to the stage of the detection result to the user's face included in the captured image, and is characterized in that it comprises an arrangement means for arranging a basic range indicating a range corresponding to the part of the face to be detected on the user's face included in the captured image, a range transformation means for transforming the basic range arranged on the user's face into a shape corresponding to the user's face, a range identification means for identifying the range transformed from the basic range by the range transformation means as a user-specific range, a detection means for detecting skin condition in stages for the user-specific range identified by the range identification means, an image generation means for generating a skin condition display image by applying a color corresponding to the stage of the detection result to the part within the user-specific range detected by the detection means, and a display processing means for performing processing related to the display of the skin condition display image generated by the image generation means.

[0013] The present invention is also characterized in that the basic ranges include item-specific basic ranges for each item having a different skin condition, the placement means places each of the item-specific basic ranges on the user's face, the range transformation means transforms each of the item-specific basic ranges, the range identification means identifies the ranges transformed from each of the item-specific basic ranges as user-specific ranges for each item, the detection means performs detection for each of the item-specific user-specific ranges, the image generation means generates an item-specific skin condition presentation image, and the display processing means performs processing related to the display of the item-specific skin condition presentation image for each item.

[0014] Furthermore, in the present invention, the image generation means is configured to generate a skin condition presentation image in which a contour line indicating the periphery of the user-specific range is shown in a color different from the color corresponding to the stage related to the detection result, and the display processing means is characterized in performing processing related to the display of the skin condition presentation image in which the contour line is shown.

[0015] Furthermore, in the present invention, the skin condition presentation image is displayed on a display screen, the user-specific range shown in the skin condition presentation image displayed on the display screen is capable of accepting a selection operation, and when a selection operation of the user-specific range is accepted, a means is provided for enlarging and displaying the user-specific range for which a selection operation has been accepted so that it is displayed in accordance with the vertical or horizontal dimensions of the display screen.

[0016] In the present invention, the basic range has a plurality of partial ranges corresponding to each part of the face, and the range transformation means is configured to transform the plurality of partial ranges of the basic range, and when an overlap occurs between ranges transformed from the partial ranges due to the transformation by the range transformation means, the range identification means identifies the continuous outermost contour ranges of the overlapping ranges as a single user-specific range.

[0017] The present invention also provides a skin condition display method in which a skin condition display device detects a skin condition by stages based on a captured image including a user's face, and performs processing related to display by applying a color corresponding to the stage related to the detection result to the user's face included in the captured image. The skin condition display device includes the steps of: arranging a basic range indicating a range corresponding to a part of the face to be detected on the user's face included in the captured image; transforming the basic range arranged on the user's face into a shape corresponding to the user's face; specifying the range transformed from the basic range as a user-specific range; detecting skin conditions by stages for the user-specific range; generating a skin condition display image by applying a color corresponding to the stage related to the detection result to the detected part in the user-specific range; and performing processing related to display of the generated skin condition display image. The present invention is characterized by carrying out the following steps.

[0018] Furthermore, the present invention provides a computer program for causing a computer to detect skin condition in stages based on a captured image including a user's face, and to execute processing related to displaying the user's face included in the captured image by applying a color corresponding to the stage of the detection result, the computer is characterized by executing the steps of: placing, on the user's face included in the captured image, a basic range indicating a range corresponding to the part of the face to be detected, a basic range placed on the user's face, a shape corresponding to the user's face, identifying the range transformed from the basic range as a user-specific range, detecting skin condition in stages for the user-specific range, applying a color corresponding to the stage of the detection result to the detected part within the user-specific range to generate a skin condition presentation image, and performing processing related to displaying the generated skin condition presentation image.

[0019] In the present invention, the basic range is transformed into a shape that corresponds to the user's face to identify a user-specific range, detection is performed within that user-specific range, and a skin condition presentation image is generated and displayed with a color gradient that corresponds to the detection result. Therefore, rather than performing detection within a uniform range and displaying the detection result in color, a user-specific detection range is identified according to the configuration of the user's face (configuration related to the size, shape, etc. of the user's face and each part of the face), so appropriate detection is performed within a range that matches the configuration of the user's face and the detection results can be displayed within that appropriate range, making it possible to more clearly express the skin condition detection results in accordance with the user's own face compared to conventional technologies.

[0020] In the present invention, a basic range for each different skin condition item is prepared, and a series of various processes is performed for each different item, so that a skin condition presentation image for each item can finally be displayed. This makes it possible to present the skin condition detection results for each skin condition item in a way that is tailored to the user's own face.

[0021] In the present invention, the contour line surrounding the user-specific range is shown in the skin condition presentation image, making it possible to visually confirm the user-specific range that is the target of detection on the user's face, making it easier to focus on the range that is the target of detection, and making it possible to present the detection results to the user by expressing the range that is the target of detection relative to the entire face.

[0022] In the present invention, when a selection operation of a user-specific range is accepted, the user-specific range for which the selection operation has been accepted is enlarged and displayed in accordance with the vertical or horizontal dimensions of the display screen, making it possible for the user to easily enlarge the range of the detection target desired by the user with a one-touch operation, and making it easier than ever to check the details of the skin condition.

[0023] In the present invention, when overlap between adjacent ranges occurs in a range deformed from a partial range corresponding to each part due to deformation to match the user's facial structure, the continuous outermost contour ranges of the overlapping ranges are identified as a unified user-specific range, thereby avoiding doubly identifying the overlapping area as a user-specific range, making it possible to execute efficient detection processing and enable high-precision and uniform detection accuracy to match the user's facial structure. Effect of the Invention

[0024] In the present invention, a user-specific detection range is identified according to the configuration of the user's face, and detection and display of the detection results are performed. Therefore, skin condition detection and display of the detection results can be performed in accordance with the configuration of the user's face, making it easier to understand the skin condition detection results in accordance with the shape, size, etc. of each user's face.

[0025] Furthermore, in the present invention, a basic range for each different skin condition item is stored, and a series of various processes are performed for each different item, so that an item-specific skin condition presentation image can finally be displayed on the screen, making it easier for the user to grasp the skin condition detection results for each skin condition item with content tailored to the user's own face.

[0026] Furthermore, in the present invention, the contour line of the user-specific range is shown in the skin condition presentation image, making it possible to visually confirm the user-specific range that is the target of detection within the user's face, making it easier to focus on the range that is the target of detection within the entire face, and allowing for comprehensive recognition of the skin condition in relation to its location within the entire face.

[0027] Furthermore, in the present invention, the user-specific range that has been selected is enlarged and displayed to match the vertical or horizontal dimensions of the display screen, so that the range that the user wishes to check can be easily enlarged with a one-touch operation, enabling the user to smoothly check the details of their skin condition.

[0028] Furthermore, in the present invention, even if overlap occurs between adjacent ranges due to deformation to match the user's facial structure, the continuous outermost contour ranges of the overlapping ranges are identified as a unified user-specific range, so that the detection conditions according to the user's facial structure can be standardized, thereby making the detection-related processing more efficient and helping to improve detection accuracy. [Brief description of the drawings]

[0029] [Figure 1] 1 is a schematic diagram showing an overall system configuration of a skin condition display system according to an embodiment of the present invention. [Diagram 2] FIG. 2 is a block diagram showing the main internal configuration of a communication terminal device (smartphone, skin condition presentation device). [Diagram 3] FIG. 1A is a schematic diagram showing the front side of a smartphone, and FIG. 1B is a schematic diagram showing a smartphone with a shooting screen including a user's face displayed on a display screen. [Figure 4] 1 is a diagram showing a portion of a basic range table. [Diagram 5] 13 is a diagram showing another portion of the basic range table. [Figure 6] 13A and 13B are schematic diagrams showing examples of situations in which the basic range is deformed. [Figure 7] 13(a) and 13(b) are schematic diagrams showing other examples of modifying the basic range. [Figure 8]13A-13C are schematic diagrams showing examples of portions of user-specific range images by item. [Figure 9] 13A to 13C are schematic diagrams showing other examples of user-specific range images by item. [Figure 10] 13(a)-(c) are schematic diagrams showing examples of some of the images presenting skin conditions by category. [Figure 11] 13(a)-(c) are schematic diagrams showing other examples of skin condition display images by category. [Figure 12] FIG. 13 is a schematic diagram showing an example of an analysis result screen displayed on a display screen. [Figure 13] FIG. 2A is a schematic diagram showing an example of a pore image screen, and FIG. 2B is a schematic diagram showing an example of a texture image screen. [Figure 14] FIG. 1A is a schematic diagram showing an example of a screen showing an image of fine lines, and FIG. 1B is a schematic diagram showing an example of a screen showing an image of dark circles. [Figure 15] FIG. 1(a) is a schematic diagram showing an example of a blemish image screen, and FIG. 1(b) is a schematic diagram showing an example of a wrinkle image screen. [Figure 16] 13A and 13B are schematic diagrams showing an example of an enlarged display of a user-specific range. [Figure 17] 3 is a flowchart showing a main processing procedure of a skin condition display method according to the present invention. [Figure 18] 2 is a block diagram showing a main internal configuration of a service management server. [Figure 19] 13 is a diagram showing a part of the contents of a registered user table. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS EXAMPLES

[0030] 1 shows an overall overview of a skin condition display system 1 according to an embodiment of the present invention. The skin condition display system 1 photographs a user's face using a communication terminal device (e.g., a smartphone 10) having a photographing function, detects the skin condition of the user's face in stages based on the photographed image, and provides a service that allows the user to check the detection results and uploads and stores the detection results in a cloud system 2. The present invention is characterized in that detection is performed according to the configuration of the user's face (the size, shape, etc. of the face and each part of the face) and the detection results are displayed.

[0031] In FIG. 1, in order to prevent the contents of the figure from becoming complicated, only one smartphone 10 used by one user is shown, but in an actual operation scene, there are many users, and each user uses various communication terminal devices including a smartphone. In addition, in order to enable each user to receive services from the skin condition display system 1, in this embodiment, the user is assumed to register in advance for the service provided by the skin condition display system 1 by transmitting user information such as the user's name and email address to a service management server 5 that manages the cloud system 2, and when the user registration is completed, a user ID (user identification information) for identifying each user is issued from the service management server 5. Details of the embodiment of the present invention will be described below, focusing on the smartphone 10 (communication terminal device) that plays an important role in the present invention.

[0032] FIG. 2 shows the internal configuration of a smartphone 10, which is an example of a communication terminal device that can be used in the present invention, and shows the parts related to the present invention. The smartphone 10 according to the present invention is required to have a photographing function for photographing the face of a user. Note that the user terminal that can be used in this embodiment is not limited to the smartphone 10, and a portable terminal such as a tablet having a photographing function (corresponding to a type of computer device) can be applied, and a stationary terminal having a photographing function can be applied as a communication terminal device that can be used in the present invention. In the following description, the case where the smartphone 10 is used will be described, but the parts related to the present invention of various terminals having the above-mentioned photographing function other than smartphones will be common to the internal configuration shown in FIG. 2.

[0033] The smartphone 10 (communication terminal device) is a device (a type of computer-like device equipped with communication means, shooting means, storage means, etc.) that performs various processes according to a program, and is configured by connecting various devices such as a communication / call module 10b (corresponding to communication means), RAM 10c, ROM 10d, screen connection processing unit 10e, shooting processing unit 10f, acceleration sensor 10g, operation unit 13, and storage unit (corresponding to storage means) 10h, to a CPU 10a (processor 10a) that performs overall control and various processes, via an internal connection line 10i.

[0034] The communication and call module 10b of the smartphone 10 has a wireless communication function via a network, and enables communication with the cloud system 2 of the above-mentioned skin condition presentation system 1, etc., and also has a function of making a call to a specified telephone number (call out function) and receiving a call (call in function) under the control of the CPU 10a. The RAM 10c temporarily stores contents, files, data, etc. associated with the processing of the CPU 10a. The ROM 10d stores programs and the like that define the basic processing contents of the CPU 10a, and also stores identification information (UID) and the like that identifies the smartphone. Note that this UID is included in the transmission contents when the above-mentioned communication and call module 10b communicates (transmits) (for example, the UID is included in the header of the transmission packet and transmission is performed).

[0035] The screen connection processing unit 10e is connected to a rectangular display screen 12 having a touch panel function, which is arranged on the front side 11a of the thin plate-like housing 11 of the smartphone 10 shown in FIG. 3(a), and performs processing related to the display of a screen (see each screen shown in FIG. 12, 13, etc.) including various images generated by the control processing of the CPU 10a (e.g., output processing to the display screen 12), so that the image of the screen, etc. is displayed on the display screen 12. The display screen 12 uses a liquid crystal panel or an organic EL panel, etc., and has a predetermined resolution with multiple pixels arranged in the horizontal and vertical directions.

[0036] The screen connection processing unit 10e also performs processing for transmitting to the CPU 10a operation contents received when the user performs operations such as tapping, swiping, flicking, etc. on the surface of the display screen 12. Note that the operation contents received when the user performs operations such as tapping, swiping, flicking, etc. on the surface of the display screen 12 change appropriately depending on the displayed screen contents.

[0037] The photographing processing unit 10f is a device equivalent to a processing engine that performs photographing control processing and processing of photographed images, and is connected to the sub-camera 14a arranged on the front side 11a of the housing 11 and the main camera 14b arranged on the rear side of the housing 11. The photographing processing unit 10f obtains a photographed image from the imaging element by controlling the photographing control processing for the imaging elements (CCD or CMOS) included in the sub-camera 14a and the main camera 14b based on the instruction of the CPU 10a, and performs a predetermined processing (color balance processing, etc.) on the obtained photographed image (for example, a photographed still image) to generate image data (image data corresponding to the photographed image) in a predetermined file format (for example, a JPEG format file, etc.).

[0038] The acceleration sensor 10g is a motion sensor for detecting the posture of the smartphone 10, and the CPU 10a determines whether the smartphone 10 is being held vertically by the user (portrait holding) or horizontally (landscape holding) based on the detection value of the acceleration sensor 10g.

[0039] The operation unit 13 is a hard button provided on the housing 11, and when the operation unit 13 is operated, the operation is notified to the CPU 10a. The meaning of the operation of the operation unit 13 varies depending on the processing status of the smartphone 10. For example, when the operation unit 13 is operated while an app is running, an operation to close the app is performed. In this case, the operation of the operation unit 13 is equivalent to receiving an instruction from the user to close the app.

[0040] The storage unit 10h is a storage means using an SSD or the like, and stores (installs) programs such as the OS program P1, the skin condition management application 15, and various other applications. The OS program P1 is a basic program equivalent to an operating system, and defines the processing of the CPU 10a for the smartphone 10 to function as a type of computer. One of the basic processes defined by the OS program P1 is to display a home screen on the display screen 12 first when the smartphone 10 is ready for use, and on this home screen, icons and the like corresponding to various applications installed in the storage unit 10h are also arranged by the processing defined by the OS program P1.

[0041] The skin condition management application 15 corresponds to a computer program that defines various processing contents to be performed by the CPU 10a in relation to the present invention, and the CPU 10a performs processing based on the definitions of the skin condition management application 15, whereby the CPU 10a executes predetermined processing as various means, and the smartphone 10 functions as the skin condition presentation device of the present invention. The skin condition management application 15 is installed in the storage unit 10h by being downloaded to the smartphone 10 from a server that distributes various applications. When the skin condition management application 15 is stored in the storage unit 10h of the smartphone 10 and installation is completed, an icon for the skin condition management application 15 is arranged selectably on the home screen (not shown).

[0042] The main processing contents defined by the skin condition management application 15 of this embodiment include a login process to the service provided by the skin condition presentation system 1, a photographing process, a skin condition analysis process, a skin condition presentation process, and an upload process of the analysis results. The skin condition management application 15 also defines screen contents corresponding to each of these processes to be displayed appropriately on the display screen 12. The skin condition management application 15 of this embodiment also includes screen data for various screens to be displayed on the display screen 12, and a basic range table 20 used in the skin condition analysis. The contents of each of the above-mentioned processing of the skin condition management application 15 will be described later, and the basic range table 20 will be described first.

[0043] The basic range table 20 stores basic ranges indicating ranges corresponding to the parts of the face to be detected, and basic ranges for each item with different skin conditions are stored. There are six types of basic range items stored in the basic range table 20, namely, "pores," "texture," "fine lines," "dark circles," "blemishes," and "wrinkles" (see Figs. 4 and 5). The basic ranges 21-26 for each item stored in the basic range table 20 are for an average face (the face indicated by the dashed line in Figs. 4 and 5 corresponds to the average face), and indicate ranges for each item corresponding to the shape and position of each part to be detected in an average face derived based on a large number of sampled faces.

[0044] 4 shows a part of the contents of the basic range table 20, including basic ranges 21-23 for the three items of "pores," "texture," and "fine wrinkles." Each of these basic ranges 21-23 is graphic data that is read from the basic range table 20 (storage unit 10h) and placed on a photograph of the user's face when the smartphone 10 (CPU 10a) detects (analyzes) the user's skin condition for the items of "pores," "texture," or "fine wrinkles."

[0045] In the basic range table 20 shown in Fig. 4, the basic range 21 for pores stored in the "pores" item has a total of five partial ranges (partial ranges corresponding to the arrangement of each part of the face) corresponding to each part of the face. The partial ranges corresponding to each of these five parts include a forehead partial range 21a for pores, an eyebrow partial range 21b for pores, a nose partial range 21c for pores, a left cheek partial range 21d for pores, and a right cheek partial range 21e for pores.

[0046] The forehead partial area 21a for pores is arranged on the forehead of the face, and is a substantially trapezoidal area that spreads horizontally to match the shape of the forehead. The eyebrow partial area 21b for pores is arranged on the eyebrow of the face, and is a reverse trapezoidal area with a long upper side and a short lower side. The nose partial area 21c for pores is arranged on the nose of the face, and is a vertically elongated trapezoidal area that matches the shape of the nose. The left cheek partial area 21d for pores is arranged on the left cheek of the face, and is a substantially fan-shaped area that covers the left cheek. The right cheek partial area 21e for pores is arranged on the right cheek of the face, and is a symmetrical area to the left cheek partial area 21d for pores, and is also symmetrical in shape.

[0047] As shown in Fig. 4, in the arrangement of each partial area 21a of the basic area 21 for pores, the upper part of the eyebrow partial area 21b for pores enters into the forehead partial area 21a for pores, and an overlap occurs between the adjacent forehead partial area 21a for pores and the eyebrow partial area 21b for pores. In detail, the eyebrow partial area 21b for pores is formed to be extended upward more than necessary, and enters into the forehead partial area 21a for pores. By adopting such a shape arrangement, it is possible to avoid a gap between the two even if a deformation process is performed according to the user's facial structure (face size, etc.) described later, and the above overlap can be adjusted to make the detection process highly accurate and uniform.

[0048] In the basic range table 20 of FIG. 4, the basic range for texture 22 stored in the "texture" item is composed of a total of three partial ranges corresponding to each part of the face (partial ranges corresponding to placement on each part of the face). The three partial ranges are a forehead partial range for texture 22a, a left cheek partial range for texture 22b, and a right cheek partial range for texture 22c. The forehead partial range for texture 22a is equivalent to the forehead partial range for pores 21a described above, and is placed on the forehead part of the face. The left cheek partial range for texture 22b is equivalent to the left cheek partial range for pores 21d described above, and is intended for placement on the left cheek part of the face. The right cheek partial range for texture 22c is equivalent to the right cheek partial range for pores 21e described above, and is intended for placement on the right cheek part of the face.

[0049] In the basic range table 20 in Fig. 4, the basic range 23 for fine lines stored in the item "fine lines" is composed of a total of six partial ranges (partial ranges corresponding to the arrangement of each part of the face) corresponding to each part of the face. The six partial ranges are a left temple partial range 23a for fine lines, a left under-eye partial range 23b for fine lines, a left cheek side partial range 23c for fine lines, a right temple partial range 23d for fine lines, a right under-eye partial range 23e for fine lines, and a right cheek side partial range 23f for fine lines.

[0050] The left temple partial area 23a for fine lines is arranged in the area of ​​the left temple in the face, and indicates a substantially triangular area with the apex pointing upward. The left under-eye area partial area 23b for fine lines is arranged in the area below the left eye in the face, and indicates a curved band-like area from the inner corner of the left eye through the lower part of the eye to the side of the outer corner of the eye. The left cheek side partial area 23c for fine lines is arranged in the area beside the left cheek from the left cheek toward the left sideburns, and indicates a substantially triangular area with the apex pointing downward. In addition, the right temple partial area 23d for fine lines, the right under-eye partial area 23e for fine lines, and the right cheek side partial area 23f for fine lines each indicate a range of an area symmetrical to the above-mentioned left temple partial area 23a for fine lines, the left under-eye partial area 23b for fine lines, and the left cheek side partial area 23c for fine lines, and are also symmetrical in shape.

[0051] As shown in FIG. 4, in the arrangement of each partial area 23a of the basic area 23 for fine lines, the lower part of the left temple partial area 23a for fine lines enters into the left eye partial area 23b for fine lines, and the upper part of the left cheek side partial area 23c for fine lines enters into the left eye partial area 23b for fine lines, so that there is an overlap between each of the adjacent partial areas 23a-23c. This is for the same reason as the overlap between the forehead partial area 21a for pores and the eyebrow partial area 21b for pores of the basic area 21 for pores described above, and the area shapes and arrangements are designed to overlap in advance so that there is no gap between them even if they are deformed. This is also the case with the overlap between the right temple partial area 23d for fine lines and the right eye partial area 23e for fine lines, and the overlap between the right cheek side partial area 23f for fine lines and the right eye partial area 23e for fine lines.

[0052] 5 shows the remaining contents of the basic range table 20, and indicates that it includes basic ranges 24-26 for the three items of "dark circles," "spots," and "wrinkles." Each of these basic ranges 24-26 is also read from the basic range table 20 (storage unit 10h) when the smartphone 10 (CPU 10a) detects the skin condition of the user for the items of "dark circles," "spots," or "wrinkles," and becomes graphic data to be placed on the photographed user's face.

[0053] In the basic area table 20 shown in Fig. 5, the basic area for dark circles 24 stored in the "dark circles" item is composed of a total of two partial areas corresponding to each part of the face. The two partial areas are a left eye under partial area for dark circles 24a and a right eye under partial area for dark circles 24b. The left eye under partial area for dark circles 24a is placed in the area below the left eye on the face, and is shaped to represent a strip-like area below the left eye. The right eye under partial area for dark circles 24b is placed in the area below the right eye on the face, and is shaped to represent a strip-like area below the right eye.

[0054] In the basic range table 20 shown in Fig. 5, the basic range for blemishes 25 stored in the "blemishes" item is composed of a total of five partial ranges corresponding to each part of the face. The five partial ranges are a forehead partial range for blemishes 25a, an eyebrow partial range for blemishes 25b, a nose partial range for blemishes 25c, a left cheek partial range for blemishes 25d, and a right cheek partial range for blemishes 25e. Each of these partial ranges for blemishes 25a-25e is equivalent to each of the partial ranges for pores 21a-21e described above, and there is an overlap between the adjacent forehead partial range for blemishes 25a and eyebrow partial range for blemishes 25b.

[0055] In basic range table 20 shown in Fig. 5, basic range 26 for wrinkles stored in the item "wrinkles" is composed of a total of nine partial ranges corresponding to each part of the face. The nine partial ranges are forehead partial range 26a for wrinkles, upper eyebrow partial range 26b for wrinkles, upper nose partial range 26c for wrinkles, left eye lower partial range 26d for wrinkles, upper left mouth partial range 26e for wrinkles, lower left mouth partial range 26f for wrinkles, right eye lower partial range 26g for wrinkles, upper right mouth partial range 26h for wrinkles, and lower right mouth partial range 26i for wrinkles.

[0056] The forehead partial area for wrinkles 26a is equivalent to the forehead partial area for pores 21a shown in Fig. 4, and is placed on the forehead area of ​​the face. The upper eyebrow partial area for wrinkles 26b is placed on the area from the upper part of the eyebrows to the forehead, and is a wider area than the eyebrow partial area for pores 21b shown in Fig. 4. The upper nose partial area for wrinkles 26c is placed on the area that will be the upper part of the nose (around the base of the nose) of the face, and is an inverted trapezoidal area with a long upper side and a short lower side.

[0057] The left eye lower partial area 26d for wrinkles is placed from the inner corner of the left eye to the upper left cheek below the left eye, and is a polygonal area formed by modifying a parallelogram. The upper left mouth partial area 26e for wrinkles is placed from the upper left end of the lips to the left cheek, and is a range shaped like an inverted trapezoid tilted about 45 degrees clockwise. The lower left mouth partial area 26f for wrinkles is placed from the lower left end of the lips to the side, and is a range shaped like an inverted trapezoid.

[0058] In addition, right eye lower partial area 26g for wrinkles, upper right mouth partial area 26h for wrinkles, and lower right mouth partial area 26i for wrinkles indicate areas that are symmetrical to the above-mentioned left eye lower partial area 26d for wrinkles, upper left mouth partial area 26e for wrinkles, and lower left mouth partial area 26f for wrinkles, and are also symmetrical in shape. As shown in Fig. 5, in the arrangement of each partial area 26a etc. of basic area 26 for wrinkles, the upper part of upper eyebrow partial area 26b for wrinkles enters into forehead partial area 26a for wrinkles, similar to the above-mentioned basic area 21 for pores, and overlap occurs.

[0059] Next, a description will be given of the contents of each process defined by the skin condition management application 15. Regarding the process of logging in to the service provided by the skin condition presentation system 1, when an icon for the skin condition management application 15 arranged on the home screen is tapped (selected) and the skin condition management application 15 is started, the CPU 10a performs a process of displaying a login screen (not shown) on the display screen 12 based on the definition of the started skin condition management application 15. This login screen is provided with input fields for a user ID and a password, and also with a selectable login button.

[0060] While this login screen is displayed, the CPU 10a accepts input of a user ID (e.g., an email address) and a password and determines whether a login button has been selected. When the CPU 10a accepts input of a user ID and a password and determines that the login button has been selected, the CPU 10a transmits the accepted user ID and password to the service management server 5.

[0061] In response to the transmission of the user ID and password, the service management server 5 sends to the smartphone 10 a notification of login completion or login failure. When the smartphone 10 receives a login failure notification, the CPU 10a displays a login screen indicating that login was not possible because the user ID or password does not match, and provides the user with an opportunity to input the user ID and password again.

[0062] On the other hand, when the smartphone 10 receives a login completion notification, the CPU 10a performs a process of switching the display on the display screen 12 to the photography screen 30, and then the CPU 10a performs a process related to photography. When past analysis results of the skin condition of the user who has completed login (such as the average analysis results for the past 30 days) are stored in the service management server 5, the login completion notification is sent including the past analysis results, so when the smartphone 10 receives such a login completion notification, the CPU 10a stores the past analysis results included in the notification in the RAM 10c.

[0063] 3(a) shows a photographing screen 30 displayed on the display screen 12. This photographing screen 30 is photographed by the sub-camera 14a arranged on the front side 11a of the housing 11, and a substantially elliptical guide line 30a is displayed on the photographing screen 30 to guide the size and position of the user's face to be photographed. Based on this photographing screen 30, the CPU 10a as a photographing processing means judges whether a photographing image that does not hinder the detection of the skin condition can be obtained with respect to photographing conditions such as brightness, the position of the user's face, and the direction of the user's face, and if it is judged that these photographing conditions are satisfied, it performs automatic photographing processing.

[0064] Fig. 3(a) shows a state immediately after the photographing screen 30 is displayed on the display screen 12, and the user's face is not yet included in the photographing screen 30. In addition, in the state shown in Fig. 3(a), photographing conditions such as brightness, the position of the user's face, and the direction of the user's face (facing forward) are not met, so the marks relating to the three conditions included in the upper part of the photographing screen 30 (brightness condition mark 30b, position condition mark 30c, and direction condition mark 30d) also indicate a state in which the conditions are not met (in Fig. 3(a), each of the marks 30b to 30d is shown by a dashed line to indicate a state in which the conditions are not met (a state in which they are not cleared)).

[0065] On the other hand, Fig. 3(b) shows a state where each condition of the photographing screen 30 displayed on the display screen 12 is satisfied. In this photographing screen 30, the user's face F is photographed and fits within the guide line 30a, and the conditions of brightness and direction are also satisfied (cleared). Therefore, in the photographing screen 30 of Fig. 3(b), each of the marks 30b to 30d also indicates a state where the conditions are satisfied (cleared) (in Fig. 3(b), each of the marks 30b to 30d is shown by a solid line to indicate a state where the three photographing conditions are satisfied).

[0066] To explain the specific processing content regarding the above photography, the photography data taken by the imaging element of the sub camera 14a is sent from the sub camera 14 to the photography processing unit 10f at any time, and the CPU 10a performs a process of comparing the brightness component of the pixel included in the photography data acquired by the photography processing unit 10f with a brightness threshold value defined in advance. If the result of the comparison shows that the brightness component of the pixel of the photography data exceeds the threshold value, it is determined that the brightness condition is satisfied (cleared).

[0067] Furthermore, the CPU 10a detects the contour of the user's face captured in the photographic data by detecting differences in components such as color between adjacent pixels in the photographic data, and detects whether or not the detected contour of the user's face exceeds the approximately elliptical guide line 30a included in the photographic screen 30. If the result of the detection shows that the contour of the user's face does not exceed the guide line 30a, the CPU 10a determines that the position condition is satisfied (cleared).

[0068] Furthermore, the CPU 10a identifies pixels in the photographed data where the user's eyes are assumed to be located, and detects whether the eyes are evenly positioned on the left and right based on the color and other components of the identified pixels. If the detection result indicates that the eyes are evenly positioned on the left and right, the CPU 10a determines that the user is facing forward (facing the camera), and that the orientation condition is satisfied (cleared).

[0069] When the three conditions of brightness, position, and orientation are satisfied (cleared) as described above (the state shown in FIG. 3(b)), the CPU 10a starts the actual photographing process, and after a predetermined time has elapsed (e.g., after 3 seconds) using the self-timer function, the shutter is automatically released to obtain a photographed image (image data corresponding to the photographed image) including the user's face. Note that when photographing using the self-timer, the value of the countdown seconds may be displayed on the photographing screen 30 shown in FIG. 3(b).

[0070] When the CPU 10a obtains a captured image (image data corresponding to the captured image) including the captured face of the user, the CPU 10a then performs a skin condition analysis process. In this skin condition analysis process, the CPU 10a sequentially executes a process divided into a plurality of stages. First, the CPU 10a reads out basic ranges 21-26 for each item from the basic range table 20 shown in Figs. 4 and 5, and places each of the read basic ranges 21-26 for each item on the user's face in the captured image corresponding to the image data (assuming that the image data is copied and there are six of them), to generate a total of six images for analysis by item.

[0071] Specifically, for each pixel forming the user's face contained in the image data (photographed image), CPU 10a identifies characteristic points on the face (e.g., characteristic points such as the corners of the eyes, the inner corners of the eyes, the left and right points of the lips, and the points at both ends of the left and right eyebrows) and characteristic lines on the face (e.g., characteristic lines such as the face lines indicating the outline of the cheeks, the lines indicating the left and right noses, and the outlines of the upper left and right eyebrows) based on components such as color and brightness.

[0072] Then, using the above-mentioned specified feature points and feature lines as guides, the CPU 10a performs processing to place each of the item-specific basic ranges 21-26 read from the basic range table 20 at each part of the user's face included in the captured image as a placement means. Specifically, the CPU 10a reads out the basic range 21 for pores stored in the "pores" item of the basic range table 20 in Fig. 4, and then places the forehead partial range 21a for pores, the eyebrow partial range 21b for pores, the nose partial range 21c for pores, the left cheek partial range 21d for pores, and the right cheek partial range 21e for pores, which constitute the read basic range 21 for pores, at each part of the user's face as appropriate, using the specified feature points and feature lines as guides.

[0073] Regarding the arrangement of each partial area 21a-21e of the basic area 21 for pores, for example, in the case of the forehead partial area 21a for pores, the ends of the eyebrows on both sides are characteristic points, and the upper contours of the eyebrows on both sides are characteristic lines, and the lower side of the forehead area 21a for pores is arranged to follow these characteristic points and characteristic lines. In the case of the eyebrow partial area 21b for pores, the ends of the eyebrows on both sides and the upper end of the nose are characteristic points, and the eyebrow partial area 21b for pores is arranged to match the area surrounded by these characteristic points.

[0074] Furthermore, in the case of the nose partial range 21c for pores, the upper and lower ends of the nose are characteristic points, and the lines indicating the left and right sides of the nose are characteristic lines, and the nose partial range 21c for pores is arranged according to the area surrounded by these characteristic points and characteristic lines. Furthermore, in the case of the left cheek partial range 21d for pores, the inner corner of the left eye, the outer corner of the left eye, and the left end of the lower part of the nose are characteristic points, and the left face line (the outline of the face around the left cheek) are characteristic lines, and the left cheek partial range 21d for pores is arranged according to the area surrounded by these characteristic points and characteristic lines. The right cheek partial range 21e for pores is also arranged in a similar area that is symmetrical to the above-mentioned left cheek partial range 21d for pores. By arranging each partial range 21a-21e for pores in this way, a basic range image for pores is generated, and the generated basic range image for pores is stored in the RAM 10c.

[0075] 4, the forehead partial range 22a for skin texture, the left cheek partial range 22b for skin texture, and the right cheek partial range 22c for skin texture included in the basic range 22 for skin texture are appropriately arranged on each part of the user's face. At this time, the forehead partial range 22a for skin texture is arranged similarly to the forehead partial range 21a for skin texture, the left cheek partial range 22b for skin texture is arranged similarly to the left cheek partial range 21d for skin texture, and the right cheek partial range 22c for skin texture is arranged similarly to the right cheek partial range 21e for skin texture, and a skin texture basic range image is generated (the generated basic range image for skin texture is also stored in the RAM 10c, and the same applies below).

[0076] Furthermore, when the basic range 23 for fine wrinkles is read out from the "fine wrinkles" item in the basic range table of Figure 4, the left temple partial range 23a for fine wrinkles, the left under-eye partial range 23b for fine wrinkles, the left cheek side partial range 23c for fine wrinkles, the right temple partial range 23d for fine wrinkles, the right under-eye partial range 23e for fine wrinkles, and the right cheek side partial range 23f for fine wrinkles included in the basic range 23 for fine wrinkles are appropriately positioned to each part of the user's face.

[0077] For the left temple partial area 23a for fine wrinkles, the outer edge of the left eyebrow and the outer corner of the left eye are characteristic points, and the face line around the left temple are characteristic lines, and the left temple partial area 23a for fine wrinkles is arranged so as to fit the area surrounded by these characteristic points and characteristic lines. For the left under-eye partial area 23b for fine wrinkles, the inner corner of the left eye and the outer corner of the left eye are characteristic points, and the lower side of the left eye and the line on the left edge of the nose are characteristic lines, and the left under-eye partial area 23b for fine wrinkles is arranged to fit these characteristic points and characteristic lines.

[0078] Furthermore, for the left cheek side partial area 23c for fine wrinkles, the base of the left ear and the like are characteristic points, and the face line of the left cheek and the like are characteristic lines, and the left cheek side partial area 23c for fine wrinkles is arranged according to these characteristic points and characteristic lines. Furthermore, the right temple partial area 23d for fine wrinkles, the right under-eye partial area 23e for fine wrinkles, and the right cheek side partial area 23f for fine wrinkles are each arranged in a similar area that is symmetrical to the above-mentioned left temple partial area 23a for fine wrinkles, the left under-eye partial area 23b for fine wrinkles, and the left cheek side partial area 23c for fine wrinkles, respectively. By performing the above-mentioned arrangement, a basic area image for fine wrinkles is generated.

[0079] Furthermore, when the CPU 10a reads out the basic area 24 for bear stored in the "bear" item of the basic area table 20 shown in FIG. 5, the left eye-under partial area 24a for bear and the right eye-under partial area 24b for bear constituting the read basic area 24 for bear are appropriately arranged in each part of the user's face. In this case, the left eye-under partial area 24a for bear is arranged along the characteristic points and characteristic lines of the inner corner and the outer corner of the left eye, and the lower side of the left eye. The right eye-under partial area 24b for bear is arranged symmetrically and similarly to the above-mentioned left eye-under partial area 24a for bear. By arranging the partial areas 24a and 24b for bear in this way, a basic area image for bear is generated.

[0080] Furthermore, when the basic range 25 for blemishes is read out from the "blemishes" item in the basic range table of Figure 5, each of the forehead partial range 25a for blemishes, eyebrow partial range 25b for blemishes, nose partial range 25c for blemishes, left cheek partial range 25d for blemishes, and right cheek partial range 25e for blemishes included in the basic range 25 for blemishes is placed on the user's face in the same manner as the forehead partial range 21a for pores, eyebrow partial range 21b for pores, nose partial range 21c for pores, left cheek partial range 21d for pores, and right cheek partial range 21e for pores that constitute the basic range 21 for pores described above, and a basic range image for blemishes is generated.

[0081] Finally, when wrinkle basic range 26 is read from the "Wrinkles" item in the basic range table of Fig. 5, six partial ranges 26a-26i consisting of a total of six partial ranges included in wrinkle basic range 26 are appropriately arranged at each part of the user's face using the specified feature points and feature lines as a guide. Specifically, wrinkle forehead partial range 26a is arranged at the forehead part of the user's face, similar to the above-mentioned pores forehead partial range 21a. Wrinkle upper eyebrow partial range 26b is also arranged at the eyebrow part of the user's face, similar to the above-mentioned pores for eyebrow partial range 21b.

[0082] The upper nose area for wrinkles 26c is located in an area surrounded by the inner corners of the left and right eyes and the upper base of the nose, which are characteristic points. The lower left eye area for wrinkles 26d is located in an area surrounded by the inner corners of the left eye and the lower edge of the left eye and the left side of the nose, which are characteristic lines. The upper left mouth area for wrinkles 26e is located in an area surrounded by the inner corners of the left eye and the lower edge of the left eye and the left side of the nose, which are characteristic points. The lower left mouth area for wrinkles 26f is located in an area surrounded by the inner corners of the left eye and the upper side of the nose, which are characteristic points.

[0083] Further, right eye lower partial range for wrinkles 26g, upper right mouth partial range for wrinkles 26h, and lower right mouth partial range for wrinkles 26i are arranged symmetrically with the above-mentioned left eye lower partial range for wrinkles 26d, upper left mouth partial range for wrinkles 26e, and lower left mouth partial range for wrinkles 26f, respectively. By arranging each partial range for wrinkles 26a-24b in this manner, a basic range image for wrinkles is generated.

[0084] After generating basic range images for each item as described above, the CPU 10a then performs processing as a range transformation means to transform each of the item-specific basic ranges 21-26 placed on the user's face in these basic range images into a shape that corresponds to the user's face.

[0085] That is, as described above, basic range 21-26 is intended for an average face, and since the face of an individual user has a different configuration (size, shape, etc.) compared to the average face, there will be discrepancies such as the ranges corresponding to each part to be detected indicated by basic range 21-26 including parts of an individual user's face that would not normally be subject to detection, or conversely, not including parts that should be subject to detection.

[0086] In order to eliminate such a misalignment, the CPU 10a performs a process of transforming each basic range 21-26 included in the item-specific basic range image into a shape corresponding to the face of each individual user. Since this transformation process can be performed in various ways depending on the size and shape of each user's face, or the size and shape of parts of each user's face, several examples of transformation situations will be described below.

[0087] FIG. 6(a) shows an example of a situation in which the left cheek partial range 21d for pores of the basic range 21 for pores is deformed in the basic range image for pores. As described above, the left cheek partial range 21d for pores is arranged to match the area surrounded by feature points such as the inner corner of the left eye, the outer corner of the left eye, and the left end of the lower part of the nose, and feature lines such as the face line showing the contour of the face around the left cheek. However, when the size of the user's face is large in the horizontal direction or when the contour of the user's face is angular, a gap S1 is generated between the cheek side perimeter line L1 on the side opposite to the face line F1 of the left cheek partial range 21d for pores and the face line F1 around the left cheek of the user. Note that the partial ranges arranged on the left and right of the left cheek partial range 21d for pores are basically aligned with the feature points on the center side of the face such as the inner corner of the left eye and the left end of the lower part of the nose as a priority guide, so when the size of the user's face is large in the horizontal direction, a gap S1 as shown in FIG. 6(a) is generated.

[0088] When the CPU 10a generates a basic range image for pores as described above, the CPU 10a detects the presence or absence of the gap S1 as described above in the generated basic range image for pores, and when it detects that a gap S1 exists, it deforms the left cheek partial range 21d for pores so as to eliminate the gap S1.

[0089] Specifically, the CPU 10a performs a process of deforming the left cheek partial area for pores 21d in a direction of expanding the area so that the cheek side peripheral line L1 of the left cheek partial area for pores 21d is positioned at a position where it overlaps with the face line F1. When performing such expansion, the CPU 10a also appropriately deforms the peripheral line constituting the periphery of the left cheek partial area for pores 21d. For example, the degree of curvature of the cheek side peripheral line L1 is deformed so that it becomes a curved line similar to the face line F1, resulting in a deformed cheek side peripheral line L1'. Through such a transformation process, the user-specific left cheek partial area for pores 21d' is formed by expansively transforming the left cheek partial area for pores 21d.

[0090] Fig. 6(b) is another example of the deformation, and shows an example of a situation in which the left cheek partial area for pores 21d of the basic area for pores 21 in the basic area for pores image is deformed in a direction (a direction in which the area is reduced) different from the deformation situation in Fig. 6(a) described above. In this example, when the size of the user's face is small in the lateral direction (horizontal direction) or when the contour of the user's face (particularly the contour around the cheek) is thin and sharp, a part of the left cheek partial area for pores 21d protrudes outward from the face line F1 of the left cheek.

[0091] The CPU 10a also detects whether or not there are any parts of each basic range that protrude from the user's facial contour (face line F1) as described above for the generated basic range image for pores, and when it detects that any protruding parts have occurred, it performs processing as range transformation means to transform the left cheek partial range 21d for pores so as to eliminate the protruding parts.

[0092] Specifically, the CPU 10a performs a process of deforming the left cheek partial area for pores 21d in a direction of shrinking so that the cheek side perimeter line L1 of the left cheek partial area for pores 21d is positioned at a position overlapping the face line F1. When performing such shrinking, the CPU 10a also performs a process of appropriately deforming the perimeter line constituting the perimeter of the left cheek partial area for pores 21d, as in the case of the expansion deformation described in Fig. 6(a) (the deformed cheek side perimeter line L1' is also generated). In the case of the example shown in Fig. 6(b), the deformed user-specific left cheek area for pores 21d' is also formed through the shrinking deformation process.

[0093] In addition, the exemplary processing method for transforming left cheek partial range 21d for pores shown in Figures 6(a) and (b) can basically be applied as an equivalent processing method when transforming left cheek partial range 22b for texture included in basic range 22 for texture, left temple partial range 23a for fine lines included in basic range 23 for fine lines, left under-eye partial range 23b for fine lines, and left cheek side partial range 23c for fine lines, left under-eye partial range 24a for dark circles included in basic range 24 for dark circles, left cheek partial range 25d for blemishes included in basic range 25 for blemishes, and left under-eye partial range 26d for wrinkles, upper left mouth partial range 26e for wrinkles, and lower left mouth partial range 26f for wrinkles included in basic range 26 for wrinkles, for example, when the horizontal size of the user's face is different from the average.

[0094] Furthermore, the exemplary processing method for transforming the left cheek partial range 21d for pores shown in Figures 6(a) and (b) can basically be applied as a processing method that is symmetrical and equivalent when transforming the right cheek partial range 22c for texture included in the basic range 22 for texture, the right temple partial range 23d for fine lines included in the basic range 23 for fine lines, the right under-eye partial range 23e for fine lines, and the right cheek side partial range 23f for fine lines, the right under-eye partial range 24b for dark circles included in the basic range 24 for dark circles, the right cheek partial range 25e for blemishes included in the basic range 25 for blemishes, and the right under-eye partial range 26g for wrinkles, the upper right mouth partial range 26h for wrinkles, and the lower right mouth partial range 26i for wrinkles included in the basic range 26 for wrinkles, for example when the horizontal size of the user's face is different from the average.

[0095] 7(a) shows an example of a situation in which the nose partial area 21c for pores of the basic area 21 for pores is deformed in the basic area image for pores. As described above, the upper and lower ends of the nose are characteristic points, and lines indicating the left and right of the nose are characteristic lines, and the nose partial area 21c for pores is arranged according to the area surrounded by these characteristic points and characteristic lines, but when the size of the user's nose is large in the lateral direction (horizontal direction), gaps S2 and S3 are generated between the left and right inclined peripheral lines L2 and L3 on both the left and right sides of the nose partial area 21c for pores and the left and right vertical lines H1 and H2 of the nose H.

[0096] The CPU 10a detects the presence or absence of the above-mentioned gaps S2, S3 in the generated basic range image for pores, and when it detects that the gaps S2, S3 exist, performs a process of deforming the nose part range 21c for pores as range deformation means so as to eliminate the gaps S2, S3.

[0097] Specifically, the CPU 10a performs a process of deforming the nose partial area for pores 21c in a direction of expanding it in the left and right lateral direction (horizontal direction) so that the left and right inclined peripheral lines L2, L3 of the nose partial area for pores 21c are positioned at the points where they meet the left and right vertical lines H1, H2 of the nose H. When performing such expansion, the CPU 10a also appropriately deforms the peripheral lines constituting the other peripheries of the nose partial area for pores 21c, as in the above-mentioned cases of Figures 6(a) and (b). Through such a transformation process, a user-specific nose partial area for pores 21c' that is expanded and deformed from the nose partial area for pores 21c is formed.

[0098] Fig. 7(b) is yet another example of the deformation, showing an example of a situation in which the nose partial area 21c for pores is deformed in a direction (a direction in which the area is reduced) different from the deformation situation in Fig. 7(a) described above in the basic area image for pores. When the size of the user's nose is small in the lateral direction (horizontal direction), as shown in Fig. 7(b), the left and right inclined peripheral lines L2, L3 on both the left and right sides of the nose partial area 21c for pores exceed the left and right vertical lines H1, H2 of the nose H, or overlap with each other.

[0099] The CPU 10a detects whether there is overlap or excess between the periphery of the nose partial area 21c for pores and facial features such as the nose for the generated basic area image for pores, as described above, and when it detects that overlap or the like has occurred, it performs processing to deform the nose partial area 21c for pores as area deformation means so as to eliminate such overlap or the like.

[0100] Specifically, the CPU 10a performs a process of deforming the nose partial area for pores 21c in a direction of shrinking it from the left and right in the lateral (horizontal) direction so that the left and right inclined perimeter lines L2, L3 of the nose partial area for pores 21c are positioned between the left and right vertical lines H1, H2 of the nose H. When performing such shrinkage, the CPU 10a also appropriately deforms the perimeter lines constituting the perimeter of the nose partial area for pores 21c, as described above. Through the above shrinkage deformation process, the nose partial area for pores 21c is formed as a deformed nose partial area for pores 21c'.

[0101] In addition, the exemplary processing method for transforming the nose partial range 21c for pores shown in Figures 7(a) and (b) can be applied as an equivalent processing method when transforming the forehead partial range 21a for pores and the eyebrow partial range 21b for pores included in the basic range 21 for pores, the forehead partial range 22a for texture included in the basic range 22 for texture, the forehead partial range 25a for blemishes, the eyebrow partial range 25b for blemishes, and the nose partial range 25c for blemishes included in the basic range 25 for blemishes, as well as the forehead partial range 25a for blemishes, the eyebrow partial range 25b for blemishes, and the nose partial range 25c for blemishes included in the basic range 25 for blemishes when the horizontal size of the user's face is different from the average.

[0102] As described above, by performing a transformation process on the item-specific basic ranges 21-26 based on the item-specific basic range image, the CPU 10a finally generates a user-specific range image including a range (referred to as a transformed basic range) transformed to fit the user's face (generating an item-specific user-specific range image). Then, in the user-specific range image generated as described above, the CPU 10a, as a range identification means, identifies each item-specific range (transformed basic range) transformed to fit the user's face as described above as a user-specific detection target range (referred to as a user-specific range) by item.

[0103] In addition, if the facial structure of the photographed user is similar to the average facial structure, the above-mentioned transformation processing may be completely unnecessary, in which case the CPU 10a will identify each of the item-specific basic ranges 21-26 as the user-specific range as is, and use the item-specific basic range image as the user-specific range image as is.

[0104] As described above, each of the basic ranges 21-26 by item has a plurality of partial ranges 21a-26i corresponding to each part of the face, but in the above transformation process, depending on the configuration of the user's face to be processed, a transformed partial range and an untransformed partial range may be mixed among a plurality of partial ranges (e.g., each partial range 21a-21e) included in one basic range by item (e.g., basic range 21). Even in the case where both a transformed partial range and an untransformed partial range are included, the untransformed partial range is used as it is, and the CPU 10a specifies each of these partial ranges as a user-specific range, and the image including such a user-specific range is made into a user-specific range image. Note that such a user-specific range image (user-specific range image by item) is stored in the RAM 10c for the next skin condition detection process.

[0105] 8(a)-(c) and 9(a)-(c) show the user-specific range images 101-106 for each item obtained through the above process. FIG. 8(a) shows the user-specific range image 101 for pores, which includes the user-specific range 121 for pores arranged on the user's face F. The user-specific range 121 for pores has a forehead user-specific range 121a for pores, an eyebrow user-specific range 121b for pores, a nose user-specific range 121c for pores, a left cheek user-specific range 121d for pores, and a right cheek user-specific range 121e for pores. As described above, each of the user-specific ranges 121a-121e for pores arranged on each part of the user's face F is a modified or directly used partial range 21a-21e of the basic range 21 for pores shown in FIG. 4.

[0106] As shown in FIG. 8(a), even after transformation of each partial range 21a of the basic range 21 for pores, overlap occurs between the adjacent forehead user-specific range 121a for pores and the between-eyebrow user-specific range 121b for pores. However, in the user-specific ranges 121a, 121b of each part where such overlap occurs, the continuous outermost contour range connecting the adjacent user-specific ranges 121a, 121b is identified by the CPU 10a as the integrated user-specific range of the relevant part.

[0107] Similarly, in the state after transformation shown in Fig. 8(a), the user-specific range 121c for nose pores overlaps with the user-specific range 121d for left cheek pores at the corner of the left eye, and the user-specific range 121c for nose pores overlaps with the user-specific range 121e for right cheek pores at the corner of the right eye. Therefore, the CPU 10a specifies the outermost periphery connecting the adjacent user-specific range 121c for nose pores, user-specific range 121d for left cheek pores, and user-specific range 121e for right cheek pores as the integrated user-specific range of the relevant part. This makes it possible to make the detection accuracy of these parts highly accurate and uniform.

[0108] 8(b) shows the user-specific range image 102 for texture, which includes a user-specific range 122 for texture arranged on the user's face F. The user-specific range 122 for texture has a forehead user-specific range 122a for texture, a left cheek user-specific range 122b for texture, and a right cheek user-specific range 122c for texture. Each of the user-specific ranges 122a-122c for texture arranged on each part of the user's face F is a modified version of the partial ranges 22a-22c of the basic range 22 for texture shown in FIG. 4, or is used as is.

[0109] 8(c) shows a user-specific range image 103 for fine wrinkles, which includes a user-specific range 123 for fine wrinkles arranged on the user's face F. The user-specific range 123 for fine wrinkles includes a user-specific range 123a for left temple for fine wrinkles, a user-specific range 123b for left eye for fine wrinkles, a user-specific range 123c for left cheek for fine wrinkles, a user-specific range 123d for right temple for fine wrinkles, a user-specific range 123e for right eye for fine wrinkles, and a user-specific range 123f for right cheek for fine wrinkles. Each of the user-specific ranges 123a-123f for fine wrinkles arranged on each part of the user's face F is a modified version of the partial ranges 23a-23f of the basic range 23 for fine wrinkles shown in FIG. 4, or is used as is.

[0110] 8(c), there is an overlap between the adjacent user specific ranges 123a for the left temple for fine lines, 123b for the left eye for fine lines, and 123c for the left cheek for fine lines, so the CPU 10a specifies the continuous outermost contour ranges of these user specific ranges 123a-123c as the integrated user specific ranges of each part. The same is true for the overlap between the adjacent user specific ranges 123d for the right temple for fine lines, 123e for the right eye for fine lines, and 123f for the right cheek for fine lines, so the CPU 10a specifies the continuous outermost contour ranges of these user specific ranges 123d-123f as the integrated user specific ranges of each part.

[0111] 9(a) shows a user-specific range image for a bear 104, which includes a user-specific range for a bear 124 arranged on the user's face F. The user-specific range for a bear 124 has a user-specific range for a bear under the left eye 124a and a user-specific range for a bear under the right eye 124b. As described above, each of these user-specific ranges for a bear 124a-124b is a modified version of each of the partial ranges 24a-24b of the basic range for a bear 24 shown in FIG. 5, or is used as is.

[0112] 9(b) shows a user-specific range image for blemishes 105, which includes a user-specific range for blemishes 125 arranged on the user's face F. The user-specific range for blemishes 125 has a forehead user-specific range for blemishes 125a, an eyebrow user-specific range for blemishes 125b, a nose user-specific range for blemishes 125c, a left cheek user-specific range for blemishes 125d, and a right cheek user-specific range for blemishes 125e. As described above, each of these user-specific ranges for blemishes 125a-125e is a modified version of each of the partial ranges 25a-25e of the basic range for blemishes 25 shown in FIG. 5, or is used as is.

[0113] In FIG. 9(b), there is also overlap between the adjacent forehead user-specific range 125a for blemishes and the interbrow user-specific range 125b for blemishes, and in the user-specific ranges 125a, 125b of each part where such overlap occurs, the continuous outermost contour ranges of the adjacent user-specific ranges 125a, 125b are identified by the CPU 10a as the integrated user-specific range of the relevant part.

[0114] 9(b), there is an overlap at the inner corner of the left eye between the nose user-specific range 125c for blemishes and the left cheek user-specific range 125d for blemishes, and there is also an overlap at the inner corner of the right eye between the nose user-specific range 125c for blemishes and the right cheek user-specific range 125e for blemishes. Therefore, the CPU 10a specifies the outermost periphery connecting the adjacent user-specific ranges 125c-125e for blemishes as the integrated user-specific range of the relevant part.

[0115] 9(c) shows a user-specific range image 106 for wrinkles, which includes a user-specific range 126 for wrinkles arranged on the user's face F. The user-specific range 126 for wrinkles includes a user-specific range 126a for forehead wrinkles, a user-specific range 126b for between-the-eyebrow wrinkles, a user-specific range 126c for above-the-nose wrinkles, a user-specific range 126d for below-the-left-eye wrinkles, a user-specific range 126e for upper-left mouth wrinkles, a user-specific range 126f for lower-left mouth wrinkles, a user-specific range 126g for below-the-right-eye wrinkles, a user-specific range 126h for upper-right mouth wrinkles, and a user-specific range 126i for lower-right mouth wrinkles. The user-specific ranges 126a-126i for each part of the wrinkles are modified or directly used from the partial ranges 26a-26i of the basic range 26 for wrinkles shown in FIG.

[0116] In FIG. 9(c), there is also overlap between the adjacent forehead user-specific range 126a for wrinkles and between-the-brow user-specific range 126b for wrinkles, and even in the user-specific ranges 126a, 126b of each part where such overlap occurs, the continuous outermost contour ranges of the adjacent user-specific ranges 126a, 126b are identified by the CPU 10a as the integrated user-specific range of the relevant part.

[0117] 9(c), there is an overlap at the inner corner of the left eye between user specific range 126c above nose for blemishes and user specific range 126d above cheek for wrinkles, and there is also an overlap at the inner corner of the right eye between user specific range 126c above nose for wrinkles and user specific range 126e above cheek for wrinkles. Therefore, CPU 10a specifies the outermost periphery connecting adjacent user specific ranges 126c-126e for wrinkles as the integrated user specific range of the relevant part.

[0118] Then, the CPU 10a performs a skin condition analysis process on the user specific ranges 121a-126i corresponding to the respective parts included in the user specific ranges 121-126 included in the item-specific user specific images 101-106 obtained by the above process.

[0119] First, as a premise, the captured image obtained by capturing an image with the image sensor of the sub-camera 14a is composed of a number of pixels (multiple pixels arranged horizontally and vertically in a rectangular shape overall) according to the resolution of the image sensor, and each of these pixels has a color component value (a value related to RGB or a value related to CMYK), a brightness value, and a value indicating its position in the image (coordinate values ​​based on the horizontal and vertical directions). Therefore, the item-specific user-specific images 101-106 shown in Figures 8(a)-(c) and 9(a)-(c) are also composed of a collection of a large number of pixels arranged horizontally and vertically in a rectangular shape overall.

[0120] Therefore, the CPU 10a identifies pixels included in the user specific ranges 121a-126i corresponding to each body part in the user specific images 101-106 by item. Then, the CPU 10a applies the color component values ​​and brightness values ​​of the identified pixels to a predetermined formula to perform a calculation process to obtain a numerical value. The predetermined formula used in this embodiment is a formula that obtains a numerical value of 0-100 as a calculation result (a numerical value of 0-100 is obtained for each pixel).

[0121] As described above, after calculating the numerical value for each pixel, the CPU 10a next calculates an average value for each user-specific range 121a-126i for each part by item from the sum of the calculated numerical values ​​of the pixels included in the range. Furthermore, the CPU 10a calculates an average value for the entire user-specific range 121-126 for each item from the average values ​​calculated for the user-specific range 121a-126i for each part by item, and specifies the calculated overall average value as the judgment value for that item.

[0122] Specifically, the user-specific range for pores 121 has a forehead user-specific range for pores 121a, a between-eyebrow user-specific range for pores 121b, a nose user-specific range for pores 121c, a left cheek user-specific range for pores 121d, and a right cheek user-specific range for pores 121e. If the average values ​​of the user-specific ranges 121a-121e for each body part are as follows: forehead user-specific range for pores 121a is "86", between-eyebrow user-specific range for pores 121b is "84", nose user-specific range for pores 121c is "83", left cheek user-specific range for pores 121d is "91", and right cheek user-specific range for pores 121e is "91", the overall average value of the user-specific range for pores 121 is "87" (in this embodiment, decimals are rounded off). Therefore, the CPU 10a specifies the overall average value "87" of the user specific range for pores 121 thus obtained as the determination result of the numerical value for the "pores" item of the photographed user.

[0123] Similarly, an overall average value is calculated for each of user-specific range for texture 122, user-specific range for fine lines 123, user-specific range for dark circles 124, user-specific range for age spots 125, and user-specific range for wrinkles 126, and each of the calculated overall average values ​​is identified as the numerical value judgment result for each of the items "texture," "fine lines," "dark circles," "age spots," and "wrinkles" for the user photographed (for example, through this process, the numerical values ​​for the "texture" item of the photographed user are identified as "78," the "wrinkles" item as "80," the "fine lines" item as "99," the "dark circles" item as "99," the "age spots" item as "85," and the "wrinkles" item as "80").

[0124] In this embodiment, for the "sagging" item, which is not included in the above-mentioned items, a numerical value is calculated not by the above-mentioned process, but by partially utilizing the judgment results of each of the items "pores," "texture," "fine lines," "dark circles," "age spots," and "wrinkles."

[0125] The CPU 10a also performs processing to calculate an estimated skin age of the photographed user by substituting the values ​​of each of the items "pores," "texture," "fine lines," "dark circles," "spots," "wrinkles," and "sagging" calculated as described above into another formula. The values ​​of each item and the value indicating the estimated skin age thus obtained are stored in the RAM 10c, and are read out and used when various screens are displayed on the display screen 12 (for example, the values ​​of each item are used in the analysis result screen 35 in FIG. 12 described later, etc.).

[0126] Furthermore, when displaying the detection results for each item of the skin condition of the user's face, the CPU 10a performs processing to display the judgment result of the skin condition in a form that is easy to visually recognize, using the result of obtaining the numerical value of 0-100 for each pixel described above. That is, since each pixel has a value (coordinate value) indicating its position in the image, for example, the numerical value of 0-100 of the pixel can be assigned to each position (location) of the pixel included in the user specific range 121a for pores by the value indicating the position of each pixel included in the user specific range 121a for pores in the user specific range 121 for pores.

[0127] Then, the CPU 10a, as a detection means, performs a stepwise detection process of the skin condition for the user specific ranges 121a-126i corresponding to each part by performing a discrimination based on a threshold value for the numerical value of 0-100 possessed by the pixel. In this embodiment, the skin condition management application 15 uses a first threshold value and a second threshold value in advance as threshold values ​​(first threshold value<second threshold value), and performs a process of comparing the numerical value of each pixel with the first threshold value and the second threshold value, thereby performing a stepwise detection of the part corresponding to the pixel in a total of three steps, such as a first state that is problematic, a second state that is slightly problematic, and a third state that is not problematic for each item of the skin condition.

[0128] In this embodiment, as an example, the first threshold value is set to "60" and the second threshold value is set to "30" (it is of course possible to use values ​​other than the above for the first and second threshold values). The CPU 10a compares the value of each pixel with each threshold value, and if the value of the pixel is less than the second threshold value, it determines that the pixel and the location of that pixel are in a problematic first state (bad state), if the value is equal to or greater than the second threshold value and less than the first threshold value, it determines that the pixel and the location of that pixel are in a slightly problematic second state (slightly bad state), and if the value is equal to or greater than the first threshold value, it detects and determines that the pixel and the location of that pixel are in an unproblematic third state.

[0129] The CPU 10a performs the above-mentioned three-stage discrimination detection process on all pixels included in the user-specific ranges 121a-126i for each part by item, thereby discriminating each pixel into three stages. Then, based on the discrimination results, the CPU 10a, as an image generating means, performs a coloring process on the locations where each pixel is located in the user-specific range images 101-106 by item, thereby generating a skin condition presentation image by item (the generated skin condition presentation image is stored in the RAM 10c).

[0130] In this embodiment, for pixels on the user's face F that are determined to be in a problematic first state, the pixel position (location of the pixel) is colored dark blue, for pixels that are determined to be in a slightly problematic second state, the pixel position (location of the pixel) is colored light blue, and for pixels that are determined to be in a non-problematic third state, no color is added, and the color of the user's skin captured is shown as is. Furthermore, in this embodiment, the user-specific ranges 121a-126i for each part by item that are arranged on the user's face F in the user-specific range image by item 101-106 are also shown in a color different from the colors added to the pixels in the first and second states, and the lines showing the peripheral contours of the user-specific ranges 121a-126i are shown in white.

[0131] 10(a)-(c) and 11(a)-(c) show skin condition presentation images 201-206 by item generated by the above-mentioned processing of the CPU 10a. FIG. 10(a) shows a skin condition presentation image 201 for pores, in which the user-specific ranges 121 for pores, consisting of the user-specific ranges 121a-121e corresponding to the parts, are arranged on the user's face F in the same manner as the user-specific range image 101 for pores shown in FIG. 8(a) described above. In addition, the CPU 10a shows the contours around the user-specific ranges 121a-121e arranged on each part in the skin condition presentation image 201 for pores in white (shown in black in FIG. 10(a) and the same below).

[0132] When such a skin condition presentation image for pores 201 is displayed on the display screen 12, the user can easily understand that the user-specific ranges 121a-121e shown in white on the user's face F are areas related to pores when checking the condition of the pores. Therefore, the ranges shown in white on the user-specific ranges 121a-121e located on the forehead, between the eyebrows, above the nose, and on the left and right cheeks serve as guides showing the areas the user should focus on when checking the condition of the pores.

[0133] Furthermore, within the user-specific ranges 121a-121e arranged for each body part, the positions of pixels determined to be in the first state (bad state) are colored dark blue (black in FIG. 10(a) ; same below), and the positions of pixels determined to be in the second state (slightly bad state) are colored light blue (gray in FIG. 10(a) ; same below). This allows the positions of noticeable pores and the distribution of pores that require attention to be grasped at a glance, which is useful for pore care.

[0134] Fig. 10(b) shows a skin condition presentation image for texture 202, in which the user specific range for texture 122 consisting of the user specific ranges 122a-122c corresponding to the body parts is arranged on the user's face F, similar to the user specific range image for texture 102 shown in Fig. 8(b) described above. The CPU 10a also shows the contours of the user specific ranges 122a-122c corresponding to the body parts in white.

[0135] When such a skin condition presentation image for texture 202 is displayed on the display screen 12, the user can easily understand that the user-specific ranges 122a-122c shown in white on the user's face F are areas related to the texture. Therefore, the ranges shown in white on the user-specific ranges 122a-122e arranged on the forehead and left and right cheeks can serve as guides showing the areas the user should pay attention to when checking the texture condition. In addition, in the user-specific ranges 122a-122e arranged on each part, the positions of pixels determined to be in the first state (bad state) are colored dark blue, and the positions of pixels determined to be in the second state (slightly bad state) are colored light blue. This allows the positions of poor texture and the distribution of texture that requires attention to be understood at a glance, which is useful for skin care.

[0136] Fig. 10(c) shows a skin condition presentation image for fine wrinkles 203, in which the user specific range for texture 123 consisting of each user specific range 123a-123f for fine wrinkles corresponding to the part is arranged on the user's face F, similarly to the user specific range image for fine wrinkles 103 shown in Fig. 8(c) described above. The CPU 10a also shows the contours of each user specific range 123a-123f arranged on each part in white.

[0137] When such a skin condition display image 203 for fine wrinkles is displayed on the display screen 12, the user can easily understand that the user-specific ranges 123a-123f shown in white on the user's face F are areas related to the texture. Therefore, the ranges shown in white on the user-specific ranges 123a-123e located below the left and right eyes and around the left and right temples can be used as guides to indicate the areas the user should pay attention to when checking the state of fine wrinkles. In addition, in the user-specific ranges 123a-123f of each part, the positions of pixels determined to be in the first state (bad state) are colored in dark blue, and the positions of pixels determined to be in the second state (slightly bad state) are colored in light blue. This allows the user to grasp at a glance the positions where fine wrinkles are noticeable and the distribution of fine wrinkles that require attention, which is useful for fine wrinkle care.

[0138] 11(a) shows a bear skin condition presentation image 204, in which the bear user specific range 124, consisting of the bear user specific ranges 124a, 124b corresponding to the body parts, is arranged on the user's face F, similar to the bear user specific range image 104 shown in Fig. 9(a) described above. The CPU 10a also shows the contours of the user specific ranges 124a, 124b arranged on each body part in white.

[0139] When such a skin condition presentation image 204 for dark circles is displayed on the display screen 12, the user can easily understand that the user-specific ranges 124a, 124b shown in white on the user's face F are areas related to dark circles. Therefore, the ranges shown by the user-specific ranges 124a, 124b arranged along the lower left and right eyes serve as guides showing the areas the user should focus on when checking the condition of dark circles. In addition, in the user-specific ranges 124a, 124b arranged on each part, the positions of pixels determined to be in the first state (bad state) are colored dark blue, and the positions of pixels determined to be in the second state (slightly bad state) are colored light blue. This allows the user to understand at a glance where dark circles stand out, which is useful for treating dark circles.

[0140] Fig. 11(b) shows a skin condition presentation image for blemishes 205, in which the user specific ranges for blemishes 125, consisting of the user specific ranges 125a-125e for blemishes corresponding to the body parts, are arranged on the user's face F, similarly to the user specific range image for blemishes 105 shown in Fig. 9(b) described above. The CPU 10a also shows the contours of the user specific ranges 125a-125e arranged on each body part in white.

[0141] When such a skin condition presentation image for blemishes 205 is displayed on the display screen 12, the user can easily understand that the user-specific ranges 125a-125e shown in white on the user's face F are areas related to blemishes. As a result, the ranges shown by the user-specific ranges 125a-125e arranged on the forehead, between the eyebrows, above the nose, and on the left and right cheeks can be used as guides showing the areas the user should pay attention to when checking the condition of blemishes. In addition, in the user-specific ranges 125a-125e arranged on each part, the positions of pixels determined to be in the first state (bad state) are colored dark blue, and the positions of pixels determined to be in the second state (slightly bad state) are colored light blue. This allows the user to understand at a glance the positions where blemishes are noticeable and their distribution, which is useful for blemish care.

[0142] 11(c) shows skin condition presentation image for wrinkles 206, in which user specific ranges for wrinkles 126 consisting of user specific ranges for wrinkles 126a-126i corresponding to parts are arranged on the user's face F, similar to user specific range image for wrinkles 106 shown in Fig. 9(c) described above. CPU 10a also shows the contours of user specific ranges 126a-126i arranged on each part in white.

[0143] When the skin condition presentation image for wrinkles 206 is displayed on the display screen 12, the user can easily understand that the user-specific ranges 126a-126i shown in white on the user's face F are areas related to spots. As a result, the ranges shown by the user-specific ranges 126a-126i arranged on the forehead, between the eyebrows, above the nose, on the left and right of the nose, and on the left and right of the mouth can be used as guides to indicate the areas the user should pay attention to when checking the state of wrinkles. In addition, among the user-specific ranges 126a-126i arranged on each part, the positions of pixels determined to be in the first state (bad state) are colored in dark blue, and the positions of pixels determined to be in the second state (slightly bad state) are colored in light blue. This allows the user to understand at a glance the positions where wrinkles are noticeable and the distribution of wrinkles, which is useful for wrinkle care.

[0144] As mentioned above, for the "sagging" item, no detection process (judgment process) is performed directly, and the judgment results of other items are partially used. Therefore, even in generating the skin condition presentation image for sagging, part of the skin condition presentation image for other items is used, and part of the skin condition presentation image for pores, dark circles under the eyes, etc., including the areas where the corners of the eyes, nasolabial folds (lines from the bottom left and right of the nose to the left and right of the mouth), marionette lines (lines from the left and right of the mouth diagonally downward and outward) are located, is used.

[0145] As described above, when the user's face F is photographed based on the photographing screen 30 shown in FIG. 3(b), a skin condition analysis process including the above-mentioned processing contents is performed to obtain numerical values ​​and estimated skin ages for each item, and an image presenting a skin condition for each item is generated. The CPU 10a then generates a screen displaying these contents and performs control processing to switch the display from the photographing screen 30 to the display screen 12.

[0146] 12 shows an analysis result screen 35 that is automatically displayed on the display screen 12 following the photographing screen 30 when the skin condition analysis process is completed. This analysis result screen 35 is generated by the CPU 10a when the skin condition analysis process is completed, and is configured to arrange a display switching operation unit 40 in an upper range 35a at the top of the screen, and to arrange result contents 36 in a main range 35b below the upper range 35a.

[0147] The display switching operation unit 40 has eight tappable (selectable) buttons, the first button 40a to the seventh button 40g, arranged according to the skin condition items (pores, texture, fine lines, dark circles, age spots, wrinkles, and sagging skin), and the numerical values ​​of the items obtained in the above-mentioned skin condition analysis process are arranged on the buttons. The eighth button 40 corresponds to the estimated skin age, and the numerical value indicating the estimated skin age of the user obtained in the above-mentioned skin condition analysis process is arranged on the buttons. In addition, in the analysis result screen 35 shown in FIG. 12, the eighth button 40h is in a selected state (the outline of the selected button 40h is shown with a thicker outline than the outlines of the other buttons in the figure. The same applies to the other buttons), and the remaining first button 40a to the seventh button 40g are in a state where they can be tapped (selected).

[0148] Moreover, the result content 36 arranged in the main area 35b includes an estimated skin age display section 36a at the top and a radar chart 36b below it. The estimated skin age display section 36a displays the user's estimated skin age obtained in the skin condition analysis process in a large font size, allowing the user to understand their own estimated skin age at a glance.

[0149] The radar chart 36b shows the values ​​of a total of seven items (pores, texture, fine lines, dark circles, age spots, wrinkles, and sagging) obtained by the above-mentioned skin condition analysis process based on the base line of a regular heptagon, and this radar chart 36b allows the user to visually understand which items are good and which items are not good. If the smartphone 10 has received the past analysis results of the user (the analysis results that are the average of the past 30 days) at the time of completing the login, the CPU 10a generates the radar chart 36b including the past analysis results. This allows the user to easily compare the current analysis results with the past analysis results.

[0150] Fig. 13(a) shows a pore image screen 41 displayed on the display screen 12. For example, when the first button 40a in the display switching operation unit 40 in the analysis result screen 35 in Fig. 12 is tapped (selected) by the user, the pore image screen 41 is switched from the analysis result screen 35, and is displayed on the display screen 12 by the CPU 10a executing a process related to the display of the pore image screen 41 including the skin condition presentation image for pores 201.

[0151] The pore image screen 41 is generated by the processing of the CPU 10a when the smartphone 10 (CPU 10a) accepts a tap operation (selection operation) of the first button 40a. In this generation process, the CPU 10a arranges the display switching operation unit 40 common to each screen in the upper range 41a at the top of the screen, and reads and arranges the skin condition presentation image 201 for pores from the skin condition presentation images 201-206 for each item, etc., generated by the above-mentioned process and stored in the RAM 10c, in the main range 41b below the upper range 41a.

[0152] Fig. 13(b) shows a texture image screen 42 displayed on the display screen 12. This texture image screen 42 is switched and displayed, for example, when the user taps the second button 40b in the display switching operation unit 40 on the analysis result screen 35 in Fig. 12 or the pore image screen 41 in Fig. 13(a). The texture image screen 42 is also generated by the CPU 10a, like the pore image screen 41, and the common display switching operation unit 40 is arranged in the upper area 42a, and the texture skin condition presenting image 202 read out from the RAM 10c is arranged in the main area 42b.

[0153] 14(a) shows fine wrinkle image screen 43 displayed on display screen 12. This fine wrinkle image screen 43 is also switched and displayed by, for example, the user tapping third button 40c in display switching operation unit 40 on each of screens 35, 41, 42, etc. Fine wrinkle image screen 43 is also generated by CPU 10a, like each of the above image screens 41 and 42, with a common display switching operation unit 40 disposed in upper range 43a and fine wrinkle skin condition presenting image 203 read out from RAM 10c disposed in main range 43b.

[0154] 14(b) shows a bear image screen 44 displayed on the display screen 12. This bear image screen 44 is also switched and displayed by, for example, tapping the fourth button 40d in the display switching operation unit 40 on each of the screens 35, 41-43, etc. The bear image screen 44, like each of the other image screens 41-43, is generated by the CPU 10a, with the common display switching operation unit 40 disposed in the upper area 44a and the bear skin condition presenting image 204 read out from the RAM 10c disposed in the main area 44b.

[0155] 15(a) shows a blemish image screen 45 displayed on the display screen 12. This blemish image screen 45 is also switched and displayed by tapping the fifth button 40e in the display switching operation unit 40 on each of the screens 35, 41-44, etc. The blemish image screen 45 is also generated by the CPU 10a, like each of the other image screens 41-44, etc., and a common display switching operation unit 40 is arranged in the upper area 45a, and a blemish skin condition presenting image 205 read out from the RAM 10c is arranged in the main area 45b.

[0156] 15(b) shows wrinkle image screen 46 displayed on display screen 12. This wrinkle image screen 46 is also switched and displayed by tapping sixth button 40f in display switching operation unit 40 on each of screens 35, 41-45, etc. Wrinkle image screen 46 is also generated by CPU 10a, like each of the other image screens 41-45, etc., and a common display switching operation unit 40 is arranged in upper area 46a, and wrinkle skin condition presenting image 206 read out from RAM 10c is arranged in main area 46b. Note that an illustration of the sagging image screen is omitted.

[0157] As shown in Figures 13-15, by selecting each button 40a-40g of the display switching operation unit 40, the image screens 41-46 etc. of each item displayed on the display screen 12 can be switched as appropriate, thereby allowing the user to smoothly check the detection results (analysis results) of the skin condition of the item that he or she wishes to check.

[0158] In addition, the skin condition presenting images 201-206 etc. included in the image screens 41-46 etc. of each displayed item visually show the results of analysis in the range of each part modified to match the configuration of the user's face F, so that the actual skin condition according to the user's individual face shape, face size etc. can be better presented to the user by narrowing it down to the range related to the item. Moreover, the analysis results show the photographed skin as it is for areas with no particular problems, areas with slight problems are shown in light blue, and areas with problems are shown in dark blue, so that the user can visually recognize the location and extent of the problem at once, making it easier to carry out subsequent care etc.

[0159] In addition, the image screens 41-46 etc. of each item displayed on the display screen 12 can be enlarged by performing a pinch out operation on the display screen 12 having a touch panel function, and thereby the details of each part of the skin condition presenting images 201-206 etc. included in the screens 41-46 etc. of each item can be appropriately confirmed. Furthermore, for the enlarged display, the CPU 10a performs processing related to a one-touch (one-tap) enlargement function, in which the user-specific ranges 126a-126i arranged in each part included in the skin condition presenting images 201-206 etc. are enlarged so that the user-specific ranges 126a-126i are displayed in a size corresponding to the vertical or horizontal dimension of the display screen 12 by touching (tapping) inside the user-specific ranges 126a-126i arranged in each part included in the skin condition presenting images 201-206 etc.

[0160] Fig. 16(a) is a left cheek enlarged screen 50 for pores showing an example of an enlarged display by the one-touch enlargement function described above. This left cheek enlarged screen 50 for pores is switched to be displayed from the pore image screen 41 of Fig. 13(a) when a portion surrounded by a contour line of the left cheek user-specific range 121d for pores located on the left cheek of the user's face F is touched (tapped) on the pore image screen 41 displayed on the display screen 12 of Fig. 13(a).

[0161] In the process of the CPU 10a relating to the enlarged display such as the enlarged screen shown in Fig. 16(a) described above, when the skin condition presenting images 201-206 of each item shown in Fig. 13-15 are displayed on the display screen 12, the CPU 10a detects whether the user-specific ranges 126a-126i of each part included in these skin condition presenting images 201-206 are touched (tapped, selected) or not. When the CPU 10a detects the receipt of a tap operation, the CPU 10a determines whether the smartphone 10 is held vertically (a posture in which the longitudinal direction of the smartphone 10 is aligned vertically) or horizontally (a posture in which the longitudinal direction of the smartphone 10 is aligned horizontally) based on the detection value from the acceleration sensor 10g.

[0162] Then, the CPU 10a enlarges the skin condition presenting image (for example, the skin condition presenting image 201 for pores shown in FIG. 13(a)) included in the display screen until either the vertical or horizontal dimension of the user specific range that was the target of the touch (tap) (for example, the user specific range 121d for pores on the left cheek shown in FIG. 13(a)) becomes compatible (matches) with either the vertical or horizontal dimension of the display screen 12 in the determined attitude of the smartphone 10. At this time, the common display switching operation unit 40 included in the displayed skin condition presenting images 201-206, etc. is hidden (removed) from the display screen 12 so that the enlarged area is displayed full-screen on the display screen 12.

[0163] In the example shown in the left cheek enlarged screen 50 for pores in FIG. 16(a), the smartphone 10 is held vertically, and therefore the rectangular display screen 12 is also displayed vertically long in FIG. 16(a). In this case, the CPU 10a enlarges the original broken state presentation image 201 for pores shown in FIG. 13(a) with the touched (tapped) left cheek user-specific range 121d for pores as the center. When enlarged, the horizontal dimension (horizontal dimension in the figure) of the left cheek user-specific range 121d for pores first matches the short side dimension 12a, which is the horizontal direction of the display screen 12, so the CPU 10a stops the enlargement at the matched state, and sets this stopped state to the left cheek enlarged screen 5 for pores in FIG. 16(a).

[0164] Therefore, in the left cheek enlarged screen 50 for pores shown in FIG. 16(a), the left cheek user-specific range 121d for pores is enlarged and displayed with one touch (one tap) to match the screen size of the display screen 12, which is useful when you want to check the inside of the left cheek user-specific range 121d for pores in detail.

[0165] In addition, the left cheek enlarged screen 50 for pores can be further enlarged by pinching out on the display screen 12, while pinching in on the display screen 12 reduces the display (similar to enlargement of other items). In addition, by swiping from one end 12c of the longitudinal direction of the display screen 12 displaying the left cheek enlarged drawing 50 for pores toward the center of the screen, the hidden display switching operation unit 40 appears on the screen, and by selecting any one of the buttons 40a-40f of the display switching operation unit 40, the display can be switched to each image screen 42-46, etc., shown in FIG. 13-15 (not enlarged state).

[0166] Also, Fig. 16(b) is a pores forehead enlargement screen 51 showing another example by the one-touch enlargement function described above. When the part surrounded by the outline of the pores forehead user specific range 121a located on the forehead of the user's face F is touched (tapped) on the pores image screen 41 displayed on the display screen 12 in Fig. 13(a), the display is switched to the pores for cheeks enlargement screen 51 in Fig. 16(b).

[0167] However, in the example of Fig. 16(b), since the smartphone 10 is held horizontally after tapping, the horizontal dimension (horizontal dimension in the figure) of the pores forehead user-specific range 121a is enlarged to match the long side dimension 12b (horizontal dimension in the figure) of the display screen. In the pores forehead enlarged cheek image 51 shown in Fig. 16(b), the pores forehead user-specific range 121a is enlarged with one touch (one tap) to match the display screen 12, so the details of the forehead skin condition can be easily confirmed.

[0168] The flowchart shown in Fig. 17 summarizes an example of main processing steps performed by the CPU 10a of the smartphone 10 described above as a skin condition presenting device (corresponding to the processing content of the skin condition presenting method). Note that this flowchart shows processing from a state in which the icon of the skin condition management app 15 is selected on the home screen displayed on the display screen 12, the skin condition management app 15 is started, login processing is completed (with the completion of login, the service management server 5 transmits the average analysis result of the logged-in user for the past 30 days to the smartphone 10), and the photographing screen 30 shown in Fig. 3(a) is displayed on the display screen 12.

[0169] First, when photographing the user's face, the CPU 10a judges whether or not all three photographing conditions, namely brightness, position, and direction, are satisfied based on the photographing data from the sub-camera 14a (S1). If all the photographing conditions are not satisfied (S1: NO), the system waits for all the conditions to be satisfied. On the other hand, if all the conditions are satisfied (S1: YES), the CPU 10a performs photographing processing based on the self-timer function to obtain a photographed image including the user's face photographed by the image sensor of the sub-camera 14a (S2).

[0170] Next, the CPU 10a reads out the item-specific basic ranges 21-26 from the basic range table 20 and arranges them on the user's face included in the acquired photographed image to generate an item-specific analysis image (S3). At this time, the CPU 10a generates a number of copies of the acquired photographed image according to the items, and arranges each of the item-specific basic ranges 21-26 for each of the generated photographed images, thereby generating a plurality of item-specific analysis images according to the number of the basic ranges 21-26.

[0171] Then, the CPU 10a transforms the basic range included in the generated multiple item-specific analysis images according to the user's facial structure (S4). Note that, by this transformation, the item-specific user-specific range images 101-106 are generated, and the range shown in the user-specific range image (the range obtained by transforming the basic range, the transformed basic range) is specified as the detection range as the user-specific range.

[0172] Then, the CPU 10a detects the skin condition by stage for the detection range (user specific range) included in the user specific range image by item (S5). From this detection result, the CPU 10a calculates the value for each item and also calculates the estimated skin age of the photographed user, and stores the calculated values ​​(detection results) in the RAM 10c.

[0173] Furthermore, the CPU 10a performs a coloring process on the arrangement positions of the pixels in the user-specific range images 101-106 for each item based on the detection results of the step S5, and generates skin condition presentation images 201-206 for each item, etc. (S6). The generated skin condition presentation images 201-206, etc. are also stored in the RAM 10c.

[0174] Then, in response to the numerical values ​​of the detection results stored in the RAM 10c and the completion of login, the CPU 10a generates an analysis result screen 35 (see FIG. 12) including a radar chart or the like using the average analysis results of the past 30 days acquired from the service management server 5, and displays it on the display screen 12 (S7). Then, the CPU 10a determines whether or not a selection operation of each of the buttons 40a-40g of the display switching operation unit 40 included in the displayed analysis result screen 35 has been accepted (S8).

[0175] When a selection operation of any of the buttons 40a-40g is received (S8: YES), the CPU 10a reads out a skin condition presentation image for the item corresponding to the button for which the selection operation was received, generates an image screen for that item (for example, each image screen 41-46 shown in Figures 13-15) and displays it on the display screen 12 (S9).

[0176] After this step S9, or when the selection operation of any of the buttons 40a-40g is not accepted (S8: NO), the CPU 10a judges whether or not an operation to end the skin condition management application 15 has been accepted (S10), and if an operation to end the application has not been accepted (S10: NO), the process returns to step S8. Therefore, once an image screen of some item is displayed at step S9, if there is no selection operation of a button of another item thereafter (S8: NO) and no operation to end the application (S10: NO), the steps S8 and S10 are looped, and the display of the image screen displayed at step S9 is continued. Also, if there is a selection operation of a button of another item during the display (S8: YES), the display is switched to an image screen of an item corresponding to the selected button (S9).

[0177] Then, when an operation to end the skin condition management application 15 is received (S10: YES), the CPU 10a ends the activation of the skin condition management application 15 (S11). Note that the CPU 10a transmits (uploads) the current analysis result to the service management server 5 during the above-mentioned process, and uploads the analysis result, for example, at the timing when the analysis result is calculated or when the activation of the skin condition management application 15 is ended.

[0178] Finally, the service management server 5, which is included in the cloud system 2 in the skin condition presentation system 1 shown in FIG. 1 and manages the skin condition presentation service according to the present invention, will be described.

[0179] 18 shows the main internal configuration of the service management server 5 constituting the skin condition presentation system 1. The service management server 5 is composed of a general server computer, but it is also possible to configure the service management server 5 by combining multiple server computers or the like by performing distributed processing or the like for the various processes performed by the service management server 5, and such a configuration using multiple devices also corresponds to the service management server 5 in the present invention.

[0180] In this embodiment, the service management server 5 has an MPU 5a (control unit 5a) that performs overall control and various processing, and various devices etc. that are connected to the MPU 5a via internal connection lines 5h. The various devices etc. include a communication module 5b, a RAM 5c, a ROM 5d, an input interface 5e, an output interface 5f, and a storage system (HDD system) 5g.

[0181] The communication module 5b is a communication device (communication means) equivalent to a connection module with a network, and conforms to a required communication standard (for example, a LAN module based on the IEEE communication standard). The communication module 5b is connected to a network (such as the internal network of the cloud system 2 and an external network NW) via a required communication device (not shown; for example, a router), and enables communication with the smartphone 10 used by the user and other communication terminal devices.

[0182] The RAM 5c temporarily stores contents and files associated with the processing of the MPU 5a, and the ROM 5d stores programs and the like that define the basic processing contents of the MPU 5a. The input interface 5e is connected to a keyboard 5i, a mouse, and the like that accept operation instructions from a system administrator of the skin condition display system 1, and transmits the operation instructions accepted from the system administrator to the MPU 5a. The output interface 5f is connected to a display 5j, and outputs contents associated with the processing of the MPU 5a to the display 5j, allowing the system administrator to visually check the current processing contents, etc.

[0183] The storage system 5g stores programs, tables, etc., and in this embodiment, stores a server system program P10, a service management program P11, a registered user table 6, etc. The server system program P10 prescribes various processes according to the operating system (OS) for the server, and the service management server 5 performs the basic functions as a server computer by having the MPU 5a execute processes based on the contents of these prescribes.

[0184] The service management program P11 prescribes the user registration process, login process, user analysis result management process, etc., performed by the MPU 5a in implementing the service according to the present invention. In the user registration process, when the service management server 5 receives user information (user name, phone number, email address, password, etc.) required for user registration from a terminal used by the user (e.g., the smartphone 10), the MPU 5a records the received user information in the registered user table 6.

[0185] In the login process, when the service management server 5 receives login information (email address, password, etc.) required for login from a terminal used by a user (for example, the smartphone 10), the MPU 5a judges whether or not the received login information is recorded in the registered user table 6. If the MPU 5a judges that the received login information is not recorded in the registered user table 6, it transmits a notification that login is not possible to the user terminal that sent the login information.

[0186] Furthermore, if the MPU 5a determines that the received login information is recorded in the registered user table 6, it transmits a notification that login is possible (login completed) to the user terminal that transmitted the login information. At this time, if past analysis results (average analysis results for the past 30 days) for the user who is the target of the login process are recorded in the registered user table 6, the MPU 5a reads out the past analysis results and transmits them to the user terminal together with a notification of login completion.

[0187] In the user analysis result management process, when an analysis result is sent (uploaded) from the terminal of a logged-in user, the MPU 5a associates the sent analysis result with the logged-in user and records it in the registered user table 6. In addition, when an analysis result is stored in the registered user table 6, the MPU 5a also performs a process of calculating an average value for the most recent 30 days at any time and recording the calculated result in the registered user table 6 as the average analysis result for the past 30 days.

[0188] 19 shows an overview of the contents of the above-mentioned registered user table 6. The registered user table 6 is a table that stores various information (user information) of registered users who use the services provided by the skin condition presentation system 1. In conjunction with user registration, a user ID for identifying the user is assigned (issued) to the user by the service management server 5, and the registered user table 6 stores various information in association with the assigned user ID.

[0189] Specifically, as shown in Fig. 19, the registered user table 6 stores the user's name, phone number, email address, etc. for each user ID, as well as the uploaded analysis results and the average analysis results for the past 30 days, etc. The user information such as the name stored in the registered user table 6 in this manner is provided by the user at the time of user registration, and the contents of the registered user table 6 are appropriately updated by the service operator in response to the registration of a new user or the withdrawal of a registered user. Note that in this embodiment, the registered email address is used as the login ID for logging in, and the password uses alphanumeric information registered by the user at the time of user registration.

[0190] As described above, in the present invention, the range for each item to be detected for skin condition is transformed (adjusted) according to the user's facial structure (shape, size, etc.), and within that transformed range, the gradual detection results are displayed using shades of color. This eliminates the need to perform skin condition detection processing on the entire face unnecessarily, allowing for efficient detection processing, and also allows the detection results to be presented in accordance with the user's facial structure, helping to better grasp the user's skin condition.

[0191] It should be noted that the present invention is not limited to the above description, and various modified examples are conceivable. For example, in the above description, a login process is performed before starting the process according to the present invention, but it is also possible to omit such a login process and execute the process according to the present invention without the login process. When the login process is omitted in this way, it is also possible to complete the process according to the present invention only by a terminal device used by a user, such as the smartphone 10 (standalone). When the login process is omitted as described above, it is also assumed that the login process is performed when uploading the analysis result to the service management server 5 or when downloading the past analysis result from the service management server 5.

[0192] In the above description, the items related to the detection of skin condition are "pores", "texture", "fine lines", "dark circles", "spots", "wrinkles", and "sagging". However, the present invention is not limited to these items, and the number of these items can be reduced or other items can be added as appropriate. For example, if the analysis (detection) of two items such as "sagging" and "texture" is not necessary, these two items may be omitted. In addition, items that can be added include "squeezing", "moisture content", "keratin plug amount", and "dryness", and these items may also be included in the detection targets.

[0193] Furthermore, in the above description, the skin condition presenting image is colored using blue, with dark blue being applied to the pixel position of the first state (worst state) and light blue being applied to the pixel position of the second state (slightly worse than normal state), but the color used is not limited to blue, and other colors such as red, orange, yellow, green, purple, etc. can also be used (however, it is safer to avoid colors that are close to skin color in color because they are difficult to distinguish from skin). In addition, the detection stages may be distinguished by using different colors for each stage, other than using different shades of color. For example, it is envisioned to use blue for the first state and a different red color from blue for the second state.

[0194] Furthermore, in the above description, the detection of skin condition is classified into three states: state 1 (poor condition with a problem), state 2 (poor condition with some problem), and state 3 (no problem state); however, the number of states to be classified is not limited to three, and a minimum of two states (a problematic state and a no problem state) can be applied, and it is also possible to classify into more than three states.

[0195] In the above description, the lines showing the contours of the user-specific ranges 121a-126i included in each item-specific skin condition presentation image are white, but it is of course possible to use a color other than white (for example, black) as long as the color is different from the skin condition detection result. Furthermore, the lines showing the contours of the user-specific ranges 121a-126i may be omitted from each item-specific skin condition presentation image. In this case, the user-specific ranges 121a-126i are not shown, so there are no lines on the user's face F, resulting in a simple representation, which is a suitable representation method for users who do not like complicated representations.

[0196] Furthermore, when each of the above-mentioned itemized skin condition presentation images is displayed on the display screen 12, advice regarding the skin condition may also be added, and further, content such as a guide to products (recommended products) that will help improve the user's skin condition may be displayed.

[0197] In addition, the present invention can be envisioned in a form in which the service management server 5 opens a website on the network that provides the present service, and the service management server 5 performs the process (skin condition presentation method process) shown in the flowchart of FIG. 17, so that the results of the process of the service management server 5 can be viewed via the website on the smartphone 10 used by the user. In this case, the service management program P11 installed in the service management server 5 includes programming content that defines the process shown in the flowchart of FIG. 17, and causes the MPU 5a to execute the process shown in the flowchart of FIG. 17 as various means, so that the service management server 5 corresponds to the skin condition presentation device, and the smartphone 10 used by the user remains a viewing terminal. When the service according to the present invention is provided to the user via the website in this way, the processing load on the smartphone 10 used by the user can be reduced, and the service management server 5 has the advantage of being able to perform the process centrally. [Industrial Applicability]

[0198] The present invention detects an area that has been deformed in accordance with the configuration of the user's face, and shows the gradual detection results of the skin condition in such a deformed area using color, and can therefore be suitably used to visually represent the skin condition in accordance with the shape, size, etc. of the user's face. [Explanation of symbols]

[0199] 1. Skin condition display system 5 Service Management Server 5a MPU 10 Smartphone (skin condition display device) 10a CPU 12 Display Screen 15 Skin condition management app 20 Basic Range Table 101-106 User-specific range images by item 121a-126i User-specific ranges for each part of the body 201-206 Images showing skin condition by category

Claims

1. A skin condition display device detects a skin condition in stages based on a captured image including a user's face, and applies a color corresponding to the stage of the detection result to the user's face included in the captured image and performs a display process, a placement means for placing a basic range indicating a range corresponding to a part of a face to be detected on a face of a user included in a photographed image; an area transformation means for transforming a basic area arranged on a user's face into a shape corresponding to the user's face; a range specifying means for specifying a range transformed from a basic range by the range transforming means as a user specific range; a detection means for detecting a skin condition in stages for the user specific range identified by the range identification means; an image generating means for generating a skin condition display image by applying a color corresponding to a stage of a detection result to a portion detected by the detecting means within the user specific range; A display processing means for performing processing related to displaying the skin condition presentation image generated by the image generating means; A skin condition display device comprising:

2. The basic ranges include basic ranges for different skin conditions, The placement means places each of the basic ranges for each item on a face of a user, The range modification means modifies each of the basic ranges for each item, the range specification means specifies a range modified from each of the basic ranges for each item as a user-specific range for each item, The detection means performs detection for each of the user-specific ranges for each item, The image generating means generates an image presenting a skin condition by each item, The skin condition display device according to claim 1 , wherein the display processing means performs processing related to displaying a skin condition display image for each item.

3. the image generating means generates a skin condition presenting image in which a contour line indicating a periphery of the user specific area is displayed in a color different from a color corresponding to a stage related to the detection result, 3 . The skin condition display device according to claim 1 , wherein the display processing means performs processing related to display of a skin condition display image in which the contour line is indicated.

4. The skin condition display image is displayed on a display screen, the user-specific range shown in the skin condition presentation image displayed on the display screen is capable of accepting a selection operation; The skin condition display device according to claim 3, further comprising a means for enlarging and displaying the user-specific range for which a selection operation of the user-specific range is accepted so that the user-specific range for which the selection operation is accepted is displayed in accordance with the vertical or horizontal dimensions of the display screen.

5. the basic area has a plurality of subareas corresponding to each part of the face, the range modification means modifies a plurality of partial ranges of the basic range, The skin condition presentation device of claim 1 or claim 2, wherein when the deformation by the range deformation means results in overlapping between ranges in the range transformed from the partial range, the range identification means identifies the continuous outermost contour ranges of the overlapping ranges as a unified user-specific range.

6. A skin condition display method in which a skin condition display device detects a skin condition by stage based on a captured image including a user's face, and applies a color corresponding to the stage of the detection result to the user's face included in the captured image and performs a display process, The skin condition display device comprises: A step of arranging a basic range indicating a range corresponding to a part of a face to be detected on a face of a user included in a photographed image; A step of transforming the basic area arranged on the user's face into a shape corresponding to the user's face; identifying a range that is modified from the basic range as a user-specific range; detecting a skin condition according to a stage for the user specific range; generating a skin condition presentation image by applying a color corresponding to a stage of the detection result to the detected portion within the user specific range; A step of performing a process related to displaying the generated skin condition presentation image. A skin condition display method comprising:

7. A computer program for causing a computer to execute a process of detecting a skin condition by stages based on a captured image including a user's face, and applying a color corresponding to the stage of the detection result to the user's face included in the captured image and displaying the color, The computer includes: A step of arranging a basic range indicating a range corresponding to a part of a face to be detected on a face of a user included in a photographed image; A step of transforming the basic area arranged on the user's face into a shape corresponding to the user's face; identifying a range that is modified from the basic range as a user-specific range; detecting a skin condition according to a stage for the user specific range; generating a skin condition presentation image by applying a color corresponding to a stage of the detection result to the detected portion within the user specific range; A step of performing a process related to displaying the generated skin condition presentation image. A computer program characterized by:

Images