Program, server apparatus, and client apparatus

Abstract

To estimate potential for improving sagging.SOLUTION: Acquiring (i) first face data that is an image or three dimensional data of a face of a target person when an angle formed by an axial direction of the face of the target person and a vertical direction is a first angle, and (ii) second face data that is an image or three dimensional data of the face of the target person when an angle formed by the axial direction of the face of the target person and the vertical direction is a second angle; And a step of specifying a region in which a difference in a second face shape that is a shape of the face of the target person indicated by the second face data is larger than a predetermined first degree and / or a region in which the difference is smaller than a predetermined second degree.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a program, a server device, and a client device. [Background technology]

[0002] BACKGROUND ART Skin care using cosmetics, beauty appliances, beauty methods, etc. that improve sagging that occurs on the face (for example, sagging under the eyes, nasolabial folds, marionette lines, etc.) are known. Summary of the Invention [Problem to be solved by the invention]

[0003] For example, if a person who practices skin care knows whether or not their own sagging skin will improve (i.e., the potential to improve sagging skin), it is thought that their motivation for skin care will increase. Also, if beauty consultants and others know the potential to improve sagging skin for their customers, they will be able to recommend appropriate cosmetics and other products.

[0004] Therefore, an object of the present invention is to estimate the potential for improving sagging. [Means for solving the problem]

[0005] A program according to one embodiment of the present invention causes a computer to execute the steps of: (i) acquiring first face data, which is an image or three-dimensional data of the subject's face when the angle between the axis direction of the subject's face and the vertical direction is a first angle; and (ii) acquiring second face data, which is an image or three-dimensional data of the subject's face when the angle between the axis direction of the subject's face and the vertical direction is a second angle; and identifying areas where the difference between the first face shape, which is the shape of the subject's face represented by the first face data, and the second face shape, which is the shape of the subject's face represented by the second face data, is greater than a predetermined first degree, and / or areas where the difference is smaller than a predetermined second degree. [Effects of the Invention]

[0006] According to the present invention, it is possible to estimate the potential for reducing sagging. [Brief explanation of the drawings]

[0007] [Figure 1] FIG. 1 is a diagram for explaining an overview of the present invention. [Figure 2] 1 is an example of an overall configuration according to an embodiment of the present invention. [Figure 3] FIG. 2 is a functional configuration diagram of a server device and a client device according to an embodiment of the present invention. [Figure 4] 10 is a flowchart of a process for estimating a potential for reducing slack according to an embodiment of the present invention. [Figure 5] 10 is an example of an image displayed on a client device according to an embodiment of the present invention. [Figure 6] 10 is an example of an image displayed on a client device according to an embodiment of the present invention. [Figure 7] 10 is an example of an image displayed on a client device according to an embodiment of the present invention. [Figure 8] 10A and 10B are diagrams for comparing sagging in a sitting position and a supine position in various lighting environments according to an embodiment of the present invention. [Figure 9] 10A and 10B are diagrams for comparing sagging in a sitting position and a supine position in various lighting environments according to an embodiment of the present invention. [Figure 10] FIG. 2 is a hardware configuration diagram of a server device and a client device according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0008] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0009] <Summary> Fig. 1 is a diagram for explaining an overview of the present invention. Fig. 1a to 1d show the sagging area of ​​a person's face when the person is sitting (hereinafter also referred to as the seated position) and the sagging area of ​​a person's face when the person is lying on their back (hereinafter also referred to as the supine position). Note that although the present specification describes the case of the seated position, the same applies to the case of the standing position.

[0010] [Age-related sagging] When a person is in a sitting position, the skin and fat of the face sag due to gravity, causing sagging. When a person is in a supine position, the skin and fat of the face are pulled toward the back of the head, eliminating or reducing sagging. In other words, if there is a large difference between the shape of the face when the person is in a sitting position and the shape of the face when the person is in a supine position, it is estimated that this is sagging due to aging. It is believed that there is a high potential for improvement of sagging due to aging.

[0011] Areas such as a that are sagging when sitting but not when lying on their backs are presumed to be areas of sagging due to aging.

[0012] Areas such as b, which show a high degree of sagging in the sitting position but a low degree of sagging in the supine position, are presumed to be areas of sagging due to aging.

[0013] [Facial Features] When a person is in a sitting position, sagging skin (that is, at first glance it appears to be sagging skin due to aging) may actually be due to the person's facial features. In other words, if there is no or only a small difference between the shape of the face when the person is in a sitting position and the shape of the face when the person is in a supine position, it is presumed to be due to the facial features. Facial features are considered to have low potential for improvement.

[0014] Areas that are sagging in both the sitting and supine positions, such as c, are presumed to be areas of the face that appear sagging.

[0015] Areas that do not show sagging in either the sitting or supine position, such as d, are presumed to be areas of the face that do not appear to have sagging.

[0016] <Overall configuration example> FIG. 2 shows an example of the overall configuration according to one embodiment of the present invention.

[0017] [Configuration example 1] The information processing system 1 includes a server device 10 and a client device 20. Each of these will be described below.

[0018] <<Server device>> The server device 10 is a device that estimates the potential for improving sagging skin of the subject 30. The server device 10 is composed of one or more computers.

[0019] Specifically, the server device 10 uses an image or three-dimensional data (hereinafter also referred to as face data) of the face of the subject 30 to identify areas of sagging due to aging on the face of the subject 30 (i.e., areas with a high potential for improvement in sagging) and areas of facial features (i.e., areas with a low potential for improvement in sagging). For example, the server device 10 receives the face data of the subject 30 from the client device 20, and transmits an image indicating the areas of sagging due to aging on the face of the subject 30 and the areas of facial features to the client device 20.

[0020] <<Client device>> The client device 20 transmits facial data of the subject 30 to the server device 10, and receives and displays an image showing areas of sagging skin due to aging and areas of facial features on the face of the subject 30 from the server device 10. The client device 20 may execute part of the processing of the server device 10 described in this specification. For example, the client device 20 is a personal computer, a tablet terminal, a smartphone, or the like. The client device 20 may include an imaging means.

[0021] [Configuration example 2] In configuration example 2, the client device 20, rather than the server device 10, estimates the potential for improving sagging of the subject 30. Specifically, the client device 20 uses facial data of the subject 30 to identify and display areas of aging sagging on the subject's face (i.e., areas with high potential for improving sagging) and areas of facial features (i.e., areas with low potential for improving sagging). The client device 20 is a personal computer, a tablet terminal, a smartphone, or the like. The client device 20 may include an imaging means.

[0022] <Functional configuration> FIG. 3 is a functional configuration diagram of the server device 10 and the client device 20 according to an embodiment of the present invention.

[0023] The server device 10 can include a face data acquisition unit 101, an estimation unit 102, and an image generation unit 103. The server device 10 can function as the face data acquisition unit 101, the estimation unit 102, and the image generation unit 103 by executing a program.

[0024] The client device 20 can include a reception unit 201, an image acquisition unit 202, and a display unit 203. The client device 20 can function as the reception unit 201, the image acquisition unit 202, and the display unit 203 by executing a program.

[0025] Note that Fig. 3 shows the case of [Configuration Example 1] in Fig. 2. In the case of [Configuration Example 2] in Fig. 2, the client device 20 includes a reception unit 201, an image acquisition unit 202, a display unit 203, a face data acquisition unit 101, an estimation unit 102, and an image generation unit 103.

[0026] The face data acquisition unit 101 acquires face data (i.e., a face image or three-dimensional data (point cloud data)) of the subject 30. The face data acquisition unit 101 may acquire face data measured by the client device 20, or may acquire face data that has already been measured and stored in the server device 10 or the client device 20.

[0027] Specifically, the face data acquisition unit 101 acquires first face data, which is an image or three-dimensional data of the face of the subject 30 when the angle between the axial direction of the face of the subject 30 and the vertical direction is a first angle, and second face data, which is an image or three-dimensional data of the face of the subject 30 when the angle between the axial direction of the face of the subject 30 and the vertical direction is a second angle.

[0028] [Facial axis] Here, the axis of the face will be described. The axis of the face may be an axis that passes through any point on the face, and may be, for example, any of three mutually perpendicular axes. For example, if the eyes are at the top, the mouth is at the bottom, the face is at the front, and the back of the head is at the back, the axis may be an axis in the up-down direction, an axis in the left-right direction, or an axis in the front-to-back direction. Note that the axis direction is not limited to this, and may be any direction (for example, an axis that goes from diagonally upper right to diagonally lower left of the face).

[0029] The estimation unit 102 uses the facial data acquired by the facial data acquisition unit 101 to estimate the potential for improving sagging (i.e., identify areas of facial sagging due to aging (i.e., areas with high potential for improving sagging) and areas of facial features (i.e., areas with low potential for improving sagging)).

[0030] Specifically, the estimation unit 102 identifies areas where the difference between the first facial shape, which is the facial shape of the subject 30 indicated by the first facial data, and the second facial shape, which is the facial shape of the subject 30 indicated by the second facial data, is greater than a predetermined first degree, and / or areas where the difference is smaller than a predetermined second degree.

[0031] [1st and 2nd degree] Here, the first degree and the second degree will be explained. The first degree and the second degree may be the same or different.

[0032] If the first degree and the second degree are the same, the estimation unit 102 identifies two areas: an area where the difference between the first face shape and the second face shape is greater than the first degree and the second degree (i.e., an area of ​​sagging due to aging), and an area where the difference is smaller than the first degree and the second degree (i.e., an area of ​​facial features).

[0033] If the first degree and the second degree are different, the estimation unit 102 identifies an area where the difference between the first facial shape and the second facial shape is greater than the first degree (i.e., an area of ​​sagging due to aging), an area where the difference is less than the first degree and greater than the second degree (i.e., an area of ​​sagging due to aging), and an area where the difference is less than the second degree (i.e., an area of ​​facial features).

[0034] The image generating unit 103 generates an image showing the potential for improving sagging. Specifically, the image generating unit 103 generates an image showing the areas of sagging caused by aging on the face of the subject 30 and the areas of facial features (for example, an image in which the areas identified by the estimation unit 102 for each facial part are highlighted).

[0035] The receiving unit 201 receives input from a user who operates the client device 20 .

[0036] The image acquisition unit 202 acquires the image generated by the image generation unit 103 from the server device 10 .

[0037] The display unit 203 displays the image acquired by the image acquisition unit 202 in accordance with the input received by the reception unit 201 .

[0038] <Example> Examples will be described below.

[0039] (1) The imaging means of the client device 20 photographs the face of the subject 30 from the front when the midline of the subject's face is parallel to the vertical direction (for example, when the subject is sitting or standing), and photographs the face of the subject 30 from the front when the midline of the subject's face is perpendicular to the vertical direction (for example, when the subject is lying on their back). (2) The face data acquisition unit 101 acquires an image of the face of the subject 30 photographed in (1) when the midline of the face is parallel to the vertical direction (referred to as the first face image), and an image of the face of the subject 30 when the midline of the face is perpendicular to the vertical direction (referred to as the second face image). (3) The estimation unit 102 identifies areas of sagging in the first facial image and areas of sagging in the second facial image. For example, the estimation unit 102 identifies areas of sagging based on areas showing shadows in the first facial image and the second facial image by image analysis that detects wrinkles and the like that occur on the skin surface due to sagging. The estimation unit 102 then generates an image showing areas with high potential for improving sagging (i.e., areas where the difference between the sagging area in the first facial image and the sagging area in the second facial image is greater than a predetermined first degree (areas of sagging due to aging)), areas with medium potential for improving sagging (i.e., areas where the difference between the sagging area in the first facial image and the sagging area in the second facial image is less than the first degree and greater than a second degree (areas of sagging due to aging)), and areas with low potential for improving sagging (i.e., areas where the difference between the sagging area in the first facial image and the sagging area in the second facial image is less than a predetermined second degree (areas of facial features)).

[0040] <Method> FIG. 4 is a flowchart of a process for estimating the potential for reducing slack according to an embodiment of the present invention.

[0041] In step 1 (S1), the face data acquisition unit 101 acquires (i) first face data, which is an image or three-dimensional data of the face of the subject 30 when the angle between the axial direction of the face of the subject 30 and the vertical direction is a first angle, and (ii) second face data, which is an image or three-dimensional data of the face of the subject 30 when the angle between the axial direction of the face of the subject 30 and the vertical direction is a second angle.

[0042] In step 2 (S2), the estimation unit 102 identifies areas where the difference between the first facial shape, which is the facial shape of the subject 30 indicated by the first facial data acquired in S1, and the second facial shape, which is the facial shape of the subject 30 indicated by the second facial data acquired in S1, is greater than a predetermined first degree, and / or areas where the difference is smaller than a predetermined second degree.

[0043] <User Interface> 5 to 7 are examples of images displayed on the client device 20 according to one embodiment of the present invention.

[0044] <<Displaying sagging areas>> For example, the sagging areas in the image of the face of the subject 30 taken when the subject 30 is in a sitting position are displayed so as to be comparable to the sagging areas in the image of the face of the subject 30 taken when the subject 30 is in a supine position. For example, as shown in Fig. 5, for each facial part (sagging under the eyes, nasolabial folds, and marionette lines in the example of Fig. 5), the sagging areas in the image of the face of the subject 30 taken when the subject 30 is in a sitting position are displayed so as to be comparable to the sagging areas in the image of the face of the subject 30 taken when the subject 30 is in a supine position.

[0045] In this way, it is possible to display areas with high potential for improving sagging (i.e., areas that are sagging when sitting but not when lying on your back), areas with medium potential for improving sagging (i.e., areas where there is a difference between the degree of sagging when sitting and the degree of sagging when lying on your back), and areas with low potential for improving sagging (i.e., areas that are sagging both when sitting and lying on your back).

[0046] <<Cheek position display>> For example, the position of the cheeks in an image of the face of the subject 30 taken when the subject 30 is in a sitting position is displayed so as to be comparable to the position of the cheeks in an image of the face of the subject 30 taken when the subject 30 is in a supine position. For example, as shown in Fig. 6, the position of the cheeks when the subject 30 is in a sitting position is displayed in the image of the face of the subject 30 taken when the subject 30 is in a sitting position, and the position of the cheeks when the subject 30 is in a supine position and the position of the cheeks when the subject 30 is in a sitting position are displayed in the image of the face of the subject 30 taken when the subject 30 is in a supine position.

[0047] In this way, it is possible to display how much the position of the cheeks changes as sagging improves.

[0048] <<Comparison with the average age-related sagging of people of the same age>> For example, the sagging skin due to aging of the subject 30 is displayed so as to be comparable to the average sagging skin due to aging of people of the same age as the subject 30. For example, as shown in Fig. 7, the sagging skin due to aging of the subject 30 is displayed so as to be comparable to the average sagging skin due to aging of people of the same age as the subject 30 for each part of the face (in the example of Fig. 7, sagging skin under the eyes, nasolabial folds, and marionette lines).

[0049] In this way, it is possible to display the results of comparing true sagging (i.e., sagging due to aging) with that of people of the same age, without taking into account the original facial features.

[0050] <Light environment> 8 and 9, we will explain whether the lighting environment affects the results of facial data measurement. If facial data is measured when a subject changes their posture under light that is always irradiated in a fixed direction from a fixed light source (e.g., ceiling lighting), the angle of the light irradiating the face will differ, and the shading on the face will also differ. How this difference in shading affects the results of facial data measurement will be described below.

[0051] 8 and 9 are diagrams for comparing sagging in a sitting position and a supine position in various lighting environments according to an embodiment of the present invention.

[0052] Figure 8 shows the amount of facial sagging for [Subject A], [Subject B], and [Subject C] when each subject is in a sitting position and when they are in a supine position under various lighting conditions (under conditions where ambient light is excluded, under ambient light, and under harsh conditions).

[0053] Under the ambient light exclusion condition, an image of the subject's face was taken in a dark room while the subject was in a sitting position and an image of the subject's face while the subject was in a supine position without changing the relative positions of the lighting, camera, and face. In this case, only changes in the shape of the face due to the influence of gravity are reflected in the results.

[0054] Under ambient lighting, an image of the subject's face was taken when the subject was in a sitting position and an image of the subject's face when the subject was in a supine position (with no unnecessary shadows on the face) in a location with ceiling lighting.

[0055] The harsh conditions were when an image of the subject's face was taken in a location with ceiling lighting installed, and an image of the subject's face when the subject was in a sitting position and an image of the subject's face when the subject was in a supine position (with unnecessary shadows cast on the face).

[0056] As shown in FIG. 8, all subjects had less sagging when in the supine position than when in the sitting position, not only under the ambient light exclusion condition but also under ambient light.

[0057] Figure 9 shows the amount of facial sagging when each subject (subject A, subject B, subject C) was lying supine under various lighting conditions (living environment light exclusion conditions, living environment light conditions, and harsh conditions) minus the amount of facial sagging when the subject was sitting. For all subjects, the amount of facial sagging was less when lying supine than when sitting, not only under living environment light exclusion conditions but also under living environment light.

[0058] From the results of Figures 8 and 9, it can be said that the potential for improving sagging can be estimated even with face data measured in a living environment. It can also be said that it is necessary to prevent unwanted shadows from being cast on the face. Therefore, when photographing the face of the subject 30, the client device 20 may notify the subject 30 to prevent unwanted shadows from being cast on the face (for example, notify the subject that an unwanted shadow is cast on the face), and may fire a flash if it is dark.

[0059] <Other embodiments> Other embodiments will be described below.

[0060] In one embodiment of the present invention, the gravity environment may be changed by applying excessive gravity or creating a zero-gravity state. This can lead to effects such as accelerating or decelerating the estimation of the potential to improve sagging, or adjusting the accuracy and sensitivity. For example, facial data may be acquired while shaking the face up and down or from side to side, or while sitting on a rotating chair (e.g., before and after sudden acceleration or deceleration).

[0061] In one embodiment of the present invention, facial data may be acquired before and after a handstand.

[0062] In one embodiment of the present invention, facial data may be acquired while rotating around a front-to-back axis (tilting the head), a left-to-right axis (looking up or down), or a top-to-bottom axis (looking right or left) without changing body position (e.g., while sitting or standing).

[0063] In one embodiment of the present invention, facial data may be acquired when the person is sleeping on a bed that is placed horizontally, and facial data may be acquired when the person is sleeping on a bed that is tilted at an angle of 45 degrees.

[0064] In one embodiment of the present invention, the acceleration sensor, gyro sensor, and angular velocity sensor of client device 20 acquire tilt or tilt change, and when it is detected that the degree of tilt change exceeds a threshold while a face is captured, client device 20 may display a screen to prompt the user to take a photo, or may output specific imaging conditions (exposure, aperture) and / or lighting conditions to the imaging means.

[0065] In one embodiment of the present invention, in order to prevent the user from changing their facial expression, a message may be displayed, a previously photographed face may be displayed dimmed, or the eyes may be opened or closed.

[0066] In one embodiment of the present invention, the user may be prompted to change facial expressions, or to change imaging conditions, posture, or the like.

[0067] <Presenting cosmetics and beauty information> In one embodiment of the present invention, the server device 10 or the client device 20 can present cosmetics and beauty information based on the results of estimating the potential to improve sagging (for example, if the sagging is due to aging, cosmetics and beauty information for improving sagging due to aging).

[0068] <Effects> In this way, in one embodiment of the present invention, by distinguishing between true sagging and original facial features, it is possible to visualize the potential for improving sagging. Specifically, it is possible to show areas with high potential for improving sagging (i.e., areas of sagging due to aging) and areas with low potential for improving sagging (i.e., areas of facial features).

[0069] <Hardware configuration> 10 is a hardware configuration diagram of a server device 10 and a client device 20 according to an embodiment of the present invention. The server device 10 and the client device 20 can include a control unit 1001, a main memory unit 1002, an auxiliary memory unit 1003, an input unit 1004, an output unit 1005, and an interface unit 1006. Each of these will be described below.

[0070] The control unit 1001 is a processor (for example, a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), etc.) that executes various programs installed in the auxiliary storage unit 1003.

[0071] The main memory unit 1002 includes a non-volatile memory (Read Only Memory (ROM)) and a volatile memory (Random Access Memory (RAM)). The ROM stores various programs, data, etc. required for the control unit 1001 to execute various programs installed in the auxiliary memory unit 1003. The RAM provides a working area into which the various programs installed in the auxiliary memory unit 1003 are expanded when executed by the control unit 1001.

[0072] The auxiliary storage unit 1003 is an auxiliary storage device that stores various programs and information used when the various programs are executed.

[0073] The input unit 1004 is an input device through which the operators of the server device 10 and the client device 20 input various instructions to the server device 10 and the client device 20 .

[0074] The output unit 1005 is an output device that outputs the internal states of the server device 10 and the client device 20, etc.

[0075] The interface unit 1006 is a communication device for connecting to a network and communicating with other devices.

[0076] Although the examples of the present invention have been described in detail above, the present invention is not limited to the specific embodiments described above, and various modifications and variations are possible within the scope of the gist of the present invention as set forth in the claims. [Explanation of symbols]

[0077] 1. Information Processing Systems 10 Server device 20 Client Device 30 Target Audience 101 Face data acquisition unit 102 Estimation part 103 Image generation unit 201 Reception 202 Image acquisition unit 203 Display section 1001 control section 1002 Main memory 1003 Auxiliary storage unit 1004 Input section 1005 Output section 1006 Interface section

Claims

1. On the computer, (i) acquiring first face data, which is an image or three-dimensional data of the face of the subject when the angle between the axis direction of the face of the subject and the vertical direction is a first angle, and (ii) acquiring second face data, which is an image or three-dimensional data of the face of the subject when the angle between the axis direction of the face of the subject and the vertical direction is a second angle; a step of identifying an area where a difference between a first face shape, which is the face shape of the subject represented by the first face data, and a second face shape, which is the face shape of the subject represented by the second face data, is greater than a predetermined first degree and / or an area where the difference is smaller than a predetermined second degree; A program to execute.

2. The program according to claim 1 , wherein the area where the difference is greater than the first degree is age-related sagging.

3. The program according to claim 1 , wherein the area where the difference is smaller than the first degree and larger than the second degree is age-related sagging.

4. The program of claim 1 , wherein the region where the difference is less than the second degree is a facial feature of the subject.

5. The computer, The program according to claim 1 , further causing the computer to execute the step of generating an image in which the identified area is highlighted for each of the facial parts.

6. The computer, The program according to claim 1, further causing a notification to be given to prevent unwanted shadows from appearing in the subject's face when photographing the subject's face, and for a flash to be emitted if it is dark.

7. (i) a face data acquisition unit that acquires first face data, which is an image or three-dimensional data of the face of the subject when the angle between the axis direction of the face of the subject and the vertical direction is a first angle, and (ii) second face data, which is an image or three-dimensional data of the face of the subject when the angle between the axis direction of the face of the subject and the vertical direction is a second angle; an estimation unit that identifies an area where a difference between a first face shape, which is the face shape of the subject represented by the first face data, and a second face shape, which is the face shape of the subject represented by the second face data, is greater than a predetermined first degree and / or an area where the difference is smaller than a predetermined second degree; A server device comprising:

8. a first face shape that is the shape of the face of the subject represented by first face data that is an image or three-dimensional data of the face of the subject when an angle formed between an axial direction of the face of the subject and a vertical direction is a first angle; a second face shape that is the shape of the face of the subject represented by second face data that is an image or three-dimensional data of the face of the subject when an angle formed between an axial direction of the face of the subject and a vertical direction is a second angle; an image acquisition unit that acquires an image in which a region where the difference is greater than a predetermined first degree and / or a region where the difference is smaller than a predetermined second degree is highlighted; a reception unit that receives input from a user; a display unit that displays the image in response to the input received by the receiving unit; A client device comprising:

Images