Evaluation system, evaluation method and program

Abstract

To quantitatively and easily evaluate symmetry of a face.SOLUTION: An evaluation system includes an imaging control unit that guides a user terminal to capture an image of a user's facial expression corresponding to each of a plurality of facial expressions, an image acquisition unit that acquires from the user terminal, each of the user's facial images captured in response to the guidance of the imaging control unit, and a score calculation unit that calculates an index value representing a degree of facial asymmetry for every facial image acquired by the image acquisition unit.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an evaluation system, an evaluation method, and a program. [Background technology]

[0002] Facial symmetry is evaluated. For example, Non-Patent Document 1 discloses a subjective evaluation method in which an examiner visually judges the subject, and an objective evaluation method in which measurements and computer analysis are performed using photographs, video images, etc., as methods for evaluating the degree of facial distortion due to facial nerve paralysis. [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] “Diagnosis and Evaluation”, [online], Tokyo Dental College, Ichikawa General Hospital, Department of Plastic Surgery, Facial Paralysis Specialist Clinic, [Retrieved April 26, 2024], Internet<URL:https: / / www.tdc.ac.jp / hospital / section / facialpalsy / diagnosis / diagnosis_02.html> Summary of the Invention [Problem to be solved by the invention]

[0004] However, subjective assessment methods such as those described above are difficult to perform unless performed by a specialist such as a speech-language-hearing therapist or physical therapist, and there is a possibility that the assessment results will vary depending on the assessor. Objective assessment methods require long-term testing using specialized, expensive measuring equipment. Since a decrease in muscle mass due to lifestyle habits or aging, not only in cases of facial paralysis but also in healthy individuals, can manifest as facial distortion, it is desirable to be able to easily assess facial symmetry on a daily basis.

[0005] In view of the above-mentioned problems, an object of the present invention is to provide an evaluation system, an evaluation method, and a program that can quantitatively and easily evaluate facial symmetry. [Means for solving the problem]

[0006] In order to solve the above-mentioned problems, one aspect of the present invention is an evaluation system comprising an imaging control unit that guides a user terminal to capture images of the user's facial expressions corresponding to each of a plurality of facial expressions, an image acquisition unit that acquires from the user terminal each of the user's facial images captured in accordance with the guidance of the imaging control unit, and a score calculation unit that calculates an index value indicating the degree of left-right difference of the face for each of the facial images acquired by the image acquisition unit.

[0007] One aspect of the present invention is an evaluation method performed by an evaluation system that is a computer, in which an imaging control unit guides a user terminal to capture images of the user's facial expressions corresponding to each of a plurality of facial expressions, an image acquisition unit acquires each of the user's facial images captured in accordance with the guidance of the imaging control unit from the user terminal, and a score calculation unit calculates an index value indicating the degree of left-right difference of the face for each of the facial images acquired by the image acquisition unit.

[0008] One aspect of the present invention is a program that causes an evaluation system, which is a computer, to guide a user terminal to capture images of the user's facial expressions corresponding to each of a plurality of facial expressions, acquire each of the user's facial images captured in accordance with the guidance from the user terminal, and calculate an index value indicating the degree of laterality of the face for each of the acquired facial images. [Effects of the Invention]

[0009] According to the present invention, facial symmetry can be quantitatively and easily evaluated. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is a block diagram illustrating an example of the configuration of an evaluation system according to an embodiment. [Figure 2] FIG. 2 is a diagram for explaining a process performed by the evaluation system according to the embodiment. [Figure 3] FIG. 2 is a diagram for explaining a process performed by the evaluation system according to the embodiment. [Figure 4] FIG. 2 is a diagram for explaining a process performed by the evaluation system according to the embodiment. [Figure 5] FIG. 2 is a diagram for explaining a process performed by the evaluation system according to the embodiment. [Figure 6] FIG. 2 is a diagram for explaining a process performed by the evaluation system according to the embodiment. [Figure 7] FIG. 2 is a diagram for explaining a process performed by the evaluation system according to the embodiment. [Figure 8] FIG. 2 is a diagram for explaining a process performed by the evaluation system according to the embodiment. [Figure 9] FIG. 2 is a diagram for explaining a process performed by the evaluation system according to the embodiment. [Figure 10] FIG. 10 is a sequence diagram showing the flow of processing performed by the evaluation server according to the embodiment. [Figure 11] FIG. 10 is a diagram for explaining a process performed by an evaluation system according to a second modified example of the embodiment. [Figure 12] FIG. 10 is a diagram for explaining a process performed by an evaluation system according to a second modified example of the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. In the following description, a case where the facial paralysis state of a patient with facial paralysis is quantitatively evaluated will be described as an example, but the present invention is not limited to this. This embodiment can be applied not only to facial paralysis but also to cases where facial distortion is quantitatively evaluated in healthy individuals who are interested in beauty and health.

[0012] <1. About the evaluation system> The configuration of the evaluation system according to this embodiment will be described with reference to Fig. 1. Fig. 1 is a block diagram showing an example of the configuration of the evaluation system according to this embodiment. As shown in Fig. 1, the evaluation system 1 includes a user terminal 10 and an evaluation server 20. The user terminal 10 and the evaluation server 20 are communicably connected via a communication network NW. The evaluation system 1 is a system that provides a service for evaluating facial symmetry (hereinafter referred to as the evaluation service). In the evaluation system 1, all or part of the functions of the evaluation server 20 may be provided in the user terminal 10. For example, an application (an evaluation application described later) installed in the user terminal 10 may have the functions of the evaluation server (an imaging control unit 23, an image determination unit 24, a feature point detection unit 25, a score calculation unit 26, and an output unit 27 described later). In this case, the communication network and the evaluation server 20 can be omitted.

[0013] The user terminal 10 is a terminal used by a user who uses the evaluation service. The user terminal 10 is, for example, a smartphone, a tablet terminal, a PC (Personal Computer), etc. The user terminal 10 is communicably connected to the evaluation server 20 via a communication network NW.

[0014] The evaluation server 20 is a server that evaluates the facial symmetry of a user who uses the evaluation service. The evaluation server 20 is configured from one or more servers, such as a cloud server.

[0015] In the evaluation system 1, a dedicated application (hereinafter referred to as "evaluation app") is distributed to the user. The user installs the evaluation app distributed from the evaluation system 1 into the user terminal 10. The user terminal 10 performs user registration with the evaluation server 20 via the evaluation app. This allows the user to use the evaluation service. Note that if an evaluation app equipped with the functions of the evaluation server (an imaging control unit 23, an image determination unit 24, a feature point detection unit 25, a score calculation unit 26, and an output unit 27, which will be described later) is installed into the user terminal 10, user registration is not necessarily required.

[0016] In the above, the case where the rating service is provided by installing the rating application on the user terminal 10 has been exemplified, but the present invention is not limited to this. The rating service may be provided by a Web system (i.e., a Web application). When the rating application is provided by a Web system, the rating service is provided via a Web browser.

[0017] 2 is a diagram for explaining the processing performed by the evaluation system according to the embodiment. In FIG. 2, an outline of the flow of the evaluation service is shown from the left to the center and to the right as viewed from the front of FIG. 2. First, a user U who operates the user terminal 10 follows the instructions displayed on the user terminal 10 to capture images of a plurality of facial expressions on his or her face. The multiple facial expressions here are expressions for evaluating facial symmetry, and are multiple expressions with different movements of facial parts such as eyebrows, eyes, nose, cheeks, mouth, etc. In this embodiment, six facial expressions are used as the multiple facial expressions: resting, furrowing the forehead, slightly closing the eyes, whistling, showing teeth, and smiling. When facial images corresponding to each of a plurality of facial expressions are captured, the facial symmetry is evaluated based on the facial images. Here, the evaluation is performed based on a central axis extracted from the facial image, and the angular difference between the left and right sides of each facial feature based on the central axis is calculated. The specific method for calculating the angular difference will be described in detail later. The degree of opening of the left and right eyes is also calculated. The degree of opening of the mouth is also calculated with respect to the central axis. Furthermore, the distance from the central axis to the left and right corners of the mouth is calculated. As an evaluation result, the calculated angle difference, degree of opening, and distance are each displayed in association with facial features in the face image. Furthermore, scores are calculated for each of the facial features, namely, eyebrows, eyes, nose, cheeks, and mouth, as individual evaluation scores. These scores are numerical representations of the degree of left-right symmetry. Furthermore, a comprehensive score for the face is calculated based on the individual evaluation scores for the eyebrows, eyes, nose, cheeks, and mouth. Specific methods for calculating the individual evaluation scores and the comprehensive evaluation score will be described in detail later.

[0018] <2. User Devices> Returning to FIG. 1, the user terminal 10 includes a communication unit 11, a storage unit 12, a control unit 13, an imaging unit 14, a display unit 15, and an operation input unit 16. The communication unit 11 communicates with the user terminal 10 . The storage unit 12 stores various types of information. The storage unit 12 is configured by a storage medium provided as hardware in the user terminal 10, such as a hard disk drive (HDD), a solid state drive (SSD), a flash memory, an electrically erasable programmable read-only memory (EEPROM), a random access read / write memory (RAM), a read-only memory (ROM), or any combination of these storage media. The storage unit 12 stores programs for realizing functions provided in the user terminal 10 (including functions related to the control unit 13, particularly the evaluation application). The control unit 13 realizes the functions of the user terminal 10 by causing a CPU provided as hardware in the user terminal 10 to execute a program stored in the storage unit 12. For example, the control unit 13 controls the imaging unit 14, the display unit 15, and the operation input unit 16. For example, in response to a user's operation input to the operation input unit 16, the control unit 13 communicates with the evaluation server 20, launches an evaluation app, and displays a screen of the evaluation app in response to the operation on the display unit 15. Furthermore, in response to being guided via the evaluation app to capture an image of the user's facial expression, the control unit 13 launches the camera function of the imaging unit 14 and displays the captured image on the display unit 15. The imaging unit 14 includes a camera module and takes an image under the control of the control unit 13 . The display unit 15 includes a display device such as a liquid crystal display, and displays an image according to the control of the control unit 13 . The operation input unit 16 acquires an operation input from the user and outputs the acquired operation input to the control unit 13 .

[0019] <3.About the evaluation server> As shown in FIG. 1, the evaluation server 20 includes a communication unit 21, a storage unit 22, an imaging control unit 23, an image determination unit 24, a feature point detection unit 25, a score calculation unit 26, and an output unit 27. The communication unit 21 communicates with the user terminal 10 . The storage unit 22 stores various information. The storage unit 22 is configured by a storage medium provided as hardware in the assessment server 20, such as an HDD, SSD, flash memory, EEPROM, RAM, ROM, or any combination of these storage media. The storage unit 22 stores programs for realizing the functions provided in the assessment server 20 (including, in particular, an imaging control unit 23, an image determination unit 24, a feature point detection unit 25, a score calculation unit 26, and an output unit 27 as functions of the assessment application). The functions provided in the assessment server 20 are realized by causing a CPU provided as hardware in the assessment server 20 to execute the programs stored in the storage unit 22.

[0020] The imaging control unit 23 controls the imaging of the facial expressions of the user by the user terminal 10 in the evaluation application. The imaging control unit 23 guides the user terminal 10 to capture images of the facial expressions of the user corresponding to each of a plurality of facial expressions. For example, the imaging control unit 23 causes the user terminal 10 to display an expression image showing an expression that the user wants to capture, and guides the user to capture an expression corresponding to the expression image. More specifically, the imaging control unit 23 first causes the user terminal 10 to display an expression image showing an expression corresponding to "resting," and guides the user to capture an expression corresponding to "resting" on the user's face.

[0021] When a user operating the user terminal 10 performs an operation to capture an image of their own facial expression in accordance with this guidance, the imaging control unit 23 displays the captured facial image on the user terminal 10 and also displays selection buttons labeled "Take Again" and "Decide." When the user operating the user terminal 10 selects the "Take Again" selection button, the imaging control unit 23 prompts the user to capture the image again. On the other hand, when the user selects the "Decide" selection button, the captured facial image is transmitted from the user terminal 10 to the evaluation server 20. When the evaluation server 20 receives a facial image capturing the user's facial expression corresponding to "resting," the imaging control unit 23 causes the user terminal 10 to display an expression image corresponding to "close your eyes," which is the expression corresponding to the next imaging target, and prompts the user to capture an image of the user's facial expression corresponding to "close your eyes." By repeating this guidance sequentially for each of the multiple facial expressions, the imaging control unit 23 guides the user terminal 10 in capturing an image.

[0022] The image determination unit 24 determines whether or not the face image captured by the user terminal 10 is an appropriate image. For example, if the image determination unit 24 cannot detect facial features such as eyebrows, eyes, nose, cheeks, and mouth from the face image transmitted from the user terminal 10 to the evaluation server 20, the image determination unit 24 determines that the face image captured by the user terminal 10 is inappropriate. The image determination unit 24 may also determine whether the composition of a captured face image is appropriate. Specifically, the image determination unit 24 determines whether the composition (e.g., position, size, angle, etc.) of the face part captured in the face image is appropriate. For example, the image determination unit 24 extracts the face contour from the face image, and determines that the position of the face part in the face image is appropriate if the distance between the center of gravity of the extracted contour and the center of the face image is within a threshold value. The image determination unit 24 also extracts the face contour from the face image, and determines that the size of the face part in the face image is appropriate if the ratio of the area of ​​the extracted contour to the total area of ​​the face image is within a predetermined range (e.g., 60% to 75%). The image determination unit 24 also extracts multiple features (e.g., eyebrows, eyes, nose, cheeks, etc.) on the left and right sides of the face from the face image. If the extracted features are asymmetric and tend to be asymmetric to the same degree, the image determines that the face in the face image is tilted to either the left or right and is inappropriate. The image determination unit 24 also extracts features on the top and bottom of the face image, such as the forehead as an upper feature and the chin as a lower feature, or the eyebrows as an upper feature and the mouth as a lower feature. If the ratio of the area of ​​the lower feature to the area of ​​the extracted upper feature is equal to or greater than a threshold, the image determination unit 24 determines that the face has been captured in a tilted position, looking up from below, and is therefore inappropriate. If the ratio of the area of ​​the upper feature to the area of ​​the extracted lower feature is equal to or greater than a threshold, the image determination unit 24 determines that the face has been captured in a panoramic position, looking down from above, and is therefore inappropriate. When the image determination unit 24 determines that the facial image captured by the user terminal 10 is inappropriate, it requests the imaging control unit 23 to guide the user terminal 10 to capture an image again. In this case, the imaging control unit 23 causes the user terminal 10 to again display an expression image showing the facial expression for which the facial image is determined to be inappropriate, and guides the user to capture an image again.

[0023] The feature point detection unit 25 detects feature points in the face image. Feature points are pixels in the face image that correspond to the contours of each of the eyebrows, eyes, nose, cheeks, and mouth, which are facial features. The feature point detection unit 25 detects multiple feature points for each feature. For example, for the "eyebrows" as a facial feature, feature points corresponding to the beginning, ridge, and end of the eyebrows, which correspond to the contours of the eyebrows, are detected. For the "eyes," feature points corresponding to the inner corner, center of the upper eyelid, center of the lower eyelid, and outer corner of the eye, which correspond to the contours of the eyes, are detected. Similarly, for the nose, cheeks, and mouth, multiple feature points corresponding to the contours of the nose, cheeks, and mouth are detected.

[0024] The score calculation unit 26 calculates the individual evaluation scores and the overall evaluation score. First, the score calculation unit 26 extracts the central axis of the face in the face image. For example, the score calculation unit 26 determines the central axis to be a line segment connecting the center point of the forehead to the center point of the bridge of the nose.

[0025] Next, the score calculation unit 26 calculates a measurement value that measures the degree of left-right symmetry for each feature point. The score calculation unit 26 calculates the angle difference for each facial feature as a measurement value. The score calculation unit 26 calculates the angle difference for each facial feature. The score calculation unit 26 calculates the angles at the vertices (feature points) of a rectangle formed by connecting each of the feature points extracted along the contours of the facial features, and calculates the difference between the angles corresponding to the calculated left and right feature points as the angle difference. For example, for the "eyebrows," the score calculation unit 26 calculates three angles corresponding to the feature points serving as the vertices of a triangle formed by connecting pixels corresponding to the beginning, the arch, and the end of the eyebrows. The score calculation unit 26 calculates the three angles corresponding to the vertices of the triangle for each of the left and right eyebrows. The score calculation unit 26 calculates the difference between the angles corresponding to the beginning, the arch, and the end of the eyebrows as the angle difference for each of the three feature points of the eyebrows. Then, the score calculation unit 26 calculates one of the three calculated angle differences that is considered to be easily recognizable by the user, for example, the angle difference for the arch of the eyebrows among the beginning, the arch, and the end of the eyebrows, as the "angle difference" serving as an index value for evaluating the symmetry of the "eyebrows" as a facial feature. Furthermore, for the "eyes," the score calculation unit 26 calculates, for each of the left and right eyes, the angles of the vertices of a diamond formed by connecting pixels corresponding to the inner corner of the eye, the center of the upper eyelid, the center of the lower eyelid, and the outer corner of the eye as feature points. For each of the left and right eyes, the score calculation unit 26 calculates four angles corresponding to the vertices of the diamond. The score calculation unit 26 calculates the differences between the angles corresponding to the inner corner of the eye, the center of the upper eyelid, the center of the lower eyelid, and the outer corner of the eye as four angle differences for the eyes. Then, the score calculation unit 26 calculates one of the four calculated angle differences that is considered to be easy for the user to recognize, for example, the angle difference of the "outer corner of the eye" among the inner corner of the eye, the center of the upper eyelid, the center of the lower eyelid, and the outer corner of the eye, as the "angle difference" serving as an index value for evaluating the symmetry of the "eyes" as a facial feature. Similarly, for the nose, cheeks, and mouth, an angle formed by connecting multiple feature points detected from the parts is calculated, and the difference between the angles calculated corresponding to each of the left and right feature points is taken as the angle difference.

[0026] Furthermore, for the "eyes" and "mouth," the score calculation unit 26 calculates the difference between the left and right degrees of opening as a measurement value. For example, for the "eyes," the score calculation unit 26 calculates the degree of opening of each of the left and right eyes by using the length of a line connecting pixels corresponding to the centers of the upper eyelid and the lower eyelid, which are feature points, as the degree of opening of the eyes. Similarly, for the mouth, the degree of opening of each of the left and right eyes is calculated as a measurement value. The score calculation unit 26 calculates the distance from the central axis to each of the left and right corners of the mouth as the measurement value for the "mouth."

[0027] The score calculation unit 26 calculates a standardized score using the calculated measurement value (any of the angle difference, the degree of opening, or the distance). The standardized score is an index value that indicates the degree to which the measurement value deviates from a standard value. The standardized score is calculated, for example, by equation (1). STSCR in equation (1) is the standardized score. x is the measurement value (any of the angle difference, the degree of opening, or the distance) calculated by the score calculation unit 26.

[0028] STSCR(x)={x-AVE(x#)} / STDEV(x#) ...Formula (1) however, AVE(x#) is the average value of x#. STDEV(x#) is the standard deviation of x#. x# is a measurement value (angle difference, degree of opening, or distance) calculated from each face image of a standard group of people.

[0029] Here, any group of people may be set as the "standard group of people" that forms the basis of the average value and standard deviation used in equation (1). For example, the standard group of people is a group consisting of multiple people who are not facially paralyzed and are so-called healthy individuals. In this case, AVE(x#) is the average value of the measurement values ​​calculated from the facial expressions of healthy individuals. STDEV(x#) is the standard deviation of the measurement values ​​calculated from the facial expressions of healthy individuals. It is assumed that the average value of the measurement values ​​calculated from the facial expressions of healthy individuals and the standard deviation or variance of the measurement values ​​calculated from the facial expressions of healthy individuals have been calculated in advance.

[0030] The score calculation unit 26 calculates a feature point evaluation score, which is an index that indicates the degree of left-right difference for each feature point, calculated by subtraction. For example, the score calculation unit 26 calculates the feature point evaluation score by subtracting a numerical value corresponding to the standardized score of the feature point from the highest score (for example, 100 points). The score calculation unit 26 calculates the feature point evaluation score using equation (2). In equation (2), FSCR is the feature point evaluation score, STSCR is the standardized score of the feature point, and C is a coefficient (for example, 10).

[0031] FSCR=100-(STSCR×C) …Formula (2)

[0032] When multiple feature points are detected for each part, the score calculation unit 26 calculates a weighted average value calculated by multiplying the corresponding feature point evaluation scores by weighting coefficients, and uses this weighted average value as the individual evaluation score for that part. For example, suppose three feature points are detected for the "eyebrows" as a facial feature: the beginning, the arch, and the end of the eyebrows, and a standardized score is calculated for each of the detected feature points. In this case, the score calculation unit 26 calculates the individual evaluation score of "eyebrows" as a facial feature using formula (3). PTSCR_MAYU in formula (3) is the individual evaluation score, and in this case, is the individual evaluation score of "eyebrows". C is a coefficient.

[0033] PTSCR = {(100-x1×C)r1 +(100-x2×C)r2 +(100-x3×C)r3} / (r1+r2+r3) ...Formula (3) however, x1 is the standardized score of the inner eyebrow as a feature point. x2 is the standardized score of the brow ridge as a feature point. x3 is the standardized score of the tail of the eyebrow as a feature point. r1 is a weighting coefficient for the inner corner of the eyebrow as a feature point. r2 is the weighting coefficient for the brow ridge as a feature point. r3 is a weighting coefficient for the tail of the eyebrow as a feature point.

[0034] Here, it is assumed that the weighting coefficients r (r1, r2, r3) are set in advance. For example, the weighting coefficients r are set in advance depending on whether or not the feature points correspond to regions where a significant difference between the left and right sides is likely to occur.

[0035] The score calculation unit 26 calculates a comprehensive evaluation score as a comprehensive score indicating the degree of facial symmetry. The score calculation unit 26 calculates the comprehensive evaluation score based on the individual evaluation scores of the eyebrows, eyes, nose, cheeks, and mouth as facial features. For example, the score calculation unit 26 calculates the comprehensive evaluation score using equation (4).

[0036] CPSCR = {PTSCR(1) × w1 +PTSCR(2)×w2 +PTSCR(3)×w3 +PTSCR(4)×w4 +PTSCR(5)×w5} / (w1+w2+w3+w4+w5) ...Formula (4) however, PTSCR(1) is an individual evaluation score for "eyebrows." PTSCR(2) is an individual evaluation score for the "eyes." PTSCR(3) is an individual evaluation score for "cheeks." PTSCR(4) is an individual rating score for "nose." PTSCR(5) is the individual assessment score for "mouth." w1 is the weighting coefficient for the "eyebrows." w2 is the weighting factor for the "eyes". w3 is the weighting factor for the "cheeks". w4 is the weighting factor for the "nose". w5 is the weighting factor for "mouth".

[0037] It is assumed that the weighting coefficients w (w1, w2, w3, w4, w5) are set in advance. For example, the weighting coefficients w are set so that, among the facial features, a weighting coefficient is increased for a feature that is particularly prone to showing a difference between the left and right sides. For example, the ratio of the weighting coefficients w is set to (w1:w2:w3:w4:w5) = (3:5:1:1:5). This is because, among the facial features, the features (eyebrows, eyes, mouth) in particular tend to show a difference between the left and right sides more easily than the features (cheeks, nose).

[0038] The output unit 27 outputs the measurement values, individual evaluation scores, overall evaluation scores, etc. as the evaluation results, and causes the user terminal 10 to display the evaluation results.

[0039] Here, the processing performed by the evaluation system will be described with reference to Figures 3 to 9. Figures 3 to 9 are diagrams for explaining the processing performed by the evaluation system according to the embodiment. Figures 3 to 9 show examples of screens displayed on the user terminal 10 via the evaluation app.

[0040] 3 shows an example of a screen displayed on the user terminal 10 when the evaluation app is launched. This figure shows an example in which multiple menus are presented in response to the launch of the evaluation app. The multiple menus are menus of services provided by the evaluation app, including massage, training, recording of laterality, history, and games (i.e., I-Woop games). When a user operating the user terminal 10 selects the menu button MB showing "Record left-right asymmetry" from among the multiple menus displayed on the user terminal 10, recording of left-right asymmetry, that is, measurement and evaluation of the degree of facial symmetry, is performed.

[0041] Fig. 4 shows an example of a screen that is displayed on the user terminal 10 when the menu button MB in Fig. 3 is selected. As shown in Fig. 4, when the menu button MB in Fig. 3 is selected, the imaging unit 14 is activated and preparations are made to capture an image of the user's facial expression. In addition, guidance is provided on which facial expression to capture from among a plurality of facial expressions to be captured. In this figure, an imaging window WG, a facial expression image HG, and an operation button SB are displayed. The imaging window WG is a display window that displays the facial expression of the user captured as a subject by the imaging unit 14. As shown in the example of this figure, the imaging window WG may display a guide image that indicates, by dotted lines or the like, the area in which the user's facial portion is to be imaged. In this case, when guiding the user terminal 10 to image the user's facial expression, the imaging control unit 23 displays, in the imaging window WG, a guide image that indicates, by dotted lines or the like, the area in which the user's facial portion is to be imaged, and guides the imaging so that the face fits within the area indicated by this guide image. This makes it possible to guide the user terminal 10 to capture a facial image in which the position and size of the facial portion in the image are appropriate. The facial expression image HG is an image that guides the user to capture a specific facial expression to be captured, in this case, a facial expression corresponding to "resting." The facial expression image HG may also display a character string indicating the facial expression to be captured, in this case, the character string "resting." The facial expression image HG may also display a percentage indicating the number of the facial expression being captured out of the total number of facial expressions to be captured. In this figure, "1 / 6" is displayed, indicating that the facial expression corresponding to "resting" will be captured first out of the six facial expressions to be captured. The operation button SB is a button that indicates "take a photo" and is operated when capturing an image. When the operation button SB is tapped, the facial expression of the user that is displayed in the imaging window WG at the time of operation (the facial expression corresponding to "resting") is captured by the imaging unit 14.

[0042] Fig. 5 shows an example of a screen displayed on the user terminal 10 when the operation button SB in Fig. 4 is selected. As shown in Fig. 5, when the operation button SB in Fig. 4 is operated, a captured image PG, a facial expression image HG, and the operation button SB are displayed. The captured image PG displays a captured image of the user's face. The facial expression image HG displays an image similar to the display image in FIG. The operation button SB is a button that allows the user to select either "take a new photograph" or "confirm." When the "take again" operation button is selected, the captured image PG switches to the imaging window WG shown in Figure 4, and the operation button switches to the operation button SB shown in Figure 4, returning to the screen shown in Figure 4 and prompting the user to take another image. When the "Decide" operation button is selected, the facial image displayed in the captured image PG is transmitted from the user terminal 10 to the evaluation server 20. When the facial image is received by the evaluation server 20, a screen is displayed on the user terminal 10 to guide the user to capture the next facial expression, for example, the "lightly closed eyes" facial expression, which is the second facial expression to be captured.

[0043] Figure 6 shows an example of a screen displayed on the user terminal 10 instructing the user to capture the third facial expression to be captured, the "Yee" facial expression with teeth showing, out of the six facial expressions to be captured. In this figure, an imaging window WG, a facial expression image HG, and an operation button SB are displayed in the same manner as in Fig. 4. Of these, the imaging window WG and the operation button SB are the same as in Fig. 4. The facial expression image HG is an image that guides the user to capture a specific facial expression to be captured, in this case, the facial expression of showing teeth and saying "Yeeee." The facial expression image HG may also show a character string indicating the facial expression to be captured, in this case, the character string "showing teeth and saying "Yeeee." Furthermore, the facial expression image HG may show "3 / 6," indicating that the facial expression of showing teeth and saying "Yeeee" is to be captured third out of the six facial expressions to be captured.

[0044] FIG. 7 shows an example in which the evaluation results of an individual evaluation based on a facial image capturing a specific facial expression (here, an expression corresponding to "resting") out of six facial expressions to be evaluated are displayed on the user terminal 10.

[0045] In this figure, the individual evaluation results are displayed as individual assessments based on facial images capturing facial expressions corresponding to "resting," including measurements calculated from the facial images (angle difference, left-right difference in degree of opening, and left-right difference in distance), individual evaluation scores for each part of the face, and an overall evaluation score.

[0046] The angle differences calculated as measurements from the facial image are displayed in area D1. Area D1 shows that the angle difference of the eyebrows is △0.79°, the angle difference of the eyes is ▽0.73°, the angle difference of the cheeks is △0.03°, the angle difference of the nose is ▽0.22°, and the angle difference of the mouth is △1.47°. △ indicates a positive value, and ▽ indicates a negative value. The degree of "eye opening" as a measurement calculated from the facial image is shown in area D2. Area D2 shows that the degree of opening of the left eye is 11.77 and the degree of opening of the right eye is 11.26. Here, the degree of opening of the left and right eyes is displayed, but the difference between the degrees of opening may also be displayed as a measurement indicating symmetry. The degree of "mouth" opening as a measurement calculated from the facial image is shown in area D3. Area D3 shows that the degree of mouth opening for the left side is 0.26 and the degree of mouth opening for the right side is 0.39. Here, the degree of mouth opening for the left and right sides is displayed, but the difference between the degrees of mouth opening may also be displayed as a measurement indicating symmetry. The distance to the corners of the mouth as a measurement value calculated from the face image is shown in area D4. Area D4 shows that the distance from the central axis to the left corner of the mouth is 21.64, and the distance from the central axis to the right corner of the mouth is 22.91. Here, the distances to the feature points corresponding to the left and right corners of the mouth are displayed, but the difference in distance may also be displayed as a measurement value indicating symmetry.

[0047] In the area D5, a character string indicating the individual evaluation calculated from the facial image corresponding to which facial expression, here the character string "resting", is displayed as the evaluation result. Area D6 shows the overall evaluation score calculated based on the individual evaluation scores of the facial features of the eyebrows, eyes, nose, cheeks, and mouth, which is "100 points" in this case. Area D7 shows the individual evaluation scores for each of the facial features: eyebrows, eyes, nose, cheeks, and mouth, showing that the eyebrows are rated 100 points, the eyes are rated 97 points, the cheeks are rated 100 points, the nose is rated 100 points, and the mouth is rated 99 points.

[0048] Also shown in this figure is an operation button SB, which is a button that allows the user to select either "redo" or "record." When the "Redo" operation button is selected, the measurement values, individual evaluation scores, and overall evaluation score as the evaluation results displayed in Figure 7 are erased without being recorded, and a screen instructing the user to capture an image of the user's facial expression corresponding to "resting," such as the screen shown in Figure 4, is displayed on the user terminal 10. When the "Record" operation button is selected, the measurement values, individual evaluation scores, and overall evaluation scores displayed as the evaluation results in Figure 7 are stored in association with the date and time of the image capture. By storing the evaluation results, the user can check the details of the evaluation results at any time. Furthermore, if the left-right difference is continuously recorded, the user can refer to the history of the evaluation results to understand areas of improvement and realize the effects.

[0049] FIG. 8 shows an example in which the evaluation results of an individual evaluation based on a facial image capturing a specific facial expression (here, the "Yee" facial expression showing teeth) out of six facial expressions to be evaluated are displayed on the user terminal 10.

[0050] In this figure, the individual evaluation results are displayed as measurements calculated from the facial image (angle difference, difference in left / right opening, and difference in left / right distance) based on a facial image of a person showing their teeth and making a "Yee" expression, as well as individual evaluation scores for each part of the face and an overall evaluation score.

[0051] The angle differences calculated as measurements from the facial image are displayed in area D11. Area D1 shows that the angle difference of the eyebrows is △0.56°, the angle difference of the eyes is ▽0.51°, the angle difference of the cheeks is △0.04°, the angle difference of the nose is △0.27°, and the angle difference of the mouth is △1.34°. △ indicates a positive value, and ▽ indicates a negative value. The degree of "eye opening" as a measurement value calculated from the facial image is shown in area D12. Area D12 shows that the degree of opening of the left eye is 11.72 and the degree of opening of the right eye is 11.06. The "mouth" opening degrees as measured values ​​calculated from the facial image are shown in area D13. Area D13 shows that the left mouth opening degree is 11.25 and the right mouth opening degree is 11.63. The distances to the corners of the mouth as measured values ​​calculated from the face image are shown in area D14. Area D14 shows that the distance from the central axis to the left corner of the mouth is 28.74, and the distance from the central axis to the right corner of the mouth is 29.42.

[0052] In area D15, a character string indicating the individual evaluation calculated from the facial image corresponding to which facial expression is displayed as the evaluation result, in this case, the character string "showing teeth and saying 'Good'." Area D16 shows the overall evaluation score calculated based on the individual evaluation scores of the facial features of the eyebrows, eyes, nose, cheeks, and mouth, which is "100 points" in this case. Area D17 shows the individual evaluation scores for each of the facial features: eyebrows, eyes, nose, cheeks, and mouth, showing that the eyebrows are 100 points, the eyes are 100 points, the cheeks are 100 points, the nose is 100 points, and the mouth is 92 points.

[0053] 7. The operation button SB in this figure is the same as the operation button SB in FIG.

[0054] FIG. 9 shows an example in which the evaluation results of the overall evaluation based on the facial images in which each of the six facial expressions to be evaluated is captured are displayed on the user terminal 10.

[0055] In this figure, the evaluation results are displayed as an overall evaluation based on each image capturing each facial expression, including the measurement values ​​calculated from each facial image (angle difference, difference in degree of opening between the left and right, and difference in distance between the left and right), the individual evaluation scores for each part of the face, and the average of the overall evaluation score.

[0056] The angle difference as the average value of the measurement values ​​calculated from each face image is displayed in area D21. Area D21 shows that the average angle difference of the eyebrows is △1.06°, the average angle difference of the eyes is ▽0.53°, the average angle difference of the cheeks is ▽0.07°, the average angle difference of the nose is ▽0.18°, and the average angle difference of the mouth is △0.18°. △ indicates a positive value, and ▽ indicates a negative value. The average value of the measurements calculated from each face image, which indicates the degree of "eye opening," is shown in area D22. Area D12 shows that the average value of the degree of left eye opening is 12.29, and the average value of the degree of right eye opening is 11.57. The "mouth opening" as the average value of the measurement values ​​calculated from each face image is shown in area D23. Area D23 shows that the average value of the left mouth opening is 6.86 and the average value of the right mouth opening is 7.12. The distance to the corners of the mouth as the average of the measured values ​​calculated from each face image is shown in area D24. Area D14 shows that the average value of the distance from the central axis to the left corner of the mouth is 27.14, and the average value of the distance from the central axis to the right corner of the mouth is 28.03.

[0057] Area D26 shows the average overall evaluation score calculated based on the individual evaluation scores of the facial features of eyebrows, eyes, nose, cheeks, and mouth, which is "100 points" in this case. Area D27 shows the average individual evaluation scores for each of the facial features: eyebrows, eyes, nose, cheeks, and mouth, showing that the eyebrows are 89 points, the eyes are 100 points, the cheeks are 100 points, the nose is 100 points, and the mouth is 100 points.

[0058] 7. The operation button SB in this figure is the same as the operation button SB in FIG.

[0059] Here, the flow of processing according to this embodiment will be described with reference to Fig. 10. Fig. 10 is a sequence diagram showing the flow of processing performed by the evaluation server according to this embodiment. First, the evaluation server 20 starts recording the laterality (step S10). When the evaluation application is started on the user terminal 10 and the menu button MB indicating "Record Laterality" is selected, the evaluation server 20 starts recording the laterality. Next, the evaluation server 20 guides the user terminal 10 to capture an image of the specified facial expression to be captured (step S11). The evaluation server 20 displays an image capture window WG, a facial expression image HG, and an operation button SB on the user terminal 10 as shown in Fig. 4, and guides the user to capture an image of the facial expression corresponding to the facial expression image. The user operates the user terminal 10 according to the facial image displayed on the user terminal 10, and captures an image of their own facial expression corresponding to the facial image (step S12). When the captured facial image is sent from the user terminal 10 to the evaluation server 20, it is determined whether the captured facial image is appropriate (step S13). If the facial image is not appropriate, the process returns to step S11, and the user is prompted to capture another facial image. If the facial image is appropriate, the evaluation server 20 determines whether there are any facial expressions that have not yet been captured (step S14). If there are any facial expressions that have not yet been captured, the process returns to step S11, and the user is prompted to capture a facial image corresponding to the next facial expression. On the other hand, if there are no facial expressions that have not yet been captured, the evaluation server 20 evaluates the left-right difference in the facial image (step S15). The evaluation server 20 calculates measurement values ​​(angle difference, degree of opening, and distance), individual evaluation scores, and overall evaluation scores for each facial image. The evaluation server 20 also calculates the average of the measurement values ​​(angle difference, openness, and distance) for each face image, the individual evaluation score, and the overall evaluation score as an overall evaluation. The evaluation server 20 outputs the evaluation result to the user terminal 10 (step S16).

[0060] As described above, the evaluation system 1 according to the embodiment is an example of an evaluation device. The evaluation system 1 includes an imaging control unit 23, a communication unit 21, and a score calculation unit 26. The imaging control unit 23 guides the user terminal 10 to capture images of the user's facial expressions corresponding to each of a plurality of facial expressions. The communication unit 21 is an example of an image acquisition unit, and acquires each of the user's facial images captured in response to the guidance of the imaging control unit 23 from the user terminal 10. The score calculation unit 26 calculates an index value indicating the degree of laterality in the face based on the facial images acquired by the communication unit 21 (image acquisition unit). The index value here is, for example, a measurement value for each facial image, an individual evaluation score, and a comprehensive evaluation score, an average value of the measurement values ​​for each facial image, an average value of the individual evaluation scores, and an average value of the comprehensive evaluation score. As a result, the evaluation system 1 according to the embodiment can calculate an index value indicating the degree of laterality in the face based on the images, thereby enabling quantitative and easy evaluation of facial symmetry.

[0061] Furthermore, in the evaluation system 1 according to the embodiment, the score calculation unit 26 extracts facial feature points in a face image, detects the central axis of the face based on the positions of the extracted feature points, and calculates an index value based on the detected central axis. As a result, the evaluation system 1 according to the embodiment can divide the face into left and right halves based on the central axis, and can easily evaluate the left-right symmetry quantitatively.

[0062] Furthermore, in the evaluation system 1 according to the embodiment, the score calculation unit 26 measures the angle at each feature point corresponding to each vertex of the shape formed by connecting multiple feature points extracted corresponding to the contours of any one of the facial features, i.e., eyebrows, eyes, nose, cheeks, and mouth, and calculates the difference in angle at each corresponding feature point on the left and right sides relative to the central axis as an index value. As a result, the evaluation system 1 according to the embodiment can calculate the difference in shape between the left and right sides of any one of the facial features, i.e., eyebrows, eyes, nose, cheeks, and mouth, as an angle difference, thereby enabling quantitative and easy evaluation of facial symmetry.

[0063] Furthermore, in the evaluation system 1 according to the embodiment, the score calculation unit 26 extracts two feature points corresponding to any one of the facial features, i.e., the eyebrows, eyes, nose, cheeks, or mouth, that are aligned in the left-right direction relative to a central axis, and calculates an index value based on the difference in distance from each of the two extracted feature points to the central axis. For example, the evaluation system 1 calculates the distance from the central axis to feature points corresponding to the left and right corners of the mouth, and calculates the difference in these distances as an index value. As a result, the evaluation system 1 according to the embodiment can calculate the difference in distance from two feature points that are aligned in the left-right direction relative to the central axis as an index value, thereby enabling quantitative and easy evaluation of facial symmetry.

[0064] Furthermore, in the evaluation system 1 according to the embodiment, the score calculation unit 26 calculates an index value based on the difference between the left and right sides in the degree of opening of either the eyes or the mouth as a facial feature, as a feature point. As a result, the evaluation system 1 according to the embodiment can calculate the difference between the left and right sides in the degree of opening of the eyes or the mouth as an index value, and can quantitatively and easily evaluate facial symmetry.

[0065] Furthermore, in the evaluation system 1 according to the embodiment, the score calculation unit 26 calculates an overall index value using a value obtained by multiplying the index values ​​corresponding to each of the facial features (eyebrows, eyes, nose, cheeks, and mouth) by a weighting coefficient for each facial feature. An example of the overall index value here is an "overall evaluation score." As a result, the evaluation system 1 according to the embodiment can evaluate overall symmetry based on various facial expressions, and can quantitatively and easily evaluate facial symmetry.

[0066] Furthermore, in the evaluation system 1 according to the embodiment, the imaging control unit 23 displays an expression image showing an expression that the user wants to capture on the user terminal 10, and guides the user to capture an expression corresponding to the expression image. In this way, the evaluation system 1 according to the embodiment can guide the user by showing the expression that the user wants to capture in a diagram, and can support the user so that the user can easily create the expression, allowing for smooth imaging.

[0067] (Modification 1 of the embodiment) Here, a first modification of the embodiment will be described. In this modification, high-speed imaging is performed, and a plurality of (for example, 10) face images are captured consecutively at approximately the same imaging timing.

[0068] Generally, it is difficult for ordinary people without special training to maintain a specific facial expression, and even if they intend to maintain a specific facial expression, the facial expression often changes subtly. For this reason, when capturing a specific facial expression, the positions of feature points obtained from the facial image often vary depending on the timing of the capture. Variation in the positions of feature points can cause a decrease in the accuracy of evaluating facial symmetry. For example, even if there is no significant change in the user's facial symmetry between the previous and current captures, the evaluation results may differ significantly.

[0069] To address this issue, the imaging control unit 23 performs high-speed imaging at the timing when the user performs an imaging operation, and successively captures multiple (e.g., 10) facial images. For example, 10 facial images can be captured by imaging for approximately 0.33 seconds at a speed of capturing 30 images per second (30 fps).

[0070] The feature point detection unit 25 extracts feature points from each of multiple face images captured by high-speed imaging, and determines the average value of the positions of the extracted feature points as the position of the feature points in the face images captured at that imaging timing. The score calculation unit 26 calculates measurement values ​​(angle difference, degree of opening, and distance), individual evaluation scores, and a comprehensive evaluation score based on the positions of the feature points detected by the feature point detection unit 25. That is, the score calculation unit 26 calculates measurement values ​​and the like as index values ​​for evaluating the left-right symmetry of the face based on the average values ​​of the positions of the feature points extracted from each of the multiple face images obtained by high-speed imaging.

[0071] As described above, in the evaluation system 1 according to the first modification of the embodiment, the imaging control unit 23 causes the user terminal 10 to perform high-speed imaging at the imaging timing when the user performs an imaging operation, thereby capturing multiple facial images continuously. The score calculation unit 26 calculates a measurement value or the like as an index value based on the average value of the positions of feature points extracted from each of the multiple facial images captured at the imaging timing by the user. As a result, the evaluation system 1 according to the first modification of the embodiment can suppress variation in the positions of feature points detected from facial images even when the user's facial expression changes slightly, thereby enabling accurate evaluation.

[0072] (Modification 2 of the embodiment) Here, a second modification of the embodiment will be described. In this modification, guidance is provided so that a facial image can be captured appropriately. 11 and 12 are diagrams for explaining the processing performed by the evaluation system according to the second modification of the embodiment. Illustrated in Fig. 11 and 12 are examples of screens displayed on the user terminal 10 via the evaluation app.

[0073] 11 shows an example of a screen displayed on the user terminal 10 before displaying the imaging window WG for capturing a face image in this modified example. In this figure, "Notes when capturing images" include capturing a front view of the face, capturing an image while aligning with a cross guide image (see guide images GD1 and GD2 in FIG. 12), and capturing an image in a well-lit location. In addition, as an example where a front view of the face can be captured, a "circle" is shown in a diagram showing a positional relationship where the subject (face) and the camera (user terminal 10) are horizontal. As an example where a front view of the face cannot be captured, a "cross" is shown in a diagram showing a positional relationship where the subject (face) and the camera (user terminal 10) are not horizontal.

[0074] An example of an imaging window WG in this modification is shown in Fig. 12. In this figure, the imaging window WG indicates an area (face area) for capturing an image of the user's face using a dotted line or the like, and further includes a cross guide image that divides the face area into upper, lower, left, and right halves. More specifically, a guide image GD1 indicating a vertical line that divides the face area into equal halves left and right, and a guide image GD2 indicating a horizontal line that divides the face area into equal halves top and bottom, are provided.

[0075] The imaging control unit 23 causes the user terminal 10 to display a screen such as that shown in FIG. 11 before displaying the imaging window WG, and then causes the user terminal 10 to display the imaging window WG such as that shown in FIG.

[0076] 12 illustrates an example in which a cross-shaped guide image is provided to divide the face area into four, but the present invention is not limited to this. In this modification, it is sufficient to display an image that can at least provide guidance (guidance) for capturing an image of the face from the front, that is, an image that guides the position and orientation of the face as a subject so that the face is captured from the front. For example, a guide image that divides the face area into two or three parts may be provided, or a guide image that divides the face area into five or more parts may be provided. Furthermore, multiple guide images showing vertical lines only may be provided, or multiple guide images showing horizontal lines only may be provided.

[0077] As described above, in the evaluation system 1 according to the second modification of the embodiment, the imaging control unit 23 displays a guide image on the user terminal 10, which guides at least one of the position and orientation of the face as a subject. This allows the user to adjust the position or orientation of the face in accordance with the guide image when capturing a facial image, and enables capturing an appropriate facial image in which the face is captured from the front. Capturing an appropriate facial image allows for accurate evaluation.

[0078] The above describes an embodiment of the present invention. Note that some or all of the functions of the evaluation system 1 and evaluation server 20 in the above-described embodiment may be implemented by a computer. In this case, a program for implementing these functions may be recorded on a computer-readable recording medium, and the program may be loaded into a computer system and executed. Note that the term "computer system" as used herein includes hardware such as an operating system and peripheral devices. Furthermore, the term "computer-readable recording medium" refers to portable media such as flexible disks, optical magnetic disks, ROMs, and CD-ROMs, as well as storage devices such as hard disks built into a computer system. Furthermore, the term "computer-readable recording medium" may also include devices that dynamically store programs for a short period of time, such as communication lines used when transmitting programs via networks such as the Internet or telephone lines, or devices that store programs for a fixed period of time, such as volatile memory within a computer system serving as a server or client. The program may be designed to implement some of the functions described above, or may be capable of implementing the functions in combination with a program already stored in the computer system, or may be implemented using a programmable logic device such as an FPGA (Field Programmable Gate Array).

[0079] The embodiments of the present invention have been described in detail above with reference to the drawings, but the specific configuration is not limited to that described above, and various design changes can be made within the scope of the gist of the present invention. [Explanation of symbols]

[0080] 1. Rating System 10 User terminal 11 Communications Department 12 Storage section 13 Control Unit 14 Imaging unit 15 Display 16 Operation input section 20 Evaluation Server 21 Communications Department 22 Memory section 23 Imaging control unit 24 Image judgment unit 25 Feature point detection unit 26 Score calculation section 27 Output section

Claims

1. an imaging control unit that guides the user terminal to capture images of the user's facial expressions corresponding to each of a plurality of facial expressions; an image acquisition unit that acquires, from the user terminal, each of the facial images of the user captured in response to guidance from the imaging control unit; a score calculation unit that calculates an index value indicating the degree of laterality of the face for each of the face images acquired by the image acquisition unit; An evaluation system comprising:

2. the imaging control unit causes the user terminal to perform high-speed imaging at an imaging timing when an imaging operation is performed by the user, thereby successively capturing a plurality of the face images; the score calculation unit calculates the index value based on an average value of positions of facial feature points extracted from each of the plurality of face images captured at the imaging timing; The evaluation system of claim 1 .

3. the score calculation unit uses a plurality of feature points extracted corresponding to the contours of facial features to measure angles at each vertex of a shape formed by connecting the feature points, and calculates, as the index value, a difference between angles at two corresponding feature points on the left and right sides of the face; The evaluation system of claim 1 .

4. the score calculation unit calculates the index value based on a central axis of the face detected based on positions of facial feature points extracted from the face image. The evaluation system of claim 1 .

5. the score calculation unit calculates the index value based on a difference in distance from each of two corresponding feature points on the left and right sides of the face to the central axis using feature points extracted corresponding to facial features; The evaluation system according to claim 4 .

6. the score calculation unit calculates the index value based on a difference in the degree of opening of either the eyes or the mouth between the left and right sides using feature points extracted corresponding to either the eyes or the mouth as facial features; The evaluation system of claim 1 .

7. the score calculation unit calculates the overall index value for the face by using a value obtained by multiplying the index value corresponding to each of a plurality of facial features by a weighting coefficient for each facial feature; The evaluation system of claim 1 .

8. The features include at least one of eyebrows, eyes, nose, cheeks, and mouth. The evaluation system according to claim 3, claim 5, or claim 7.

9. An evaluation method performed by a computer in an evaluation system, comprising: the imaging control unit instructs the user terminal to capture images of the user's facial expressions corresponding to each of the plurality of facial expressions; an image acquisition unit acquires, from the user terminal, each of the facial images of the user captured in response to the guidance of the imaging control unit; a score calculation unit calculates an index value indicating the degree of laterality of the face for each of the face images acquired by the image acquisition unit; Evaluation method.

10. The evaluation system computer prompting the user terminal to capture images of the user's facial expressions corresponding to each of the plurality of facial expressions; acquiring, from the user terminal, each of the facial images of the user captured in response to the guidance; calculating an index value indicating a degree of laterality of the face for each of the acquired face images; program.

Images