Information processing device, information processing method, and program

Abstract

An information processing device (2) is provided with: a moving image acquisition unit (231) that acquires a moving image in which a user's face is captured; a detection unit (232) that detects, on the basis of the moving image acquired by the moving image acquisition unit (231), one or more expression patterns of the user and expression intensities indicating the degree of the intensity of the expression in each of the one or more expression patterns; and an output unit (235) that outputs beauty information relating to the beauty of the user's face on the basis of the detection result of the detection unit (232). The information processing device (2) further comprises an estimation unit (234) that estimates the risk of occurrence of wrinkles on the face of the user on the basis of the frequency of occurrence and expression intensity of each of the one or more expression patterns detected by the detection unit (232), and an output unit (235) outputs cosmetic information corresponding to the risk of occurrence estimated by the estimation unit (234).

Description

Technical Field

[0001] The present invention relates to an information processing apparatus, an information processing method, and a program. Background Art

[0002] Conventionally, a system that provides information related to the beauty of a user's face (facial) has been known. In Patent Document 1, a technique for providing information corresponding to the skin type of a user determined from an image of the user's skin is disclosed.

[0003] Prior Art Documents

[0004] Patent Documents

[0005] Patent Document 1: Japanese Unexamined Patent Application Publication No. 2018-160230 Summary of the Invention

[0006] Problems to be Solved by the Invention

[0007] However, in the above technique, there is a problem that only information corresponding to the current skin state of the user can be provided.

[0008] Therefore, the present invention has been completed in view of these points, and an object thereof is to provide information that takes into account possible future skin changes of the user.

[0009] Means for Solving the Problems

[0010] An information processing apparatus according to a first aspect of the present invention includes: a moving image acquisition unit that acquires a moving image of a user's face; a detection unit that detects one or more expression patterns of the user and an expression intensity indicating the intensity of the expression represented by each of the one or more expression patterns based on the moving image acquired by the moving image acquisition unit; and an output unit that outputs beauty information related to the beauty of the user's face based on the detection result of the detection unit.

[0011] The information processing apparatus may further include an estimation unit that estimates a generation risk of wrinkles occurring on the user's face based on the appearance frequency of each of the one or more expression patterns detected by the detection unit and the expression intensity, and the output unit may output the beauty information corresponding to the generation risk estimated by the estimation unit.

[0012] The detection unit may detect the expression pattern and the expression intensity at a predetermined interval, and the information processing apparatus may further include: a calculation unit that calculates the sum of the expression intensities detected at the predetermined interval; and an estimation unit that estimates a generation risk of wrinkles occurring on the user's face based on a predetermined threshold and the sum.

[0013] The information processing apparatus may also include a storage unit that stores one or more expression patterns in association with each part of the face. The calculation unit may also calculate, for each part, the sum of the expression intensities of the expression patterns detected by the detection unit, that is, the one or more expression patterns associated with that part. The estimation unit may also estimate the generation risk for each part based on the threshold value corresponding to that part and the sum corresponding to that part.

[0014] The output unit may also output, as the beauty information, an index indicating the generation risk of wrinkles on the user's face.

[0015] The output unit may also output, as the beauty information, information about a product for coping with the generation risk of wrinkles on the user's face.

[0016] The product may be at least any one of cosmetics, beauty equipment, and food.

[0017] The information processing apparatus may also include a history acquisition unit that acquires the purchase history of the product by the user. The output unit may also refer to the purchase history and output, after a predetermined period has elapsed since the date when the user purchased the product, information for requesting transmission of the moving image.

[0018] The output unit may also output, as the beauty information, information about a beauty method for coping with the generation risk of wrinkles on the user's face.

[0019] The information processing apparatus may also include a history acquisition unit that acquires the implementation history of the beauty method by the user. The output unit may also refer to the implementation history and output, after a predetermined period has elapsed since the date when the user implemented the beauty method, information for requesting transmission of the moving image.

[0020] The output unit may also output, as the beauty information, at least any one of the scene of the moving image when an expression pattern whose appearance frequency is equal to or higher than a predetermined threshold value among the one or more expression patterns detected by the detection unit is detected, and the scene of the moving image when an expression pattern whose expression intensity is equal to or higher than a predetermined threshold value among the one or more expression patterns detected by the detection unit is detected.

[0021] The output unit may also output, as the beauty information, a result obtained by comparing the risk of wrinkle formation of the user's face estimated based on the first moving image which is the moving image with the risk of wrinkle formation estimated based on a second moving image captured before the first moving image was captured.

[0022] The output unit may also output a question to the user before the moving image acquisition unit acquires the moving image, and the moving image acquisition unit may acquire the moving image after the output unit outputs the question to the user.

[0023] The information processing method according to the second aspect of the present invention includes the following steps executed by a computer: a step of acquiring a moving image of the user's face; a step of detecting, based on the acquired moving image, one or more expression patterns of the user and the degree of expression intensity indicating the intensity of the expression for each of the one or more expression patterns; and a step of outputting, based on the detection result, beauty information related to the beauty of the user's face.

[0024] The program according to the third aspect of the present invention causes a computer to function as the following units: a moving image acquisition unit that acquires a moving image of the user's face; a detection unit that detects, based on the moving image acquired by the moving image acquisition unit, one or more expression patterns of the user and the degree of expression intensity indicating the intensity of the expression for each of the one or more expression patterns; and an output unit that outputs, based on the detection result of the detection unit, beauty information related to the beauty of the user's face.

[0025] Advantageous Effects of the Invention

[0026] According to the present invention, there is an effect that information considering possible future skin changes of the user can be provided. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 is a diagram for explaining the outline of the information processing system.

[0028] Figure 2 is a diagram for explaining the outline of the information processing system.

[0029] Figure 3 is a diagram showing the configuration of the information processing apparatus.

[0030] Figure 4 is a diagram showing an example of the structure of the correspondence table.

[0031] Figure 5 is a diagram showing an example of the structure of the beauty management database.

[0032] Figure 6This is a diagram schematically showing the detection results obtained by the detection unit.

[0033] Figure 7 This is a diagram schematically showing a display screen on which beauty information is displayed.

[0034] Figure 8 This is a flowchart showing the processing flow of the information processing device. Detailed implementation mode

[0035] [Overview of information processing system S]

[0036] Figure 1 And Figure 2 This is a diagram for explaining the overview of the information processing system S. The information processing system S is a system for providing an information processing service to a user. The information processing service is a service for providing beauty information to the user. The beauty information is information related to the beauty of the user's face, such as beauty-related products (e.g., cosmetics, beauty equipment, or foods, etc.) or beauty methods, etc.

[0037] As emotions (feelings) change, users will make various expressions. Generally speaking, for each user, the pattern of the expression corresponding to each emotion is determined, and when a user shows a certain emotion on the face, the user will make an expression in the pattern corresponding to that emotion. On the user's skin, when the user makes an expression in the pattern corresponding to a certain emotion, lines (wrinkles) will be generated along with the habit of that expression, and when returning to a serious face (expressionless face), the wrinkles generated along with the habit of that expression will disappear. However, if the user repeatedly makes the same pattern of expression, then even when returning to a serious face, a part of the wrinkles temporarily generated on the user's skin along with the habit of the expression will remain, and the remaining wrinkles may become static wrinkles.

[0038] Figure 2 (a) of this shows the user's normal expression (e.g., serious face), Figure 2 (b) of this shows the expression when the user smiles. On the user's face, Figure 2 there are no wrinkles in the normal expression shown in (a) of this, and if the user repeatedly makes the expression shown in (b) of this every time he / she smiles, then in the future, the wrinkles at the corners of the eyes and the nasolabial folds will not go back, and the wrinkles may become static wrinkles. In addition, on the user's face, if Figure 2 the expression shown in (b) of this is made repeatedly, then in the future, the wrinkles at the corners of the eyes and the nasolabial folds will not go back, and the wrinkles may become static wrinkles. In addition, on the user's face, if Figure 2As the degree of the expression as shown in (b) becomes stronger, the wrinkles at the corners of the eyes and the nasolabial folds become deeper, and the wrinkles are likely to become static wrinkles. In addition, the user not only consciously makes a certain expression as described above, but sometimes also unconsciously makes a certain expression. Due to repeatedly making a certain expression unconsciously, it is also possible for the wrinkles to remain on the user's face permanently. Therefore, in the information processing service, it is desirable to provide the user with beauty information that takes into account possible skin changes such as wrinkles that the user may develop in the future.

[0039] Then, the information processing system S provides the user with beauty information that takes into account possible skin changes that may occur on the user's face in the future. Figure 1 The information processing system S shown has a user terminal 1 and an information processing device 2.

[0040] The user terminal 1 is a terminal used by the user, such as a smart phone, a tablet terminal, or a personal computer. The user terminal 1 has: an operation unit such as a touch panel and a keyboard for accepting the user's operations; a display unit such as a liquid crystal display for displaying information; a photographing unit such as a camera for photographing an image; a voice output unit such as a speaker for outputting sound; and a voice input unit such as a microphone for collecting sound. The user terminal 1 can communicate with the information processing device 2 via a network.

[0041] The information processing device 2 is a device that manages the information processing service, such as a server. The information processing device 2 can communicate with the user terminal 1 via a network.

[0042] Hereinafter, Figure 1 A description will be given of the processing performed by the information processing system S to provide information for maintaining or improving the beautiful appearance of the user's face.

[0043] First, when the user photographs a dynamic image of the user's face with the user terminal 1, the user terminal 1 sends the photographed dynamic image to the information processing device 2 ( Figure 1 in (1)). Based on the dynamic image obtained from the user terminal 1, the information processing device 2 detects one or more expression patterns of the user and the expression intensity of each of the one or more expression patterns ( Figure 1 in (2)).

[0044] The expression pattern is, for example, a pattern such as "raising the inner part of the eyebrows" or "lifting the cheeks", and various expression patterns are predetermined. The expression pattern is, for example, an action unit (AU: Action Unit) defined in the Facial Action Coding System (FACS). The expression intensity represents the degree of the intensity of the expression, for example, a numerical value representing the intensity of the expression change of the user's face from the normal expression to the predetermined expression pattern.

[0045] Then, based on the detection result, the information processing device 2 sends beauty information to the user terminal 1 ( Figure 1 in (3) of). After that, the user terminal 1 causes the display unit to display the beauty information.

[0046] In this way, the information processing system S can provide information considering skin changes that the user may develop in the future due to conscious or unconscious expressions. For example, the user can, based on the provided information, delay the progression of skin state changes or extend the period during which the current skin state of the user is maintained. Additionally, for example, by notifying the user's expression habits as information considering future skin changes that the user may develop, the information processing system S can enable the user to predict wrinkles that may develop in the future (wrinkles that may remain permanently on the user's skin).

[0047] Next, the configuration of the information processing device 2 will be described.

[0048] [Configuration of Information Processing Device 2]

[0049] Figure 3 FIG. is a diagram showing the configuration of the information processing device 2. The information processing device 2 includes a communication unit 21, a storage unit 22, and a control unit 23. The control unit 23 includes a moving image acquisition unit 231, a detection unit 232, a calculation unit 233, an estimation unit 234, an output unit 235, and a history acquisition unit 236.

[0050] The communication unit 21 is an interface for connecting to a network and is configured to include, for example, a communication controller. The storage unit 22 is a storage medium such as a ROM (Read Only Memory), a RAM (Random Access Memory), and a hard disk. The storage unit 22 stores programs executed by the control unit 23.

[0051] The storage unit 22 stores a correspondence table that associates expression patterns with parts of the face. Figure 4 FIG. is an example diagram showing the structure of the correspondence table. In Figure 4 the example shown, the expression patterns are associated with the presence or absence of each part of the face (forehead, eyebrows, eyes, nose, mouth, etc.). For example, in expression pattern "1", "forehead" among multiple parts of the face is associated.

[0052] In addition, the storage unit 22 stores a beauty management database that manages information related to beauty recommended for the user. Figure 5 FIG. is an example diagram showing the structure of the beauty management database. In Figure 5 the example shown, the beauty management database stores by associating parts of the face, metrics, products, and beauty methods.

[0053] An index is a value indicating the risk of wrinkle formation (hereinafter referred to as "wrinkle formation risk") in or around a part of a user's face. A product is recommended to the user according to the degree of the wrinkle formation risk of a part of the face or the like to reduce the wrinkle formation risk. The product is, for example, at least any one of cosmetics, beauty devices, and foods. A beauty method is an implementation method such as massage recommended to the user according to the degree of the wrinkle formation risk of a part of the face or the like to reduce the wrinkle formation risk. The beauty method is, for example, at least any one of an image or a moving image introducing an implementation method such as massage performed by the user himself / herself and information of a store such as a beauty salon where massage or the like is performed.

[0054] Return to Figure 3 The control unit 23 is, for example, a CPU (Central Processing Unit). The control unit 23 functions as a moving image acquisition unit 231, a detection unit 232, a calculation unit 233, an estimation unit 234, an output unit 235, and a history acquisition unit 236 by executing a program stored in the storage unit 22.

[0055] The moving image acquisition unit 231 acquires a moving image obtained by photographing a user's face from the user terminal 1. The moving image acquisition unit 231 may also acquire identification information (ID: identifier) for identifying the user.

[0056] The moving image acquisition unit 231 acquires, for example, a moving image in which the user is photographed during a conversation. Specifically, first, before the moving image acquisition unit 231 acquires the moving image, the output unit 235 outputs a question to the user. Then, the moving image acquisition unit 231 acquires the moving image after the output unit 235 outputs the question to the user.

[0057] For example, when the user accesses the website of the information processing service using the user terminal 1, the output unit 235 causes the user terminal 1 to display the display screen of the information processing service. A photographing button for causing the photographing unit of the user terminal 1 to photograph a moving image is provided in the display screen. When the user presses the photographing button in the display screen displayed on the user terminal 1, the photographing unit of the user terminal 1 starts photographing the moving image.

[0058] In addition, when the shooting button is pressed, the output unit 235 causes the user terminal 1 to output, by voice, a question predetermined for the user. For example, the shooting unit of the user terminal 1 shoots the expression of the user when listening to the question and the expression of the user when answering the question, etc., during the period from when the shooting button is pressed until a predetermined period (for example, several minutes) has elapsed. Further, when the predetermined period has elapsed since the shooting button was pressed, the user terminal 1 transmits the captured moving image to the information processing device 2. By doing so, the information processing device 2 can capture the change in the face of the user.

[0059] In the above, an example of the display screen of the information processing service being the display screen displayed according to the accessed website has been described, but it is not limited thereto. For example, the display screen of the information processing service may also be the display screen displayed when a dedicated application (hereinafter referred to as "dedicated APP") for providing the information processing service is started. For example, when a dedicated APP is installed in the user terminal 1 and the user performs an operation to start the dedicated APP on the user terminal 1, the user terminal 1 causes the display unit to display the display screen of the information processing service.

[0060] The detection unit 232 detects one or more expression patterns of the user and the expression intensity of each of the one or more expression patterns based on the moving image acquired by the moving image acquisition unit 231. Specifically, the detection unit 232 detects one or more expression patterns and the expression intensity of each of the one or more patterns at a predetermined interval in the moving image acquired by the moving image acquisition unit 231.

[0061] Figure 6 is a diagram schematically showing the detection result obtained by the detection unit 232. In Figure 6 the example shown, for each frame of the moving image, values indicating the expression intensity of each of a plurality of expression patterns are shown. For example, when a certain expression pattern is detected, the expression intensity is represented by a value greater than 0, and when a certain expression pattern is not detected, it is represented by 0. In this way, the detection unit 232 detects the expression intensity of each of a plurality of predetermined expression patterns for each frame of the moving image.

[0062] The detection unit 232 may also use an external service to detect the expression pattern of the user and the expression intensity of the expression pattern. For example, the external service is a service that analyzes the face of the user reflected in the moving image and provides a detection result obtained by detecting the expression intensity of each of a plurality of predetermined expression patterns at a predetermined interval (for example, the frame of the moving image). In this case, the detection unit 232 obtains the detection result output by the external service by sending the moving image to the external service.

[0063] The output unit 235 outputs beauty information related to the beauty of the user's face based on the detection result obtained by the detection unit 232. Specifically, the output unit 235 causes the user terminal 1 to display the beauty information by sending the beauty information to the user terminal 1. For example, the information processing device 2 may also have a display unit (not shown), and the output unit 235 may cause the display unit to display the beauty information.

[0064] The output unit 235 outputs at least any one of the information indicating the product and the information indicating the beauty method as beauty information. The output unit 235 outputs, for example, beauty information corresponding to the risk of wrinkle formation based on the detection result obtained by the detection unit 232. Specifically, the information processing device 2 outputs beauty information corresponding to the risk of wrinkle formation by performing the following two steps.

[0065] As a first step, the estimation unit 234 estimates the risk of future change in the user's skin based on the detection result detected by the detection unit 232. Specifically, the estimation unit 234 estimates the risk of wrinkle formation based on the appearance frequency of each of the one or more detected expression patterns detected by the detection unit 232 and the expression intensity of each of the one or more detected expression patterns detected by the detection unit 232.

[0066] More specifically, the estimation unit 234 classifies the risk of wrinkle formation into indexes based on the appearance frequency of each of the one or more detected expression patterns detected by the detection unit 232 and the expression intensity of each of the one or more detected expression patterns detected by the detection unit 232, thereby estimating the risk of wrinkle formation.

[0067] For example, the indexes indicating the risk of wrinkle formation include "low" and "high". In this case, the lower the appearance frequency of the expression pattern, or the weaker the expression intensity of the expression pattern (the smaller the value indicating the expression intensity), the estimation unit 234 classifies the risk of wrinkle formation as the index "low". In addition, the higher the appearance frequency of the expression pattern, or the stronger the expression intensity of the expression pattern (the larger the value indicating the expression intensity), the estimation unit 234 classifies the risk of wrinkle formation as the index "high".

[0068] The estimation unit 234 may also estimate the risk of wrinkle formation using a predetermined threshold value. Specifically, first, the calculation unit 233 calculates the sum of the expression intensities detected by the detection unit 232 at predetermined intervals. The calculation unit 233 calculates, for example, the sum of the expression intensities of each expression pattern detected by the detection unit 232 for each frame of the moving image.

[0069] The estimation unit 234 estimates the risk of wrinkle formation based on the sum of the expression intensities calculated by the calculation unit 233 (hereinafter referred to as the "sum value") and a sum threshold that is a predetermined threshold. For example, when the sum value is less than the sum threshold, the estimation unit 234 classifies the risk of wrinkle formation as the index "low", and when the sum value is equal to or greater than the sum threshold, the estimation unit 234 classifies the risk of wrinkle formation as the index "high".

[0070] The estimation unit 234 may also use a plurality of sum thresholds to estimate the risk of wrinkle formation. For example, in the information processing apparatus 2, a plurality of sum thresholds (for example, two thresholds, a first sum threshold and a second threshold greater than the first sum threshold) for classifying into any one of "low", "medium", and "high" included in the index are determined.

[0071] In this case, when the sum value is less than the first sum threshold, the estimation unit 234 classifies the risk of wrinkle formation as the index "low". Further, when the sum value is equal to or greater than the first sum threshold and less than the second sum threshold, the estimation unit 234 classifies the risk of wrinkle formation as the index "medium". Further, when the sum value is equal to or greater than the second sum threshold, the estimation unit 234 classifies the risk of wrinkle formation as the index "high".

[0072] As a second step, the output unit 235 outputs beauty information corresponding to the risk of wrinkle formation estimated by the estimation unit 234. Specifically, the output unit 235 outputs at least any one of information on products for coping with the risk of wrinkle formation on the user's face and information on beauty methods for coping with the risk of wrinkle formation on the user's face as beauty information corresponding to the estimated risk of wrinkle formation.

[0073] For example, when the risk of wrinkle formation is classified as the index "high", the output unit 235 outputs information on at least any one of products and beauty methods associated with the index "high" in the beauty management database. By doing so, the information processing apparatus 2 can provide information for delaying the progress of wrinkles that may occur on the user's face in the future.

[0074] The output unit 235 may also output beauty information corresponding to at least any one of multiple parts of the face. For example, in the information processing device 2, a total threshold is determined for each part of the face. In this case, first, the calculation unit 233 refers to the correspondence table stored in the storage unit 22 and calculates, for each part of the face, the sum of the expression intensities of each of the expression patterns detected by the detection unit 232, that is, one or more expression patterns associated with that part. For example, in the correspondence table, if the part "forehead" of the face is associated with the expression patterns "1" and "2", the calculation unit 233 calculates the sum of the expression intensities of the face part "forehead" by adding the numerical values representing the expression intensities of the expression patterns "1" and "2" detected by the detection unit 232 at predetermined intervals.

[0075] The estimation unit 234 estimates the risk of wrinkle formation for each part of the face based on one or more total thresholds corresponding to that part and the total value corresponding to that part. Moreover, the output unit 235 outputs beauty information corresponding to at least any one of multiple parts of the face based on the risk of wrinkle formation corresponding to each part.

[0076] The output unit 235 outputs, for example, beauty information corresponding to multiple parts of the face respectively. The output unit 235 may also output beauty information corresponding to a part of the multiple parts of the face with a relatively high risk of wrinkle formation. For example, as a part with a relatively high risk of wrinkle formation, the output unit 235 outputs beauty information corresponding to the part of the face where the index among the indexes corresponding to multiple parts of the face is above the index threshold (for example, the index "medium"). As a part with a relatively high risk of wrinkle formation, the output unit 235 may also output beauty information corresponding to the part of the face where the index with the highest risk of wrinkle formation (for example, the index "high") among the indexes corresponding to multiple parts of the face. In this way, the information processing device 2 can provide information for delaying the progression of wrinkles that may occur in parts with a high risk of wrinkle formation in the future.

[0077] In the above, an example is described in which the output unit 235 outputs either the information indicating the product or the information indicating the beauty method as beauty information, but it is not limited to this. For example, the output unit 235 may also output the index classified by the estimation unit 234 as beauty information.

[0078] The output unit 235 may output the indexes of all parts of the face or may output the indexes of a part of the parts of the face. In this way, the information processing device 2 can enable the user to recognize how high the risk of wrinkles occurring in the user's face in the future is.

[0079] The output unit 235 may also output, as beauty information, at least any one of the scenes (frames) of the moving image when an expression pattern with a relatively high occurrence frequency is detected and the scenes of the moving image when an expression intensity relatively high in the expression pattern is detected. Specifically, the output unit 235 outputs, as beauty information, at least any one of the scenes of the moving image when an expression pattern with an occurrence frequency equal to or higher than a predetermined frequency threshold among one or more expression patterns detected by the detection unit 232 is detected, and the scenes of the moving image when an expression pattern with an expression intensity equal to or higher than a predetermined intensity threshold among one or more expression patterns detected by the detection unit 232 is detected.

[0080] The output unit 235 may also output, as beauty information, at least any one of the scenes of the moving image when an expression pattern with a relatively high occurrence frequency (for example, the expression pattern with the highest occurrence frequency) among one or more expression patterns detected by the detection unit 232 is detected, and the scenes of the moving image when an expression pattern with a relatively strong expression intensity (for example, the expression pattern with the strongest expression intensity) among one or more expression patterns detected by the detection unit 232 is detected. By doing so, the information processing device 2 can enable the user to recognize the user's expression habits. As a result, the user can recognize which expressions will increase the risk of wrinkle formation. In addition, the user can predict the wrinkles that may occur in the future based on their own expression habits.

[0081] The output unit 235 may also output, as beauty information, the result obtained by comparing the wrinkle formation risk estimated this time with the wrinkle formation risk estimated last time. Specifically, the output unit 235 outputs, as beauty information, the comparison result obtained by comparing the wrinkle formation risk estimated based on the first moving image (hereinafter referred to as the "first wrinkle formation risk") with the wrinkle formation risk estimated based on the second moving image (hereinafter referred to as the "second wrinkle formation risk"). The first moving image is the moving image captured this time. The second moving image is the moving image captured last time, specifically, the moving image captured before the first moving image.

[0082] For example, in the storage unit 22, the user ID is stored in association with information indicating the second wrinkle generation risk estimated by the past estimation unit 234 based on the second moving image of the user. In this case, the output unit 235 outputs a comparison result obtained by comparing the first wrinkle generation risk estimated based on the first moving image of the user acquired by the moving image acquisition unit 231 with the second wrinkle generation risk stored in association with the user ID in the storage unit 22. For example, when the index of the first wrinkle generation risk is "medium" and the index of the second wrinkle generation risk is "high", the output unit 235 outputs a message indicating that the wrinkle generation risk is decreasing as the comparison result. In this way, the information processing device 2 can enable a user who repeatedly uses the information processing service to identify whether there is an improvement in the wrinkle generation risk.

[0083] Figure 7 is a diagram schematically showing a display screen displaying beauty information. In Figure 7 beauty information is respectively displayed in a plurality of display areas (R1, R2, R3). In the display area R1, a predetermined scene of the moving image (a scene of the moving image when a relatively frequently occurring expression pattern is detected, or a scene of the moving image when a relatively high expression intensity is detected in the expression pattern) is displayed. In the display area R2, an index indicating the wrinkle generation risk around the part "eyes" of the face and a message indicating the comparison result of comparing the first wrinkle generation risk with the second wrinkle generation risk are displayed. In the display area R3, information on cosmetics for coping with the wrinkle generation risk around the part "eyes" of the face is displayed. For example, when the user presses the "purchase" button on the display screen of the beauty information displayed on the user terminal 1, the user can purchase the cosmetics displayed in the display area R3.

[0084] After the user purchases a product represented by the beauty information provided according to the use of the information processing service, the information processing device 2 may prompt the user to use the information processing service again after a certain period of time has elapsed. Specifically, first, the history acquisition unit 236 acquires a purchase history of the user's purchase of a product. The purchase history is, for example, a history of products purchased by the user via the information processing service, associating the purchased product with the date of purchase. The purchase history may be stored in the storage unit 22 or managed by an external server.

[0085] Then, the output unit 235 refers to the purchase history and outputs information for requesting the transmission of a moving image after a predetermined period (for example, one month, etc.) has elapsed since the date of the user's purchase of the product. In this way, the information processing device 2 can diagnose whether there is an improvement in the wrinkle generation risk.

[0086] The information processing device 2 can also prompt the user to use the information processing service again after a certain period has elapsed when the user has performed the beauty method indicated by the beauty information provided according to the use of the information processing service. Specifically, first, the history acquisition unit 236 acquires the implementation history of the user's performance of the beauty method. The implementation history of the beauty method is, for example, the browsing history of an image introducing the implementation method of massage or the like, the viewing history of a moving image introducing the implementation method of massage or the like, or the utilization history of a store such as a beauty salon where massage or the like is performed, and associates the performed beauty method with the date of performing the beauty method.

[0087] Then, the output unit 235 refers to the implementation history and outputs information for requesting the transmission of a moving image after a predetermined period has elapsed since the date when the user performed the beauty method. By doing so, the information processing device 2 can diagnose whether there is an improvement in the risk of wrinkle formation.

[0088] [Processing of the information processing device 2]

[0089] Next, the processing flow of the information processing device 2 will be described. Figure 8 It is a flowchart showing the processing flow of the information processing device 2. This flowchart starts (S1) on the occasion that the moving image acquisition unit 231 has acquired a moving image from the user terminal 1.

[0090] The detection unit 232 detects one or more expression patterns and the expression intensity of each of the one or more patterns in the moving image acquired by the moving image acquisition unit 231 at predetermined intervals (S2). The calculation unit 233 calculates the sum of the expression intensities detected by the detection unit 232 at predetermined intervals (S3).

[0091] The estimation unit 234 estimates the risk of wrinkle formation based on the sum value (the sum of the expression intensities calculated by the calculation unit 233) and the sum threshold (S4). Then, the output unit 235 outputs beauty information corresponding to the risk of wrinkle formation estimated by the estimation unit 234 (S5).

[0092] [Effects of this embodiment]

[0093] As described above, the information processing device 2 detects one or more expression patterns of the user and the expression intensity of each of the one or more expression patterns based on the user's moving image, and outputs beauty information based on the detected results. By doing so, the information processing device 2 can provide information considering skin changes such as wrinkles that the user may develop in the future due to conscious or unconscious expressions of the user.

[0094] As described above, the present invention has been described using embodiments, but the technical scope of the present invention is not limited to the scope described in the above embodiments, and various modifications and changes can be made within the scope of its gist. For example, all or part of the device can be functionally or physically dispersed / merged in any unit to form. In addition, new embodiments generated by any combination of multiple embodiments are also included in the embodiments of the present invention. The effects of the new embodiments generated by the combination have the effects of the original embodiments both.

[0095] Description of Reference Numerals

[0096] 1: User terminal

[0097] 2: Information processing device

[0098] 21: Communication unit

[0099] 22: Storage unit

[0100] 23: Control unit

[0101] 231: Moving image acquisition unit

[0102] 232: Detection unit

[0103] 233: Calculation unit

[0104] 234: Estimation unit

[0105] 235: Output unit

[0106] 236: History acquisition unit

[0107] S: Information processing system

Claims

1. An information processing apparatus, comprising: a moving image acquisition unit that acquires a moving image of a user's face; a detection unit that detects one or more expression patterns of the user and an expression intensity indicating the intensity of the expression represented by each of the one or more expression patterns, based on the moving image acquired by the moving image acquisition unit; and an output unit that outputs beauty information related to the beauty of the user's face, based on the detection result of the detection unit.

2. The information processing apparatus according to claim 1, further comprising an estimation unit that estimates a risk of wrinkle formation on the user's face, based on the occurrence frequency of each of the one or more expression patterns detected by the detection unit and the expression intensity, wherein the output unit outputs the beauty information corresponding to the risk of formation estimated by the estimation unit.

3. The information processing apparatus according to claim 1, wherein the detection unit detects the expression pattern and the expression intensity at a predetermined interval, and the information processing apparatus further comprises: a calculation unit that calculates the sum of the expression intensities detected at each of the predetermined intervals; and an estimation unit that estimates a risk of wrinkle formation on the user's face, based on a predetermined threshold and the sum.

4. The information processing apparatus according to claim 3, further comprising a storage unit that stores one or more expression patterns in association with each part of the face, wherein the calculation unit calculates, for each part, the sum of the expression intensities of the expression patterns detected by the detection unit, that is, each of the one or more expression patterns associated with the part, and the estimation unit estimates, for each part, the risk of formation based on the threshold corresponding to the part and the sum corresponding to the part.

5. The information processing apparatus according to claim 1, wherein the output unit outputs an index indicating the risk of wrinkle formation on the user's face as the beauty information.

6. The information processing apparatus according to claim 1, wherein the output unit outputs information on a product for coping with the risk of wrinkle formation on the user's face as the beauty information.

7. The information processing apparatus according to claim 6, wherein the product is at least one of cosmetics, beauty equipment, and food.

8. The information processing apparatus according to claim 6 or 7, further comprising a history acquisition unit that acquires a purchase history of the product purchased by the user, wherein the output unit refers to the purchase history and outputs information for requesting transmission of the moving image after a predetermined period has elapsed since the date on which the user purchased the product.

9. The information processing apparatus according to claim 1, wherein the output unit outputs information on a beauty method for coping with the risk of wrinkle formation on the user's face as the beauty information.

10. The information processing apparatus according to claim 9, further comprising a history acquisition unit that acquires an implementation history of the beauty method implemented by the user, The output unit refers to the implementation history record and outputs information for requesting transmission of the moving image after a predetermined period has elapsed since the date when the user implemented the beauty method.

11. The information processing apparatus according to claim 1, When a facial expression pattern with an appearance frequency equal to or higher than a predetermined threshold among one or more facial expression patterns detected by the detection unit is detected, the output unit outputs, as the beauty information, at least one of the scenes of the moving image when the facial expression pattern is detected and the scenes of the moving image when a facial expression pattern with a facial expression intensity equal to or higher than a predetermined threshold among one or more facial expression patterns detected by the detection unit is detected.

12. The information processing apparatus according to claim 1, The output unit outputs, as the beauty information, a result obtained by comparing the risk of wrinkle formation in the user's face estimated based on a first moving image that is the moving image with the risk of wrinkle formation estimated based on a second moving image captured before the first moving image was captured.

13. The information processing apparatus according to claim 1, Before the moving image acquisition unit acquires the moving image, the output unit outputs a question to the user, and the moving image acquisition unit acquires the moving image after the output unit outputs the question to the user.

14. An information processing method, comprising the following steps executed by a computer: a step of acquiring a moving image of a user's face; a step of detecting, based on the acquired moving image, one or more facial expression patterns of the user and the facial expression intensity indicating the degree of the intensity of each of the one or more facial expression patterns; and a step of outputting, based on the detection result, beauty information related to the beauty of the user's face.

15. A program for causing a computer to function as the following units: a moving image acquisition unit that acquires a moving image of a user's face; a detection unit that detects, based on the moving image acquired by the moving image acquisition unit, one or more facial expression patterns of the user and the facial expression intensity indicating the degree of the intensity of each of the one or more facial expression patterns; and an output unit that outputs, based on the detection result of the detection unit, beauty information related to the beauty of the user's face.

Images