Medium, information processing apparatus, quantification method, and information processing system

Abstract

A program is used to enable a computer to function to analyze animation data and output a value for quantifying the makeup actions of the characters reflected in the animation data. The program enables the computer to function as the following units: a first detection unit that detects a facial area reflecting the face of the character from the animation data; a second detection unit that detects a hand area reflecting the hand of the character from the animation data; and an output unit that outputs a value for quantifying the makeup actions of the characters reflected in the animation data based on the changes in the detected facial area and hand area.

Description

Technical Field

[0001] The present invention relates to a medium, an information processing device, a quantification method, and an information processing system. Background Art

[0002] As is well known, makeup effects vary depending on the cosmetics used and makeup techniques. Conventionally, most users of makeup have few opportunities to see the makeup techniques of others. In recent years, many animations of makeup (makeup animations) have been shared on websites such as animation websites. Through such makeup animations, the opportunities for users to confirm makeup techniques are increasing (for example, refer to Non-Patent Document 1).

[0003] Prior Art Documents

[0004] Non-Patent Documents

[0005] Non-Patent Document 1: "Lesson in Animation" ("Animation Course"), [online], Shiseido Co., Ltd., [searched on April 20, 2018], Internet <URL:https: / / www.shiseido.co.jp / beauty / dictionary / lesson / index.html> Summary of the Invention

[0006] Problems to be Solved by the Invention

[0007] By watching the above-mentioned makeup animation, the user can see the cosmetics and makeup actions used by the person shown in the makeup animation. However, it is difficult for the user to correctly imitate the makeup actions of the person shown in the makeup animation only by watching the makeup actions. In this regard, if the makeup actions can be quantified based on the makeup animation, it is easy to compare the makeup actions of the user who is imitating and the makeup actions of the person shown in the makeup animation, which is convenient for use.

[0008] An object of an embodiment of the present invention is to provide a program capable of quantifying the makeup actions of a person shown in animation data.

[0009] Means for Solving the Problems

[0010] For the purpose of achieving the above object, a program according to an embodiment of the present invention is a program for enabling a computer to analyze animation data and output a value obtained by quantifying the makeup actions of a person depicted in the animation data. The program causes the computer to function as the following respective units: a first detection unit that detects a face region depicting the face of the person from the animation data; a second detection unit that detects a hand region depicting the hand of the person from the animation data; and an output unit that outputs a value obtained by quantifying the makeup actions of the person depicted in the animation data based on changes in the detected face region and hand region.

[0011] Effect of the Invention

[0012] According to an embodiment of the present invention, it is possible to quantify the makeup actions of a person depicted in animation data. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1A It is a structural diagram of an example of the information processing system of the present embodiment.

[0014] Figure 1B It is a structural diagram of an example of the information processing system of the present embodiment.

[0015] Figure 2 It is a hardware structural diagram of an example of the computer of the present embodiment.

[0016] Figure 3A It is a distribution diagram of an example of the measurement result.

[0017] Figure 3B It is a distribution diagram of an example of the measurement result

[0018] Figure 4 It is a functional block diagram of an example of the information processing system of the present embodiment.

[0019] Figure 5 It is a structural diagram of an example of the region detection unit.

[0020] Figure 6 It is a conceptual diagram of an example of the detection process of the face region of a person depicted in a frame image.

[0021] Figure 7 It is a flowchart of an example of the facial feature point detection process.

[0022] Figure 8 It is a conceptual diagram of an example of the detection process of the hand region of a person depicted in a frame image.

[0023] Figure 9 It is a structural diagram of an example of the region detection unit.

[0024] Figure 10 It is a conceptual diagram of an example of the processing of the face area detection unit and the hand area detection unit.

[0025] Figure 11 It is a conceptual diagram of an example of the processing of the force field calculation unit and the channel calculation unit. Detailed Implementation Modes

[0026] Hereinafter, the implementation modes of the present invention will be described in detail.

[0027] [First Implementation Mode]

[0028] [System Structure]

[0029] Figure 1A and Figure 1B are structural diagrams of an example of the information processing system of the present implementation mode. Figure 1A The information processing system of has a single information processing device 1. The information processing device 1 is a PC operated by a user, a smart phone, a tablet computer, a dedicated machine for quantifying makeup actions for home or business use, etc.

[0030] In addition, Figure 1B in the information processing system of, one or more client terminals 2 are connected to the server device 3 through a network 4 such as the Internet. The client terminal 2 is a terminal device such as a PC, a smart phone, or a tablet computer operated by a user, a dedicated machine for quantifying makeup actions for home or business use, etc. The server device 3 performs processing related to the quantification of makeup actions implemented in the client terminal 2, etc.

[0031] As described above, the present invention can be applied not only to the client-server type information processing system as shown in Figure 1B but also to the single information processing device 1 as shown in Figure 1A . Here, Figure 1A and Figure 1B The information processing systems of are only examples, and of course, there can be various system structure examples according to the use and purpose. For example, Figure 1B the server device 3 of can also be composed of a plurality of distributed computers.

[0032] [Hardware Structure]

[0033] Figure 1A and Figure 1B The information processing device 1, the client terminal 2, and the server device 3 of are implemented by a computer having a hardware structure as shown in Figure 2 . Figure 2 It is a hardware structure diagram of an example of the computer of the present implementation mode.

[0034] Figure 2The computer includes an input device 501, an output device 502, an external I / F 503, a RAM 504, a ROM 505, a CPU 506, a communication I / F 507, an HDD 508, etc., which are respectively connected by a bus B.

[0035] The input device 501 is a keyboard, a mouse, etc. for input operations. The output device 502 is composed of a display such as a liquid crystal or an organic EL that can display a screen, a speaker for outputting audio data such as sound or music, etc. The communication I / F 507 is an interface for connecting the computer to the network 4. The HDD 508 is an example of a non-volatile storage device for storing programs or data.

[0036] The external I / F 503 is an interface for connecting to an external device. The computer can read and / or write to the recording medium 503a through the external I / F 503. The recording medium 503a can be a DVD, an SD memory card, a USB storage device, etc.

[0037] The CPU 506 is an arithmetic device that reads programs and data from storage devices such as the ROM 505 or the HDD 508 onto the RAM 504 and processes them to achieve overall control and functions of the computer. The information processing device 1, the client terminal 2, and the server device 3 in the present embodiment can achieve various functions by executing programs in a computer having the above hardware structure.

[0038] Here, Figure 2 is an example of a hardware structure. Of course, various structural examples can be formed according to uses and purposes. For example Figure 2 in the computer of, the input device 501 may further have a camera function capable of shooting videos.

[0039] 〈Research on Quantification of Makeup Movements〉

[0040] As a method for quantifying makeup movements, for example, there is a method using a sensor. In the method for quantifying makeup movements using a sensor, a person performing makeup wears a sensor and performs makeup movements. In the method for quantifying makeup movements using a sensor, the makeup movements of a person can be quantified based on the data output by the sensor, but the makeup movements of a person reflected in the captured makeup video cannot be quantified.

[0041] If it is possible to analyze the post-shot makeup animation and quantify the makeup actions of the person reflected in the makeup animation, it is possible to use the makeup animations on websites such as animation websites without wearing sensors or the like, and thus it is expected to quantify natural makeup actions. Here, in the present embodiment, in order to quantify the makeup actions of the person reflected in the post-shot makeup animation, discussions have been made on grasping the object obtained from the makeup animation and using the object for quantifying the makeup actions.

[0042] <<Grasping the Object Obtained from the Makeup Animation>>

[0043] In order to grasp the object obtained from the makeup animation, a motion capture device was used to measure the changes in the makeup actions of the subject. Here, the measurement sites were a total of 6 locations including the tip of the right middle finger, the base of the middle finger, the center of the back of the hand, the center of the wrist, the elbow, and the forehead. The analysis objects were the coordinates (displacements), velocities, accelerations, and angular velocities of the measurement sites. The principal components of the changes of the subject after the above analysis were presumed to be the main change factors in the makeup actions.

[0044] Figure 3A and Figure 3B is a distribution diagram showing an example of the measurement results. Figure 3A is a distribution diagram showing an example of the fineness of the hand movements and the linkage between the hand and the elbow in each makeup action. Figure 3B is a distribution diagram showing an example of the fineness of the hand movements and the facial changes in each makeup action. As Figure 3A and Figure 3B shown, the principal components of the changes of the subject are the fineness of the hand movements, the linkage between the hand and the elbow, and the facial changes. Here, in the present embodiment, the coordinates and velocities of the hand and the coordinates and velocities of the face are set as the objects obtained from the makeup animation.

[0045] <<Quantifying the Makeup Actions Using the Obtained Object>>

[0046] As a method for obtaining the coordinates and velocities of the hand and the coordinates and velocities of the face in the makeup actions of the person reflected in the makeup animation, image recognition using a convolutional neural network (hereinafter referred to as CNN) can be cited. In image recognition using CNN, it is possible to detect the facial area and the hand area from a two-dimensional image. Therefore, by tracking the facial area and the hand area detected from the frame images of the makeup animation, it is possible to obtain the coordinates and velocities of the hand and the coordinates and velocities of the face in the makeup actions of the person reflected in the makeup animation. Hereinafter, the image recognition using CNN in the present embodiment will be described in detail.

[0047] <Software Structure>

[0048] <<Function block>>

[0049] The software structure of the information processing system according to this embodiment will be described below. Here, the information processing device 1 Figure 1A shown will be described as an example. Figure 4 is a functional block diagram of an example of the information processing system according to this embodiment. The information processing device 1 realizes the functions of the operation reception unit 10, the area detection unit 12, the quantification unit 14, the post-processing unit 16, and the animation data storage unit 18 by executing a program.

[0050] The operation reception unit 10 receives various operations from the user. The animation data storage unit 18 stores makeup animations. Here, the animation data storage unit 18 may also be provided outside the information processing device 1. The makeup animations stored in the animation data storage unit 18 and the makeup animations captured using the camera function are input to the area detection unit 12. The area detection unit 12 performs the following detection on the face area and hand area of the person reflected in each frame image constituting the input makeup animation.

[0051] The quantification unit 14 quantifies the makeup actions of the person reflected in the makeup animation by obtaining the coordinates and speeds of the hands and the coordinates and speeds of the face in the makeup actions of the person reflected in the makeup animation from the face area and hand area detected by the area detection unit 12. The post-processing unit 16 performs post-processing on the processing results of the area detection unit 12 and the quantification unit 14 and outputs them to the output device 502 or the like.

[0052] For example, the post-processing unit 16 performs post-processing to enclose the face area and hand area in the makeup actions of the person reflected in the makeup animation with a rectangle. In addition, the post-processing unit 16 performs post-processing to visually represent the fineness of the hand movements, the linkage between the hand and the elbow, and the facial changes based on the coordinates and speeds of the hands and the coordinates and speeds of the face in the makeup actions of the person reflected in the makeup animation.

[0053] In addition, the post-processing unit 16 can quantify and compare the makeup actions of the persons reflected in two makeup animations and output the comparison result. For example, a user using the information processing system according to this embodiment can easily understand the difference from their own makeup skills by quantifying and comparing their own makeup actions with the makeup actions of a person with high makeup skills such as a makeup artist. Thus, the information processing system according to this embodiment can provide a service for improving the user's makeup skills.

[0054] The area detection unit 12 for detecting the face area and hand area of the person reflected in the frame image of the makeup animation Figure 4 has, for example, Figure 5 the structure shown. Figure 5This is a structural diagram of an example of the area detection unit. Figure 5 The area detection unit 12 includes a frame conversion unit 20, a face area detection unit 22, a hand area detection unit 24, and a facial feature point detection unit 26.

[0055] The frame conversion unit 20 provides the input makeup animation to the face area detection unit 22 and the hand area detection unit 24 in units of frame images. The face area detection unit 22 includes a face area learning model including a facial organ area learning model.

[0056] Here, through machine learning using a two-dimensional image in which the hand area overlaps the face area as teaching data, the face area learning model included in the face area detection unit 22 is created. Based on a two-dimensional image in which the hand is mapped as the face foreground, teaching data in which the hand area overlaps the face area is created. With respect to the hand area learning data set (used for creating teaching data) with annotations, an image of the face area learning data set with annotations attached in such a way that the hand is the foreground can be used to create teaching data in which the hand area overlaps the face area.

[0057] The face area detection unit 22 uses the face area learning model obtained through CNN learning to achieve highly effective face area detection for the case where the face area and the hand area overlap. The CNN learning data set uses teaching data in a state where a part of the face area is occluded by the hand area (occlusion environment).

[0058] The hand area detection unit 24 includes a hand area learning model including a fingertip position area learning model. The hand area learning model included in the hand area detection unit 24 is created using a two-dimensional image of the hand during makeup as teaching data.

[0059] Here, the teaching data of the hand during makeup is created from a hand area learning data set with annotations specialized for the shape of the hand during makeup and a fingertip position area learning data set with annotations specialized for the fingertip position during makeup. The hand area detection unit 24 uses the hand area learning model obtained through CNN learning to achieve hand area detection with high detection accuracy for the hand with various shape changes and the fingertip position during the makeup action. The CNN learning data set uses the above teaching data.

[0060] In addition, the facial feature point detection unit 26 has a facial feature point learning model, which includes a facial organ feature point learning model. The facial feature point detection unit 26 detects the facial feature points of the entire face by using the facial feature point learning model. Then, the facial feature point detection unit 26 detects the facial feature points of the facial organs by region by using the facial organ feature point learning model. The facial feature point detection unit 26 detects the facial feature points of the eyes included in the facial organs, and corrects the positions of the facial feature points (including the contour) of the parts other than the eyes based on the positions of the facial feature points of the eyes, so as to achieve high-precision facial feature point detection even in low-resolution or occluded environments.

[0061] <Processing>

[0062] <<Facial Region Detection and Facial Feature Point Detection>>

[0063] The region detection unit 12 performs a process of detecting the facial region of a person reflected in the frame image, for example, in the manner Figure 6 shown. Figure 6 is a conceptual diagram of an example of the process of detecting the facial region of a person reflected in the frame image.

[0064] The facial region detection unit 22 of the region detection unit 12 detects the facial region of the person's face in the frame image 1000 as a rectangle 1002 by using the above-mentioned facial region learning model. The facial region detection unit 22 detects the facial organ region from the region of the rectangle 1002 by using the above-mentioned facial organ region learning model, and corrects the rectangular ratio of the rectangle 1002 centered on the nose into the form of a rectangle 1004.

[0065] The facial feature point detection unit 26 detects the facial feature points as shown in the rectangular region image 1006 from the region of the rectangle 1004 by using the above-mentioned facial feature point learning model. In addition, the facial feature point detection unit 26 detects the feature points of the facial organs as shown in the rectangular region image 1008 by region from the rectangular region image 1006 by using the above-mentioned facial organ feature point learning model.

[0066] Here, Figure 5 the region detection unit 12 can achieve high-precision facial feature point detection even in low-resolution and occluded environments by performing the Figure 7 processing shown in the flowchart. Figure 7 is a flowchart of an example of the facial feature point detection process.

[0067] In step S11, the facial feature point detection unit 26 of the region detection unit 12 detects the facial feature points from the facial region (the entire face) detected by the facial region detection unit 22 by using the above-mentioned facial feature point learning model, and estimates the head pose.

[0068] Proceed to step S12. The facial feature point detection unit 26 takes into consideration the head pose estimated in step S11, and uses the facial organ feature point learning model to detect the eyes detected by the facial area detection unit 22, thereby correcting the estimation of the eye positions to improve the accuracy of the estimated eye positions.

[0069] Proceed to step S13. The facial feature point detection unit 26 takes into consideration the corrected estimated eye positions in step S12, and uses the above-mentioned facial organ feature point learning model to detect the feature points (including the contours) of the facial organs other than the eyes, and corrects the estimated positions of the facial organs other than the eyes. Figure 7 The processing of the flowchart is effective, for example, when the facial contour is blocked by a hand.

[0070] <<Hand area detection>>

[0071] The area detection unit 12 performs the process of detecting the hand area of the person reflected in the frame image, for example, in the manner shown by Figure 8 as follows. Figure 8 It is a conceptual diagram of an example of the detection process of the hand area of the person reflected in the frame image.

[0072] The hand area detection unit 24 of the area detection unit 12 uses the above-mentioned hand area learning model to detect the hand areas of the left and right hands of the person reflected in the frame image 1100 with a rectangle 1102. In addition, the hand area detection unit 24 uses the above-mentioned fingertip position area learning model to detect the areas 1112 and 1114 of the left and right hands and the fingertip positions 1116 and 1118 of the left and right hands of the person in the frame image 1110 from the area of the rectangle 1102.

[0073] <<Output>>

[0074] The information processing apparatus 1 of the present embodiment can calculate and output the fineness of the hand movement, the linkage between the hand and the elbow, and the facial changes, for example, based on the coordinates and speeds of the hands and the coordinates and speeds of the face in the makeup animation of the person reflected. The output of the fineness of the hand movement, the linkage between the hand and the elbow, and the facial changes can be used for research on makeup movements and the like.

[0075] In addition, the information processing apparatus 1 of the present embodiment can quantify and compare the makeup actions of the characters shown in two makeup animations. Therefore, it is possible to quantify and compare the makeup actions of users who want to learn makeup techniques and those of people with high makeup skills such as makeup artists. The comparison results can be scored and provided to the user, or the differences between the user's makeup actions and those of people with high makeup skills such as makeup artists can be visually presented to the user. In addition, the information processing apparatus 1 of the present embodiment can also provide makeup techniques to the user based on the comparison results so that the user's makeup actions are closer to those of people with high makeup skills such as makeup artists.

[0076] For example, when applying makeup products such as blush / foundation that are applied over a large area, the information processing apparatus 1 of the present embodiment can provide the user with guidance on the application area judgment and application range. In addition, when using makeup products that are more difficult in terms of techniques such as eyeliner / eyeshadow / concealer, the information processing apparatus 1 of the present embodiment can provide the user with judgment on the correctness of the actions and guidance. Furthermore, the information processing apparatus 1 of the present embodiment can also provide the user with guidance on the application method of hair wax (hair care product), the application method of skin care products, or the massage method.

[0077] In addition, makeup methods such as the application method of blush (rouge) vary depending on users with more pointed bone structures and those with rounder ones. Therefore, the information processing apparatus 1 of the present embodiment can also provide a recommended makeup look suitable for the user's face shape and technical guidance for achieving the makeup look.

[0078] [Second Embodiment]

[0079] In the information processing system of the second embodiment, other structures are the same as those of the information processing system of the first embodiment except for a part of the structures. Therefore, it is appropriate to omit the description. In the structure of the information processing system of the second embodiment, Figure 9 the area detection unit 12 is replaced with Figure 5 the area detection unit 12. Figure 9 It is a structural diagram of an example of the area detection unit.

[0080] Figure 9 The structure of the area detection unit 12 of

[0081] The framer 50 provides the input makeup animation to the skin color area extraction unit 52 in units of frame images. The skin color area extraction unit 52 extracts the skin color area from the frame image. The area segmentation unit 54 divides the skin color area extracted by the skin color area extraction unit 52 into candidate blocks and marks the candidate blocks. The area segmentation unit 54 provides the marked candidate blocks to the face area detection unit 56 and the hand area detection unit 58.

[0082] The face area detection unit 56 classifies the candidate blocks of the face area (detects the face area) according to the marks of the candidate blocks provided above (the features of the segmented skin color area). In addition, the hand area detection unit 58 classifies the candidate blocks of the hand area (detects the hand area) according to the marks of the candidate blocks provided by the area segmentation unit 54 (the features of the segmented skin color area) and the candidate blocks of the face area classified by the face area detection unit 56.

[0083] Figure 9 The face area detection unit 56 and the hand area detection unit 58, as Figure 10 shown, first classify the candidate blocks of the face area by the face area detection unit 56, remove the candidate blocks of the face area, and classify the candidate blocks of the hand area by the hand area detection unit 58. Therefore, false detection of the hand area can be prevented in this embodiment.

[0084] When the face area detection unit 56 cannot classify the candidate blocks of the face area, the force field calculation unit 60 determines that interference has occurred between the face area and the hand area and performs the following processing. The candidate blocks of the face area and the hand area of the previous frame (t - 1) and the marked candidate blocks of the current frame (t) are provided to the force field calculation unit 60.

[0085] The force field calculation unit 60, as Figure 11 shown, sets a plurality of channels (Channels) in the image of the candidate block according to the force field (Force Field). The channel calculation unit 62 calculates the moving distance relative to the previous frame for each channel, and sets the candidate block of the channel with the large moving distance as the candidate block of the moving hand, thereby being able to classify the candidate blocks of the hand area and the face area.

[0086] Here, in addition to the magnitude of the moving distance, the force field calculation unit 60 and the channel calculation unit 62 of the area detection unit 12 can further prevent false detection of the candidate blocks of the hand area and the face area by pre - clustering (clustering) the changes of similar hand areas.

[0087] (Summary)

[0088] As described above, according to the present embodiment, it is possible to quantify the makeup actions of a person reflected in the captured makeup animation without equipping sensors or the like, and provide or instruct makeup techniques. The present invention is not limited to the specifically disclosed embodiments above, and various modifications or changes can be made as long as the scope of the claims is not departed from. For example, in the present embodiment, two-dimensional animation data is taken as an example for explanation, and three-dimensional animation data can also be used. According to the present embodiment, by the same data analysis as the two-dimensional animation data, or by combining the analysis of two-dimensional animation data and three-dimensional information, it is possible to quantify the makeup actions of a person reflected in the three-dimensional animation data and provide or instruct makeup techniques.

[0089] As described above, the present invention has been described based on the embodiments, but the present invention is not limited to the above embodiments, and various modifications can be made within the scope described in the claims. This application claims priority based on the basic application No. 2018-199739 filed with the Japan Patent Office on October 24, 2018, and incorporates the entire contents of the said basic application by reference.

[0090] Symbol Explanation

[0091] 1 Information processing device

[0092] 2 Client terminal

[0093] 3 Server device

[0094] 4 Network

[0095] 10 Operation reception unit

[0096] 12 Region detection unit

[0097] 14 Quantification unit

[0098] 16 Post-processing unit

[0099] 18 Animation data storage unit

[0100] 20 Framing unit

[0101] 22 Facial region detection unit

[0102] 24 Hand region detection unit

[0103] 26 Facial feature point detection unit

[0104] 50 Framing unit

[0105] 52 Skin color region extraction unit

[0106] 54 Region segmentation unit

[0107] 56 Facial region detection unit

[0108] 58 Hand region detection unit

[0109] 60 Force field calculation unit

[0110] 62 Channel calculation unit

Claims

1. A recording medium having stored thereon a program for analyzing the makeup actions of a person depicted in animation data, the program causing a computer to execute the following steps: A first detection step of detecting a face area depicting the face of the person from the animation data; A second detection step of detecting a hand area depicting the hand of the person from the animation data; and An output step of obtaining and outputting the coordinates and speed of the hand and the coordinates and speed of the face in the makeup actions of the person depicted in the animation data based on the detected face area and hand area and on the changes in the face area and hand area, In the output step, an image visually representing the difference between the makeup actions of the person depicted in the two animation data is output according to the result of comparing the coordinates and speed of the hand and the coordinates and speed of the face in the makeup actions of the person depicted in the animation data obtained based on the two animation data.

2. The recording medium according to claim 1, wherein The first detection process includes the following steps: Using a facial area learning model, detect the facial area of the person from the animation data, and using a facial part area learning model, detect the facial part area from the detected facial area; and Using a facial feature point learning model, detect facial feature points from the facial area, and using a facial part feature point learning model, detect the feature points of the facial part from the facial part area, The second detection process includes the following steps: Using the hand area learning model of the makeup action, detect the hand area of the person from the animation data, and using the fingertip position area learning model of the makeup action, detect the fingertip position area from the hand area.

3. The recording medium according to claim 2, wherein The facial area learning model is a convolutional neural network obtained by using the supervised data in a state where a part of the facial area is blocked by the hand area for learning in the facial area learning dataset.

4. The recording medium according to claim 2, wherein The hand area learning model is a convolutional neural network obtained by using the supervised data of the hand shape during makeup for learning in the hand area learning dataset, The fingertip position area learning model is a convolutional neural network obtained by using the supervised data of the fingertip position during makeup for learning in the fingertip position area learning dataset.

5. The recording medium according to claim 1, wherein The computer is also made to execute the following processes: An extraction process of extracting the skin color area from the animation data; and A segmentation process of segmenting the skin color area into segmented areas, In the first detection process, according to the feature amount of the segmented area, detect the segmented area that reflects the face of the person from the skin color area as the facial area, In the second detection process, according to the feature amount of the segmented area, detect the segmented area that reflects the hand of the person from the skin color area other than the segmented area that is the facial area as the hand area.

6. The recording medium according to claim 5, wherein The computer is also made to execute the following processes: A third detection process. When the first detection process fails to detect the facial area that reflects the face of the person from the skin color area, the third detection process detects the moving distance of the segmented area between the frame images constituting the animation data, and assumes the segmented area with a larger moving distance as the hand area that reflects the hand of the person.

7. An information processing apparatus analyzes the makeup actions of a person depicted in animation data. The information processing apparatus includes: A first detection unit that detects the facial area that reflects the face of the person from the animation data; A second detection unit that detects the hand area that reflects the hand of the person from the animation data; and An output unit that, based on the detected facial area and hand area, obtains and outputs the coordinates and speed of the hand and the coordinates and speed of the face in the makeup action of the person reflected in the animation data based on the changes in the facial area and the hand area, The output unit outputs an image visually representing the difference in the makeup actions of the persons reflected in the two animation data based on the result of comparing the coordinates and speed of the hand and the coordinates and speed of the face in the makeup action of the person reflected in the animation data obtained based on the two animation data.

8. A quantification method performs a process of analyzing the makeup actions of a person depicted in animation data in an information processing apparatus. The quantification method includes: The first detection step of detecting a facial region that shows the face of the person from the animation data; The second detection step of detecting a hand region that shows the hand of the person from the animation data; and An output step of obtaining and outputting the coordinates and speed of the hand and the coordinates and speed of the face in the makeup action of the person shown in the animation data based on the changes in the detected facial region and hand region; In the output step, according to the result of comparing the coordinates and speed of the hand and the coordinates and speed of the face in the makeup action of the person shown in the animation data obtained based on two pieces of the animation data, an image visually representing the difference in the makeup actions of the person shown in the two pieces of the animation data is output.

9. An information processing system includes a client terminal and a server device. The client terminal accepts operations from a user, and the server device analyzes the makeup actions of a person depicted in animation data according to the operations accepted by the client terminal from the user. The server device includes: A receiving unit that receives information on an operation received by the client terminal from the user; A first detection unit that detects a facial region that shows the face of the person from the animation data according to the operation received by the user; A second detection unit that detects a hand region that shows the hand of the person from the animation data according to the operation received by the user; An output unit that obtains and outputs the coordinates and speed of the hand and the coordinates and speed of the face in the makeup action of the person shown in the animation data based on the changes in the detected facial region and hand region; and A sending unit that sends the output result of the output unit to the client terminal, The output unit outputs an image visually representing the difference in the makeup actions of the person shown in the two pieces of the animation data according to the result of comparing the coordinates and speed of the hand and the coordinates and speed of the face in the makeup action of the person shown in the animation data obtained based on two pieces of the animation data.

Images