Apparatus and program for determining type of emotion in catchlight, and apparatus and program for generating pupil image data reflecting catchlight

Abstract

When communicating, people obtain a great deal of information from their faces. Eyes, in particular, are said to provide a great deal of information. Catchlights reflected in the pupils of eyes have already developed into an expression technique that supports or enhances character expression in the media and are widely used as an indispensable non-verbal tool. This invention aims to technologically classify the shapes of catchlights of characters depicted in the media, thereby enabling the emotional expressions of people or characters to be determined by distinguishing the shapes of the catchlights, rather than by language or facial expressions. [Solution] Using a device for determining the type of emotion represented by a catchlight image and a device for generating a catchlight image representing the type of emotion, the type of emotion is determined from the shape of the catchlight in the pupil image, and a catchlight image representing the type of emotion is generated from a predetermined type of emotion.

Description

[Technical Field]

[0001] The present invention relates to a device for determining the type of emotion expressed by the shape of catchlights reflected in the pupils. [Background technology]

[0002] There was no technological technology available to determine the type of emotion based on the shape of the catchlight reflected in the eyes. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Published Patent Application (A) JP2024-45247A [Non-patent literature] [Non-patent literature] "Emotional Expressions through Robot Eye Lighting Patterns Influence Human Social Decision-Making," HAI Symposium 2013, pp. 147-150 Summary of the Invention [Problem to be solved by the invention]

[0004] With the maturation of computer science and the rapid advancement of AI, humanoid characters, avatars, and humanoids are beginning to play an important role in society as communication media interfaces. It is known that humans obtain a great deal of information from their faces, especially from their eyes, when communicating. In traditional media content such as manga and anime, catchlights reflected in the pupils of eyes have developed as an expressive technique that supports and enhances the expression of characters' emotions and worldviews, and are now widely used in everyday life as an indispensable nonverbal tool. In the future, such catchlight-based expressions are expected to be widely incorporated into humanoid characters, avatars, and humanoids in digital media, as well as next-generation media interfaces such as mixed reality goggles, mixed reality glasses, and mixed reality contact lenses, becoming an important communication feature. Furthermore, when communicating with media such as robots, whose faces are made of solid materials such as plastic and cannot express emotions, language is used exclusively. However, the inability to manipulate facial expressions means that people are unable to fully read the robot's emotions, not only when the robot is silent, but even when it is speaking, which creates the problem that people cannot help but feel uneasy about this type of communication. [Means for solving the problem]

[0005] The basic coding theory and emotion discrimination theory of the present invention are based on the inventor's findings in media engineering and psychology research on catchlights. Based on the findings, it was discovered that there is a clear correspondence between the shape of catchlights and emotions. Based on this correspondence, the invention aims to enable highly automated discrimination of the type of emotion of a person or character from the shape of catchlights. Using this catchlight emotion discrimination device, the emotional expression of a person or character can be discerned by discerning the shape of catchlights in the information space or semantic space via the above-mentioned media, without using language or facial expressions, thereby enabling emotional communication using catchlights. Catchlights are formed by combining six types of shape elements, as shown in Figures 21 and 22. A code representing these shape elements is called a "shape element code." Predetermined manipulation of these six shape elements results in the expression of positive emotions. From the relationship between these shape elements and emotions, combinations of six catchlight shape elements can be derived to express the six basic human emotions: (1) surprise, (2) fear, (3) anger, (4) disgust, (5) sadness, and (6) joy. The rules for relating the type of emotion, shape element code, and catchlight shape are shown in Figure 23. According to this table, the type of emotion can be determined from the shape of the catchlight.Here, the present invention is characterized by comprising an image input device that takes in an pupil image as an electrical signal and converts it into a processable format, feature extraction processing means that receives image data from the image input device and extracts a feature portion image by comparing it with predetermined pupil data based on feature portion extraction conditions, shape element extraction processing means that extracts shape elements from the feature portion image based on predetermined shape element-characteristic portion association rules to obtain catch-light shape element information, shape element encoding processing means that performs shape element encoding processing on the shape element information based on predetermined shape element code association rules to calculate a shape element code, and emotion type discrimination means that receives the predetermined shape element code and discriminates the type of emotion based on the association rules between predetermined emotion types, shape element codes, and catch-light shapes to specify the type of emotion, and is also characterized by comprising an arithmetic processing device having storage means for storing the shape element codes and emotion types, and an emotion type information output device that outputs the shape element codes and emotion types stored in the arithmetic processing device.

[0006] The basic decoding theory and emotion type identification theory of the present invention are based on the inventor's findings in media engineering and psychology research on catchlights. Catchlights are formed by combining six types of shape elements shown in Figures 21 and 22, and a code representing these shape elements is called a "shape element code." Performing specific operations on these six types of shape elements results in the expression of positive emotions. From the relationship between these shape elements and emotions, combinations of catchlight shape elements can be derived to represent the six basic human emotions: (1) surprise, (2) fear, (3) anger, (4) disgust, (5) sadness, and (6) joy. The rules for the relationship between emotion types, shape element codes, and catchlight shapes are shown in Figure 23. By following this table, emotions can be identified from the shape of catchlights. Here, the present invention is characterized by comprising a code input device that takes in the catch-light shape element code output by the catch-light shape encoding device as an electrical signal and obtains predetermined catch-light shape element code data, shape element decoding processing means that receives the predetermined catch-light shape code data from the code input device and performs shape element decoding processing based on predetermined shape element code association rules to calculate catch-light shape element information, emotion discrimination means that receives the predetermined shape element code and performs emotion discrimination processing based on a predetermined emotion type, shape element code, and catch-light shape association rules to identify the emotion, further comprising processing means that obtains predetermined catch-light image data from the obtained emotion type information, and a calculation processing device that is also equipped with storage means for storing pupil image data reflecting the catch-light, and a pupil image data output device reflecting the catch-light that outputs the image data saved in the calculation processing device. [Effects of the Invention]

[0007] The encoding process of the present invention makes it possible to technologically classify and encode the shapes of catchlights reflected in pupils. This enables simple, accurate, and rapid processing with only necessary, sufficient, and minimal information processing, allowing for rapid and accurate extraction of the desired shape element code. Furthermore, the decoding process of the present invention makes it possible to create and display the desired catchlight shape from the shape element code. Furthermore, if the shape element code is stored in a storage device, the catchlight shape can be accurately restored. The catchlight emotion determination device of the present invention quickly and accurately extracts the desired shape element code from the catchlight reflected in pupils and determines the emotion. Furthermore, it can form and output catchlights that express the desired emotion from the shape element code. Thus, communication technologies and devices utilizing catchlight shapes can be applied, for example, to characters and avatars in media, the eyes of robots and humanoids, or to MR goggles, MR glasses, and MR contact lenses. These technologies and devices are not only expected to enrich communication for healthy people, but also to help people with speech or hearing disabilities, or people with facial muscle impairments, in reading the emotions of others and expressing their own. This invention will enable richer and more accurate communication between people, between people and AI, and between AI and AI. Catchlights usually have the same shape reflected in both eyes. Therefore, for ease of explanation, only one eye is shown in the diagram. The up, down, left, and right directions shown in the diagram are those when the target eye is viewed from the outside, front position. [Brief explanation of the drawings]

[0008] [Figure 1] <Number of Shape Elements> This is a diagram of CUs showing shape elements of the characteristic parts of the number of catchlights of the present invention. (a) One catchlight (CU101), (b) Two catchlights (CU102), (c) Three catchlights (CU103) [Figure 2]This is a diagram showing the shape elements of the characteristic parts of the catchlight position in the eye that has a realistic shape such as a photograph or video, according to the present invention. [Figure 3] This is a diagram showing the shape elements of the characteristic parts of the catchlight position in eyes with shapes created in manga, anime, etc., according to the present invention. [Figure 4] <Shape element of position> This is a diagram of CU showing the shape element of the characteristic part of the catch light position of the present invention. [Figure 5] <Shape Elements of Position> of the Present Invention When a catchlight is displayed across multiple areas, this is a diagram showing shape elements of characteristic parts of the position of the catchlight in the area where the center point of the catchlight is located. [Figure 6] <Shape Elements of Position> This is a diagram showing the shape elements of the characteristic part of the position of the catchlight on the line where the center point of the catchlight is located when the center point of the catchlight is on the boundary line of the area. [Figure 7] <Shape Elements of Position> This is a diagram showing shape elements of the characteristic part of the position of the catchlight at the center point of the catchlight when the center point of the catchlight is on the center point of the area. [Figure 8] <Shape Elements> of the present invention: A diagram of CU showing the shape elements of the characteristic parts of the shape of the catchlight. (a) Catchlight round (CU31), (b) Catchlight oval (CU32), (c) Catchlight square (CU33), (d) Catchlight hexagonal (CU34), (e) Catchlight octagonal (CU35), (f) Catchlight rectangle (CU36), (g) Catchlight design shape (CU37). [Figure 9] <Size and Shape Elements> This is a diagram of CUs showing the shape elements of characteristic parts of the size of catchlights of the present invention. (a) Large catchlight (CU41), (b) Medium catchlight (CU42), (c) Medium catchlight (CU43), (d) Medium catchlight (CU44), (e) Small catchlight (CU45). [Figure 10]<Brightness Shape Element> This is a diagram of CUs showing the shape elements of the characteristic parts of the brightness of the catchlight of the present invention. (a) Extremely bright catchlight (CU51), (b) Bright catchlight (CU52), (c) Slightly dark catchlight (CU53), (d) Dark catchlight (CU54). [Figure 11] <Color Shape Elements> This is a diagram of CUs showing the shape elements of the characteristic parts of the color of catchlights of the present invention. (a) Catchlight Gray (CU61), (b) Catchlight Pink (CU62), (c) Catchlight Red (CU63), (d) Catchlight Purple (CU64), (e) Catchlight Blue (CU65), (f) Catchlight Cyan (CU66), (g) Catchlight Green (CU67), (h) Catchlight Yellow (CU68), (i) Catchlight Orange (CU69). [Figure 12] FIG. 10 is a diagram showing the shape of a catchlight generated by decoding a code encoded using the catchlight encoding device of the present invention. [Figure 13] 1 is a graph showing an algorithm for outputting emotion type information of catchlights according to the present invention; [Figure 14] 1 is a flowchart of the catchlight emotion type information output device of the present invention. [Figure 15] 1 is a catchlight image output algorithm of the present invention. [Figure 16] 14 is a flowchart of the catchlight image output device of the present invention. (1A) is the inverse process of (1) in Fig. 14, (2A) is the inverse process of (2) in Fig. 14, (3A) is the inverse process of (3) in Fig. 14, and (4A) is the inverse process of (4) in Fig. 14. [Figure 17] This is pupil image data input by an image input device. [Figure 18] This is an image of a feature portion extracted using a fixed coordinate system. [Figure 19] This is a pupil image without catchlight for comparative measurement of pupil images. [Figure 20] This is an image of the extracted characteristic area (rectangular screen). [Figure 21]This is the association rules between shape elements, characteristic parts, and shape element codes (part 1). [Figure 22] This is the association rules between shape elements, characteristic parts, and shape element codes (part 2). [Figure 23] These are rules relating to the type of emotion, shape element code, and catchlight shape. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, an embodiment of the present invention will be described. 1. Catchlight emotion type information output algorithm The relationship between emotion types and shape element codes is shown in Figure 23. A catchlight image of the pupil is supplied from image input device 131 to arithmetic processing device 132 as digital data that can be processed by the arithmetic processing device, and this data is stored and held as pupil image data in the image storage area of ​​arithmetic processing device 132. Then, emotion type information (i) is obtained from the catchlight image data (a) according to the flowchart in Figure 14. 2. Catchlight image output algorithm The relationship between emotion types and shape element codes is shown in Figure 23. Codes are supplied from emotion type information input device 151 to arithmetic processing device 152 as digital data that can be processed by the arithmetic processing device, and this data is stored as code data in a code storage area of ​​arithmetic processing device 152. Then, catchlight image data (a) is obtained from the emotion type information (i) according to the flowchart in Figure 16. 3. Characteristic part extraction processing and characteristic part extraction reverse processing The feature portion extracted image shown in Figure 18 is obtained by extracting the portions necessary for recognizing catchlights from the pupil image data shown in Figure 17 according to the feature portion extraction conditions, and by comparing it with Figure 19, the feature portion extraction conditions are applied and extraction is performed using the fixed coordinate method. As a result, the feature portion image (rectangular screen) shown in Figure 20 can be cut out. The reverse process of this process is called the feature portion extraction reverse process. 4. Shape element extraction processing and shape element extraction reverse processing The characteristic part (c) is associated with the shape element according to the association rule (d) between the shape element and the characteristic part image, and the change in the characteristic part determined based on the shape constituent factors of the catchlight identified from media engineering research, and the shape element information (e) is extracted. The reverse process of this process is the characteristic part extraction reverse process. Tables for explaining specific examples of the association rule (d) between the shape element and the characteristic part are shown in Figures 21 and 22. 5. Shape element encoding process and shape element decoding process The code assigned to each shape element is called a catchlight unit. In other words, a shape element code is a unique code assigned to six shape elements: (1) number, (2) position, (3) shape, (4) size, (5) brightness, and (6) color. For example, if there is one catchlight, the code would be "CU101." Furthermore, by performing detailed analysis of the characteristic portion image (c) for each shape element code determined by the shape element code association rules according to the flowcharts in Figures 14 and 16, several measurement values ​​for each shape element code are determined as shape element information (e). The association rules between catchlight shape elements and characteristic portions, and the association rules between shape element codes called catchlight units (CU) and shape elements are shown below. (1) <Number of Shape Elements> As the rules and means for assessing the change in the number of catchlights between shape elements and characteristic features, the number of catchlights is calculated and assessed by comparing with a required image without catchlights and applying the characteristic feature extraction conditions. Furthermore, an individual identification symbol is assigned to each extracted catchlight as a unique ID. <Number shape element> The CU that evaluates the number of catchlights and the individual identification symbol are shown below. 0 pieces (CU100), 1 piece (CU101), 1st piece A, 2 pieces (CU102), 1st piece A, 2nd piece B 3 pieces (CU103), 1st piece A, 2nd piece B, 3rd piece C 4 pieces (CU104), 1st piece A, 2nd piece B, 3rd piece C, 4th piece D 5 pieces (CU105), 1st piece A, 2nd piece B, 3rd piece C, 4th piece D, 5th piece E 6 pieces (CU106), 1st piece A, 2nd piece B, 3rd piece C, 4th piece D, 5th piece E, 6th piece F 7 pieces (CU107), 1st piece A, 2nd piece B, 3rd piece C, 4th piece D, 5th piece E, 6th piece F, 7th piece G 8 pieces (CU108), 1st piece A, 2nd piece B, 3rd piece C, 4th piece D, 5th piece E, 6th piece F, 7th piece G, 8th piece H 9 pieces (CU109), 1st piece A, 2nd piece B, 3rd piece C, 4th piece D, 5th piece E, 6th piece F, 7th piece G, 8th piece H, 9th piece J *1 The letter I is not used because it is easily confused with the number 1 and the letter O is not used because it is easily confused with the number 0. *2 If the number of catchlights is 24 or more, the individual identification code will start again with A, with the number of turns at the beginning of the alphabet. (Example) If there are 30 catchlights, it will be 2F. *3 The CU for 100 catchlights is (CU200). (2) <Shape elements of position> As the rules and means for relating shape elements and characteristic features to evaluate changes in the position of catchlights, catchlights with individual identification symbols are arranged on a two-axis coordinate system with the pupil of the eye as the center point, and the position of the catchlight is evaluated by applying the characteristic feature extraction conditions. <Shape element of position> The CU for evaluating the position of the catchlight is shown below. Upper left (CU21), middle left (CU22), lower left (CU23), upper center (CU24), middle center (CU25), lower center (CU26), upper right (CU27), middle right (CU28), lower right (CU29) *1 If a catchlight is displayed across multiple areas, the position of the catchlight is classified and evaluated in the area where the center point of the catchlight is located (Figure 5, in this case the catchlight position is in the upper left). *2 If the center point of the catchlight is on the boundary line of the area, evaluate the position of the catchlight on the line where the center point of the catchlight is located (Figure 6, in this case the catchlight position is the top center). *3 If the center point of the catchlight is on the center point of the area, evaluate the position of the catchlight at the center point of the catchlight (Figure 7, in this case the position of the catchlight is in the center). (3) <Shape elements of shape> As the rules and means for relating shape elements and characteristic parts to evaluate changes in the shape of catchlights, the shape of catchlights is evaluated by applying characteristic part extraction conditions that determine the approximate shape of catchlights given individual identification symbols. <Shape element> The CU for evaluating the shape of the catchlight is shown below. Round (CU31), oval (CU32), square (CU33), hexagonal (CU34), octagonal (CU35), rectangular (CU36), design (CU37) *1 All shapes that are not (CU31) to (CU36) are considered design shapes (CU37). (4) <Shape element of size> As the rules and means for relating shape elements and characteristic parts to evaluate changes in the size of catchlights, the size of catchlights is evaluated by applying characteristic part extraction conditions for measuring and judging the size of catchlights that have been given individual identification symbols. <Size and shape element> The CU for assessing the size of catchlights is shown below. Large (CU41): Large enough to cover the entire pupil or almost the entire pupil. Slightly large (CU42): A size between large and medium. Medium (CU43): Pupil size is approximately 2mm to 4.5mm. Slightly small (CU44): A size between medium and small. Small (CU45): The smallest visible size. (5) <Shape element of brightness> As the rules and means for relating the shape element and characteristic part for evaluating the brightness of a catchlight, the brightness of the catchlight is evaluated by applying the characteristic part extraction conditions for determining the light intensity of brightness for a catchlight given an individual identification symbol. <Brightness shape element> The CU for rating the brightness of catchlights is shown below. Extremely bright (CU51): Bright enough to produce a glow around it. Bright (CU52): Completely white. Slightly dark (CU53): A brightness that is between light and dark. Dark (CU54): The minimum brightness that can be perceived as light. (6) <Color shape element> As the rules and means for relating the shape element and characteristic parts to evaluate the color of a catchlight, the color of the catchlight is evaluated by applying the characteristic part extraction conditions for determining the color of the catchlight using a color chart for the catchlight that has been given an individual identification symbol. <Color shape element> The CU for evaluating the color of catchlights is shown below. Gray (CU61), Pink (CU62), Red (CU63), Purple (CU64), Blue (CU65), Cyan (CU66), Green (CU67), Yellow (CU68), Orange (CU69) The reverse process of the above is the shape element combination process. 6. Emotion type discrimination processing and reverse emotion type discrimination processing The catchlight emotion type information output algorithm identifies the emotion type by matching the shape element code obtained from the catchlight image with the emotion type based on the association rules between the shape element code and the catchlight shape. This processing process is shown in the flowchart of Figure 14. The reverse processing of this is called the emotion type identification reverse processing. [Example]

[0010] 13 and 14 show an embodiment of the "device for determining the type of emotion expressed by a catchlight image" of the present invention, with reference to FIG. This device comprises an image input device 131 that takes in the pupil image as an electrical signal and converts it into a processable format, an arithmetic processing device 132 that analyzes the pupil image and determines catchlight element information from the input from the image input device, encodes and stores this information in an arithmetic processing device, and an output device 133 that outputs emotion type information calculated by the arithmetic processing device. Image input device 131 may be a video camera and a computer arithmetic processing device connected to it, arithmetic processing device 132 may be a computer arithmetic processing device equipped with storage means for storing programs and calculated data, and emotion type information output device 133 may be a numerical / symbol display device, a parallel or serial digital signal output device, or the like. (Specific example of processing to determine the type of emotion expressed by a catchlight image) The result of shape element coding (Figure 14(3)) for a pupil image (first row from the top, third column from the left in Figure 23) that has one catchlight in the pupil, is as follows: 101, A25, A36, A43, A52, A61 (first row from the top, second row from the left in Figure 23) The results of the emotion type determination process (FIG. 14(4)) based on this combination of shape element codes are as follows: "Surprise" (first row from the top, first column from the left in Figure 23) [Example] 15 and 16 show an embodiment of the "device for generating a catchlight image representing a type of emotion" of the present invention, with reference to FIG. This device comprises an emotion type information input device 151 that takes in emotion type information as an electrical signal and obtains predetermined emotion type information, an arithmetic processing device 152 that processes the input from the information input device to generate decoded pupil image data with catchlights and store it, and an image output device 153 that outputs the pupil image data stored in the arithmetic processing device. The emotion type information input device 151 may be a numerical / symbol display device, a parallel or serial digital signal input device, a video camera and a computer arithmetic processing device connected thereto, etc., the arithmetic processing device 152 may be a computer arithmetic processing device equipped with storage means for storing programs and calculated data, etc., and the image output device 153 may be a video output monitor and a computer arithmetic processing device connected thereto, etc. (Example of a process for generating catchlight images that represent different emotions) The result of the reverse emotion type determination process (FIG. 16(4A)) based on the emotion type information "anger" (third row from the top, first column from the left in FIG. 23) is as follows: 101, A26, A33, A44, A53, A63 (Third row from the top, second row from the left in Figure 23) This combination of shape element codes is subjected to shape element decoding processing (FIG. 16(3A)), and the data of the generated pupil image reflecting the catchlight becomes “anger” (third row from the top, third column from the left in FIG. 23). [Explanation of symbols]

[0011] 1 eye 2 Eyelids 3. Catchlight 4 Location determination area table 5 Catchlight center point 6 Area Boundaries 7 Area center point and area boundary center 8 Area boundary line, left center 9 Area boundary line, center right 10 Area boundary line, top center 11 Area boundary line bottom center 131 Image input device 132 Processing Unit 133 Emotion type information output device 151 Emotion type information input device 152 Processing Unit 153 Image output device 201 Feature Area Image (Square Screen) 202 Feature Area Image (Square Screen) 203 Feature Area Image (Square Screen)

Claims

1. an image input device that captures a catchlight image reflected in the pupil as an electrical signal and obtains predetermined catchlight image data; an arithmetic processing device having feature extraction processing means for receiving predetermined catchlight image data from the image input device and extracting a feature portion image based on specific portion extraction conditions, shape element extraction processing means for extracting shape element information from the feature portion image based on predetermined shape element and feature portion association rules to obtain catchlight shape element information, shape element encoding processing means for encoding the shape element information based on predetermined shape element codes and shape element association rules to calculate a shape element code, emotion type discrimination processing means for discriminating the emotion type by arithmetic processing the shape element code based on predetermined emotion types and association rules between shape element codes and catchlight shapes, and storage means for storing the emotion type information; an emotion type information output device that outputs the emotion type information stored in the arithmetic processing device; A device for determining the type of emotion of a catchlight, comprising:

2. The device for determining the emotion type of a catchlight according to claim 1, wherein the shape elements include six elements: (1) number, (2) position, (3) shape, (4) size, (5) brightness, and (6) color.

3. 2. The catchlight emotion type determination device according to claim 1, wherein the emotion type information includes six elements: (1) surprise, (2) fear, (3) anger, (4) disgust, (5) sadness, and (6) joy.

4. an information input device that receives the emotion type information determined by the determination device of claim 1 as an electrical signal and obtains predetermined emotion type information; an arithmetic processing device comprising: type determination inverse processing means for receiving predetermined emotion type information from the emotion type information input device, and performing emotion type determination inverse processing based on association rules between the predetermined emotion type, shape element code, and catchlight shape to calculate shape element code data; shape element decoding processing means for performing shape element decoding processing on the obtained shape element code data based on the association rules between shape element codes and shape elements to calculate catchlight shape element information; shape element extraction inverse processing means for performing shape element extraction processing on the obtained shape element information based on the association rules between shape elements and feature portions to obtain a predetermined feature portion image; processing means for performing feature portion extraction inverse processing on the feature portion image based on predetermined feature extraction conditions to generate pupil image data reflecting the catchlight; and storage means for storing the pupil image data. an emotion type image output device that outputs image data of the catchlight stored in the arithmetic processing device; A device for generating pupil image data reflecting catchlight, comprising:

5. An emotion type discrimination program that discriminates the type of emotion from the shape of catchlights reflected in the eyes, A computer operating device, an input means for inputting image data of the catchlight; a feature extraction processing means for extracting a feature portion image based on a specific portion extraction condition of the image data; a shape element extraction processing means for extracting shape elements from the characteristic portion image based on predetermined association rules between shape elements and characteristic portions, and obtaining shape element information of the catchlight; a shape element encoding processing means for encoding the shape element information based on predetermined shape element codes and shape element association rules to calculate a shape element code; an emotion type discrimination processing means for discriminating emotion type information by performing emotion type discrimination processing based on a predetermined emotion type from the shape element code and an association rule between the shape element code and the shape of the catchlight; a catchlight emotion type discrimination program for functioning as an output means for outputting the emotion type information;

6. A generation program for generating a shape of a catchlight in response to emotion type information of the catchlight determined by the emotion type determination program of claim 5, A computer operating device, emotion type information input means for inputting the emotion type information; an emotion type determination inverse processing means for determining an emotion type from the emotion type information based on a rule relating a predetermined emotion type, a shape element code, and a catchlight shape to calculate a shape element code; a shape element decoding processing means for decoding the shape element code based on a predetermined rule for associating the shape element with the shape element code to calculate shape element information of the catch light; a shape element extraction reverse processing means for obtaining a predetermined feature portion image from the shape element information based on a rule for associating predetermined shape elements with feature portions; a processing means for performing a feature extraction reverse process on the feature portion image based on predetermined feature extraction conditions to generate pupil image data reflecting catchlight; a generating program for generating pupil image data reflecting a catchlight, for functioning as an output means for outputting the pupil image data;

Images