Information processing device, information processing method, and program

Abstract

An information processing device (100) comprises: a scoring unit (130) that sets, on the basis of feature information of toys and a keyword list indicating a list of keywords corresponding to each of a plurality of suitability fields, field-specific scores for the respective toys; an acquisition unit (140) that acquires toy usage information including image information obtained by imaging the situation of usage of a toy by a child; an identification unit (150) that uses a toy detection model (151) to identify the toy used by the child from the acquired image information; a calculation unit (160) that uses a recognition model (161) to recognize a facial expression of the child from the acquired image information, and calculates a favorability level for the identified toy on the basis of the recognized facial expression; a determination unit (170) that determines a suitability field for which the child is suitable on the basis of the field-specific scores for the identified toy and the calculated favorability level for the toy; and an output unit (180) that outputs determination result information including the determined suitability field.

Description

Information Processing Apparatus, Information Processing Method, and Program

[0001] The present invention relates to an information processing apparatus, an information processing method, and a program.

[0002] Attention has been paid to determining a child's aptitude and making use of it in education. For example, Patent Document 1 discloses an information processing apparatus that selects a toy to be provided according to a child's growth situation and determines an aptitude field in which the child has an aptitude based on the degree of use of the toy by the child who has received the toy.

[0003] Japanese Unexamined Patent Application Publication No. 2021-89700

[0004] The information processing apparatus of Patent Document 1 stores a field-specific score indicating the degree to which each toy contributes to the development of the abilities of each of a plurality of aptitude fields, and uses the favorite degree input according to the usage situation of the child's toy and the field-specific score of the toy to obtain a score for each aptitude field of the child to be determined. The field-specific score and the favorite degree are set or input by an administrator, a child's guardian, or the like. Therefore, there is a problem that the objectivity of determining a child's aptitude is not sufficient.

[0005] The present invention has been made in view of the above problems, and an object thereof is to provide an information processing apparatus, an information processing method, and a program capable of improving the objectivity of determining a child's aptitude.

[0006] To achieve the above objective, the information processing device according to the present invention is an information processing device that determines the fields in which a child using a toy has an aptitude, according to the usage status of the toy, and includes: a scoring unit that sets a field-specific score for each toy indicating the degree to which each toy contributes to the development of each ability in the field of aptitude, based on characteristic information of each toy and a keyword list showing a list of keywords corresponding to each of a plurality of aptitude fields; an acquisition unit that acquires toy usage information including image information of the child's use of the toy; an identification unit that identifies the toy used by the child from the image information acquired by the acquisition unit using a pre-trained toy detection model that detects the toy used by the child using the image information as input; a calculation unit that recognizes the child's facial expression from the image information acquired by the acquisition unit using a pre-trained recognition model that recognizes facial expressions and calculates a favorability score for the identified toy based on the recognized facial expression; and a determination unit that determines the fields in which the child has an aptitude by obtaining an aptitude score indicating the level of aptitude for each field of the child who is the subject of determination, based on the field-specific score of the toy identified by the identification unit and the favorability score for the toy calculated by the calculation unit. The system includes an output unit that outputs judgment result information including the aptitude field determined by the judgment unit.

[0007] According to the present invention, it is possible to improve the objectivity of determining a child's aptitude.

[0008] This is a block diagram showing the configuration of an information processing system according to an embodiment of the present invention. This is a block diagram showing an example of the configuration of an information processing device according to an embodiment. This is a diagram showing an example of a toy database shown in Figure 2. This is a diagram showing an example of a keyword list. This is a diagram showing an example of a method for calculating the aptitude score calculated by the judgment unit shown in Figure 2. This is a diagram showing an example of an evaluation report output to the output unit shown in Figure 2. This is an explanatory diagram showing the physical configuration of the information processing device. This is a flowchart showing the scoring process flow of the information processing device according to an embodiment. This is a flowchart showing the judgment process flow of the information processing device according to an embodiment. This is a diagram showing an example of a personality keyword list. This is a diagram showing an example of a toy database according to a modified example. This is a diagram showing an example of a graph showing the trend of personality element scores generated by the output unit. This is a diagram showing an example of a talent discovery program sheet.

[0009] An information processing apparatus, an information processing method, and a program according to embodiments of the present invention will be described in detail with reference to the drawings. In the drawings, the same or corresponding parts are denoted by the same reference numerals.

[0010] An example of applying the information processing device 100 according to an embodiment of the present invention to the information processing system 1 shown in Figure 1 will be described. As shown in the figure, the information processing system 1 comprises the information processing device 100 and a terminal device 200, and the information processing device 100 and the terminal device 200 are connected via a communication network 300.

[0011] Information processing system 1 is a system that determines which of six fields (hereinafter referred to as "aptitude fields") a child has a high aptitude for, based on their use of toys and other factors. The fields include artist, engineer, liberal arts, communicator, athlete, and natural science.

[0012] The information processing device 100 is composed of one or more server computers. The information processing device 100 includes a toy database that stores information on multiple toys. Based on characteristic information (e.g., instruction manuals, specifications, etc.) that describes the characteristics of each toy registered in the toy database, the information processing device 100 automatically sets a category-specific score that indicates the degree to which each toy contributes to the development of each of the six appropriate areas of ability.

[0013] Furthermore, the information processing device 100 collects image information (still images or videos) of the child playing with toys, identifies the toys used by the child based on the collected image information, and evaluates the child's preference for the toys. The preference is calculated, for example, by recognizing emotions such as enjoyment or happiness based on the child's facial expressions while playing with the toys.

[0014] The information processing device 100 calculates an aptitude score indicating the level of suitability for each appropriate field, based on the child's preference for the toy identified from the image information and the toy's score in each field. The information processing device 100 generates an evaluation report including the trend of the aptitude score and outputs it to the terminal device 200.

[0015] The terminal device 200 is an information terminal such as a smartphone, tablet, or PC, and is used by users such as the child's guardian. The terminal device 200 transmits image information of the child playing with toys to the information processing device 100. The terminal device 200 also displays the evaluation report output by the information processing device 100.

[0016] The communication network 300 may include various types of networks. For example, local area networks (LANs), wide area networks (WANs) such as the Internet, telecommunications networks such as public switched telephone networks (PSTNs), wireless networks, public switched networks, satellite networks, cellular networks, public land mobile communications networks (PLMNs), metropolitan area networks (MANs), private networks, ad hoc networks, intranets, fiber optic-based networks, etc., or any combination of these or other types of networks.

[0017] Next, the functional configuration of the information processing device 100 will be explained using Figure 2. As shown in the figure, the information processing device 100 includes a toy DB 110, a user DB 120, a scoring unit 130, an acquisition unit 140, a identification unit 150, a calculation unit 160, a determination unit 170, and an output unit 180.

[0018] The toy database 110 stores information about each toy. The toy database 110 may also store characteristic information such as the instruction manual and specifications of each toy, associated with the toy ID. Figure 3 shows an example of the toy database 110. As shown in the figure, the toy database 110 includes "basic information" such as the toy name, toy ID which indicates information that uniquely identifies the toy, toy category which indicates the category of the toy, manufacturer name which indicates the name of the manufacturer that produces the toy, and target age of children (minimum and maximum age); "toy image" which shows the appearance of the toy; "suitability field score" which includes the field (attribute) to which the toy belongs and the score for each of the six suitability fields mentioned above; and "toy characteristics" which indicates the elements that contribute to the development of children when each toy is used. The "toy image" may store multiple images of the toy taken from various angles.

[0019] The "Toy Characteristics Information" includes items such as "Developmental Contributing Elements," which indicate elements of development in a child's aptitude field, and "Presence / Absence," which indicates the degree to which the toy contributes to the development of each element. The degree to which the toy contributes to the development of each element is not limited to presence / absence; it may be divided into any number of levels. The "Aptitude Field Score" includes a "Toy Category," which indicates the attributes of the toy, and one or more categories such as artist, engineer, liberal arts, communicator, athlete, and natural science are entered. Each score in the "Aptitude Field Score" is set by the scoring unit 130.

[0020] In the example of Toy DB110 in Figure 3, the "Toy Name" is "Toy A," the "Toy ID" is "1766," and the "Target Age" is "36 months to 60 months old." Furthermore, this toy contributes to the development of "three-dimensional spatial awareness" as a "Developmental Contributing Element," the "Toy Category" is "Engineer," and the "Aptitude Field Score" indicates that the aptitude field for artist is "0," the aptitude field for engineer is "4," the aptitude field for liberal arts is "0," the aptitude field for communicator is "0," the aptitude field for athlete is "0," and the aptitude field for natural science is "0."

[0021] Returning to Figure 2, the user database 120 stores user information such as the child's name, gender, age, and date of birth, as well as the child's facial image. The user database 120 also stores image information transmitted by the terminal device 200, which shows the child playing with toys. Furthermore, the user database 120 stores evaluation reports previously output by the output unit 180. Each piece of data is assigned a user ID to identify the user, and each piece of data is managed on a per-user basis.

[0022] The scoring unit 130 sets a category-specific score for each of the six suitability categories for each toy registered in the toy database 110. Specifically, the scoring unit 130 sets a score for each of the six suitability categories according to the characteristics of each toy. For example, the scoring unit 130 includes a keyword list 131 that defines keywords corresponding to each suitability category, as illustrated in Figure 4. The scoring unit 130 searches for keywords included in the keyword list from characteristic information such as the instruction manual and specifications of each toy. The scoring unit 130 calculates a category-specific score for each suitability category based, for example, on the frequency of occurrence of keywords in each suitability category, and stores it in the toy database 110. Details of the processing by the scoring unit 130 will be described later.

[0023] The acquisition unit 140 acquires image information of a child playing with a toy. For example, a user (such as a child's guardian) takes a picture of the child playing with a toy using the camera on the terminal device 200 and transmits the image information to the information processing device 100. The acquisition unit 140 stores the received image information in the user database 120, associating it with the user ID. Note that the image information is an example of toy usage information.

[0024] The identification unit 150 identifies toys used by children from image information. The identification unit 150 includes a toy detection model 151 generated by machine learning, deep learning, etc., which takes image information as input to detect toys used by children. For example, the toy detection model 151 uses a convolutional neural network (CNN) to detect children (people) and toys in the image and analyze the contact state between the child and the toy to identify the toy. The toy detection model 151 is constructed using supervised learning. For learning, training data, which is a dataset of various pre-labeled images including children playing with toys, is used. Each image clearly indicates the regions of the human body and toys, as well as identification information of the toy the child is playing with (toy name or toy ID). As a result, the toy detection model 151 learns features such as the shape, color, and size of the toys and can identify which toys are being used by analyzing the contact state between the child and the toy.

[0025] The toy detection model 151 may be trained to detect the position of a hand and identify the toy closest to the detected hand position, or it may detect the child to be processed from image information using facial recognition.

[0026] The calculation unit 160 recognizes the child's facial expression from the image information and calculates the child's preference for the toy identified by the identification unit 150 based on the recognized facial expression. The calculation unit 160 includes a recognition model 161 generated by machine learning, deep learning, etc., which takes the image information as input and recognizes the child's facial expression contained in the image information. The recognition model 161 recognizes the child's facial expression by identifying the most plausible one from a predetermined set of multiple types of facial expressions. For example, the types of facial expressions are classified into multiple types by words that indicate emotions such as "happy," "joyful," "rejoicing," "surprised," "angry," and "neutral." The recognition model 161 is trained using a supervised learning method that uses training data, which is a set of various image datasets that have been pre-labeled with types of facial expressions, to automatically extract features corresponding to each facial expression. The calculation unit 160 calculates the favorability score from the recognized facial expression based on pre-set rules, such as calculating a favorability score of "2" if the recognized facial expression is "happy," "joyful," or "rejoicing," calculating a favorability score of "0.5" if the recognized facial expression is "surprised" or "angry," and calculating a favorability score of "1" if "neutral" or any other facial expression is recognized.

[0027] The recognition model 161 may also recognize facial expressions, including the degree of each emotion. In that case, the calculation unit 160 may calculate a degree of liking based on the emotion and its degree recognized by the recognition model 161.

[0028] Furthermore, the calculation unit 160 may calculate the level of liking based not only on facial expressions, but also on the time and frequency with which the child plays with each toy, and the voice they make while playing. Specifically, for example, the calculation unit 160 may set a higher level of liking for toys that the child plays with for a long time or frequently. In addition, the calculation unit 160 may recognize emotions from the voice using a machine learning model that detects emotions from the voice based on the tone and volume of the voice, and if, for example, a positive emotion is detected, it may set a higher level of liking.

[0029] Furthermore, if multiple still images including the same toy are input, the calculation unit 160 may calculate a favorability score corresponding to the facial expression recognized by the recognition model 161 for each still image, and calculate the average of these favorability scores as the favorability score for the target toy. Also, if a video is input, the calculation unit 160 may set a weight for each facial expression according to the proportion of time spent on each recognized facial expression for each toy, and calculate a favorability score by multiplying the favorability score corresponding to each facial expression by the weight and adding them together.

[0030] The determination unit 170 calculates an aptitude score indicating the degree of aptitude for each aptitude field, based on the likeability score of the toy calculated by the calculation unit 160 and the aptitude field score of the toy set by the scoring unit 130. Specifically, the determination unit 170 calculates the aptitude score for each aptitude field by multiplying the likeability score by the aptitude field score. In the calculation example illustrated in Figure 5, the determination unit 170 multiplies the "aptitude field score" of toys A to D identified by the identification unit 150 by the respective "likeability score" calculated by the calculation unit 160, adds them up for each aptitude field, and calculates an aptitude score for each aptitude field, such as "3" for artist, "8" for engineer, "2" for liberal arts, "0.5" for communicator, "1.5" for athlete, and "2" for natural science. The judgment unit 170 adds the calculated aptitude score and the previously calculated aptitude score for each aptitude field to calculate a cumulative value for each aptitude field.

[0031] Furthermore, the determination unit 170 determines the child's aptitude area based on pre-set rules. Details of the process for determining the aptitude area will be described later.

[0032] Returning to Figure 2, the output unit 180 generates an evaluation report based on the aptitude score for each aptitude field and the aptitude field determination result calculated by the determination unit 170. An example of the evaluation report 181 is shown in Figure 6. The evaluation report 181 includes the aptitude field determination result, the cumulative value of the aptitude score for each aptitude field, and evaluation comments. In the "Aptitude Field Determination Result," one or more categories such as artist, engineer, liberal arts, communicator, athlete, and natural science are entered. If no aptitude field is determined, it may be left blank, or a comment indicating that continued observation is necessary may be entered. The evaluation comments may be created by the administrator of the information processing device 100, or they may be generated by the text generation AI. When the text generation AI is used to create the comments, the administrator simply inputs a prompt to the text generation AI instructing it to generate evaluation comments based on the child's profile, the cumulative value of the aptitude score, etc., and has the text generation AI output the evaluation comments. Note that evaluation report 181 is an example of judgment result information.

[0033] (Hardware configuration of the information processing device) Figure 7 is a block diagram showing the hardware configuration of the information processing device 100. The information processing device 100 includes a CPU (Central Processing Unit) 11 that executes processing according to a program, a RAM (Random Access Memory) 12 which is volatile memory, a ROM (Read Only Memory) 13 which is non-volatile memory, a storage unit 14 that stores data, an input unit 15 that accepts information input, a display unit 16 that visualizes and displays the information, and a communication unit 17 that sends and receives information, and these are connected via an internal bus 99.

[0034] The CPU 11 controls the operation of the entire information processing device 100, is connected to each component, and exchanges control signals and data. The CPU 11 performs various processes by reading programs stored in the memory unit 14 into the RAM 12 and executing them. The CPU 11 performs the processes of the scoring unit 130, acquisition unit 140, identification unit 150, calculation unit 160, determination unit 170, and output unit 180, which are the main functions provided by the program.

[0035] RAM 12 is for temporarily storing data and programs, and holds programs and data read from the storage unit 14, as well as other data necessary for communication. RAM 12 is used as the work area of ​​the CPU 11.

[0036] ROM 13 stores control programs, BIOS (Basic Input Output System), and other data that the CPU 11 executes for the basic operation of the information processing device 100.

[0037] The storage unit 14 includes a hard disk drive, flash memory, etc., and stores programs executed by the CPU 11 and various data used during program execution. The storage unit 14 functions as a toy database 110 and a user database 120. The storage unit 14 also stores a toy detection model 151 and a recognition model 161.

[0038] The input unit 15 is a user interface equipped with a touch panel, keyboard, mouse, communication device, etc. The input unit 15 receives operation input from the user of the information processing device 100 and outputs a signal corresponding to the received operation input to the CPU 11.

[0039] The display unit 16 is a display device such as a liquid crystal display or an organic EL (Electro-Luminescence) display that visualizes and displays information.

[0040] The communication unit 17 is a network termination device or wireless communication device connected to a network, and a serial interface or LAN (Local Area Network) interface connected to them. The information processing device 100 communicates with other information processing devices, etc., via the communication unit 17. The communication unit 17 functions as an acquisition unit 140 and an output unit 180.

[0041] (Scoring Process) Next, the operation of the information processing system 1 will be explained with reference to Figures 8 and 9. First, with reference to Figure 8, the scoring process, which sets category-specific scores for each toy in the toy database 110, will be explained. The scoring process is performed, for example, when a new toy is registered in the toy database 110.

[0042] The scoring unit 130 of the information processing apparatus 100 acquires the feature information of the toy to be processed (step S101). For example, the scoring unit 130 acquires feature information such as the instruction manual and specification of the toy to be processed from the toy DB 110.

[0043] Next, the scoring unit 130 searches for keywords included in the preset keyword list 131 from the feature information acquired in step S101 (step S102). Specifically, the scoring unit 130 refers to the keyword list 131 illustrated in FIG. 4 and searches for keywords in the feature information that match the keywords included in the keyword list 131. In addition to the matching keywords, the scoring unit 130 may extract keywords that are semantically similar to the keywords included in the keyword list 131 by using techniques such as word embedding.

[0044] Next, the scoring unit 130 sets the field-specific scores for each appropriate field (step S103). For example, the scoring unit 130 sets the field-specific scores based on the frequency of occurrence of keywords for each appropriate field. Specifically, the scoring unit 130 counts the number of occurrences of the keywords searched in step S102 for each appropriate field. Next, the scoring unit 130 divides the number of occurrences of the keywords by the number of words included in the feature information for each appropriate field to calculate the frequency of occurrence. For example, when the number of occurrences of keywords in the artist field is 5 and the number of words included in the feature information is 100, 0.05 is calculated as the frequency of occurrence of keywords in the artist field. Next, the scoring unit 130 normalizes the calculated frequency of occurrence for each appropriate field to the same scale (for example, 0 to 5) to calculate the field-specific score for each appropriate field. The scoring unit 130 sets the calculated field-specific scores in the toy DB 110 and ends the process.

[0045] The method for calculating the category-specific scores by the scoring unit 130 is not limited to this, and any method may be used. For example, the scoring unit 130 may set category-specific scores based on the number of keywords searched. For example, if the keyword list 131 contains 20 keywords in the artist category, and 5 of those keywords are searched from the feature information, the matching score may be calculated as 5 / 20 = 0.25, and the matching scores calculated for each aptitude category may be normalized to the same scale to calculate the category-specific score for each aptitude category.

[0046] (Determination Process) Next, with reference to Figure 9, the determination process by which the information processing device 100 determines the suitable field will be explained. The determination of the suitable field is performed, for example, at a predetermined timing (e.g., every month). By the time of the determination, the user operates the terminal device 200 to transmit image information of the child playing with toys to the information processing device 100. The acquisition unit 140 of the information processing device 100 stores the image information transmitted from the terminal device 200 in the user DB 120, associating it with the user ID. When the information processing device 100 determines that the timing for determining the suitable field is approaching (for example, three days before the determination timing), it starts processing.

[0047] First, the information processing device 100 notifies the terminal device 200 via the communication unit 17 that the determination timing is approaching (step S201). Next, when the acquisition unit 140 detects that the determination timing has arrived, it acquires image information transmitted from the user to be processed from the toy DB 110 (step 202). For example, the acquisition unit 140 accesses the user DB 120 and refers to the shooting date included in the image information associated with the user ID of the user to be processed, and acquires image information whose shooting date is after the date of the previous determination timing, but before the date of the determination timing today. The acquisition unit 140 transmits the acquired image information to the identification unit 150.

[0048] Next, the specifying unit 150 specifies the toy used by the child from the image information acquired by the acquisition unit 140 (step S203). Specifically, the specifying unit 150 inputs the image information acquired by the acquisition unit 140 into the toy detection model 151, and obtains detection result information including the position information (bounding box) of the toy in the image and the identification information (toy name or toy ID) of the toy.

[0049] Next, the calculation unit 160 calculates the favorability of the toy specified by the specifying unit 150 (step S204). Specifically, the calculation unit 160 inputs the image information acquired by the acquisition unit 140 and the detection result information obtained in step S203 into the recognition model 161, and obtains the inference result of the expression of the child in the image. When the expression recognized by the recognition model 161 is "happy", "joyful", or "delighted", the calculation unit 160 calculates the favorability "2", and when the recognized expression is "surprised" or "angry", the calculation unit 160 calculates the favorability "0.5". When the expression "neutral" and other expressions are recognized, the calculation unit 160 calculates the favorability "1". Based on such rules, the calculation unit 160 calculates the favorability for each toy from the expression recognized by the recognition model 161.

[0050] Next, the determination unit 170 calculates an appropriateness score indicating the degree of appropriateness of the child for each appropriateness field (step S205). Specifically, the determination unit 170 accesses the toy DB 110 and acquires the score by field of the toy specified in step S203. As illustrated in FIG. 5, the determination unit 170 accumulates the acquired score by field and the favorability of the toy calculated in step S204 for each toy, and adds them for each appropriateness field to calculate the appropriateness score for each appropriateness field.

[0051] Next, the determination unit adds the appropriateness score calculated in step S205 to the cumulative value, which is the cumulative sum of the previous appropriateness scores, to update the cumulative value for each appropriateness field (step S206).

[0052] Next, the determination unit 170 determines the aptitude field (step S207). Specifically, the determination unit 170 determines whether the cumulative value of each aptitude field calculated in step S206 and the difference rate between the cumulative values ​​of the aptitude field with the highest cumulative value and the aptitude field with the second highest cumulative value are equal to or greater than predetermined thresholds. If both the cumulative value and the difference rate of each aptitude field are equal to or greater than their respective thresholds, the determination unit 170 determines that the aptitude field with the highest cumulative value is the aptitude field of the child being processed. On the other hand, if either or both of the cumulative value and the difference rate of each aptitude field are not equal to or greater than the threshold, the determination unit 170 determines that no aptitude field is to be determined. The determination unit 170 notifies the output unit 180 of the determination result.

[0053] Next, the output unit 180 outputs an evaluation report 181. Specifically, the output unit 180 generates an evaluation report 181 as illustrated in Figure 6, based on the cumulative value of the aptitude score for each aptitude field calculated in step S206, the cumulative value of past aptitude scores, the judgment result from step S207, and comments generated by the administrator or the text generation AI. The generated evaluation report 181 is then sent to the terminal device 200 (step S208), and the process ends. In the illustrated example, the case where an engineer is determined to be an aptitude field is shown, but if it is determined in step S207 that there is no aptitude field, the "Aptitude Field Judgment Result" item may be left blank, or a comment indicating that continued observation is necessary may be entered.

[0054] As explained above, the information processing device 100 automatically sets a category-specific score indicating the degree to which each toy contributes to the development of each ability in the appropriate field, based on the characteristic information of each toy. The information processing device 100 also identifies the toy used by the child from image information of the child playing with the toy and recognizes the child's facial expression. Based on the recognized facial expression, the information processing device 100 calculates the child's liking for the toy. Based on the calculated liking score of the toy and the category-specific score of the toy, the information processing device 100 calculates an aptitude score for each aptitude field and determines the aptitude field in which the child has an aptitude. Therefore, the objectivity of determining the child's aptitude can be improved. In addition, since it eliminates the need for manual calculation and setting of category-specific scores for each toy, and judgment and input of the child's liking for the toy, the child's aptitude can be determined more efficiently.

[0055] (Variation) In addition to determining a child's aptitude, the information processing device 100 may also diagnose the child's personality. Specifically, the information processing device 100 may classify the elements that constitute a child's personality into five categories (for example, agreeableness, conscientiousness, sensitivity, openness, and extroversion), and calculate a score for each classified personality element based on the child's use of toys. Note that the personality elements are not limited to five, and any number may be classified, and the names and types of personality elements may also be set arbitrarily.

[0056] In this case, the scoring unit 130 further includes a personality keyword list 132 that defines keywords corresponding to each personality element, as illustrated in Figure 10. Based on the characteristic information of each toy, the scoring unit 130 can calculate the element score for each personality element using a calculation method similar to that used for the field-specific scores of the aptitude field, for example, based on the frequency of occurrence of keywords for each personality element.

[0057] An example of the toy DB 110A in this case is shown in Figure 11. As shown in the figure, the toy DB 110A includes the "basic information," "toy image," and "aptitude field score" stored in the toy DB 110, as well as the "personality element score" item which stores the personality score for each personality element calculated by the scoring unit 130. Furthermore, the "toy characteristics" may include information indicating the elements that contribute to the development of a child's personality when each toy is used.

[0058] In step S205, the judgment unit 170 calculates the aptitude score for the aptitude field, as well as the personality score for each personality element. Specifically, the judgment unit 170 accesses the toy DB 110 and obtains the personality element scores for the toys identified in step S203. The judgment unit 170 then calculates the personality score for each personality element by summing the obtained personality element scores and the toy's likeability score calculated in step S204 for each toy, and adding them together for each personality element. The output unit 180 can also generate a transition graph 182 showing the transition of personality element scores, as illustrated in Figure 12, and include it in the evaluation report 181.

[0059] Furthermore, the judgment unit 170 may determine the child's aptitude based on the determined aptitude field and diagnosed personality, using the aptitude discovery program sheet 171, which defines the child's aptitude consisting of a combination of aptitude fields and personality, as illustrated in Figure 13. As shown in the figure, the aptitude discovery program sheet includes information on "combinations of personality elements" indicating whether each of the five personality elements is high or low, and "the child's aptitude" determined by the combination of aptitude fields, personality, and aptitude fields determined by the judgment unit 170. For example, the judgment unit 170 compares the calculated scores for each personality element with predetermined standard values ​​and determines whether each of the personality elements—agreeableness, conscientiousness, sensitivity, openness, and extroversion—is high or low. If the judgment unit 170 determines that all personality elements are high and that the aptitude field is engineering, it determines that the child has the aptitude of an "innovation leader," as shown in the figure, based on these combinations.

[0060] In the above embodiment, the determination unit 170 calculates the cumulative value for each aptitude field, determines whether the cumulative value for each aptitude field is above a predetermined threshold, and determines whether the difference rate between the cumulative value of the aptitude field with the highest cumulative value and the cumulative value of the aptitude field with the second highest cumulative value is above a predetermined threshold, and if it determines that there are aptitude fields that satisfy each threshold, it determines that the aptitude fields that satisfy the conditions are the child's aptitude fields, but it is not limited to this. For example, the determination unit 170 may calculate the difference rate (deviation) between the cumulative value for each aptitude field and the average value of the cumulative values ​​of other aptitude fields, determine whether these are above a predetermined threshold, and if it determines that there are aptitude fields that satisfy each threshold, it may determine that one or more aptitude fields that satisfy the conditions are the child's aptitude fields. In addition, the growth rate of the cumulative value at a certain point in time may be used to determine whether or not there are aptitude fields. Furthermore, each threshold may be a different value for each month of age.

[0061] Furthermore, the information processing device 100 may also acquire, in addition to image information capturing the usage status of children's toys, text information entered by parents regarding toys that children frequently played with and how children played with them, and calculate the likeability score. In this case, the calculation unit 160 may calculate the likeability score by any method, such as multiplying the likeability score calculated based on the image information by a predetermined coefficient for the likeability of the toy derived from the text information.

[0062] Furthermore, the information processing device 100 can be implemented using a regular computer, not necessarily a dedicated device. For example, the information processing device 100 that performs the above-mentioned processing may be configured by installing a program for performing any of the above-mentioned operations from a recording medium stored on the computer to the computer. Alternatively, multiple computers may cooperate to form a single information processing device 100 or terminal device 200.

[0063] Furthermore, if the above-mentioned functions are realized through a division of labor between the OS (Operating System) and the application, or through collaboration between the OS and the application, then only the parts other than the OS may be stored on the medium.

[0064] Furthermore, it is possible to superimpose a program onto a carrier wave and distribute it via a communication network. For example, the program could be posted on a bulletin board system (BBS) on the communication network and distributed via the network. These programs could then be launched and executed under the control of the operating system, similar to other application programs, thereby enabling the execution of the aforementioned processes.

[0065] Furthermore, the information stored in the memory unit 14 is centrally managed by a cloud server located on the network, and the information processing device 100 may access the cloud server as needed to read and write information. In this case, the information processing device 100 does not need to have a toy DB 110 and a user DB 120. Also, the scoring process and judgment process performed by the information processing device 100 may be executed on the cloud using the information stored on the cloud server.

[0066] This invention allows for various embodiments and modifications without departing from the broad spirit and scope of the invention. Furthermore, the embodiments described above are for illustrative purposes only and do not limit the scope of the invention. In other words, the scope of this invention is indicated not by the embodiments, but by the claims. Various modifications made within the scope of the claims and the equivalent scope of the meaning of the invention are considered to be within the scope of this invention.

[0067] 1 Information processing system, 100 Information processing device, 200 Terminal device, 300 Network, 110, 110A Toy DB, 120 User DB, 130 Scoring unit, 131 Keyword list, 132 Personality keyword list, 140 Acquisition unit, 150 Identification unit, 151 Toy detection model, 160 Calculation unit, 161 Recognition model, 170 Judgment unit, 171 Talent discovery program sheet, 180 Output unit, 181 Evaluation report, 182 Transition graph, 11 CPU, 12 RAM, 13 ROM, 14 Storage unit, 15 Input unit, 16 Display unit, 17 Communication unit, 99 Internal bus

Claims

1. An information processing device for determining the fields in which a child has aptitude based on the usage status of a toy, comprising: a scoring unit that sets a field-specific score for each toy indicating the degree to which each toy contributes to the development of each ability in the field of aptitude, based on characteristic information of each toy and a keyword list showing a list of keywords corresponding to each of a plurality of aptitude fields; an acquisition unit that acquires toy usage information including image information of the child's use of the toy; an identification unit that identifies the toy used by the child from the image information acquired by the acquisition unit using a pre-trained toy detection model that detects the toy used by the child using the image information as input; a calculation unit that recognizes the child's facial expression from the image information acquired by the acquisition unit using a pre-trained recognition model that recognizes facial expressions and calculates a favorability score for the identified toy based on the recognized facial expression; a determination unit that determines the fields in which the child has aptitude by obtaining an aptitude score indicating the level of aptitude for each field of the child who is the subject of determination, based on the field-specific score of the toy identified by the identification unit and the favorability score for the toy calculated by the calculation unit; and an output unit that outputs determination result information including the aptitude fields determined by the determination unit. An information processing device equipped with the following features.

2. The toy detection model is generated by machine learning on training data in which the regions of the human body and the toy used by the child and identification information that uniquely identifies the toy are assigned as labels to the image information, and the identification unit identifies the toy used by the child by inputting the image information acquired by the acquisition unit into the toy detection model, the information processing apparatus according to claim 1.

3. The information processing apparatus according to claim 1 or 2, wherein the recognition model is generated by machine learning on training data to which one of a predetermined number of facial expressions is assigned as a label, and the calculation unit recognizes a child's facial expression by inputting the image information acquired by the acquisition unit into the recognition model.

4. The information processing apparatus according to any one of claims 1 to 3, wherein the determination unit obtains the suitability score for each field of the child being judged by multiplying the field-specific score of the toy identified by the identification unit by the favorability score of the toy calculated by the calculation unit, and if the cumulative value of the field with the highest cumulative value obtained by accumulating the suitability scores obtained over a predetermined period and the difference between the cumulative value of the field with the highest cumulative value and the cumulative value of the field with the second highest cumulative value are both greater than or equal to a predetermined threshold, the determination unit determines that the field with the highest cumulative value is the suitable field.

5. The information processing apparatus according to any one of claims 1 to 4, wherein the scoring unit further sets a personality element score for each toy, indicating the degree to which each toy contributes to the development of each of the personality elements, based on the characteristic information of each toy and a personality keyword list showing a list of keywords corresponding to each of the multiple personality elements that constitute the child's personality, and the determination unit diagnoses the child's personality by determining the personality score for each of the personality elements of the child who is the subject of determination, based on the personality element score of the toy identified by the identification unit and the favorability rating for the toy calculated by the calculation unit.

6. The information processing apparatus according to claim 5, wherein the determination unit refers to a talent discovery program sheet that defines a child's talent consisting of a combination of aptitude fields and personality traits, and determines the child's talent based on the determined aptitude fields and the calculated personality scores for each personality element.

7. An information processing method comprising: a computer that determines fields in which a child has an aptitude based on the usage status of a toy, sets a field-specific score for each toy indicating the degree to which each toy contributes to the development of each ability in the field of aptitude, based on characteristic information of each toy and a keyword list showing a list of keywords corresponding to each of several aptitude fields; acquires toy usage information including image information of the child's use of the toy; identifies the toy used by the child from the acquired image information using a pre-trained toy detection model that detects toys used by the child using the image information as input; recognizes the child's facial expression from the acquired image information using a pre-trained recognition model that recognizes facial expressions, and calculates a degree of liking for the identified toy based on the recognized facial expression; determines the fields in which a child has an aptitude by obtaining an aptitude score indicating the degree of aptitude for each field of the child being judged, based on the field-specific score of the identified toy and the calculated degree of liking for the toy; and outputs judgment result information including the determined fields of aptitude.

8. A program that causes a computer to determine the fields in which a child has aptitudes based on the usage of a toy, to perform the following steps: setting a field-specific score for each toy, indicating the degree to which each toy contributes to the development of each ability in the field of aptitude, based on characteristic information of each toy and a keyword list showing a list of keywords corresponding to each of several aptitude fields; acquiring toy usage information, including image information of the child's use of the toy; identifying the toy used by the child from the acquired image information using a pre-trained toy detection model that detects toys used by the child using the image information as input; recognizing the child's facial expressions from the acquired image information using a pre-trained recognition model that recognizes facial expressions, and calculating a degree of liking for the identified toy based on the recognized facial expressions; determining the fields in which a child has aptitudes by calculating an aptitude score indicating the level of aptitude for each field of the child being judged, based on the field-specific score of the identified toy and the calculated degree of liking for the toy; and outputting judgment result information including the determined fields of aptitude.

Images