Information processing device, information processing method, and program

Abstract

The information processing device (100) includes a scoring unit (130) that sets a category-specific score for each toy based on characteristic information of each toy and a keyword list showing a list of keywords corresponding to each of a plurality of aptitude categories; an acquisition unit (140) that acquires toy usage information including image information capturing a situation in which a child is using the toy; an identification unit (150) that identifies the toy used by the child from the acquired image information using a toy detection model (151); a calculation unit (160) that recognizes the child's facial expression from the acquired image information using a recognition model (161) and calculates a likeability rating for the identified toy based on the recognized facial expression; a determination unit (170) that determines an aptitude category for which the child has aptitude based on the category-specific score of the identified toy and the calculated likeability rating for the toy; and an output unit (180) that outputs determination result information including the determined aptitude category.

Description

[Technical field]

[0001] The present invention relates to an information processing device, an information processing method, and a program. [Background technology]

[0002] Attention is being paid to determining a child's aptitude and utilizing it in education. For example, Patent Document 1 discloses an information processing device that selects toys to be provided to a child according to the child's developmental state, and determines the aptitude field for the child based on the degree to which the child uses the toys provided. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2021-89700 Summary of the Invention [Problem to be solved by the invention]

[0004] The information processing device of Patent Document 1 stores a category score indicating the degree to which each toy contributes to the development of abilities in each of a plurality of aptitude categories, and uses the favorite degree input according to the child's use of the toy and the category score of the toy to determine the score for each aptitude category of the child to be judged. The category score and favorite degree are set or input by an administrator or a guardian of the child, respectively. Therefore, there is a problem in that the judgment of the child's aptitude is not sufficiently objective.

[0005] The present invention has been made in consideration of the above-mentioned problems, and aims to provide an information processing device, an information processing method, and a program that can improve the objectivity of determining a child's aptitude. [Means for solving the problem]

[0006] In order to achieve the above object, an information processing device according to the present invention comprises: An information processing device that determines a field in which a child using a toy has aptitude according to a usage situation of the toy, a scoring unit that sets, for each toy, a field-specific score that indicates the degree to which each toy contributes to the development of each ability in the aptitude field, based on the characteristic information of each toy and a keyword list that indicates a list of keywords corresponding to each of a plurality of aptitude fields; an acquisition unit that acquires toy usage information including image information capturing a situation in which the child is using the toy; an identification unit that identifies a toy used by a child from the image information acquired by the acquisition unit using a pre-trained toy detection model that detects toys used by a child using the image information as an input; a calculation unit that recognizes a facial expression of the child from the image information acquired by the acquisition unit using a recognition model that has been trained in advance to recognize facial expressions, and calculates a liking rating for the specified toy based on the recognized facial expression; a determination unit that determines an aptitude field for which the child has aptitude by calculating an aptitude score that indicates a level of aptitude for each of the fields of the child to be determined based on the field-specific scores of the toys identified by the determination unit and the likeability rating for the toys calculated by the calculation unit; an output unit that outputs judgment result information including the aptitude field judged by the judgment unit; Equipped with. Effect of the Invention

[0007] According to the present invention, it is possible to improve the objectivity of determining a child's aptitude. [Brief description of the drawings]

[0008] [Figure 1] 1 is a block diagram showing a configuration of an information processing system according to an embodiment of the present invention. [Diagram 2] 1 is a block diagram showing an example of a configuration of an information processing device according to an embodiment; [Diagram 3] 3 is a diagram showing an example of a toy DB shown in FIG. 2. FIG. [Figure 4]FIG. 13 is a diagram illustrating an example of a keyword list. [Diagram 5] 3 is a diagram showing an example of a method for calculating an aptitude score calculated by a determination unit shown in FIG. 2. FIG. [Figure 6] 3 is a diagram showing an example of an evaluation report outputted to an output unit shown in FIG. 2. FIG. [Figure 7] FIG. 2 is an explanatory diagram illustrating a physical configuration of an information processing device. [Figure 8] 10 is a flowchart showing the flow of a scoring process of the information processing device according to the embodiment. [Figure 9] 10 is a flowchart showing the flow of a determination process of the information processing device according to the embodiment. [Figure 10] FIG. 13 is a diagram showing an example of a personality keyword list. [Figure 11] FIG. 13 is a diagram showing an example of a toy DB according to a modified example. [Figure 12] FIG. 13 is a diagram showing an example of a transition graph of personality element scores generated by an output unit. [Figure 13] FIG. 13 is a diagram showing an example of a talent discovery program sheet. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] An information processing device, an information processing method, and a program according to an embodiment of the present invention will be described in detail with reference to the drawings. In the drawings, the same or corresponding parts are denoted by the same reference numerals.

[0010] An information processing device 100 according to an embodiment of the present invention will be described using an example in which it is applied to an information processing system 1 shown in Fig. 1. As shown in the figure, the information processing system 1 includes the information processing device 100 and a terminal device 200, and the information processing device 100 and the terminal device 200 are connected via a communication network 300.

[0011] The information processing system 1 is a system that determines the aptitude field in which a child has high aptitude from six fields (hereinafter referred to as aptitude fields): artist, engineer, liberal arts, communicator, athlete, and natural science, based on factors such as the child's use of toys.

[0012] The information processing device 100 is composed of one or more server computers. The information processing device 100 includes a toy DB (database) that stores information on a plurality of toys. The information processing device 100 automatically sets a field-specific score indicating the degree to which each toy contributes to the development of abilities in each of six aptitude fields, based on characteristic information (e.g., instructions, specifications, etc.) indicating the characteristics of each toy registered in the toy DB.

[0013] The information processing device 100 also collects image information (still images or video) of a child playing with a toy, identifies the toy used by the child based on the collected image information, and evaluates the child's likeability for the toy. The likeability is calculated by recognizing, for example, feelings such as enjoyment or happiness based on the facial expression of the child playing with the toy.

[0014] The information processing device 100 calculates an aptitude score indicating the level of aptitude for each aptitude field based on the child's liking level for the toy identified from the image information and the field-specific score of the toy. The information processing device 100 generates an evaluation report including the progress of the aptitude score and outputs it to the terminal device 200.

[0015] The terminal device 200 is an information terminal such as a smartphone, a tablet, or a PC, and is used by a user such as a child's guardian. The terminal device 200 transmits image information captured of a child playing with a toy to the information processing device 100. The terminal device 200 also displays an evaluation report output by the information processing device 100.

[0016] Communications network 300 may include various types of networks, such as a local area network (LAN), a wide area network (WAN) such as the Internet, a telecommunications network such as the public switched telephone network (PSTN), a wireless network, a public switched network, a satellite network, a cellular network, a public land mobile network (PLMN), a metropolitan area network (MAN), a private network, an ad-hoc network, an intranet, an optical fiber-based network, or any combination of these or other types of networks.

[0017] Next, a functional configuration of the information processing device 100 will be described with reference to Fig. 2. As shown in the figure, the information processing device 100 includes a toy DB 110, a user DB 120, a scoring section 130, an acquisition section 140, an identification section 150, a calculation section 160, a determination section 170, and an output section 180.

[0018] The toy DB 110 stores information about each toy. The toy DB 110 may store characteristic information such as instructions and specifications for each toy in association with a toy ID. FIG. 3 illustrates the toy DB 110. As illustrated, the toy DB 110 includes information on "basic information" including a toy name indicating the name of the toy, a toy ID indicating information that uniquely identifies the toy, a toy category indicating the category of the toy, a manufacturer name indicating the name of the manufacturer that produces the toy, and a target age for children (minimum age and maximum age), a "toy image" indicating the appearance of the toy, an "aptitude field score" including the field (attribute) to which the toy belongs and scores for each of the six aptitude fields described above, and a "toy characteristic" indicating an element that contributes to the development of a child by using each toy. The "toy image" may store a plurality of images of the toy taken from various angles.

[0019] The "toy characteristic information" includes items such as "elements contributing to development" which indicate elements of the development of a child's aptitude field, and "presence or absence" which indicates the degree to which the toy contributes to the development of each element. The degree to which the toy contributes to the development of each element is not limited to presence or absence, and may be classified into any number of levels. The "aptitude field score" includes a "toy category" which indicates the attributes of the toy, and one or more categories such as artist, engineer, liberal arts, communicator, athlete, and natural science are input. Each score in the "aptitude field score" is set by the scoring unit 130.

[0020] 3, the toy DB110 example indicates that the "toy name" is "Toy A," the "toy ID" is "1766," and the "target age" is "36 to 60 months old." Furthermore, this toy contributes to the development of "three-dimensional spatial cognition" as an "element contributing to development," the "toy category" is "engineer," and the "aptitude field scores" are "0" for artist, "4" for engineer, "0" for liberal arts, "0" for communicator, "0" for athlete, and "0" for natural science.

[0021] Returning to Fig. 2, the user DB 120 stores user information such as the name, sex, age, and date of birth of the child to be judged, as well as facial images of the child. The user DB 120 also stores image information of a child playing with a toy, transmitted by the terminal device 200. The user DB 120 also stores evaluation reports previously output by the output unit 180. Each piece of data is assigned a user ID for identifying the user, and each piece of data is managed for each user.

[0022] The scoring unit 130 sets a category score for each of the six aptitude categories for each toy registered in the toy DB 110. Specifically, the scoring unit 130 sets a score for each of the six aptitude categories according to the characteristics of each toy. For example, the scoring unit 130 includes a keyword list 131 that defines keywords corresponding to each aptitude category, as illustrated in FIG. 4. The scoring unit 130 searches for keywords included in the keyword list from characteristic information such as the instruction manual and specifications of each toy. The scoring unit 130 calculates a category score for each aptitude category based on, for example, the frequency of appearance of the keywords for each aptitude category, and stores the score in the toy DB 110. Details of the processing by the scoring unit 130 will be described later.

[0023] The acquiring unit 140 acquires image information capturing an image of a child playing with a toy. For example, a user (such as a child's guardian) captures an image of a child playing with a toy using a camera included in the terminal device 200, and transmits the image information to the information processing device 100. The acquiring unit 140 stores the received image information in the user DB 120 in association with a user ID. Note that the image information is an example of toy usage information.

[0024] The identification unit 150 identifies toys used by children from image information. The identification unit 150 includes a toy detection model 151 generated by machine learning, deep learning, or the like, which detects toys used by children using image information as input. For example, the toy detection model 151 identifies toys by detecting children (people) and toys in an image and analyzing the contact state between the child and the toys using a convolutional neural network (CNN). The toy detection model 151 is constructed using supervised learning. For learning, teacher data, which is a pre-labeled image data set including a child playing with a toy, is used. Each image clearly shows the areas of the human body and toys, and the identification information (toy name and toy ID) of the toy the child is playing with. As a result, the toy detection model 151 can identify which toy is being used by learning the characteristics of the toy, such as the shape, color, and size, and analyzing the contact state between the child and the toy.

[0025] The toy detection model 151 may be trained to detect the position of the hand and identify the toy that is closest to the detected hand position, or may detect the child to be processed from image information using facial recognition.

[0026] The calculation unit 160 recognizes the child's facial expression from the image information, and calculates the child's liking level for the toy identified by the identification unit 150 based on the recognized facial expression. The calculation unit 160 has a recognition model 161 generated by machine learning, deep learning, or the like, which uses image information as an input and recognizes the child's facial expression contained in the image information. The recognition model 161 identifies a plausible one from a plurality of predetermined types of facial expressions and recognizes the child's facial expression. For example, the types of facial expressions are classified into a plurality of types according to words indicating emotions such as "fun," "happy," "delighted," "surprise," "anger," and "neutral." The learning of the recognition model 161 uses a supervised learning method that uses teacher data, which is a data set of various images to which types of facial expressions are previously labeled, and automatically extracts features corresponding to each facial expression. The calculation unit 160 calculates a likeability rating from the recognized facial expression based on preset rules, such as calculating a likeability rating of "2" if the facial expression recognized by the recognition model 161 is "joyful," "happy," or "pleased," calculating a likeability rating of "0.5" if the recognized facial expression is "surprise" or "anger," and calculating a likeability rating of "1" if "neutral" or other facial expressions are recognized.

[0027] Note that the recognition model 161 may recognize facial expressions including the degree of each emotion. In this case, the calculation unit 160 may calculate a favorability rating according to the emotion recognized by the recognition model 161 and the degree of the emotion.

[0028] Furthermore, the calculation unit 160 may calculate the likeability rating based on the time and frequency that the child plays with each toy, and the voice of the child while playing, in addition to the facial expression. Specifically, for example, the calculation unit 160 may set a high likeability rating for a toy that is played with for a long time or frequently. Furthermore, the calculation unit 160 may recognize emotions from the voice using a machine learning model that detects emotions in the voice from the tone, volume, etc. of the voice, and may set a high likeability rating when a positive emotion is detected, for example.

[0029] Furthermore, when multiple still images including the same toy are input, the calculation unit 160 may calculate the likeability rating corresponding to the facial expression recognized by the recognition model 161 for each still image, and calculate the average of the likeability ratings as the likeability rating of the target toy. Furthermore, when a video is input, the calculation unit 160 may set a weight for each facial expression according to the proportion of time that the multiple facial expressions recognized for each toy are shown, and calculate the likeability rating as a score obtained by multiplying the likeability ratings corresponding to each facial expression by the weight and adding them up.

[0030] The determination unit 170 calculates an aptitude score indicating the level of aptitude for each aptitude field based on the likeability of the toy calculated by the calculation unit 160 and the aptitude field score of the toy set by the scoring unit 130. Specifically, the determination unit 170 calculates an aptitude score for each aptitude field by multiplying the likeability by the value of the aptitude field score. In the calculation example illustrated in FIG. 5, the determination unit 170 multiplies the "aptitude field score" of toys A to D identified by the identification unit 150 by the value of each "likeability" calculated by the calculation unit 160, and adds them up for each aptitude field to calculate an aptitude score for each aptitude field such as artist "3", engineer "8", liberal arts "2", communicator "0.5", athlete "1.5", and natural science "2". The determination section 170 adds up the calculated aptitude score and the previously calculated aptitude score for each aptitude field, and calculates a cumulative value for each aptitude field.

[0031] Furthermore, the determination unit 170 determines the child's aptitude field based on a preset rule. The process of determining the aptitude field will be described in detail later.

[0032] Returning to FIG. 2, the output unit 180 generates an evaluation report based on the aptitude score for each aptitude field calculated by the judgment unit 170 and the judgment result of the aptitude field. FIG. 6 shows an example of the evaluation report 181. The evaluation report 181 includes the judgment result of the aptitude field, the transition of the cumulative value of the aptitude score for each aptitude field, and an evaluation comment. In the "aptitude field judgment result", any one or more of the categories of artist, engineer, liberal arts, communicator, athlete, and natural science is input. If the aptitude field is not judged, it may be left blank, or a comment that continuous observation is necessary may be input. In addition, the evaluation comment may be created by the administrator of the information processing device 100 or may be generated by the text generation AI. When the comment is created by the text generation AI, the administrator inputs a prompt to the text generation AI to instruct the generation of the evaluation comment based on the profile of the target child, the transition of the cumulative value of the aptitude score, and the like, and causes the text generation AI to output the evaluation comment. In addition, the evaluation report 181 is an example of judgment result information.

[0033] (Hardware configuration of information processing device) 7 is a block diagram showing a hardware configuration of the information processing device 100. The information processing device 100 includes a CPU (Central Processing Unit) 11 that executes processing according to a program, a RAM (Random Access Memory) 12 that is a volatile memory, a ROM (Read Only Memory) 13 that is a non-volatile memory, a storage unit 14 that stores data, an input unit 15 that accepts input of information, a display unit 16 that visualizes and displays information, and a communication unit 17 that transmits and receives information, which are connected via an internal bus 99.

[0034] CPU 11 controls the operation of information processing device 100 as a whole, and is connected to each component to exchange control signals and data. CPU 11 executes various processes by reading a program stored in storage unit 14 into RAM 12 and executing it. CPU 11 executes each process by scoring unit 130, acquisition unit 140, identification unit 150, calculation unit 160, determination unit 170, and output unit 180 as main functions provided by the program.

[0035] The RAM 12 is for temporarily recording data and programs, and holds programs and data read from the storage unit 14, other data necessary for communication, etc. The RAM 12 is used as a work area for the CPU 11.

[0036] The ROM 13 stores a control program executed by the CPU 11 for the basic operation of the information processing device 100, a BIOS (Basic Input Output System), and the like.

[0037] The storage unit 14 includes a hard disk drive, a flash memory, etc., and stores the programs executed by the CPU 11 and various data used when executing the programs. The storage unit 14 functions as a toy DB 110 and a user DB 120. The storage unit 14 also stores a toy detection model 151 and a recognition model 161.

[0038] The input unit 15 is a user interface including a touch panel, a keyboard, a mouse, a communication device, etc. The input unit 15 receives an operation input from a user of the information processing device 100, and outputs to the CPU 11 a signal corresponding to the received operation input.

[0039] The display unit 16 is a display device such as a liquid crystal display or an organic EL (Electro Luminescence) display that visualizes and displays information.

[0040] The communication unit 17 is a network termination device or a wireless communication device that connects to a network, and a serial interface or a LAN (Local Area Network) interface that connects to them. The information processing device 100 communicates with other information processing devices, etc. through the communication unit 17. The communication unit 17 functions as an acquisition unit 140 and an output unit 180.

[0041] (Scoring process) Next, the operation of the information processing system 1 will be described with reference to Fig. 8 and Fig. 9. First, the scoring process for setting a category score for each toy in the toy DB 110 will be described with reference to Fig. 8. The scoring process is executed, for example, when a new toy is registered in the toy DB 110.

[0042] The scoring unit 130 of the information processing device 100 acquires feature information of a toy to be processed (step S101). For example, the scoring unit 130 acquires feature information such as an instruction manual or a specification sheet of the toy to be processed from the toy DB 110.

[0043] Next, the scoring unit 130 searches for keywords included in a preset keyword list 131 from the characteristic information acquired in step S101 (step S102). Specifically, the scoring unit 130 refers to the keyword list 131 illustrated in Fig. 4, and searches for keywords that match with keywords included in the keyword list 131 from the characteristic information. Note that, in addition to the matching keywords, the scoring unit 130 may extract keywords that are semantically similar to the keywords included in the keyword list 131 by using a word embedding technique or the like.

[0044] Next, the scoring unit 130 sets a category-specific score for each of the suitability fields (step S103). For example, the scoring unit 130 sets the category-specific score based on the appearance frequency of the keyword for each suitability field. Specifically, the scoring unit 130 counts the appearance frequency of the keyword searched for in step S102 for each suitability field. Next, the scoring unit 130 calculates the appearance frequency for each suitability field by dividing the appearance frequency of the keyword by the number of words included in the characteristic information. For example, if the appearance frequency of the keyword in the artist field is 5 and the number of words included in the characteristic information is 100, the scoring unit 130 calculates 0.05 as the appearance frequency of the keyword in the artist field. Next, the scoring unit 130 normalizes the appearance frequency calculated for each suitability field to the same scale (for example, 0 to 5) to calculate the category-specific score for each suitability field. The scoring unit 130 sets the calculated category-specific score in the toy DB 110 and ends the process.

[0045] The method of calculating the category scores by the scoring unit 130 is not limited to this, and any method may be used for calculation. For example, the scoring unit 130 may set the category scores based on the number of keywords searched. For example, if 20 keywords are included in the artist category in the keyword list 131 and 5 of these keywords are searched from the feature information, 5 / 20=0.25 may be calculated as the matching score, and the matching scores calculated for each aptitude category may be normalized to the same scale to calculate the category scores for each aptitude category.

[0046] (Decision making process) Next, a determination process for determining an appropriate field by the information processing device 100 will be described with reference to Fig. 9. The determination of the appropriate field is performed, for example, at a preset timing (such as once a month). The user operates the terminal device 200 to transmit image information of a child playing with a toy to the information processing device 100 by the determination timing. The acquisition unit 140 of the information processing device 100 stores the image information transmitted from the terminal device 200 in the user DB 120 in association with the user ID. When the information processing device 100 determines that the determination timing for determining the appropriate field is approaching (for example, three days before the determination timing), it starts the process.

[0047] First, the information processing device 100 notifies the terminal device 200 via the communication unit 17 that the judgment timing is approaching (step S201). Next, when the acquisition unit 140 detects that the judgment timing has arrived, it acquires image information transmitted from the user to be processed from the toy DB 110 (step 202). For example, the acquisition unit 140 accesses the user DB 120, and refers to the shooting date included in the image information associated with the user ID of the user to be processed, to acquire image information whose shooting date is after the date of the previous judgment timing and before the date of the judgment timing. The acquisition unit 140 transmits the acquired image information to the identification unit 150.

[0048] Next, the identification unit 150 identifies the toy used by the child from the image information acquired by the acquisition unit 140 (step S203). Specifically, the identification unit 150 inputs the image information acquired by the acquisition unit 140 to the toy detection model 151, and obtains detection result information including position information (bounding box) of the toy in the image and a label indicating identification information of the toy (toy name or toy ID).

[0049] Next, the calculation unit 160 calculates the likeability of the toy identified by the identification unit 150 (step S204). Specifically, the calculation unit 160 inputs the image information acquired by the acquisition unit 140 and the detection result information obtained in step S203 to the recognition model 161, and obtains an inference result of the facial expression of the child in the image. The calculation unit 160 calculates the likeability of each toy from the facial expression recognized by the recognition model 161 based on the following rules: if the facial expression recognized by the recognition model 161 is "joyful", "happy", or "delighted", the calculation unit 160 calculates a likeability of "2", if the recognized facial expression is "surprise" or "anger", the calculation unit 160 calculates a likeability of "0.5", and if "neutral" or other facial expressions are recognized, the calculation unit 160 calculates a likeability of "1".

[0050] Next, the judgment unit 170 calculates an aptitude score indicating the degree of aptitude of the child for each aptitude field (step S205). Specifically, the judgment unit 170 accesses the toy DB 110 and acquires the field-specific scores of the toys identified in step S203. As illustrated in Fig. 5, the judgment unit 170 accumulates the acquired field-specific scores and the toy's favorability ratings calculated in step S204 for each toy, and adds them up for each aptitude field to calculate an aptitude score for each aptitude field.

[0051] Next, judgment unit 170 updates the cumulative value for each aptitude field by adding the aptitude score calculated in step S205 to a cumulative value that is the cumulative total of the aptitude scores up to that point (step S206).

[0052] Next, the judgment unit 170 judges the aptitude field (step S207). Specifically, the judgment unit 170 judges whether the cumulative value of each aptitude field calculated in step S206 and the difference rate of the cumulative value between the aptitude field with the highest cumulative value and the aptitude field with the second highest cumulative value are equal to or greater than the respective predetermined thresholds. When both the cumulative value and the difference rate of each aptitude field are equal to or greater than the respective thresholds, the judgment unit 170 judges that the aptitude field with the highest cumulative value is the aptitude field of the child to be processed. On the other hand, when either or both of the cumulative value and the difference rate of each aptitude field are not equal to or greater than the thresholds, the judgment unit 170 judges that the aptitude field is not to be judged. The judgment unit 170 notifies the output unit 180 of the judgment result.

[0053] Next, the output unit 180 outputs the evaluation report 181. Specifically, the output unit 180 generates the evaluation report 181 illustrated in FIG. 6 based on the cumulative value of the aptitude score for each aptitude field calculated in step S206, the cumulative value of the past aptitude scores, the judgment result in step S207, and the comment generated by the administrator or the text generation AI, transmits the generated evaluation report 181 to the terminal device 200 (step S208), and ends the process. Note that, in the illustrated example, a case is shown in which engineering is judged as an aptitude field, but if it is judged in step S207 that the aptitude field is not judged, the item "aptitude field judgment result" may be left blank, or a comment to the effect that continued observation is necessary may be input.

[0054] As described above, the information processing device 100 automatically sets a category score indicating the degree to which each toy contributes to the development of each ability in the aptitude field based on the characteristic information of each toy. Furthermore, the information processing device 100 identifies the toy used by the child from image information capturing the child playing with the toy, and recognizes the child's facial expression. The information processing device 100 calculates the child's likeability rating for the toy based on the recognized facial expression. The information processing device 100 calculates an aptitude score for each aptitude field based on the calculated likeability rating of the toy and the category score of the toy, and determines the aptitude field for which the child has aptitude. Therefore, the objectivity of the judgment of the child's aptitude can be improved. Furthermore, the labor of manually calculating and setting the category score of each toy, judging and inputting the child's likeability rating for the toy, and the like can be more efficiently judged.

[0055] (Modification) The information processing device 100 may diagnose the child's personality in addition to judging the child's aptitude. Specifically, the information processing device 100 may classify the elements constituting the child's personality into five categories (e.g., agreeableness, conscientiousness, sensitivity, openness, and extroversion), and calculate a score for each classified personality element based on the usage status of the toy. Note that the personality elements are not limited to five, and may be classified into any number, and the names and types of the personality elements may be set arbitrarily.

[0056] In this case, the scoring unit 130 further includes a personality keyword list 132 that defines keywords corresponding to each personality element, as exemplified in Fig. 10. The scoring unit 130 may calculate an element score for each personality element based on the characteristic information of each toy using a calculation method similar to that used for the field-specific scores in the aptitude fields, for example, based on the frequency of appearance of keywords for each personality element.

[0057] An example of the toy DB 110A in this case is shown in Fig. 11. As shown in the figure, in addition to the "basic information", "toy image", and "score by aptitude field" stored in the toy DB 110, the toy DB 110A includes an item of "score by personality element" that stores the personality score by personality element calculated by the scoring unit 130. Furthermore, the "toy characteristics" may further include information indicating the elements that contribute to the development of a child's personality by using each toy.

[0058] In step S205, the determination unit 170 may calculate the aptitude scores for the aptitude fields and also calculate personality scores for each personality element. Specifically, the determination unit 170 accesses the toy DB 110 to acquire the personality element scores for the toys identified in step S203. The determination unit 170 may add up the acquired personality element scores and the toy favorability ratings calculated in step S204 for each toy, and add them up for each personality element to calculate a personality score for each personality element. In addition, the output unit 180 may generate a transition graph 182 showing the transition of the personality element scores as shown in FIG. 12, for example, and output the same by including it in the evaluation report 181.

[0059] The determination unit 170 may determine the child's aptitude from the determined aptitude field and the diagnosed personality based on a aptitude discovery program sheet 171 that defines the child's aptitude consisting of a combination of aptitude field and personality, as illustrated in FIG. 13. As illustrated, the aptitude discovery program sheet includes information on a "combination of personality elements" indicating whether each of the five personality elements is high or low, and a "child's aptitude" determined by the aptitude field determined by the determination unit 170, the personality, and the aptitude field. For example, the determination unit 170 compares the calculated personality element scores with a predetermined reference value and determines whether the personality elements of agreeableness, sincerity, sensitivity, openness, and extroversion are high or low. When the determination unit 170 determines that all personality elements are high and that the aptitude field is an engineer, it determines that the child has the aptitude of an "innovation leader" determined by the combination of these elements, as illustrated.

[0060] In the above embodiment, the judgment unit 170 calculates the cumulative value for each aptitude field, judges whether the cumulative value for each aptitude field is equal to or greater than a predetermined threshold value, and judges whether the difference rate between the cumulative value of the aptitude field with the highest cumulative value and the cumulative value of the aptitude field with the second highest cumulative value is equal to or greater than a predetermined threshold value, and if it is determined that there is an aptitude field that satisfies each threshold value, judges that the aptitude field that satisfies the condition is the aptitude field of the child, but this is not limited to the above. For example, it may calculate the difference rate (deviation degree) between the cumulative value for each aptitude field and the average value of the cumulative values ​​of the other aptitude fields, judge whether they are equal to or greater than a predetermined threshold value, and if it is determined that there is an aptitude field that satisfies each threshold value, judge one or more aptitude fields that satisfy the condition are the aptitude field of the child. In addition, the growth rate of the cumulative value at a certain point in time may be used to judge whether there is an aptitude field. Furthermore, each threshold value may be a different value for each month.

[0061] Furthermore, the information processing device 100 may calculate the likeability rating by acquiring text information entered by a guardian on the toys that the child often plays with and how the child plays with them, in addition to image information capturing the usage of the child's toys. In this case, the calculation unit 160 may calculate the likeability rating by any method, such as multiplying the likeability rating score calculated based on the image information by a predetermined coefficient for the likeability rating of the toy derived from the text information.

[0062] The information processing device 100 can be realized by using a normal computer, not a dedicated device. For example, the information processing device 100 that executes the above-mentioned processes may be configured by installing a program for executing any of the above-mentioned processes on a computer from a recording medium storing the program. Also, one information processing device 100 or terminal device 200 may be configured by multiple computers operating in cooperation with each other.

[0063] Furthermore, when the above-mentioned functions are realized by sharing the functions between an OS (Operating System) and an application, or by cooperation between the OS and the application, only the parts other than the OS may be stored on the medium.

[0064] It is also possible to superimpose the program on a carrier wave and distribute it via a communication network. For example, the program may be posted on a bulletin board (BBS, Bulletin Board System) on the communication network and distributed via the network. These programs may then be started and executed under the control of the operating system in the same way as other application programs, thereby enabling the above-mentioned processing to be performed.

[0065] Furthermore, the information stored in the storage unit 14 may be collectively managed by a cloud server present on the network, and the information processing device 100 may access the cloud server as necessary to read and write information. In this case, the information processing device 100 may not include the toy DB 110 and the user DB 120. Furthermore, the scoring process and the determination process by the information processing device 100 may be executed on the cloud using information stored in the cloud server.

[0066] Various embodiments and modifications of the present invention are possible without departing from the broad spirit and scope of the present invention. The above-described embodiments are for the purpose of explaining the present invention and do not limit the scope of the present invention. That is, the scope of the present invention is indicated by the claims, not the embodiments. Various modifications made within the scope of the claims and the scope of the invention equivalent thereto are considered to be within the scope of the present invention. [Explanation of symbols]

[0067] 1 Information processing system, 100 Information processing device, 200 Terminal device, 300 Network, 110, 110A Toy DB, 120 User DB, 130 Scoring section, 131 Keyword list, 132 Personality keyword list, 140 Acquisition section, 150 Identification section, 151 Toy detection model, 160 Calculation section, 161 Recognition model, 170 Judgment section, 171 Talent discovery program sheet, 180 Output section, 181 Evaluation report, 182 Progression graph, 11 CPU, 12 RAM, 13 ROM, 14 Memory section, 15 Input section, 16 Display section, 17 Communication section, 99 Internal bus

Claims

1. An information processing device that determines a field in which a child using a toy has aptitude according to a usage situation of the toy, a scoring unit that sets, for each toy, a field-specific score that indicates the degree to which each toy contributes to the development of each ability in the aptitude field, based on the characteristic information of each toy and a keyword list that indicates a list of keywords corresponding to each of a plurality of aptitude fields; an acquisition unit that acquires toy usage information including image information capturing a situation in which the child is using the toy; an identification unit that identifies a toy used by a child from the image information acquired by the acquisition unit using a pre-trained toy detection model that detects toys used by a child using the image information as an input; a calculation unit that recognizes a facial expression of the child from the image information acquired by the acquisition unit using a recognition model that has been trained in advance to recognize facial expressions, and calculates a liking rating for the specified toy based on the recognized facial expression; a determination unit that determines an aptitude field for which the child has aptitude by calculating an aptitude score that indicates a level of aptitude for each of the fields of the child to be determined based on the field-specific scores of the toys identified by the determination unit and the likeability rating for the toys calculated by the calculation unit; an output unit that outputs judgment result information including the aptitude field judged by the judgment unit; An information processing device comprising:

2. the toy detection model is generated by machine learning on training data in which regions of the human body and toys used by children and identification information that uniquely identifies the toys are assigned as labels to the image information; the identification unit inputs the image information acquired by the acquisition unit into the toy detection model to identify the toy used by the child. The information processing device according to claim 1 .

3. the recognition model is generated by machine learning on training data to which any one of a plurality of predetermined types of facial expressions is assigned as a label; the calculation unit inputs the image information acquired by the acquisition unit into the recognition model to recognize the child's facial expression.

3. The information processing device according to claim 1 or 2.

4. the determination unit determines the aptitude score for each category of the child to be determined by multiplying the category score of the toy identified by the determination unit by the likeability rating of the toy calculated by the calculation unit, and determines that the category with the highest cumulative value is the aptitude category when a cumulative value of the category with the highest cumulative value obtained by integrating the aptitude scores determined in a predetermined period and a difference between the cumulative value of the category with the highest cumulative value and the cumulative value of the category with the second highest cumulative value are equal to or greater than predetermined threshold values.

3. The information processing device according to claim 1 or 2.

5. the scoring unit further sets, for each toy, a personality element score indicating a degree to which each toy contributes to the development of each of the personality elements based on the characteristic information of each toy and a personality keyword list indicating a list of keywords corresponding to each of a plurality of personality elements constituting the child's personality; the determining unit determines a personality score for each personality element of the child to be determined based on the personality element scores of the toy identified by the identifying unit and the likeability rating for the toy calculated by the calculating unit, thereby diagnosing the personality of the child.

3. The information processing device according to claim 1 or 2.

6. the assessment unit refers to a talent discovery program sheet that defines the child's talent, which is a combination of aptitude fields and personality, and assesses the child's talent based on the determined aptitude fields and the calculated personality scores for each personality element; The information processing device according to claim 5 .

7. A computer that determines a field in which a child using a toy has aptitude according to a usage situation of the toy, a step of setting a category-specific score for each toy, which indicates a degree of contribution of each toy to the development of abilities in each of the aptitude categories, based on the characteristic information of each toy and a keyword list indicating a list of keywords corresponding to each of the plurality of aptitude categories; acquiring toy usage information including image information capturing a situation in which a child is using the toy; Identifying the toy used by the child from the acquired image information using a pre-trained toy detection model that detects toys used by a child using the image information as an input; a step of recognizing a facial expression of the child from the acquired image information using a pre-trained recognition model for recognizing facial expressions, and calculating a preference rating for the identified toy based on the recognized facial expression; determining an aptitude field in which the child has aptitude by calculating an aptitude score indicating a level of aptitude for each of the fields of the child to be judged based on the field-specific score of the identified toy and the calculated likeability rating for the toy; outputting judgment result information including the judged aptitude field; An information processing method comprising:

8. A computer that determines a field in which a child using a toy has aptitude according to a usage situation of the toy, A process of setting a category-specific score for each toy, which indicates the degree to which each toy contributes to the development of each ability in each aptitude field, based on the characteristic information of each toy and a keyword list indicating a list of keywords corresponding to each of a plurality of aptitude fields; A process of acquiring toy usage information including image information capturing a situation in which a child is using the toy; A process of identifying a toy used by a child from the acquired image information using a pre-trained toy detection model that detects toys used by a child using the image information as an input; a process of recognizing a facial expression of the child from the acquired image information using a pre-trained recognition model for recognizing facial expressions, and calculating a preference rating for the identified toy based on the recognized facial expression; determining an aptitude field for which the child has aptitude by calculating an aptitude score indicating a level of aptitude for each of the fields of the child to be judged based on the field-specific score of the identified toy and the calculated likeability rating for the toy; A process of outputting judgment result information including the judged aptitude field; A program that executes the following.