System

Abstract

To provide a system that allows a child to select a commodity in which the child has an interest.SOLUTION: A system includes: means that grasps the state of the emotion of a user by using emotion recognition technique; means that selects a toy or a game suitable for the user from a commodity database, on the basis of an interest, the age, and the gender of the user, and the state of the emotion of the user; means that, for the selected toy or game, generates detail information or how to play the toy or game, on the basis of a generative AI, and a prompt sentence including the interest, age, and gender of the user and instructing generating the detail information or how to play the toy or game; means that presents the generated detail information or how to play the toy or game to the user; means that receives input of images that are prepared in advance and show a situation where a person plays with the toy or game, and converts the images into images seen by the user playing with the toy or game; and means that presents the images obtained through the conversion to the user.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including a description and related instruction sentence regarding the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2022-180282 A Summary of the Invention [Problem to be solved by the invention]

[0004] When purchasing conventional toys and games, it is difficult for a child to know detailed information about the product or how to play it before actually experiencing it, making it difficult to choose a product that matches a child's preferences and interests. [Means for solving the problem]

[0005] The present invention provides a means for allowing children to experience toys and games in a virtual playground and select products that interest them by providing detailed product information and instructions on how to play the products.

[0006] As described above, the present invention solves the conventional problem of a lack of detailed product information and play instructions by providing a virtual playground application, allowing children to choose more appropriate toys and games.

[0007] A "virtual playground application" is an application that allows children to virtually experience toys and games.

[0008] A "product database" is a database that stores information about toys and games.

[0009] A "toys or games list" is a list of toys and games selected based on a user's preferences, interests, age, and gender.

[0010] A "data generation model" is an interactive AI model that uses natural language processing technology to provide information about toys and games and how to play them through dialogue with the user. One example of a "data generation model" is generative AI.

[0011] A "simulated experience" is one that gives the feeling of actually experiencing a toy or game, and refers to the visual representation of what it's like to play a toy or game.

[0012] The "actual purchase option" refers to the process or options that allow a user to actually purchase a toy or game that they like. [Brief description of the drawings]

[0013] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Diagram 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. FIG. [Diagram 3] FIG. 11 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Diagram 5]FIG. 13 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 13 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 13 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] 4 is a sequence diagram showing a process flow of the data processing system according to the first embodiment. FIG. [Figure 12] 11 is a sequence diagram showing a process flow of the data processing system in application example 1. FIG. [Figure 13] FIG. 11 is a sequence diagram showing the flow of processing of the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 11 is a sequence diagram showing the flow of processing in the data processing system in application example 2 when combined with an emotion engine. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0015] First, the terms used in the following description will be explained.

[0016] In the following embodiments, a signed processor (hereinafter simply referred to as a "processor") may be one arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be one type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), or an APU (Accelerated Processing Unit).

[0017] In the following embodiments, a signed RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0018] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0019] In the following embodiments, a communication I / F (Interface) with a code is an interface including a communication processor and an antenna. The communication I / F controls communication between multiple computers. An example of a communication standard applied to the communication I / F is a wireless communication standard including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0020] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. In addition, in this specification, the same idea as "A and / or B" is also applied when three or more things are expressed by connecting them with "and / or."

[0021] [First embodiment]

[0022] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0023] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0024] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[0025] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0026] The reception device 38 includes a touch panel 38A and a microphone 38B, and receives user input. The touch panel 38A detects contact with a pointer (e.g., a pen or a finger) to receive user input by the touch of the pointer. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0027] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (e.g., voice and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs voice according to instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a Complementary Metal-Oxide-Semiconductor (CMOS) image sensor or a Charge Coupled Device (CCD) image sensor.

[0028] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54.

[0029] FIG. 2 shows an example of main functions of the data processing device 12 and the smart device 14.

[0030] As shown in Fig. 2, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32. The specific process program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific process program 56 from the storage 32, and executes the read specific process program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific process program 56 executed on the RAM 30.

[0031] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0032] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores a reception output program 60. The reception output program 60 is used together with the specific processing program 56 by the data processing system 10. The processor 46 reads out the reception output program 60 from the storage 50, and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0033] Next, a description will be given of the specific processing by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal".

[0034] An embodiment for implementing the present invention includes the following elements.

[0035] 1. Server: Has the function of receiving information on the user's preferences, interests, age, and gender, and retrieving a list of toys and games from a product database.

[0036] 2. Generative AI: Equipped with the ability to use natural language processing AI models to provide users with detailed information about toys and games and how to play them.

[0037] 3. Terminal: Shows the user a list of toys and games, displays a video to let them experience the selected product, and also provides the option to actually purchase the product.

[0038] 4. Product Database: A database that stores information about toys and games, providing users with information to select appropriate products based on their preferences, interests, age, and gender.

[0039] 5. Virtual experience video conversion function: Equipped with a function that converts the image of playing a toy or game into an image from a child's perspective and displays it on the device.

[0040] In this way, the server, generative AI, terminal, product database, and virtual experience video conversion function work together to enable users to select appropriate toys and games, experience products through a virtual experience, and actually purchase them.

[0041] The process flow of each embodiment will be described below.

[0042] Step 1: The server receives input information from the user (preferences, interests, age, gender).

[0043] Step 2: The server uses the received information to retrieve a list of toys and games from the product database.

[0044] Step 3: The server uses generative AI to provide the user with detailed information about the toy or game and how to play it.

[0045] Step 4: The device displays a list of toys and games to the user.

[0046] Step 5: User selects the product that interests them from the list.

[0047] Step 6: The device will display a video to experience the selected product.

[0048] Step 7: The user experiences the product virtually.

[0049] Step 8: If the user finds an item they like, the device gives them the option to actually purchase it.

[0050] Step 9: The user proceeds with the purchase.

[0051] For example, if a user inputs "I want a robot toy," the server will refer to the user's information and retrieve a list of robot toys from a product database. If the user then asks the generative AI, "Please tell me how to play with this robot toy," the generative AI will provide an appropriate answer. When the user selects a robot toy that interests them, the device will display an image from a child's perspective, allowing the user to have a simulated experience. If the user likes it, they can actually purchase it.

[0052] Example 1

[0053] Next, a description will be given of Example 1. In the following description, the data processing device 12 is referred to as a "server" and the smart device 14 is referred to as a "terminal."

[0054] In conventional systems, it is difficult for users to easily find toys and games that match their preferences and interests. In addition, detailed product descriptions and instructions for playing based on a lot of information are lacking. Furthermore, the functionality for actually experiencing toys and games is limited, making it difficult to increase purchasing motivation. This results in low user satisfaction and makes it an issue to increase purchasing motivation.

[0055] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0056] In this invention, the server includes a means for acquiring a list of toys or games from a product database based on the user's preferences, interests, age, or sex, a means for providing the user with detailed product information or playing instructions using a generative AI model, a means for converting the state of playing the toy or game to the user's perspective, a means for transmitting information input by the user to the server, a means for displaying the list acquired by the server from the product database on a terminal, and a means for generating a prompt sentence based on a product selected by the user and acquiring the detailed information. This allows the user to easily find toys or games that match their preferences or interests, and to consider purchasing them after obtaining detailed information. In addition, the simulated experience increases purchasing motivation and user satisfaction.

[0057] A "server" is a computer device that receives information input by a user, retrieves an appropriate product list from a product database, and transmits the list to a terminal.

[0058] "User preferences, interests, age, and gender" are individual attribute information input by the user, and are important criteria for product selection.

[0059] The "product database" is a database that stores detailed information about various toys and games.

[0060] The "list of toys or games" is a list of toys and games obtained from a product database, and is a list that has been filtered based on the user's attribute information.

[0061] A "generative AI model" is an artificial intelligence model that generates and provides detailed product information and play instructions to users in natural language.

[0062] A "prompt sentence" is an instruction sentence that causes a generative AI model to generate detailed information.

[0063] "Means of providing detailed information or instructions for playing" refers to a function that uses a generative AI model to present users with detailed information about a product or instructions for playing the product.

[0064] "Playing with a toy or game" is visual data showing a situation in which the selected toy or game is actually being used.

[0065] The "means for converting to a user's viewpoint" is a function for converting the state of playing a toy or game into an image from the user's viewpoint.

[0066] The "means for transmitting information input by the user to the server" is a function for transmitting information input by the user using the terminal to the server via the network.

[0067] A "terminal" is a computing device through which a user enters information and displays information received from a server.

[0068] A "simulated experience video" is a video that makes the user feel as if they are actually experiencing a toy or game that they have selected.

[0069] "Purchase options" is a function that provides options for actually purchasing a product in which the user is interested.

[0070] This invention is a system that allows users to easily find toys and games that match their tastes and interests, and consider purchasing them after obtaining detailed information. Specifically, a server, a terminal, a generative AI model, a product database, and a virtual experience video conversion function work together.

[0071] The server receives information about preferences, interests, age, and gender that the user inputs using the terminal. For example, if the user inputs information such as "7-year-old boy, likes dinosaurs," the information is sent to the server via the Internet. Based on this information, the server retrieves a list of suitable toys and games from a product database. The product database contains information about a wide variety of toys and games, and returns a list that matches the criteria to the server.

[0072] The device displays the product list sent from the server to the user. The user browses the list on the device and selects a product that interests them. At that time, the device utilizes a generative AI model to provide the user with detailed information about the selected product and how to play with it in natural language. For example, this generative AI model applies a prompt sentence such as "Please explain the details of the dinosaur exploration set and how to play with it," and the AI ​​generates the relevant information.

[0073] In addition, the device is equipped with a virtual experience video conversion function that generates a video that makes it appear as if the user is actually playing the selected toy or game. This video is displayed from the user's point of view, so the user can feel as if they are playing the toy or game. For example, a realistic video of playing with a dinosaur exploration set is provided.

[0074] Finally, the device will offer the user the option to purchase the item they are interested in. The user can then complete the purchase process from the displayed options and get their hands on the selected toy or game.

[0075] The system allows users to easily find toys and games that match their tastes and interests, and then get detailed information about them to make a purchase. Using generative AI models and virtual reality videos, the user experience is improved and purchase motivation is increased. An example of a prompt sentence is "Please explain the details of the dinosaur exploration set and how to play it."

[0076] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0077] Step 1:

[0078] The user uses the device to input information about his / her preferences, interests, age, and gender. Specifically, the user enters information such as "7-year-old boy, loves dinosaurs" into a form on the screen.

[0079] Input: User attribute information (preferences, interests, age, gender)

[0080] Output: User information data entered

[0081] Step 2:

[0082] The terminal sends the information entered by the user to the server. When the user presses the send button, the entered information is sent to the server via the Internet.

[0083] Input: User information data entered

[0084] Output: User information sent to the server

[0085] Step 3:

[0086] The server retrieves a list of suitable toys and games from the product database based on the received user information. The server searches the database for the condition "age = 7 years old AND preference = dinosaurs" and retrieves the list.

[0087] Input: User information

[0088] Output: A filtered list of toys and games.

[0089] Step 4:

[0090] The terminal displays a list of toys and games sent from the server to the user. For example, the list may show a "Dinosaur Exploration Set."

[0091] Input: Product list sent from the server

[0092] Output: Product list displayed on the terminal screen

[0093] Step 5:

[0094] The user selects from the displayed list the toy or game that he or she is interested in. For example, the user selects "Dinosaur Exploration Set."

[0095] Input: Displayed product list

[0096] Output: Selected product data

[0097] Step 6:

[0098] The server sends a prompt to the generative AI model to generate detailed information about the product selected by the user and instructions on how to play with it. Specifically, the server sends the prompt to the generative AI model, "Please explain the details of the dinosaur exploration set and how to play with it."

[0099] Input: Selected product data

[0100] Output: The prompt sent to the generative AI model

[0101] Step 7:

[0102] The generative AI model generates detailed product information and instructions for playing based on the prompt text sent and returns it to the server.

[0103] Input: Prompt statement

[0104] Output: Generated details and how to play

[0105] Step 8:

[0106] The server sends the detailed information received from the generative AI model to the terminal, which then displays the information to the user, who can view the details in a natural conversational format.

[0107] Input: Detailed information received from the generative AI model

[0108] Output: Detailed information displayed on the terminal screen

[0109] Step 9:

[0110] The device uses its virtual experience conversion function to generate a virtual experience video of the selected toy or game, for example, from the perspective of a child playing with a "dinosaur exploration set."

[0111] Input: Selected product data

[0112] Output: Generated simulated experience video

[0113] Step 10:

[0114] The device displays the generated simulated experience video to the user, who feels as if he or she is actually playing the toy or game.

[0115] Input: Generated simulated experience video

[0116] Output: Simulation video displayed on the device screen

[0117] Step 11:

[0118] The terminal then provides options for purchasing the selected toy or game, and the user can proceed with the purchase from the displayed options.

[0119] Input: Selected product data

[0120] Output: A screen with purchase options

[0121] This enables the system as a whole to consistently provide products based on the user's preferences and interests, obtain detailed information, and provide purchasing support through simulated experiences.

[0122] (Application example 1)

[0123] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0124] On current online shopping sites, it is difficult for users to find toys and games that interest them, and they have to browse multiple pages to find detailed information and instructions on how to play. Another issue is that it is difficult to provide a simulated experience to increase the desire to purchase a product, and an experience from the user's perspective is particularly important for products aimed at children.

[0125] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0126] In this invention, the server includes a means for acquiring a list of toys or games from a product database based on a user's preferences, interests, age, or sex, a means for providing the user with detailed product information or play instructions using a data generation model, a means for converting a state of playing a toy or game into a view from the user's perspective, a means for displaying a list of recommended toys and games appropriate for the user, a means for generating detailed information about toys or games in natural language using generative AI, and a means for displaying the generated detailed information and recommendation list on the user's terminal. This allows the user to easily find products that interest them and check detailed information in a centralized manner, and also increases their desire to purchase through a simulated experience.

[0127] "User preferences" refers to a user's personal tastes and preferences, and is a concept that includes preferences for specific categories and themes.

[0128] "Interest" refers to a subject or area in which a user is interested, and is a concept that includes a state in which a user is interested in a particular category or topic.

[0129] "Age" refers to the number of years from the user's date of birth to the present, and is a factor that reflects an individual's stage of growth and development.

[0130] "Gender" refers to a user's biological sex, and is a factor that influences an individual's purchasing behavior and preferences.

[0131] "Product database" refers to a database that stores information about toys and games, and is a system for retrieving specific product information as needed.

[0132] "Data generation model" refers to a generative AI model used to provide users with detailed information about a product or instructions for playing the product, and is a means of using technologies such as natural language processing.

[0133] A "recommended list" is a list of toys and games selected based on a user's preferences, interests, age, and gender, and provides individually optimized product suggestions.

[0134] "Generative AI" refers to an artificial intelligence model that generates appropriate information or answers based on given prompts, and is a technology that is particularly used for generating information in natural language.

[0135] "Natural language" refers to language that humans use on a daily basis, and is a term used in a context that includes machine natural language processing.

[0136] "Terminal" refers to a device used by a user to receive and display information, including smartphones and tablets.

[0137] "Simulated experience video" refers to video that makes the user feel as if they are actually playing a toy or game, and is a means of visually providing an experience from the user's perspective.

[0138] The present invention is a system that recommends optimal toys and games based on a user's preferences, interests, age, and gender, and provides detailed information and simulated experiences. An embodiment of the system according to the present invention will be specifically described below.

[0139] The server receives information about the user's preferences, interests, age, and gender, and retrieves a list of toys and games from a product database. It then uses a data generation model to provide the user with detailed product information and how to play the product. To do this, it uses a generative AI model (e.g., ChatGPT (registered trademark)) to generate appropriate information based on a given prompt. The detailed information and recommendation list are displayed on the user's device (e.g., smartphone, tablet, etc.).

[0140] The server also has a virtual experience video conversion function that converts the state of playing a toy or game into the user's perspective. This video is used to help users understand the product more specifically. For example, a prompt sentence such as "Please tell me more about the telescope" can be input to the generative AI to generate a detailed explanation about the telescope.

[0141] Users can select toys and games that interest them from a list of suggested products based on their preferences and interests, and check out the detailed information and simulated experience video. For example, if a 7-year-old boy is interested in science, science experiment kits and telescopes will be recommended, and users can view detailed information and videos about them. At the same time, users are given the option to purchase the selected product, and can then complete the purchase procedure.

[0142] The hardware required is a terminal (smartphone, tablet, etc.) that can accept user operations, and the software required is a generative AI model (for example, a natural language processing platform for running ChatGPT), a database system, and a system with a simulated experience video conversion function.

[0143] The system allows users to easily find toys and games that match their interests and preferences, and provides detailed information and simulated experiences to make a purchase with confidence.

[0144] The flow of the specific process in the application example 1 will be described with reference to FIG.

[0145] Step 1:

[0146] A user uses a terminal to send input information such as preferences, interests, age, and gender to a server.

[0147] Input: User preferences, interests, age, gender

[0148] Output: Receives the corresponding input data on the server side

[0149] Specific operation: When the user answers questions on the application screen and presses the submit button, this information is sent to the server.

[0150] Step 2:

[0151] The server searches for and acquires a list of relevant toys and games from a product database based on the received user information.

[0152] Input: User preferences, interests, age, and gender information

[0153] Output: Product list based on user information

[0154] Specific operation: The server executes a database query to retrieve product information that matches the criteria.

[0155] Step 3:

[0156] The server uses a generative AI model (e.g. ChatGPT) to generate detailed information and instructions for each toy and game included in the product list.

[0157] Input: product list, generative AI model, prompt

[0158] Output: Detailed information about each product and how to play

[0159] Specific operation: The server inputs each product name as a prompt sentence into the generative AI and generates a detailed product description.

[0160] Step 4:

[0161] The server transmits the generated detailed information and recommendation list to the user's terminal.

[0162] Input:Detailed information, Recommendation list

[0163] Output: Product list and details displayed on the user's device

[0164] Specific operation: The generated information is sent to the user's device in JSON or XML format, and displayed on the app screen.

[0165] Step 5:

[0166] When a user browses detailed information and selects a product that interests them, the server creates a simulated experience video of that product and converts it into a view from the user's perspective.

[0167] Input: User selection, virtual experience video conversion function

[0168] Output: Virtual experience video from the user's point of view

[0169] Specific operation: Generate a play scene based on the selected product and convert it into a simulated experience video.

[0170] Step 6:

[0171] The server transmits the converted simulated experience video to the user's terminal, and the user experiences it.

[0172] Input: Simulation video

[0173] Output: Virtual experience video displayed on the user's device

[0174] Specific operation: The generated video data is sent to the user's device and played using the video player on the app.

[0175] Step 7:

[0176] When a user wishes to purchase a product, the user completes the purchase procedure via the terminal, and the server accepts the order.

[0177] Input: User's purchase intention, purchase procedure information

[0178] Output: Purchase information received and confirmed

[0179] Specific operation: The user presses the purchase button, enters the necessary information in the purchase form, and submits it. The server confirms the received purchase information and processes the order.

[0180] By following these steps, users can easily find toys and games that match their interests and preferences, and then purchase with confidence thanks to detailed information and simulated experiences.

[0181] Furthermore, an emotion engine that estimates the emotion of the user may be combined. That is, the identification processing unit 290 may estimate the emotion of the user using the emotion identification model 59, and perform identification processing using the emotion of the user.

[0182] An embodiment for implementing the present invention includes the following elements.

[0183] 1. Emotion engine: An engine for recognizing the user's emotions. It analyzes information such as voice, facial expressions, and text to grasp the user's emotional state.

[0184] 2. Server: Has the function of combining the user's preferences, interests, age, and gender information with the emotional information obtained from the emotion engine, and retrieving a list of toys and games from a product database.

[0185] 3. Generative AI: Equipped with the ability to use natural language processing AI models to provide users with detailed information about toys and games and how to play them.

[0186] 4. Terminal: Shows the user a list of toys and games, and has the ability to show a video to experience the selected product, as well as provide the option to actually purchase it.

[0187] 5. Product Database: A database that stores information about toys and games, providing information to help users choose the right product based on their preferences, interests, age, gender, and emotions.

[0188] 6. Virtual experience video conversion function: Equipped with a function that converts the image of playing a toy or game into an image from a child's perspective and displays it on the device.

[0189] In the above-described configuration, in a system that combines an emotion engine, the server, generative AI, terminal, product database, and simulated experience video conversion function work in coordination, making it possible to customize suggestions and experiences for toys and games according to the user's emotions.

[0190] The process flow of each embodiment will be described below.

[0191] Step 1: The server receives input information from the user (preferences, interests, age, gender) and recognizes the user's emotion information using an emotion engine.

[0192] Step 2: The server retrieves a list of toys and games from the product database based on the received information. Taking into account the emotion information, the server selects toys and games that match the user's emotion.

[0193] Step 3: The server uses generative AI to provide the user with detailed information about the toy or game and how to play it. It also uses emotional information to choose the right language and approach to suit the user’s emotions.

[0194] Step 4: The device displays a list of toys and games to the user, adjusting the order and presentation of the list based on the emotional information.

[0195] Step 5: The user selects the products from the list that he or she is interested in. Taking into account the emotion information, we help the user select products that match his or her emotion.

[0196] Step 6: The device displays a video for experiencing the selected product. Taking into account the emotional information, the presentation and expression of the video are adjusted.

[0197] Step 7: The user experiences the product in a simulated way. Taking into account the emotional information, the product responds and gives feedback that matches the user's emotions.

[0198] Step 8: If the user finds a product they like, the device offers them the option to actually purchase it. Taking into account emotional information, the approach is to increase their desire to purchase.

[0199] Step 9: The user proceeds with the purchase. Taking into account the emotional information, the procedure is carried out to support the user's willingness to purchase.

[0200] As described above, the server receives input information and emotional information from the user, retrieves a list from a product database, and uses generative AI to provide information. The user looks at the list on their device, selects a product, and simulates the experience of the product through an experiential video. If the user finds a product they like, they can actually purchase it. By taking emotional information into account, customization and approaches can be made to match the user's emotions.

[0201] Example 2

[0202] Next, a description will be given of Example 2. In the following description, the data processing device 12 is referred to as a "server" and the smart device 14 is referred to as a "terminal."

[0203] Current virtual playground applications simply recommend products and games based on preferences and profile information, without considering the user's emotional state. As a result, they are unable to make suggestions that reflect the user's temporary emotions and moods, which makes it difficult to improve satisfaction or increase purchasing motivation. In addition, since detailed information about specific products is not provided, it is difficult for users to deepen their understanding of the products, which can reduce their motivation to purchase. Furthermore, it is difficult for users to imagine themselves actually playing the products, making it difficult for them to make a purchasing decision through the experience.

[0204] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0205] In this invention, the server includes: a means for grasping the emotional state of the user using emotion recognition technology; a means for selecting a toy or game suitable for the user from a product database based on the user's interests, age, and sex and the user's emotional state; a means for generating detailed information or a way of playing the selected toy or game based on a prompt sentence including the user's interests, age, and sex and instructing to generate detailed information or a way of playing the selected toy or game, and a generative AI; a means for presenting the generated detailed information or a way of playing to the user; a means for inputting a prepared video showing the state of playing the toy or game into a video from the user's line of sight while playing the toy or game, and a means for presenting the converted video to the user; a means for presenting to the user a screen including a purchase button for purchasing the toy or game and a purchase form for inputting purchase information necessary for the purchase, and a means for performing an order process based on the purchase information input in the purchase form when the purchase button is pressed. This enables appropriate product suggestions that reflect the user's temporary emotions and moods.

[0206] "User preferences" are objects or categories that a user finds particularly interesting or appealing.

[0207] A "database" is an information accumulation system that systematically organizes various information and data and allows them to be efficiently managed, searched, and retrieved.

[0208] "Generative AI" is an algorithm that applies natural language processing technology to generate new text information based on input data and prompts.

[0209] "Emotion recognition technology" is a technology that analyzes information such as voice, facial expressions, and text to accurately determine a user's emotional state.

[0210] "Profile Information" is a collective term for data relating to a user's individual attributes, such as age, gender, interests, preferences, etc.

[0211] "Product suggestions" are the act of recommending appropriate toys and games based on a user's profile information and emotional information.

[0212] A "simulated experience video" is a simulation video that makes it appear as if the user is actually using the toy or game.

[0213] "Purchase Option" means the choice or process through which a User may actually purchase a suggested Toy or Game.

[0214] "Natural language processing" refers to the techniques and processes that enable computers to understand and generate human language.

[0215] The present invention relates to a system for suggesting optimal toys and games based on a user's emotional state and profile information. This system is implemented by the following specific configuration and operation.

[0216] Hardware and Software Configuration

[0217] Server: A high-performance computer server is used to manage user profile information and emotion information, execute database queries, and run data generation models. The software uses MySQL (registered trademark) as a database management system (DBMS) and Python as a scripting language.

[0218] Terminal: A device through which a user accesses the system, such as a tablet or smartphone. These terminals have built-in cameras and microphones and receive audio and video input from the user. The software uses Google (registered trademark) Cloud Speech-to-Text as the voice recognition technology and Microsoft (registered trademark) Azure (registered trademark) Face API as the face recognition technology.

[0219] Emotion engine: An AI engine for recognizing the user's emotions, analyzing multiple pieces of information such as voice, facial expressions, and text to grasp the user's emotional state. This process uses an API that operates in real time.

[0220] Generative AI: Using a data generation model, it provides users with detailed product information and how to play in natural language. ChatGPT by OpenAI (registered trademark) can be used as this model. An example of a prompt sentence is "Tell me about a dinosaur toy that a 5-year-old boy would like."

[0221] Product Database: A database that stores information about toys and games, and the server executes queries to retrieve data.

[0222] Example of operation

[0223] When a user registers with the system, he or she enters profile information such as preferences, interests, age, gender, etc. from their terminal. This information is sent to the server and stored in a database. After registration is complete, the camera and microphone capture the user's voice and video when the user starts interacting with the system.

[0224] The emotion engine analyzes these data and determines the emotion state of “user having fun.” Meanwhile, the server combines the user’s profile information with the emotion information to perform a query to select appropriate toys and games from the product database.

[0225] For example, if a 5-year-old boy is interested in dinosaur toys, the server will retrieve products from the database that match the criteria, such as "5-year-old boy, dinosaurs, having fun." Next, the generative AI will generate detailed information about the selected product and display it as text on the device. Specifically, the information created will be, "Enjoy the realistic movement and sounds of this Tyrannosaurus toy!"

[0226] Furthermore, for products that the user has shown an interest in, the virtual experience video conversion function will provide a simulation video of what it would be like to play with the toy or game, allowing the user to visually confirm the feel of the product.

[0227] Finally, the terminal presents the user with purchasing options and connects to an online shopping platform, where the user can simply press the purchase button to purchase the product.

[0228] In this way, the system takes into account the user's emotional state and provides consistent support from suggesting the most suitable toys and games to purchasing them.

[0229] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0230] Step 1:

[0231] Collection of User Information

[0232] Terminal: When a user first accesses the system, they go through a registration process in which they fill in a form with information such as preferences, interests, age, and gender.

[0233] Input: Profile information entered by the user (preferences, interests, age, gender)

[0234] Output: The entered profile information is sent to the server and stored in the database.

[0235] Specific operation: A user enters information into an HTML form displayed on the terminal screen and clicks the submit button.

[0236] Step 2:

[0237] emotion recognition

[0238] Terminal: Uses a camera and microphone to capture the user's voice and video in real time to initiate a dialogue with the system.

[0239] Input: Video and audio data of the user's face

[0240] Output: The captured video and audio data is sent to the emotion engine.

[0241] Specific actions: The user speaks to the device and directs their facial expressions to the camera.

[0242] Emotion Engine: Analyzes captured audio and video to recognize the user's emotional state.

[0243] Input: Video and audio data

[0244] Output: Analyzed emotion information (e.g. happy, excited)

[0245] How it works: Emotion recognition algorithms use voice and facial recognition technology to determine emotional state.

[0246] Step 3:

[0247] Integration of user and emotion information

[0248] Server: Integrates the emotion information sent from the emotion engine with the user's profile information collected in advance.

[0249] Input: User profile information and emotion information

[0250] Output: Consolidated user information

[0251] Specific operation: The server executes a Python script to retrieve profile information and emotion information from the database and integrate them.

[0252] Step 4:

[0253] Get product data

[0254] Server: Based on the integrated user information, selects appropriate toys and games from a product database.

[0255] Input: Integrated user information (age, gender, interests, emotional state)

[0256] Output: A list of toys and games suitable for the user.

[0257] Specific operation: The server executes the query "SELECT FROM product database WHERE age = 5 AND interest = 'dinosaur' AND mood = 'happy'".

[0258] Step 5:

[0259] Generate detailed information

[0260] Generative AI: Uses NLP models to generate detailed information about selected toys and games. Example prompt: "Tell me about a dinosaur toy that a 5-year-old boy would like."

[0261] Input: A list of suitable toys and games, a prompt

[0262] Output: Generated detailed information (e.g. "This T-Rex toy moves and sounds realistic!")

[0263] Specific operation: Generative AI generates detailed product information based on the prompt text

[0264] Step 6:

[0265] Display of proposed products and provision of experience videos

[0266] Terminal: The generated detailed information is displayed to the user, and the play scene is recreated using a virtual experience video conversion function.

[0267] Input: Detailed information and a list of toys and games, and video data prepared in advance for toys and games showing the state of playing with them

[0268] Output: Details displayed to the user, and footage from the user's point of view

[0269] Specific operation: The server inputs a prepared video showing a user playing a toy or game, and converts it into a video from the user's point of view while playing the toy or game. The terminal displays the generated text of detailed information and the video from the user's point of view on the screen. In addition, when converting into an image from the user's point of view, the server may convert video data prepared in advance for a toy or game, which represents the state of playing the toy or game, into an image from the user's point of view, based on a prompt sentence that includes the user's emotions and instructs the conversion to an image from the user's point of view while playing the toy or game, and the generative AI. This makes it possible to convert into an image that takes the user's emotions into consideration. Specifically, by instructing in the prompt sentence to convert into an image with a color tone or tempo that matches the user's emotions, it is possible to convert into an image from the user's point of view that takes the user's emotions into consideration.

[0270] Step 7:

[0271] Present purchasing options

[0272] On your device: Displays options to directly purchase suggested toys and games. Connects to online shopping platforms.

[0273] Input: Proposed product, purchase option data

[0274] Output: A screen including a purchase button for purchasing a toy or game and a purchase form for inputting the necessary purchase information.

[0275] Specific operation: The terminal displays a screen including a purchase button and a purchase form. When the user enters purchase information and clicks the purchase button, the user is redirected to the online store and the order is processed.

[0276] (Application example 2)

[0277] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0278] In current virtual stores and online shopping, it is difficult to suggest appropriate products and toys to users and to realistically convey the experience of actually using them. In particular, there is a lack of means to provide product suggestions and virtual experiences that reflect the user's emotions and interests, and this poses a challenge in increasing purchasing motivation and satisfaction. To solve this problem, a system is needed that can accurately grasp the user's emotional state and customize product suggestions and experiences based on that information.

[0279] The identification process by the identification processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means. In this invention, the server includes a means for acquiring a list of products or play equipment from an information database based on the user's preferences, interests, age, or sex, a means for providing the user with detailed information or usage methods of the products or play equipment using a natural language processing engine, a means for analyzing the user's emotional state using an emotion recognition engine, and a means for converting the experience of using the play equipment into an image expressed from the user's point of view. This makes it possible to propose products that reflect the user's emotions and interests, and the user can experience the actual feel of use through a virtual experience, which is expected to increase purchasing motivation and satisfaction.

[0280] An "emotion recognition engine" is a device or software that analyzes information such as voice, facial expressions, and text to grasp a user's emotional state.

[0281] A "natural language processing engine" is a technology that uses generative AI models and other techniques to provide users with detailed information about toys and games and how to play them in natural language.

[0282] An "information database" is a database that stores information about various products and play equipment.

[0283] "Based on the user's preferences, interests, age, or gender" means that the most suitable products or play equipment are selected taking into account the user's individual attribute information.

[0284] "A means for converting the experience of using play equipment into an image expressed from the user's perspective" is a technology that virtually recreates the experience of using play equipment or a product and displays the image from the user's perspective.

[0285] The "server" is a central management system that links emotion recognition engines, natural language processing engines, information databases, etc. to provide users with optimal product suggestions and experiences.

[0286] An embodiment of the present invention comprises a system including the following main elements:

[0287] Emotion Recognition Engine

[0288] The server uses an emotion recognition engine to analyze the user's emotional state. The emotion recognition engine analyzes data such as the user's voice, facial expressions, or text messages captured using a camera or microphone. Here, face recognition and facial expression recognition are performed using OpenCV and deep learning models. This makes it possible to grasp the user's real-time emotions.

[0289] Information Database

[0290] The server retrieves a list of appropriate products and play equipment from the information database based on the emotion data obtained from the emotion recognition engine and profile information such as the user's age, gender, interests, etc. The information database stores detailed information about each product and play equipment, recommended age, gender, features, etc.

[0291] Natural Language Processing Engine

[0292] The server uses a natural language processing engine (generative AI model) to generate detailed information and instructions for playing with each product based on a list of products and play equipment retrieved from an information database. This makes it possible to provide information to users in a natural dialogue format. ChatGPT from OpenAI is used as the generative AI model.

[0293] Terminal Display

[0294] The terminal displays a list of the acquired products and playground equipment, as well as detailed information, to the user. It also displays images that express the experience of using the playground equipment from the user's point of view. A smartphone or a head-mounted display (HMD) is used for this. Using a virtual experience image conversion function, it provides an experience that makes the user feel as if they are actually using the product.

[0295] Buying Options

[0296] The terminal provides the user with the option to purchase products and play equipment, allowing the user to purchase the product they like on the spot.

[0297] Examples and prompts

[0298] For example, when a five-year-old boy is using a smartphone or HMD, the camera reads his facial expression and the emotion recognition engine detects that he is smiling. The server is a five-year-old boy, and selects remote control cars and building block sets as product candidates based on the smiling state. The generative AI model generates detailed information about the selected products and how to play with them, explaining them to the user in natural language. The user can enjoy the experience of operating a remote control car in a virtual space with the HMD, and if they like it, they can purchase it on the spot.

[0299] Example prompt:

[0300] What are the features of the following toy? Remote control car

[0301] This makes it possible to suggest products that reflect the user's emotions and interests, and since the user can check out products through a virtual experience before purchasing, it is expected to increase purchasing motivation and satisfaction.

[0302] The flow of the specific process in the application example 2 will be described with reference to FIG.

[0303] Step 1:

[0304] The device uses a camera and microphone to capture the user's voice and facial expressions. Input includes the user's facial image and voice data. This data is transmitted to the server in real time.

[0305] Step 2:

[0306] The server starts an emotion recognition engine to analyze the transmitted voice and facial expression data. The analysis process uses OpenCV and deep learning models. As input, the user's voice and facial image data are used, and as output, the user's emotional state (e.g., happy, sad, surprised, etc.) is obtained.

[0307] Step 3:

[0308] The server queries the information database based on the emotional state obtained from the emotion recognition engine and the user's age, gender, and interests to obtain a list of products and toys. The user's emotional state, age, gender, and interests are used as inputs, and a list of suitable products and toys is obtained as output.

[0309] Step 4:

[0310] The server uses a natural language processing engine (generative AI model) to generate detailed information and usage instructions for each product from the list of acquired products and play equipment. This process uses generative AI such as ChatGPT from OpenAI. Product names and category information are used as input, and the generated detailed information and usage instructions for the product are obtained as output.

[0311] Step 5:

[0312] The server transmits the generated product information to the terminal, which displays it to the user. As input, text data of the product information is used, and as output, the product information is displayed on the user's device.

[0313] Step 6:

[0314] The terminal uses the virtual experience video conversion function to provide a virtual experience of the product selected by the user. In this process, 3D models and computer graphics technology are used. The product information selected by the user is used as input, and the virtual experience video from the user's point of view is obtained as output.

[0315] Step 7:

[0316] After the user enjoys the experience, the terminal offers purchasing options, using the user's selected product information as input and obtaining a display of purchasing options as output.

[0317] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits the voice data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0318] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by making a neural network perform deep learning. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image is input. The data generation model 58 performs inference on the input inference data according to the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0319] In the above embodiment, an example was given in which the specific process was performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0320] [Second embodiment]

[0321] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0322] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0323] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[0324] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0325] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[0326] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[0327] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54. The transmission and reception of various types of information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[0328] Fig. 4 shows an example of main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0329] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0330] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0331] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50, and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0332] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[0333] An embodiment for implementing the present invention includes the following elements.

[0334] 1. Server: Has the function of receiving information on the user's preferences, interests, age, and gender, and retrieving a list of toys and games from a product database.

[0335] 2. Generative AI: Equipped with the ability to use natural language processing AI models to provide users with detailed information about toys and games and how to play them.

[0336] 3. Terminal: Shows the user a list of toys and games, displays a video to let them experience the selected product, and also provides the option to actually purchase it.

[0337] 4. Product Database: A database that stores information about toys and games, providing users with information to choose appropriate products based on their preferences, interests, age, and gender.

[0338] 5. Virtual experience video conversion function: Equipped with a function that converts the image of playing a toy or game into an image from a child's perspective and displays it on the device.

[0339] In this way, the server, generative AI, terminal, product database, and virtual experience video conversion function work in coordination, allowing users to select appropriate toys and games, experience products through a virtual experience, and actually purchase them.

[0340] The process flow of each embodiment will be described below.

[0341] Step 1: The server receives input information from the user (preferences, interests, age, gender).

[0342] Step 2: The server uses the received information to retrieve a list of toys and games from the product database.

[0343] Step 3: The server uses generative AI to provide the user with detailed information about the toy or game and how to play it.

[0344] Step 4: The device displays a list of toys and games to the user.

[0345] Step 5: User selects the product that interests them from the list.

[0346] Step 6: The device will display a video to experience the selected product.

[0347] Step 7: The user experiences the product virtually.

[0348] Step 8: If the user finds an item they like, the device gives them the option to actually purchase it.

[0349] Step 9: The user proceeds with the purchase.

[0350] For example, if a user inputs "I want a robot toy," the server will refer to the user's information and retrieve a list of robot toys from a product database. If the user then asks the generative AI, "Please tell me how to play with this robot toy," the generative AI will provide an appropriate answer. When the user selects a robot toy that interests them, the device will display an image from a child's perspective, allowing the user to have a simulated experience. If the user likes it, they can actually purchase it.

[0351] Example 1

[0352] Next, a description will be given of Example 1. In the following description, the data processing device 12 is referred to as a "server" and the smart glasses 214 are referred to as a "terminal".

[0353] In conventional systems, it is difficult for users to easily find toys and games that match their preferences and interests. In addition, detailed product descriptions and instructions for playing based on a lot of information are lacking. Furthermore, the functionality for actually experiencing toys and games is limited, making it difficult to increase purchasing motivation. This results in low user satisfaction and makes it an issue to increase purchasing motivation.

[0354] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0355] In this invention, the server includes a means for acquiring a list of toys or games from a product database based on the user's preferences, interests, age, or sex, a means for providing the user with detailed product information or playing instructions using a generative AI model, a means for converting the state of playing the toy or game to the user's perspective, a means for transmitting information input by the user to the server, a means for displaying the list acquired by the server from the product database on a terminal, and a means for generating a prompt sentence based on a product selected by the user and acquiring the detailed information. This allows the user to easily find toys or games that match their preferences or interests, and to consider purchasing them after obtaining detailed information. In addition, the simulated experience increases purchasing motivation and user satisfaction.

[0356] A "server" is a computer device that receives information input by a user, retrieves an appropriate product list from a product database, and transmits the list to a terminal.

[0357] "User preferences, interests, age, and gender" are individual attribute information input by the user, and are important criteria for product selection.

[0358] The "product database" is a database that stores detailed information about various toys and games.

[0359] The "list of toys or games" is a list of toys and games obtained from a product database, and is a list that has been filtered based on the user's attribute information.

[0360] A "generative AI model" is an artificial intelligence model that generates and provides detailed product information and play instructions to users in natural language.

[0361] A "prompt sentence" is an instruction sentence that causes a generative AI model to generate detailed information.

[0362] "Means of providing detailed information or instructions for playing" refers to a function that uses a generative AI model to present users with detailed information about a product or instructions for playing the product.

[0363] "Playing with a toy or game" is visual data showing a situation in which the selected toy or game is actually being used.

[0364] The "means for converting into a user's viewpoint" is a function for converting the state of playing a toy or game into an image from the user's viewpoint.

[0365] The "means for transmitting information input by the user to the server" is a function for transmitting information input by the user using the terminal to the server via the network.

[0366] A "terminal" is a computing device through which a user enters information and displays information received from a server.

[0367] A "simulated experience video" is a video that makes the user feel as if they are actually experiencing a toy or game that they have selected.

[0368] "Purchase options" is a function that provides options for the user to actually purchase a product in which they are interested.

[0369] This invention is a system that allows users to easily find toys and games that match their tastes and interests, and consider purchasing them after obtaining detailed information. Specifically, a server, a terminal, a generative AI model, a product database, and a virtual experience video conversion function work together.

[0370] The server receives information about preferences, interests, age, and gender that the user inputs using the terminal. For example, if the user inputs information such as "7-year-old boy, likes dinosaurs," the information is sent to the server via the Internet. Based on this information, the server retrieves a list of suitable toys and games from a product database. The product database contains information about a wide variety of toys and games, and returns a list that matches the criteria to the server.

[0371] The device displays the product list sent from the server to the user. The user browses the list on the device and selects a product that interests them. At that time, the device utilizes a generative AI model to provide the user with detailed information about the selected product and how to play with it in natural language. For example, this generative AI model applies a prompt sentence such as "Please explain the details of the dinosaur exploration set and how to play with it," and the AI ​​generates the relevant information.

[0372] In addition, the device is equipped with a virtual experience video conversion function that generates a video that makes it appear as if the user is actually playing the selected toy or game. This video is displayed from the user's point of view, so the user can feel as if they are playing the toy or game. For example, a realistic video of playing with a dinosaur exploration set is provided.

[0373] Finally, the device will offer the user the option to purchase the item they are interested in. The user can then complete the purchase process from the displayed options and get their hands on the selected toy or game.

[0374] The system allows users to easily find toys and games that match their tastes and interests, and then get detailed information about them to make a purchase. Using generative AI models and virtual reality videos, the user experience is improved and purchase motivation is increased. An example of a prompt sentence is "Please explain the details of the dinosaur exploration set and how to play it."

[0375] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0376] Step 1:

[0377] The user uses the device to input information about his / her preferences, interests, age, and gender. Specifically, the user enters information such as "7-year-old boy, loves dinosaurs" into a form on the screen.

[0378] Input: User attribute information (preferences, interests, age, gender)

[0379] Output: User information data entered

[0380] Step 2:

[0381] The terminal sends the information entered by the user to the server. When the user presses the send button, the entered information is sent to the server via the Internet.

[0382] Input: User information data entered

[0383] Output: User information sent to the server

[0384] Step 3:

[0385] The server retrieves a list of suitable toys and games from the product database based on the received user information. The server searches the database for the condition "age = 7 years old AND preference = dinosaurs" and retrieves the list.

[0386] Input: User information

[0387] Output: A filtered list of toys and games.

[0388] Step 4:

[0389] The terminal displays a list of toys and games sent from the server to the user. For example, the list may show a "Dinosaur Exploration Set."

[0390] Input: Product list sent from the server

[0391] Output: Product list displayed on the terminal screen

[0392] Step 5:

[0393] The user selects from the displayed list the toy or game that he or she is interested in. For example, the user selects "Dinosaur Exploration Set."

[0394] Input: Displayed product list

[0395] Output: Selected product data

[0396] Step 6:

[0397] The server sends a prompt to the generative AI model to generate detailed information about the product selected by the user and instructions on how to play with it. Specifically, the server sends the prompt to the generative AI model, "Please explain the details of the dinosaur exploration set and how to play with it."

[0398] Input: Selected product data

[0399] Output: The prompt sent to the generative AI model

[0400] Step 7:

[0401] The generative AI model generates detailed product information and instructions for playing based on the prompt text sent and returns it to the server.

[0402] Input: Prompt statement

[0403] Output: Generated details and how to play

[0404] Step 8:

[0405] The server sends the detailed information received from the generative AI model to the terminal, which then displays the information to the user, who can view the details in a natural conversational format.

[0406] Input: Detailed information received from the generative AI model

[0407] Output: Detailed information displayed on the terminal screen

[0408] Step 9:

[0409] The device uses its virtual experience conversion function to generate a virtual experience video of the selected toy or game, for example, from the perspective of a child playing with a "dinosaur exploration set."

[0410] Input: Selected product data

[0411] Output: Generated simulated experience video

[0412] Step 10:

[0413] The device displays the generated simulated experience video to the user, who feels as if he or she is actually playing the toy or game.

[0414] Input: Generated simulated experience video

[0415] Output: Simulation video displayed on the device screen

[0416] Step 11:

[0417] The terminal then provides options for purchasing the selected toy or game, and the user can proceed with the purchase from the displayed options.

[0418] Input: Selected product data

[0419] Output: A screen with purchase options

[0420] This enables the system as a whole to consistently provide products based on the user's preferences and interests, obtain detailed information, and provide purchasing support through simulated experiences.

[0421] (Application example 1)

[0422] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[0423] On current online shopping sites, it is difficult for users to find toys and games that interest them, and they have to browse multiple pages to find detailed information and instructions on how to play. Another issue is that it is difficult to provide a simulated experience to increase the desire to purchase a product, and an experience from the user's perspective is particularly important for products aimed at children.

[0424] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0425] In this invention, the server includes a means for acquiring a list of toys or games from a product database based on a user's preferences, interests, age, or sex, a means for providing the user with detailed product information or play instructions using a data generation model, a means for converting a state of playing a toy or game into a view from the user's perspective, a means for displaying a list of recommended toys and games appropriate for the user, a means for generating detailed information about toys or games in natural language using generative AI, and a means for displaying the generated detailed information and recommendation list on the user's terminal. This allows the user to easily find products that interest them and check detailed information in a centralized manner, and also increases their desire to purchase through a simulated experience.

[0426] "User preferences" refers to a user's personal tastes and preferences, and is a concept that includes preferences for specific categories and themes.

[0427] "Interest" refers to a subject or area in which a user is interested, and is a concept that includes a state in which a user is interested in a particular category or topic.

[0428] "Age" refers to the number of years from the user's date of birth to the present, and is a factor that reflects an individual's stage of growth and development.

[0429] "Gender" refers to a user's biological sex, and is a factor that influences an individual's purchasing behavior and preferences.

[0430] "Product database" refers to a database that stores information about toys and games, and is a system for retrieving specific product information as needed.

[0431] "Data generation model" refers to a generative AI model used to provide users with detailed information about a product or instructions for playing the product, and is a means of using technologies such as natural language processing.

[0432] A "recommended list" is a list of toys and games selected based on a user's preferences, interests, age, and gender, and provides individually optimized product suggestions.

[0433] "Generative AI" refers to an artificial intelligence model that generates appropriate information or answers based on given prompts, and is a technology that is particularly used for generating information in natural language.

[0434] "Natural language" refers to language that humans use on a daily basis, and is a term used in a context that includes machine natural language processing.

[0435] "Terminal" refers to a device used by a user to receive and display information, including smartphones and tablets.

[0436] "Simulated experience video" refers to video that makes the user feel as if they are actually playing a toy or game, and is a means of visually providing an experience from the user's perspective.

[0437] The present invention is a system that recommends optimal toys and games based on a user's preferences, interests, age, and gender, and provides detailed information and simulated experiences. An embodiment of the system according to the present invention will be specifically described below.

[0438] The server receives information about the user's preferences, interests, age, and gender, and retrieves a list of toys and games from a product database. It then uses a data generation model to provide the user with detailed product information and how to play with the product. To do this, it uses a generative AI model (e.g. ChatGPT) to generate appropriate information based on a given prompt. The detailed information and recommendation list are then displayed on the user's device (e.g., smartphone, tablet, etc.).

[0439] The server also has a virtual experience video conversion function that converts the state of playing a toy or game into the user's perspective. This video is used to help users understand the product more specifically. For example, a prompt sentence such as "Please tell me more about the telescope" can be input to the generative AI to generate a detailed explanation about the telescope.

[0440] Users can select toys and games that interest them from a list of suggested products based on their preferences and interests, and check out the detailed information and simulated experience video. For example, if a 7-year-old boy is interested in science, science experiment kits and telescopes will be recommended, and users can view detailed information and videos about them. At the same time, users are given the option to purchase the selected product, and can then complete the purchase procedure.

[0441] The hardware required is a terminal (smartphone, tablet, etc.) that can accept user operations, and the software required is a generative AI model (for example, a natural language processing platform for running ChatGPT), a database system, and a system with a simulated experience video conversion function.

[0442] The system allows users to easily find toys and games that match their interests and preferences, and provides detailed information and simulated experiences to make a purchase with confidence.

[0443] The flow of the specific process in the application example 1 will be described with reference to FIG.

[0444] Step 1:

[0445] A user uses a terminal to send input information such as preferences, interests, age, and gender to a server.

[0446] Input: User preferences, interests, age, gender

[0447] Output: Receives the corresponding input data on the server side

[0448] Specific operation: When the user answers questions on the application screen and presses the submit button, this information is sent to the server.

[0449] Step 2:

[0450] The server searches for and acquires a list of relevant toys and games from a product database based on the received user information.

[0451] Input: User preferences, interests, age, and gender information

[0452] Output: Product list based on user information

[0453] Specific operation: The server executes a database query to retrieve product information that matches the criteria.

[0454] Step 3:

[0455] The server uses a generative AI model (e.g. ChatGPT) to generate detailed information and instructions for each toy and game included in the product list.

[0456] Input: product list, generative AI model, prompt

[0457] Output: Detailed information about each product and how to play

[0458] Specific operation: The server inputs each product name as a prompt sentence into the generative AI and generates a detailed product description.

[0459] Step 4:

[0460] The server transmits the generated detailed information and recommendation list to the user's terminal.

[0461] Input:Detailed information, Recommendation list

[0462] Output: Product list and details displayed on the user's device

[0463] Specific operation: The generated information is sent to the user's device in JSON or XML format, and displayed on the app screen.

[0464] Step 5:

[0465] When a user browses detailed information and selects a product that interests them, the server creates a simulated experience video of that product and converts it into a view from the user's perspective.

[0466] Input: User selection, virtual experience video conversion function

[0467] Output: Virtual experience video from the user's point of view

[0468] Specific operation: Generate a play scene based on the selected product and convert it into a simulated experience video.

[0469] Step 6:

[0470] The server transmits the converted simulated experience video to the user's terminal, and the user experiences it.

[0471] Input: Simulation video

[0472] Output: Virtual experience video displayed on the user's device

[0473] Specific operation: The generated video data is sent to the user's device and played using the video player on the app.

[0474] Step 7:

[0475] When a user wishes to purchase a product, the user completes the purchase procedure via the terminal, and the server accepts the order.

[0476] Input: User's purchase intention, purchase procedure information

[0477] Output: Purchase information received and confirmed

[0478] Specific operation: The user presses the purchase button, enters the necessary information in the purchase form, and submits it. The server confirms the received purchase information and processes the order.

[0479] By following these steps, users can easily find toys and games that match their interests and preferences, and then purchase with confidence thanks to detailed information and simulated experiences.

[0480] In addition, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0481] An embodiment for implementing the present invention includes the following elements.

[0482] 1. Emotion engine: An engine for recognizing the user's emotions. It analyzes information such as voice, facial expressions, and text to grasp the user's emotional state.

[0483] 2. Server: Has the function of combining the user's preferences, interests, age, and gender information with the emotional information obtained from the emotion engine, and retrieving a list of toys and games from a product database.

[0484] 3. Generative AI: Equipped with the ability to use natural language processing AI models to provide users with detailed information about toys and games and how to play them.

[0485] 4. Terminal: Shows the user a list of toys and games, and has the ability to show a video to experience the selected product, as well as provide the option to actually purchase it.

[0486] 5. Product Database: A database that stores information about toys and games, providing information to help users choose the right product based on their preferences, interests, age, gender, and emotions.

[0487] 6. Virtual experience video conversion function: Equipped with a function that converts the image of playing a toy or game into an image from a child's perspective and displays it on the device.

[0488] In the above-described configuration, in a system that combines an emotion engine, the server, generative AI, terminal, product database, and simulated experience video conversion function work in coordination, making it possible to customize suggestions and experiences for toys and games according to the user's emotions.

[0489] The process flow of each embodiment will be described below.

[0490] Step 1: The server receives input information from the user (preferences, interests, age, gender) and recognizes the user's emotion information using an emotion engine.

[0491] Step 2: The server retrieves a list of toys and games from the product database based on the received information. Taking into account the emotion information, the server selects toys and games that match the user's emotion.

[0492] Step 3: The server uses generative AI to provide the user with detailed information about the toy or game and how to play it. It also uses emotional information to choose the right language and approach to suit the user’s emotions.

[0493] Step 4: The device displays a list of toys and games to the user, adjusting the order and presentation of the list based on the emotional information.

[0494] Step 5: The user selects the products from the list that he or she is interested in. Taking into account the emotion information, we help the user select products that match his or her emotion.

[0495] Step 6: The device displays a video for experiencing the selected product. Taking into account the emotional information, the presentation and expression of the video are adjusted.

[0496] Step 7: The user experiences the product in a simulated way. Taking into account the emotional information, the product responds and gives feedback that matches the user's emotions.

[0497] Step 8: If the user finds a product they like, the device offers them the option to actually purchase it. Taking into account emotional information, the approach is to increase their desire to purchase.

[0498] Step 9: The user proceeds with the purchase. Taking into account the emotional information, the procedure is carried out to support the user's willingness to purchase.

[0499] As described above, the server receives input information and emotional information from the user, retrieves a list from a product database, and uses generative AI to provide information. The user looks at the list on their device, selects a product, and simulates the experience of the product through an experiential video. If the user finds a product they like, they can actually purchase it. By taking emotional information into account, customization and approaches can be made to match the user's emotions.

[0500] Example 2

[0501] Next, a description will be given of Example 2. In the following description, the data processing device 12 is referred to as a "server" and the smart device 14 is referred to as a "terminal."

[0502] Current virtual playground applications simply recommend products and games based on preferences and profile information, without considering the user's emotional state. As a result, they are unable to make suggestions that reflect the user's temporary emotions and moods, which makes it difficult to improve satisfaction or increase purchasing motivation. In addition, since detailed information about specific products is not provided, it is difficult for users to deepen their understanding of the products, which can reduce their motivation to purchase. Furthermore, it is difficult for users to imagine themselves actually playing the products, making it difficult for them to make a purchasing decision through the experience.

[0503] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0504] In this invention, the server includes: a means for grasping the emotional state of the user using emotion recognition technology; a means for selecting a toy or game suitable for the user from a product database based on the user's interests, age, and sex and the user's emotional state; a means for generating detailed information or a way of playing the selected toy or game based on a prompt sentence including the user's interests, age, and sex and instructing to generate detailed information or a way of playing the selected toy or game, and a generative AI; a means for presenting the generated detailed information or a way of playing to the user; a means for inputting a prepared video showing the state of playing the toy or game into a video from the user's line of sight while playing the toy or game, and a means for presenting the converted video to the user; a means for presenting to the user a screen including a purchase button for purchasing the toy or game and a purchase form for inputting purchase information necessary for the purchase, and a means for performing an order process based on the purchase information input in the purchase form when the purchase button is pressed. This enables appropriate product suggestions that reflect the user's temporary emotions and moods.

[0505] "User preferences" are objects or categories that a user finds particularly interesting or appealing.

[0506] A "database" is an information accumulation system that systematically organizes various information and data and allows them to be efficiently managed, searched, and retrieved.

[0507] "Generative AI" is an algorithm that applies natural language processing technology to generate new text information based on input data and prompts.

[0508] "Emotion recognition technology" is a technology that analyzes information such as voice, facial expressions, and text to accurately determine a user's emotional state.

[0509] "Profile Information" is a collective term for data relating to a user's individual attributes, such as age, gender, interests, preferences, etc.

[0510] "Product suggestions" are the act of recommending appropriate toys and games based on a user's profile information and emotional information.

[0511] A "simulated experience video" is a simulation video that makes it appear as if the user is actually using the toy or game.

[0512] "Purchase Option" means the choice or process through which a User may actually purchase a suggested Toy or Game.

[0513] "Natural language processing" refers to the techniques and processes that enable computers to understand and generate human language.

[0514] The present invention relates to a system for suggesting optimal toys and games based on a user's emotional state and profile information. This system is implemented by the following specific configuration and operation.

[0515] Hardware and Software Configuration

[0516] Server: A high-performance computer server is used to manage user profile information and emotion information, execute database queries, and run data generation models. The software uses MySQL (registered trademark) as a database management system (DBMS) and Python as a scripting language.

[0517] Terminal: A device through which a user accesses the system, such as a tablet or smartphone. These terminals have built-in cameras and microphones and receive audio and video input from the user. The software uses Google (registered trademark) Cloud Speech-to-Text as the voice recognition technology and Microsoft (registered trademark) Azure (registered trademark) Face API as the face recognition technology.

[0518] Emotion engine: An AI engine for recognizing the user's emotions, analyzing multiple pieces of information such as voice, facial expressions, and text to grasp the user's emotional state. This process uses an API that operates in real time.

[0519] Generative AI: Using a data generation model, it provides users with detailed product information and how to play in natural language. ChatGPT by OpenAI (registered trademark) can be used as this model. An example of a prompt sentence is "Tell me about a dinosaur toy that a 5-year-old boy would like."

[0520] Product Database: A database that stores information about toys and games, and the server executes queries to retrieve data.

[0521] Example of operation

[0522] When a user registers with the system, he or she enters profile information such as preferences, interests, age, gender, etc. from their terminal. This information is sent to the server and stored in a database. After registration is complete, the camera and microphone capture the user's voice and video when the user starts interacting with the system.

[0523] The emotion engine analyzes these data and determines the emotion state of “user having fun.” Meanwhile, the server combines the user’s profile information with the emotion information to perform a query to select appropriate toys and games from the product database.

[0524] For example, if a 5-year-old boy is interested in dinosaur toys, the server will retrieve products from the database that match the criteria, such as "5-year-old boy, dinosaurs, having fun." Next, the generative AI will generate detailed information about the selected product and display it as text on the device. Specifically, the information created will be, "Enjoy the realistic movement and sounds of this Tyrannosaurus toy!"

[0525] Furthermore, for products that the user has shown an interest in, the virtual experience video conversion function will provide a simulation video of what it would be like to play with the toy or game, allowing the user to visually confirm the feel of the product.

[0526] Finally, the terminal presents the user with purchasing options and connects to an online shopping platform, where the user can simply press the purchase button to purchase the product.

[0527] In this way, the system takes into account the user's emotional state and provides consistent support from suggesting the most suitable toys and games to purchasing them.

[0528] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0529] Step 1:

[0530] Collection of User Information

[0531] Terminal: When a user first accesses the system, they go through a registration process in which they fill in a form with information such as preferences, interests, age, and gender.

[0532] Input: Profile information entered by the user (preferences, interests, age, gender)

[0533] Output: The entered profile information is sent to the server and stored in the database.

[0534] Specific operation: A user enters information into an HTML form displayed on the terminal screen and clicks the submit button.

[0535] Step 2:

[0536] emotion recognition

[0537] Terminal: Uses a camera and microphone to capture the user's voice and video in real time to initiate a dialogue with the system.

[0538] Input: Video and audio data of the user's face

[0539] Output: The captured video and audio data is sent to the emotion engine.

[0540] Specific actions: The user speaks to the device and directs their facial expressions to the camera.

[0541] Emotion Engine: Analyzes captured audio and video to recognize the user's emotional state.

[0542] Input: Video and audio data

[0543] Output: Analyzed emotion information (e.g. happy, excited)

[0544] How it works: Emotion recognition algorithms use voice and facial recognition technology to determine emotional state.

[0545] Step 3:

[0546] Integration of user and emotion information

[0547] Server: Integrates the emotion information sent from the emotion engine with the user's profile information collected in advance.

[0548] Input: User profile information and emotion information

[0549] Output: Consolidated user information

[0550] Specific operation: The server executes a Python script to retrieve profile information and emotion information from the database and integrate them.

[0551] Step 4:

[0552] Get product data

[0553] Server: Based on the integrated user information, selects appropriate toys and games from a product database.

[0554] Input: Integrated user information (age, gender, interests, emotional state)

[0555] Output: A list of toys and games suitable for the user.

[0556] Specific operation: The server executes the query "SELECT FROM product database WHERE age = 5 AND interest = 'dinosaur' AND mood = 'happy'".

[0557] Step 5:

[0558] Generate detailed information

[0559] Generative AI: Uses NLP models to generate detailed information about selected toys and games. Example prompt: "Tell me about a dinosaur toy that a 5-year-old boy would like."

[0560] Input: A list of suitable toys and games, a prompt

[0561] Output: Generated detailed information (e.g. "This T-Rex toy moves and sounds realistic!")

[0562] Specific operation: Generative AI generates detailed product information based on the prompt text

[0563] Step 6:

[0564] Display of proposed products and provision of experience videos

[0565] Terminal: The generated detailed information is displayed to the user, and the play scene is recreated using a virtual experience video conversion function.

[0566] Input: Detailed information and a list of toys and games, and video data prepared in advance for toys and games showing the state of playing with them

[0567] Output: Details displayed to the user, and footage from the user's point of view

[0568] Specific operation: The server inputs a prepared video showing a user playing a toy or game, and converts it into a video from the user's point of view while playing the toy or game. The terminal displays the generated text of detailed information and the video from the user's point of view on the screen. In addition, when converting into an image from the user's point of view, the server may convert video data prepared in advance for a toy or game, which represents the state of playing the toy or game, into an image from the user's point of view, based on a prompt sentence that includes the user's emotions and instructs the conversion to an image from the user's point of view while playing the toy or game, and the generative AI. This makes it possible to convert into an image that takes the user's emotions into consideration. Specifically, by instructing in the prompt sentence to convert into an image with a color tone or tempo that matches the user's emotions, it is possible to convert into an image from the user's point of view that takes the user's emotions into consideration.

[0569] Step 7:

[0570] Present purchasing options

[0571] On your device: Displays options to directly purchase suggested toys and games. Connects to online shopping platforms.

[0572] Input: Proposed product, purchase option data

[0573] Output: A screen including a purchase button for purchasing a toy or game and a purchase form for inputting the necessary purchase information.

[0574] Specific operation: The terminal displays a screen including a purchase button and a purchase form. When the user enters purchase information and clicks the purchase button, the user is redirected to the online store and the order is processed.

[0575] (Application example 2)

[0576] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[0577] In current virtual stores and online shopping, it is difficult to suggest appropriate products and toys to users and to realistically convey the experience of actually using them. In particular, there is a lack of means to provide product suggestions and virtual experiences that reflect the user's emotions and interests, and this poses a challenge in increasing purchasing motivation and satisfaction. To solve this problem, a system is needed that can accurately grasp the user's emotional state and customize product suggestions and experiences based on that information.

[0578] The identification process by the identification processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means. In this invention, the server includes a means for acquiring a list of products or play equipment from an information database based on the user's preferences, interests, age, or sex, a means for providing the user with detailed information or usage methods of the products or play equipment using a natural language processing engine, a means for analyzing the user's emotional state using an emotion recognition engine, and a means for converting the experience of using the play equipment into an image expressed from the user's point of view. This makes it possible to propose products that reflect the user's emotions and interests, and the user can experience the actual feel of use through a virtual experience, which is expected to increase purchasing motivation and satisfaction.

[0579] An "emotion recognition engine" is a device or software that analyzes information such as voice, facial expressions, and text to grasp a user's emotional state.

[0580] A "natural language processing engine" is a technology that uses generative AI models and other techniques to provide users with detailed information about toys and games and how to play them in natural language.

[0581] An "information database" is a database that stores information about various products and play equipment.

[0582] "Based on the user's preferences, interests, age, or gender" means that the most suitable products or play equipment are selected taking into account the user's individual attribute information.

[0583] "A means for converting the experience of using play equipment into an image expressed from the user's perspective" is a technology that virtually recreates the experience of using play equipment or a product and displays the image from the user's perspective.

[0584] The "server" is a central management system that links emotion recognition engines, natural language processing engines, information databases, etc. to provide users with optimal product suggestions and experiences.

[0585] An embodiment of the present invention comprises a system including the following main elements:

[0586] Emotion Recognition Engine

[0587] The server uses an emotion recognition engine to analyze the user's emotional state. The emotion recognition engine analyzes data such as the user's voice, facial expressions, or text messages captured using a camera or microphone. Here, face recognition and facial expression recognition are performed using OpenCV and deep learning models. This makes it possible to grasp the user's real-time emotions.

[0588] Information Database

[0589] The server retrieves a list of appropriate products and play equipment from the information database based on the emotion data obtained from the emotion recognition engine and profile information such as the user's age, gender, interests, etc. The information database stores detailed information about each product and play equipment, recommended age, gender, features, etc.

[0590] Natural Language Processing Engine

[0591] The server uses a natural language processing engine (generative AI model) to generate detailed information and instructions for playing with each product based on a list of products and play equipment retrieved from an information database. This makes it possible to provide information to users in a natural dialogue format. ChatGPT from OpenAI is used as the generative AI model.

[0592] Terminal Display

[0593] The terminal displays a list of the acquired products and playground equipment, as well as detailed information, to the user. It also displays images that express the experience of using the playground equipment from the user's point of view. A smartphone or a head-mounted display (HMD) is used for this. Using a virtual experience image conversion function, it provides an experience that makes the user feel as if they are actually using the product.

[0594] Buying Options

[0595] The terminal provides the user with the option to purchase products and play equipment, allowing the user to purchase the product they like on the spot.

[0596] Examples and prompts

[0597] For example, when a five-year-old boy is using a smartphone or HMD, the camera reads his facial expression and the emotion recognition engine detects that he is smiling. The server is a five-year-old boy, and selects remote control cars and building block sets as product candidates based on the smiling state. The generative AI model generates detailed information about the selected products and how to play with them, explaining them to the user in natural language. The user can enjoy the experience of operating a remote control car in a virtual space with the HMD, and if they like it, they can purchase it on the spot.

[0598] Example prompt:

[0599] What are the features of the following toy? Remote control car

[0600] This makes it possible to suggest products that reflect the user's emotions and interests, and since the user can check out products through a virtual experience before purchasing, it is expected to increase purchasing motivation and satisfaction.

[0601] The flow of the specific process in the application example 2 will be described with reference to FIG.

[0602] Step 1:

[0603] The device uses a camera and microphone to capture the user's voice and facial expressions. Input includes the user's facial image and voice data. This data is transmitted to the server in real time.

[0604] Step 2:

[0605] The server starts an emotion recognition engine to analyze the transmitted voice and facial expression data. The analysis process uses OpenCV and deep learning models. As input, the user's voice and facial image data are used, and as output, the user's emotional state (e.g., happy, sad, surprised, etc.) is obtained.

[0606] Step 3:

[0607] The server queries the information database based on the emotional state obtained from the emotion recognition engine and the user's age, gender, and interests to obtain a list of products and toys. The user's emotional state, age, gender, and interests are used as inputs, and a list of suitable products and toys is obtained as output.

[0608] Step 4:

[0609] The server uses a natural language processing engine (generative AI model) to generate detailed information and usage instructions for each product from the list of acquired products and play equipment. This process uses generative AI such as ChatGPT from OpenAI. Product names and category information are used as input, and the generated detailed information and usage instructions for the product are obtained as output.

[0610] Step 5:

[0611] The server transmits the generated product information to the terminal, which displays it to the user. As input, text data of the product information is used, and as output, the product information is displayed on the user's device.

[0612] Step 6:

[0613] The terminal uses the virtual experience video conversion function to provide a virtual experience of the product selected by the user. In this process, 3D models and computer graphics technology are used. The product information selected by the user is used as input, and the virtual experience video from the user's point of view is obtained as output.

[0614] Step 7:

[0615] After the user enjoys the experience, the terminal offers purchasing options, using the user's selected product information as input and obtaining a display of purchasing options as output.

[0616] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits the voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0617] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by making a neural network perform deep learning. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image is input. The data generation model 58 performs inference on the input inference data according to the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0618] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0619] [Third embodiment]

[0620] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0621] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0622] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[0623] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0624] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[0625] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[0626] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54. The transmission and reception of various types of information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[0627] Fig. 6 shows an example of main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0628] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0629] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0630] In the headset type terminal 314, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50, and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0631] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server", and the headset type terminal 314 will be referred to as the "terminal".

[0632] An embodiment for implementing the present invention includes the following elements.

[0633] 1. Server: Has the function of receiving information on the user's preferences, interests, age, and gender, and retrieving a list of toys and games from a product database.

[0634] 2. Generative AI: Equipped with the ability to use natural language processing AI models to provide users with detailed information about toys and games and how to play them.

[0635] 3. Terminal: Shows the user a list of toys and games, displays a video to let them experience the selected product, and also provides the option to actually purchase the product.

[0636] 4. Product Database: A database that stores information about toys and games, providing users with information to select appropriate products based on their preferences, interests, age, and gender.

[0637] 5. Virtual experience video conversion function: Equipped with a function that converts the image of playing a toy or game into an image from a child's perspective and displays it on the device.

[0638] In this way, the server, generative AI, terminal, product database, and virtual experience video conversion function work together to enable users to select appropriate toys and games, experience products through a virtual experience, and actually purchase them.

[0639] The process flow of each embodiment will be described below.

[0640] Step 1: The server receives input information from the user (preferences, interests, age, gender).

[0641] Step 2: The server uses the received information to retrieve a list of toys and games from the product database.

[0642] Step 3: The server uses generative AI to provide the user with detailed information about the toy or game and how to play it.

[0643] Step 4: The device displays a list of toys and games to the user.

[0644] Step 5: User selects the product that interests them from the list.

[0645] Step 6: The device will display a video to experience the selected product.

[0646] Step 7: The user experiences the product virtually.

[0647] Step 8: If the user finds an item they like, the device gives them the option to actually purchase it.

[0648] Step 9: The user proceeds with the purchase.

[0649] For example, if a user inputs "I want a robot toy," the server will refer to the user's information and retrieve a list of robot toys from a product database. If the user then asks the generative AI, "Please tell me how to play with this robot toy," the generative AI will provide an appropriate answer. When the user selects a robot toy that interests them, the device will display an image from a child's perspective, allowing the user to have a simulated experience. If the user likes it, they can actually purchase it.

[0650] Example 1

[0651] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal".

[0652] In conventional systems, it is difficult for users to easily find toys and games that match their preferences and interests. In addition, detailed product descriptions and instructions for playing based on a lot of information are lacking. Furthermore, the functionality for actually experiencing toys and games is limited, making it difficult to increase purchasing motivation. This results in low user satisfaction and makes it an issue to increase purchasing motivation.

[0653] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0654] In this invention, the server includes a means for acquiring a list of toys or games from a product database based on the user's preferences, interests, age, or sex, a means for providing the user with detailed product information or playing instructions using a generative AI model, a means for converting the state of playing the toy or game to the user's perspective, a means for transmitting information input by the user to the server, a means for displaying the list acquired by the server from the product database on a terminal, and a means for generating a prompt sentence based on a product selected by the user and acquiring the detailed information. This allows the user to easily find toys or games that match their preferences or interests, and to consider purchasing them after obtaining detailed information. In addition, the simulated experience increases purchasing motivation and user satisfaction.

[0655] A "server" is a computer device that receives information input by a user, retrieves an appropriate product list from a product database, and transmits the list to a terminal.

[0656] "User preferences, interests, age, and gender" are individual attribute information input by the user, and are important criteria for product selection.

[0657] The "product database" is a database that stores detailed information about various toys and games.

[0658] The "list of toys or games" is a list of toys and games obtained from a product database, and is a list that has been filtered based on the user's attribute information.

[0659] A "generative AI model" is an artificial intelligence model that generates and provides detailed product information and play instructions to users in natural language.

[0660] A "prompt sentence" is an instruction sentence that causes a generative AI model to generate detailed information.

[0661] "Means of providing detailed information or instructions for playing" refers to a function that uses a generative AI model to present users with detailed information about a product or instructions for playing the product.

[0662] "Playing with a toy or game" is visual data showing a situation in which the selected toy or game is actually being used.

[0663] The "means for converting to a user's viewpoint" is a function for converting the state of playing a toy or game into an image from the user's viewpoint.

[0664] The "means for transmitting information input by the user to the server" is a function for transmitting information input by the user using the terminal to the server via the network.

[0665] A "terminal" is a computing device through which a user enters information and displays information received from a server.

[0666] A "simulated experience video" is a video that makes the user feel as if they are actually experiencing a toy or game that they have selected.

[0667] "Purchase options" is a function that provides options for actually purchasing a product in which the user is interested.

[0668] This invention is a system that allows users to easily find toys and games that match their tastes and interests, and consider purchasing them after obtaining detailed information. Specifically, a server, a terminal, a generative AI model, a product database, and a virtual experience video conversion function work together.

[0669] The server receives information about preferences, interests, age, and gender that the user inputs using the terminal. For example, if the user inputs information such as "7-year-old boy, likes dinosaurs," the information is sent to the server via the Internet. Based on this information, the server retrieves a list of suitable toys and games from a product database. The product database contains information about a wide variety of toys and games, and returns a list that matches the criteria to the server.

[0670] The device displays the product list sent from the server to the user. The user browses the list on the device and selects a product that interests them. At that time, the device utilizes a generative AI model to provide the user with detailed information about the selected product and how to play with it in natural language. For example, this generative AI model applies a prompt sentence such as "Please explain the details of the dinosaur exploration set and how to play with it," and the AI ​​generates the relevant information.

[0671] In addition, the device is equipped with a virtual experience video conversion function that generates a video that makes it appear as if the user is actually playing the selected toy or game. This video is displayed from the user's point of view, so the user can feel as if they are playing the toy or game. For example, a realistic video of playing with a dinosaur exploration set is provided.

[0672] Finally, the device will offer the user the option to purchase the item they are interested in. The user can then complete the purchase process from the displayed options and get their hands on the selected toy or game.

[0673] The system allows users to easily find toys and games that match their tastes and interests, and then get detailed information about them to make a purchase. Using generative AI models and virtual reality videos, the user experience is improved and purchase motivation is increased. An example of a prompt sentence is "Please explain the details of the dinosaur exploration set and how to play it."

[0674] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0675] Step 1:

[0676] The user uses the device to input information about his / her preferences, interests, age, and gender. Specifically, the user enters information such as "7-year-old boy, loves dinosaurs" into a form on the screen.

[0677] Input: User attribute information (preferences, interests, age, gender)

[0678] Output: User information data entered

[0679] Step 2:

[0680] The terminal sends the information entered by the user to the server. When the user presses the send button, the entered information is sent to the server via the Internet.

[0681] Input: User information data entered

[0682] Output: User information sent to the server

[0683] Step 3:

[0684] The server retrieves a list of suitable toys and games from the product database based on the received user information. The server searches the database for the condition "age = 7 years old AND preference = dinosaurs" and retrieves the list.

[0685] Input: User information

[0686] Output: A filtered list of toys and games.

[0687] Step 4:

[0688] The terminal displays a list of toys and games sent from the server to the user. For example, the list may show a "Dinosaur Exploration Set."

[0689] Input: Product list sent from the server

[0690] Output: Product list displayed on the terminal screen

[0691] Step 5:

[0692] The user selects from the displayed list the toy or game that he or she is interested in. For example, the user selects "Dinosaur Exploration Set."

[0693] Input: Displayed product list

[0694] Output: Selected product data

[0695] Step 6:

[0696] The server sends a prompt to the generative AI model to generate detailed information about the product selected by the user and instructions on how to play with it. Specifically, the server sends the prompt to the generative AI model, "Please explain the details of the dinosaur exploration set and how to play with it."

[0697] Input: Selected product data

[0698] Output: The prompt sent to the generative AI model

[0699] Step 7:

[0700] The generative AI model generates detailed product information and instructions for playing based on the prompt text sent and returns it to the server.

[0701] Input: Prompt statement

[0702] Output: Generated details and how to play

[0703] Step 8:

[0704] The server sends the detailed information received from the generative AI model to the terminal, which then displays the information to the user, who can view the details in a natural conversational format.

[0705] Input: Detailed information received from the generative AI model

[0706] Output: Detailed information displayed on the terminal screen

[0707] Step 9:

[0708] The device uses its virtual experience conversion function to generate a virtual experience video of the selected toy or game, for example, from the perspective of a child playing with a "dinosaur exploration set."

[0709] Input: Selected product data

[0710] Output: Generated simulated experience video

[0711] Step 10:

[0712] The device displays the generated simulated experience video to the user, who feels as if he or she is actually playing the toy or game.

[0713] Input: Generated simulated experience video

[0714] Output: Simulation video displayed on the device screen

[0715] Step 11:

[0716] The terminal then provides options for purchasing the selected toy or game, and the user can proceed with the purchase from the displayed options.

[0717] Input: Selected product data

[0718] Output: A screen with purchase options

[0719] This enables the system as a whole to consistently provide products based on the user's preferences and interests, obtain detailed information, and provide purchasing support through simulated experiences.

[0720] (Application example 1)

[0721] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0722] On current online shopping sites, it is difficult for users to find toys and games that interest them, and they have to browse multiple pages to find detailed information and instructions on how to play. Another issue is that it is difficult to provide a simulated experience to increase the desire to purchase a product, and an experience from the user's perspective is particularly important for products aimed at children.

[0723] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0724] In this invention, the server includes a means for acquiring a list of toys or games from a product database based on a user's preferences, interests, age, or sex, a means for providing the user with detailed product information or play instructions using a data generation model, a means for converting a state of playing a toy or game into a view from the user's perspective, a means for displaying a list of recommended toys and games appropriate for the user, a means for generating detailed information about toys or games in natural language using generative AI, and a means for displaying the generated detailed information and recommendation list on the user's terminal. This allows the user to easily find products that interest them and check detailed information in a centralized manner, and also increases their desire to purchase through a simulated experience.

[0725] "User preferences" refers to a user's personal tastes and preferences, and is a concept that includes preferences for specific categories and themes.

[0726] "Interest" refers to a subject or area in which a user is interested, and is a concept that includes a state in which a user is interested in a particular category or topic.

[0727] "Age" refers to the number of years from the user's date of birth to the present, and is a factor that reflects an individual's stage of growth and development.

[0728] "Gender" refers to a user's biological sex, and is a factor that influences an individual's purchasing behavior and preferences.

[0729] "Product database" refers to a database that stores information about toys and games, and is a system for retrieving specific product information as needed.

[0730] "Data generation model" refers to a generative AI model used to provide users with detailed information about a product or instructions for playing the product, and is a means of using technologies such as natural language processing.

[0731] A "recommended list" is a list of toys and games selected based on a user's preferences, interests, age, and gender, and provides individually optimized product suggestions.

[0732] "Generative AI" refers to an artificial intelligence model that generates appropriate information or answers based on given prompts, and is a technology that is particularly used for generating information in natural language.

[0733] "Natural language" refers to language that humans use on a daily basis, and is a term used in a context that includes machine natural language processing.

[0734] "Terminal" refers to a device used by a user to receive and display information, including smartphones and tablets.

[0735] "Simulated experience video" refers to video that makes the user feel as if they are actually playing a toy or game, and is a means of visually providing an experience from the user's perspective.

[0736] The present invention is a system that recommends optimal toys and games based on a user's preferences, interests, age, and gender, and provides detailed information and simulated experiences. An embodiment of the system according to the present invention will be specifically described below.

[0737] The server receives information about the user's preferences, interests, age, and gender, and retrieves a list of toys and games from a product database. It then uses a data generation model to provide the user with detailed product information and how to play with the product. To do this, it uses a generative AI model (e.g. ChatGPT) to generate appropriate information based on a given prompt. The detailed information and recommendation list are then displayed on the user's device (e.g., smartphone, tablet, etc.).

[0738] The server also has a virtual experience video conversion function that converts the state of playing a toy or game into the user's perspective. This video is used to help users understand the product more specifically. For example, a prompt sentence such as "Please tell me more about the telescope" can be input to the generative AI to generate a detailed explanation about the telescope.

[0739] Users can select toys and games that interest them from a list of suggested products based on their preferences and interests, and check out the detailed information and simulated experience video. For example, if a 7-year-old boy is interested in science, science experiment kits and telescopes will be recommended, and users can view detailed information and videos about them. At the same time, users are given the option to purchase the selected product, and can then complete the purchase procedure.

[0740] The hardware required is a terminal (smartphone, tablet, etc.) that can accept user operations, and the software required is a generative AI model (for example, a natural language processing platform for running ChatGPT), a database system, and a system with a simulated experience video conversion function.

[0741] The system allows users to easily find toys and games that match their interests and preferences, and provides detailed information and simulated experiences to make a purchase with confidence.

[0742] The flow of the specific process in the application example 1 will be described with reference to FIG.

[0743] Step 1:

[0744] A user uses a terminal to send input information such as preferences, interests, age, and gender to a server.

[0745] Input: User preferences, interests, age, gender

[0746] Output: Receives the corresponding input data on the server side

[0747] Specific operation: When the user answers questions on the application screen and presses the submit button, this information is sent to the server.

[0748] Step 2:

[0749] The server searches for and acquires a list of relevant toys and games from a product database based on the received user information.

[0750] Input: User preferences, interests, age, and gender information

[0751] Output: Product list based on user information

[0752] Specific operation: The server executes a database query to retrieve product information that matches the criteria.

[0753] Step 3:

[0754] The server uses a generative AI model (e.g. ChatGPT) to generate detailed information and instructions for each toy and game included in the product list.

[0755] Input: product list, generative AI model, prompt

[0756] Output: Detailed information about each product and how to play

[0757] Specific operation: The server inputs each product name as a prompt sentence into the generative AI and generates a detailed product description.

[0758] Step 4:

[0759] The server transmits the generated detailed information and recommendation list to the user's terminal.

[0760] Input:Details, Recommendation list

[0761] Output: Product list and details displayed on the user's device

[0762] Specific operation: The generated information is sent to the user's device in JSON or XML format, and displayed on the app screen.

[0763] Step 5:

[0764] When a user browses detailed information and selects a product that interests them, the server creates a simulated experience video of that product and converts it into a view from the user's perspective.

[0765] Input: User selection, virtual experience video conversion function

[0766] Output: Virtual experience video from the user's point of view

[0767] Specific operation: Generate a play scene based on the selected product and convert it into a simulated experience video.

[0768] Step 6:

[0769] The server transmits the converted simulated experience video to the user's terminal, and the user experiences it.

[0770] Input: Simulation video

[0771] Output: Virtual experience video displayed on the user's device

[0772] Specific operation: The generated video data is sent to the user's device and played using the video player on the app.

[0773] Step 7:

[0774] When a user wishes to purchase a product, the user completes the purchase procedure via the terminal, and the server accepts the order.

[0775] Input: User's purchase intention, purchase procedure information

[0776] Output: Purchase information received and confirmed

[0777] Specific operation: The user presses the purchase button, enters the necessary information in the purchase form, and submits it. The server confirms the received purchase information and processes the order.

[0778] By following these steps, users can easily find toys and games that match their interests and preferences, and then purchase with confidence thanks to detailed information and simulated experiences.

[0779] In addition, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0780] An embodiment for implementing the present invention includes the following elements.

[0781] 1. Emotion engine: An engine for recognizing the user's emotions. It analyzes information such as voice, facial expressions, and text to grasp the user's emotional state.

[0782] 2. Server: Has the function of combining the user's preferences, interests, age, and gender information with the emotional information obtained from the emotion engine, and retrieving a list of toys and games from a product database.

[0783] 3. Generative AI: Equipped with the ability to use natural language processing AI models to provide users with detailed information about toys and games and how to play them.

[0784] 4. Terminal: Shows the user a list of toys and games, and has the ability to show a video to experience the selected product, as well as provide the option to actually purchase it.

[0785] 5. Product Database: A database that stores information about toys and games, providing information to help users choose the right product based on their preferences, interests, age, gender, and emotions.

[0786] 6. Virtual experience video conversion function: Equipped with a function that converts the image of playing a toy or game into an image from a child's perspective and displays it on the device.

[0787] In the above-described configuration, in a system that combines an emotion engine, the server, generative AI, terminal, product database, and simulated experience video conversion function work in coordination, making it possible to customize suggestions and experiences for toys and games according to the user's emotions.

[0788] The process flow of each embodiment will be described below.

[0789] Step 1: The server receives input information from the user (preferences, interests, age, gender) and recognizes the user's emotion information using an emotion engine.

[0790] Step 2: The server retrieves a list of toys and games from the product database based on the received information. Taking into account the emotion information, the server selects toys and games that match the user's emotion.

[0791] Step 3: The server uses generative AI to provide the user with detailed information about the toy or game and how to play it. It also uses emotional information to choose the right language and approach to suit the user’s emotions.

[0792] Step 4: The device displays a list of toys and games to the user, adjusting the order and presentation of the list based on the emotional information.

[0793] Step 5: The user selects the products from the list that he or she is interested in. Taking into account the emotion information, we help the user select products that match his or her emotion.

[0794] Step 6: The device displays a video for experiencing the selected product. Taking into account the emotional information, the presentation and expression of the video are adjusted.

[0795] Step 7: The user experiences the product in a simulated way. Taking into account the emotional information, the product responds and gives feedback that matches the user's emotions.

[0796] Step 8: If the user finds a product they like, the device offers them the option to actually purchase it. Taking into account emotional information, the approach is to increase their desire to purchase.

[0797] Step 9: The user proceeds with the purchase. Taking into account the emotional information, the procedure is carried out to support the user's willingness to purchase.

[0798] As described above, the server receives input information and emotional information from the user, retrieves a list from a product database, and uses generative AI to provide information. The user looks at the list on their device, selects a product, and simulates the experience of the product through an experiential video. If the user finds a product they like, they can actually purchase it. By taking emotional information into account, customization and approaches can be made to match the user's emotions.

[0799] Example 2

[0800] Next, a description will be given of Example 2. In the following description, the data processing device 12 is referred to as a "server" and the smart device 14 is referred to as a "terminal."

[0801] Current virtual playground applications simply recommend products and games based on preferences and profile information, without considering the user's emotional state. As a result, they are unable to make suggestions that reflect the user's temporary emotions and moods, which makes it difficult to improve satisfaction or increase purchasing motivation. In addition, since detailed information about specific products is not provided, it is difficult for users to deepen their understanding of the products, which can reduce their motivation to purchase. Furthermore, it is difficult for users to imagine themselves actually playing the products, making it difficult for them to make a purchasing decision through the experience.

[0802] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0803] In this invention, the server includes: a means for grasping the emotional state of the user using emotion recognition technology; a means for selecting a toy or game suitable for the user from a product database based on the user's interests, age, and sex and the user's emotional state; a means for generating detailed information or a way of playing the selected toy or game based on a prompt sentence including the user's interests, age, and sex and instructing to generate detailed information or a way of playing the selected toy or game, and a generative AI; a means for presenting the generated detailed information or a way of playing to the user; a means for inputting a prepared video showing the state of playing the toy or game into a video from the user's line of sight while playing the toy or game, and a means for presenting the converted video to the user; a means for presenting to the user a screen including a purchase button for purchasing the toy or game and a purchase form for inputting purchase information necessary for the purchase, and a means for performing an order process based on the purchase information input in the purchase form when the purchase button is pressed. This enables appropriate product suggestions that reflect the user's temporary emotions and moods.

[0804] "User preferences" are objects or categories that a user finds particularly interesting or appealing.

[0805] A "database" is an information accumulation system that systematically organizes various information and data and allows them to be efficiently managed, searched, and retrieved.

[0806] "Generative AI" is an algorithm that applies natural language processing technology to generate new text information based on input data and prompts.

[0807] "Emotion recognition technology" is a technology that analyzes information such as voice, facial expressions, and text to accurately determine a user's emotional state.

[0808] "Profile Information" is a collective term for data relating to a user's individual attributes, such as age, gender, interests, preferences, etc.

[0809] "Product suggestions" are the act of recommending appropriate toys and games based on a user's profile information and emotional information.

[0810] A "simulated experience video" is a simulation video that makes it appear as if the user is actually using the toy or game.

[0811] "Purchase Option" means the choice or process through which a User may actually purchase a suggested Toy or Game.

[0812] "Natural language processing" refers to the techniques and processes that enable computers to understand and generate human language.

[0813] The present invention relates to a system for suggesting optimal toys and games based on a user's emotional state and profile information. This system is implemented by the following specific configuration and operation.

[0814] Hardware and Software Configuration

[0815] Server: A high-performance computer server is used to manage user profile information and emotion information, execute database queries, and run data generation models. The software uses MySQL (registered trademark) as a database management system (DBMS) and Python as a scripting language.

[0816] Terminal: A device through which a user accesses the system, such as a tablet or smartphone. These terminals have built-in cameras and microphones and receive audio and video input from the user. The software uses Google (registered trademark) Cloud Speech-to-Text as the voice recognition technology and Microsoft (registered trademark) Azure (registered trademark) Face API as the face recognition technology.

[0817] Emotion engine: An AI engine for recognizing the user's emotions, analyzing multiple pieces of information such as voice, facial expressions, and text to grasp the user's emotional state. This process uses an API that operates in real time.

[0818] Generative AI: Using a data generation model, it provides users with detailed product information and how to play in natural language. ChatGPT by OpenAI (registered trademark) can be used as this model. An example of a prompt sentence is "Tell me about a dinosaur toy that a 5-year-old boy would like."

[0819] Product Database: A database that stores information about toys and games, and the server executes queries to retrieve data.

[0820] Example of operation

[0821] When a user registers with the system, he or she enters profile information such as preferences, interests, age, gender, etc. from their terminal. This information is sent to the server and stored in a database. After registration is complete, the camera and microphone capture the user's voice and video when the user starts interacting with the system.

[0822] The emotion engine analyzes these data and determines the emotion state of “user having fun.” Meanwhile, the server combines the user’s profile information with the emotion information to perform a query to select appropriate toys and games from the product database.

[0823] For example, if a 5-year-old boy is interested in dinosaur toys, the server will retrieve products from the database that match the criteria, such as "5-year-old boy, dinosaurs, having fun." Next, the generative AI will generate detailed information about the selected product and display it as text on the device. Specifically, the information created will be, "Enjoy the realistic movement and sounds of this Tyrannosaurus toy!"

[0824] Furthermore, for products that the user has shown an interest in, the virtual experience video conversion function will provide a simulation video of what it would be like to play with the toy or game, allowing the user to visually confirm the feel of the product.

[0825] Finally, the terminal presents the user with purchasing options and connects to an online shopping platform, where the user can simply press the purchase button to purchase the product.

[0826] In this way, the system takes into account the user's emotional state and provides consistent support from suggesting the most suitable toys and games to purchasing them.

[0827] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0828] Step 1:

[0829] Collection of User Information

[0830] Terminal: When a user first accesses the system, they go through a registration process in which they fill in a form with information such as preferences, interests, age, and gender.

[0831] Input: Profile information entered by the user (preferences, interests, age, gender)

[0832] Output: The entered profile information is sent to the server and stored in the database.

[0833] Specific operation: A user enters information into an HTML form displayed on the terminal screen and clicks the submit button.

[0834] Step 2:

[0835] emotion recognition

[0836] Terminal: Uses a camera and microphone to capture the user's voice and video in real time to initiate a dialogue with the system.

[0837] Input: Video and audio data of the user's face

[0838] Output: The captured video and audio data is sent to the emotion engine.

[0839] Specific actions: The user speaks to the device and directs their facial expressions to the camera.

[0840] Emotion Engine: Analyzes captured audio and video to recognize the user's emotional state.

[0841] Input: Video and audio data

[0842] Output: Analyzed emotion information (e.g. happy, excited)

[0843] How it works: Emotion recognition algorithms use voice and facial recognition technology to determine emotional state.

[0844] Step 3:

[0845] Integration of user and emotion information

[0846] Server: Integrates the emotion information sent from the emotion engine with the user's profile information collected in advance.

[0847] Input: User profile information and emotion information

[0848] Output: Consolidated user information

[0849] Specific operation: The server executes a Python script to retrieve profile information and emotion information from the database and integrate them.

[0850] Step 4:

[0851] Get product data

[0852] Server: Based on the integrated user information, selects appropriate toys and games from a product database.

[0853] Input: Integrated user information (age, gender, interests, emotional state)

[0854] Output: A list of toys and games suitable for the user.

[0855] Specific operation: The server executes the query "SELECT FROM product database WHERE age = 5 AND interest = 'dinosaur' AND mood = 'happy'".

[0856] Step 5:

[0857] Generate detailed information

[0858] Generative AI: Uses NLP models to generate detailed information about selected toys and games. Example prompt: "Tell me about a dinosaur toy that a 5-year-old boy would like."

[0859] Input: A list of suitable toys and games, a prompt

[0860] Output: Generated detailed information (e.g. "This T-Rex toy moves and sounds realistic!")

[0861] Specific operation: Generative AI generates detailed product information based on the prompt text

[0862] Step 6:

[0863] Display of proposed products and provision of experience videos

[0864] Terminal: The generated detailed information is displayed to the user, and the play scene is recreated using a virtual experience video conversion function.

[0865] Input: Detailed information and a list of toys and games, and video data prepared in advance for toys and games showing the state of playing with them

[0866] Output: Details displayed to the user, and footage from the user's point of view

[0867] Specific operation: The server inputs a prepared video showing a user playing a toy or game, and converts it into a video from the user's point of view while playing the toy or game. The terminal displays the generated text of detailed information and the video from the user's point of view on the screen. In addition, when converting into an image from the user's point of view, the server may convert video data prepared in advance for a toy or game, which represents the state of playing the toy or game, into an image from the user's point of view, based on a prompt sentence that includes the user's emotions and instructs the conversion to an image from the user's point of view while playing the toy or game, and the generative AI. This makes it possible to convert into an image that takes the user's emotions into consideration. Specifically, by instructing in the prompt sentence to convert into an image with a color tone or tempo that matches the user's emotions, it is possible to convert into an image from the user's point of view that takes the user's emotions into consideration.

[0868] Step 7:

[0869] Present purchasing options

[0870] On your device: Displays options to directly purchase suggested toys and games. Connects to online shopping platforms.

[0871] Input: Proposed product, purchase option data

[0872] Output: A screen including a purchase button for purchasing a toy or game and a purchase form for inputting the necessary purchase information.

[0873] Specific operation: The terminal displays a screen including a purchase button and a purchase form. When the user enters purchase information and clicks the purchase button, the user is redirected to the online store and the order is processed.

[0874] (Application example 2)

[0875] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server", and the headset type terminal 314 will be referred to as a "terminal".

[0876] In current virtual stores and online shopping, it is difficult to suggest appropriate products and toys to users and to realistically convey the experience of actually using them. In particular, there is a lack of means to provide product suggestions and virtual experiences that reflect the user's emotions and interests, and this poses a challenge in increasing purchasing motivation and satisfaction. To solve this problem, a system is needed that can accurately grasp the user's emotional state and customize product suggestions and experiences based on that information.

[0877] The identification process by the identification processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means. In this invention, the server includes a means for acquiring a list of products or play equipment from an information database based on the user's preferences, interests, age, or sex, a means for providing the user with detailed information or usage methods of the products or play equipment using a natural language processing engine, a means for analyzing the user's emotional state using an emotion recognition engine, and a means for converting the experience of using the play equipment into an image expressed from the user's point of view. This makes it possible to propose products that reflect the user's emotions and interests, and the user can experience the actual feel of use through a virtual experience, which is expected to increase purchasing motivation and satisfaction.

[0878] An "emotion recognition engine" is a device or software that analyzes information such as voice, facial expressions, and text to grasp a user's emotional state.

[0879] A "natural language processing engine" is a technology that uses generative AI models and other techniques to provide users with detailed information about toys and games and how to play them in natural language.

[0880] An "information database" is a database that stores information about various products and play equipment.

[0881] "Based on the user's preferences, interests, age, or gender" means that the most suitable products or play equipment are selected taking into account the user's individual attribute information.

[0882] "A means for converting the experience of using play equipment into an image expressed from the user's perspective" is a technology that virtually recreates the experience of using play equipment or a product and displays the image from the user's perspective.

[0883] The "server" is a central management system that links emotion recognition engines, natural language processing engines, information databases, etc. to provide users with optimal product suggestions and experiences.

[0884] An embodiment of the present invention comprises a system including the following main elements:

[0885] Emotion Recognition Engine

[0886] The server uses an emotion recognition engine to analyze the user's emotional state. The emotion recognition engine analyzes data such as the user's voice, facial expressions, or text messages captured using a camera or microphone. Here, face recognition and facial expression recognition are performed using OpenCV and deep learning models. This makes it possible to grasp the user's real-time emotions.

[0887] Information Database

[0888] The server retrieves a list of appropriate products and play equipment from the information database based on the emotion data obtained from the emotion recognition engine and profile information such as the user's age, gender, interests, etc. The information database stores detailed information about each product and play equipment, recommended age, gender, features, etc.

[0889] Natural Language Processing Engine

[0890] The server uses a natural language processing engine (generative AI model) to generate detailed information and instructions for playing with each product based on a list of products and play equipment retrieved from an information database. This makes it possible to provide information to users in a natural dialogue format. ChatGPT from OpenAI is used as the generative AI model.

[0891] Terminal Display

[0892] The terminal displays a list of the acquired products and playground equipment, as well as detailed information, to the user. It also displays images that express the experience of using the playground equipment from the user's point of view. A smartphone or a head-mounted display (HMD) is used for this. Using a virtual experience image conversion function, it provides an experience that makes the user feel as if they are actually using the product.

[0893] Buying Options

[0894] The terminal provides the user with the option to purchase products and play equipment, allowing the user to purchase the product they like on the spot.

[0895] Examples and prompts

[0896] For example, when a five-year-old boy is using a smartphone or HMD, the camera reads his facial expression and the emotion recognition engine detects that he is smiling. The server is a five-year-old boy, and selects remote control cars and building block sets as product candidates based on the smiling state. The generative AI model generates detailed information about the selected products and how to play with them, explaining them to the user in natural language. The user can enjoy the experience of operating a remote control car in a virtual space with the HMD, and if they like it, they can purchase it on the spot.

[0897] Example prompt:

[0898] What are the features of the following toy? Remote control car

[0899] This makes it possible to suggest products that reflect the user's emotions and interests, and since the user can check out products through a virtual experience before purchasing, it is expected to increase purchasing motivation and satisfaction.

[0900] The flow of the specific process in the application example 2 will be described with reference to FIG.

[0901] Step 1:

[0902] The device uses a camera and microphone to capture the user's voice and facial expressions. Input includes the user's facial image and voice data. This data is transmitted to the server in real time.

[0903] Step 2:

[0904] The server starts an emotion recognition engine to analyze the transmitted voice and facial expression data. The analysis process uses OpenCV and deep learning models. As input, the user's voice and facial image data are used, and as output, the user's emotional state (e.g., happy, sad, surprised, etc.) is obtained.

[0905] Step 3:

[0906] The server queries the information database based on the emotional state obtained from the emotion recognition engine and the user's age, gender, and interests to obtain a list of products and toys. The user's emotional state, age, gender, and interests are used as inputs, and a list of suitable products and toys is obtained as output.

[0907] Step 4:

[0908] The server uses a natural language processing engine (generative AI model) to generate detailed information and usage instructions for each product from the list of acquired products and play equipment. This process uses generative AI such as ChatGPT from OpenAI. Product names and category information are used as input, and the generated detailed information and usage instructions for the product are obtained as output.

[0909] Step 5:

[0910] The server transmits the generated product information to the terminal, which displays it to the user. As input, text data of the product information is used, and as output, the product information is displayed on the user's device.

[0911] Step 6:

[0912] The terminal uses the virtual experience video conversion function to provide a virtual experience of the product selected by the user. In this process, 3D models and computer graphics technology are used. The product information selected by the user is used as input, and the virtual experience video from the user's point of view is obtained as output.

[0913] Step 7:

[0914] After the user enjoys the experience, the terminal offers purchasing options, using the user's selected product information as input and obtaining a display of purchasing options as output.

[0915] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input for the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0916] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by making a neural network perform deep learning. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image is input. The data generation model 58 performs inference on the input inference data according to the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0917] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[0918] [Fourth embodiment]

[0919] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[0920] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0921] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[0922] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. In addition, the microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[0923] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[0924] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[0925] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54. The transmission and reception of various types of information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[0926] The control target 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, legs, etc. The posture and behavior of the robot 414 are controlled by controlling the motors of the arms, hands, legs, etc. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[0927] Fig. 8 shows an example of main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0928] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0929] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0930] In the robot 414, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50, and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0931] Next, a description will be given of the specific processing by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal".

[0932] An embodiment for implementing the present invention includes the following elements.

[0933] 1. Server: Has the function of receiving information on the user's preferences, interests, age, and gender, and retrieving a list of toys and games from a product database.

[0934] 2. Generative AI: Equipped with the ability to use natural language processing AI models to provide users with detailed information about toys and games and how to play them.

[0935] 3. Terminal: Shows the user a list of toys and games, displays a video to let them experience the selected product, and also provides the option to actually purchase the product.

[0936] 4. Product Database: A database that stores information about toys and games, providing users with information to select appropriate products based on their preferences, interests, age, and gender.

[0937] 5. Virtual experience video conversion function: Equipped with a function that converts the image of playing a toy or game into an image from a child's perspective and displays it on the device.

[0938] In this way, the server, generative AI, terminal, product database, and virtual experience video conversion function work together to enable users to select appropriate toys and games, experience products through a virtual experience, and actually purchase them.

[0939] The process flow of each embodiment will be described below.

[0940] Step 1: The server receives input information from the user (preferences, interests, age, gender).

[0941] Step 2: The server uses the received information to retrieve a list of toys and games from the product database.

[0942] Step 3: The server uses generative AI to provide the user with detailed information about the toy or game and how to play it.

[0943] Step 4: The device displays a list of toys and games to the user.

[0944] Step 5: User selects the product that interests them from the list.

[0945] Step 6: The device will display a video to experience the selected product.

[0946] Step 7: The user experiences the product virtually.

[0947] Step 8: If the user finds an item they like, the device gives them the option to actually purchase it.

[0948] Step 9: The user proceeds with the purchase.

[0949] For example, if a user inputs "I want a robot toy," the server will refer to the user's information and retrieve a list of robot toys from a product database. If the user then asks the generative AI, "Please tell me how to play with this robot toy," the generative AI will provide an appropriate answer. When the user selects a robot toy that interests them, the device will display an image from a child's perspective, allowing the user to have a simulated experience. If the user likes it, they can actually purchase it.

[0950] Example 1

[0951] Next, a description will be given of Example 1. In the following description, the data processing device 12 is referred to as a "server" and the robot 414 is referred to as a "terminal."

[0952] In conventional systems, it is difficult for users to easily find toys and games that match their preferences and interests. In addition, detailed product descriptions and instructions for playing based on a lot of information are lacking. Furthermore, the functionality for actually experiencing toys and games is limited, making it difficult to increase purchasing motivation. This results in low user satisfaction and makes it an issue to increase purchasing motivation.

[0953] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0954] In this invention, the server includes a means for acquiring a list of toys or games from a product database based on the user's preferences, interests, age, or sex, a means for providing the user with detailed product information or playing instructions using a generative AI model, a means for converting the state of playing the toy or game to the user's perspective, a means for transmitting information input by the user to the server, a means for displaying the list acquired by the server from the product database on a terminal, and a means for generating a prompt sentence based on a product selected by the user and acquiring the detailed information. This allows the user to easily find toys or games that match their preferences or interests, and to consider purchasing them after obtaining detailed information. In addition, the simulated experience increases purchasing motivation and user satisfaction.

[0955] A "server" is a computer device that receives information input by a user, retrieves an appropriate product list from a product database, and transmits the list to a terminal.

[0956] "User preferences, interests, age, and gender" are individual attribute information input by the user, and are important criteria for product selection.

[0957] The "product database" is a database that stores detailed information about various toys and games.

[0958] The "list of toys or games" is a list of toys and games obtained from a product database, and is a list that has been filtered based on the user's attribute information.

[0959] A "generative AI model" is an artificial intelligence model that generates and provides detailed product information and play instructions to users in natural language.

[0960] A "prompt sentence" is an instruction sentence that causes a generative AI model to generate detailed information.

[0961] "Means of providing detailed information or instructions for playing" refers to a function that uses a generative AI model to present users with detailed information about a product or instructions for playing the product.

[0962] "Playing with a toy or game" is visual data showing a situation in which the selected toy or game is actually being used.

[0963] The "means for converting to a user's viewpoint" is a function for converting the state of playing a toy or game into an image from the user's viewpoint.

[0964] The "means for transmitting information input by the user to the server" is a function for transmitting information input by the user using the terminal to the server via the network.

[0965] A "terminal" is a computing device through which a user enters information and displays information received from a server.

[0966] A "simulated experience video" is a video that makes the user feel as if they are actually experiencing a toy or game that they have selected.

[0967] "Purchase options" is a function that provides options for actually purchasing a product in which the user is interested.

[0968] This invention is a system that allows users to easily find toys and games that match their tastes and interests, and consider purchasing them after obtaining detailed information. Specifically, a server, a terminal, a generative AI model, a product database, and a virtual experience video conversion function work together.

[0969] The server receives information about preferences, interests, age, and gender that the user inputs using the terminal. For example, if the user inputs information such as "7-year-old boy, likes dinosaurs," the information is sent to the server via the Internet. Based on this information, the server retrieves a list of suitable toys and games from a product database. The product database contains information about a wide variety of toys and games, and returns a list that matches the criteria to the server.

[0970] The device displays the product list sent from the server to the user. The user browses the list on the device and selects a product that interests them. At that time, the device utilizes a generative AI model to provide the user with detailed information about the selected product and how to play with it in natural language. For example, this generative AI model applies a prompt sentence such as "Please explain the details of the dinosaur exploration set and how to play with it," and the AI ​​generates the relevant information.

[0971] In addition, the device is equipped with a virtual experience video conversion function that generates a video that makes it appear as if the user is actually playing the selected toy or game. This video is displayed from the user's point of view, so the user can feel as if they are playing the toy or game. For example, a realistic video of playing with a dinosaur exploration set is provided.

[0972] Finally, the device will offer the user the option to purchase the item they are interested in. The user can then complete the purchase process from the displayed options and get their hands on the selected toy or game.

[0973] The system allows users to easily find toys and games that match their tastes and interests, and then get detailed information about them to make a purchase. Using generative AI models and virtual reality videos, the user experience is improved and purchase motivation is increased. An example of a prompt sentence is "Please explain the details of the dinosaur exploration set and how to play it."

[0974] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0975] Step 1:

[0976] The user uses the device to input information about his / her preferences, interests, age, and gender. Specifically, the user enters information such as "7-year-old boy, loves dinosaurs" into a form on the screen.

[0977] Input: User attribute information (preferences, interests, age, gender)

[0978] Output: User information data entered

[0979] Step 2:

[0980] The terminal sends the information entered by the user to the server. When the user presses the send button, the entered information is sent to the server via the Internet.

[0981] Input: User information data entered

[0982] Output: User information sent to the server

[0983] Step 3:

[0984] The server retrieves a list of suitable toys and games from the product database based on the received user information. The server searches the database for the condition "age = 7 years old AND preference = dinosaurs" and retrieves the list.

[0985] Input: User information

[0986] Output: A filtered list of toys and games.

[0987] Step 4:

[0988] The terminal displays a list of toys and games sent from the server to the user. For example, the list may show a "Dinosaur Exploration Set."

[0989] Input: Product list sent from the server

[0990] Output: Product list displayed on the terminal screen

[0991] Step 5:

[0992] The user selects from the displayed list the toy or game that he or she is interested in. For example, the user selects "Dinosaur Exploration Set."

[0993] Input: Displayed product list

[0994] Output: Selected product data

[0995] Step 6:

[0996] The server sends a prompt to the generative AI model to generate detailed information about the product selected by the user and instructions on how to play with it. Specifically, the server sends the prompt to the generative AI model, "Please explain the details of the dinosaur exploration set and how to play with it."

[0997] Input: Selected product data

[0998] Output: The prompt sent to the generative AI model

[0999] Step 7:

[1000] The generative AI model generates detailed product information and instructions for playing based on the prompt text sent and returns it to the server.

[1001] Input: Prompt statement

[1002] Output: Generated details and how to play

[1003] Step 8:

[1004] The server sends the detailed information received from the generative AI model to the terminal, which then displays the information to the user, who can view the details in a natural conversational format.

[1005] Input: Detailed information received from the generative AI model

[1006] Output: Detailed information displayed on the terminal screen

[1007] Step 9:

[1008] The device uses its virtual experience conversion function to generate a virtual experience video of the selected toy or game, for example, from the perspective of a child playing with a "dinosaur exploration set."

[1009] Input: Selected product data

[1010] Output: Generated simulated experience video

[1011] Step 10:

[1012] The device displays the generated simulated experience video to the user, who feels as if he or she is actually playing the toy or game.

[1013] Input: Generated simulated experience video

[1014] Output: Simulation video displayed on the device screen

[1015] Step 11:

[1016] The terminal then provides options for purchasing the selected toy or game, and the user can proceed with the purchase from the displayed options.

[1017] Input: Selected product data

[1018] Output: A screen with purchase options

[1019] This enables the system as a whole to consistently provide products based on the user's preferences and interests, obtain detailed information, and provide purchasing support through simulated experiences.

[1020] (Application example 1)

[1021] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1022] On current online shopping sites, it is difficult for users to find toys and games that interest them, and they have to browse multiple pages to find detailed information and instructions on how to play. Another issue is that it is difficult to provide a simulated experience to increase the desire to purchase a product, and an experience from the user's perspective is particularly important for products aimed at children.

[1023] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1024] In this invention, the server includes a means for acquiring a list of toys or games from a product database based on a user's preferences, interests, age, or sex, a means for providing the user with detailed product information or play instructions using a data generation model, a means for converting a state of playing a toy or game into a view from the user's perspective, a means for displaying a list of recommended toys and games appropriate for the user, a means for generating detailed information about toys or games in natural language using generative AI, and a means for displaying the generated detailed information and recommendation list on the user's terminal. This allows the user to easily find products that interest them and check detailed information in a centralized manner, and also increases their desire to purchase through a simulated experience.

[1025] "User preferences" refers to a user's personal tastes and preferences, and is a concept that includes preferences for specific categories and themes.

[1026] "Interest" refers to a subject or area in which a user is interested, and is a concept that includes a state in which a user is interested in a particular category or topic.

[1027] "Age" refers to the number of years from the user's date of birth to the present, and is a factor that reflects an individual's stage of growth and development.

[1028] "Gender" refers to a user's biological sex, and is a factor that influences an individual's purchasing behavior and preferences.

[1029] "Product database" refers to a database that stores information about toys and games, and is a system for retrieving specific product information as needed.

[1030] "Data generation model" refers to a generative AI model used to provide users with detailed information about a product or instructions for playing the product, and is a means of using technologies such as natural language processing.

[1031] A "recommended list" is a list of toys and games selected based on a user's preferences, interests, age, and gender, and provides individually optimized product suggestions.

[1032] "Generative AI" refers to an artificial intelligence model that generates appropriate information or answers based on given prompts, and is a technology that is particularly used for generating information in natural language.

[1033] "Natural language" refers to language that humans use on a daily basis, and is a term used in a context that includes machine natural language processing.

[1034] "Terminal" refers to a device used by a user to receive and display information, including smartphones and tablets.

[1035] "Simulated experience video" refers to video that makes the user feel as if they are actually playing a toy or game, and is a means of visually providing an experience from the user's perspective.

[1036] The present invention is a system that recommends optimal toys and games based on a user's preferences, interests, age, and gender, and provides detailed information and simulated experiences. An embodiment of the system according to the present invention will be specifically described below.

[1037] The server receives information about the user's preferences, interests, age, and gender, and retrieves a list of toys and games from a product database. It then uses a data generation model to provide the user with detailed product information and how to play with the product. To do this, it uses a generative AI model (e.g. ChatGPT) to generate appropriate information based on a given prompt. The detailed information and recommendation list are then displayed on the user's device (e.g., smartphone, tablet, etc.).

[1038] The server also has a virtual experience video conversion function that converts the state of playing a toy or game into the user's perspective. This video is used to help users understand the product more specifically. For example, a prompt sentence such as "Please tell me more about the telescope" can be input to the generative AI to generate a detailed explanation about the telescope.

[1039] Users can select toys and games that interest them from a list of suggested products based on their preferences and interests, and check out the detailed information and simulated experience video. For example, if a 7-year-old boy is interested in science, science experiment kits and telescopes will be recommended, and users can view detailed information and videos about them. At the same time, users are given the option to purchase the selected product, and can then complete the purchase procedure.

[1040] The hardware required is a terminal (smartphone, tablet, etc.) that can accept user operations, and the software required is a generative AI model (for example, a natural language processing platform for running ChatGPT), a database system, and a system with a simulated experience video conversion function.

[1041] The system allows users to easily find toys and games that match their interests and preferences, and provides detailed information and simulated experiences to make a purchase with confidence.

[1042] The flow of the specific process in the application example 1 will be described with reference to FIG.

[1043] Step 1:

[1044] A user uses a terminal to send input information such as preferences, interests, age, and gender to a server.

[1045] Input: User preferences, interests, age, gender

[1046] Output: Receives the corresponding input data on the server side

[1047] Specific operation: When the user answers questions on the application screen and presses the submit button, this information is sent to the server.

[1048] Step 2:

[1049] The server searches for and acquires a list of relevant toys and games from a product database based on the received user information.

[1050] Input: User preferences, interests, age, and gender information

[1051] Output: Product list based on user information

[1052] Specific operation: The server executes a database query to retrieve product information that matches the criteria.

[1053] Step 3:

[1054] The server uses a generative AI model (e.g. ChatGPT) to generate detailed information and instructions for each toy and game included in the product list.

[1055] Input: product list, generative AI model, prompt

[1056] Output: Detailed information about each product and how to play

[1057] Specific operation: The server inputs each product name as a prompt sentence into the generative AI and generates a detailed product description.

[1058] Step 4:

[1059] The server transmits the generated detailed information and recommendation list to the user's terminal.

[1060] Input:Details, Recommendation list

[1061] Output: Product list and details displayed on the user's device

[1062] Specific operation: The generated information is sent to the user's device in JSON or XML format, and displayed on the app screen.

[1063] Step 5:

[1064] When a user browses detailed information and selects a product that interests them, the server creates a simulated experience video of that product and converts it into a view from the user's perspective.

[1065] Input: User selection, virtual experience video conversion function

[1066] Output: Virtual experience video from the user's point of view

[1067] Specific operation: Generate a play scene based on the selected product and convert it into a simulated experience video.

[1068] Step 6:

[1069] The server transmits the converted simulated experience video to the user's terminal, and the user experiences it.

[1070] Input: Simulation video

[1071] Output: Virtual experience video displayed on the user's device

[1072] Specific operation: The generated video data is sent to the user's device and played using the video player on the app.

[1073] Step 7:

[1074] When a user wishes to purchase a product, the user completes the purchase procedure via the terminal, and the server accepts the order.

[1075] Input: User's purchase intention, purchase procedure information

[1076] Output: Purchase information received and confirmed

[1077] Specific operation: The user presses the purchase button, enters the necessary information in the purchase form, and submits it. The server confirms the received purchase information and processes the order.

[1078] By following these steps, users can easily find toys and games that match their interests and preferences, and then purchase with confidence thanks to detailed information and simulated experiences.

[1079] In addition, an emotion engine that estimates the emotion of the user may be further combined. That is, the identification processing unit 290 may estimate the emotion of the user using the emotion identification model 59, and perform identification processing using the emotion of the user.

[1080] An embodiment for implementing the present invention includes the following elements.

[1081] 1. Emotion engine: An engine for recognizing the user's emotions. It analyzes information such as voice, facial expressions, and text to grasp the user's emotional state.

[1082] 2. Server: Has the function of combining the user's preferences, interests, age, and gender information with the emotional information obtained from the emotion engine, and retrieving a list of toys and games from a product database.

[1083] 3. Generative AI: Equipped with the ability to use natural language processing AI models to provide users with detailed information about toys and games and how to play them.

[1084] 4. Terminal: Shows the user a list of toys and games, and has the ability to show a video to experience the selected product, as well as provide the option to actually purchase it.

[1085] 5. Product Database: A database that stores information about toys and games, providing information to help users choose the right product based on their preferences, interests, age, gender, and emotions.

[1086] 6. Virtual experience video conversion function: Equipped with a function that converts the image of playing a toy or game into an image from a child's perspective and displays it on the device.

[1087] In the above-described configuration, in a system that combines an emotion engine, the server, generative AI, terminal, product database, and simulated experience video conversion function work in coordination, making it possible to customize suggestions and experiences for toys and games according to the user's emotions.

[1088] The process flow of each embodiment will be described below.

[1089] Step 1: The server receives input information from the user (preferences, interests, age, gender) and recognizes the user's emotion information using an emotion engine.

[1090] Step 2: The server retrieves a list of toys and games from the product database based on the received information. Taking into account the emotion information, the server selects toys and games that match the user's emotion.

[1091] Step 3: The server uses generative AI to provide the user with detailed information about the toy or game and how to play it. It also uses emotional information to choose the right language and approach to suit the user’s emotions.

[1092] Step 4: The device displays a list of toys and games to the user, adjusting the order and presentation of the list based on the emotional information.

[1093] Step 5: The user selects the products from the list that he or she is interested in. Taking into account the emotion information, we help the user select products that match his or her emotion.

[1094] Step 6: The device displays a video for experiencing the selected product. Taking into account the emotional information, the presentation and expression of the video are adjusted.

[1095] Step 7: The user experiences the product in a simulated way. Taking into account the emotional information, the product responds and gives feedback that matches the user's emotions.

[1096] Step 8: If the user finds a product they like, the device offers them the option to actually purchase it. Taking into account emotional information, the approach is to increase their desire to purchase.

[1097] Step 9: The user proceeds with the purchase. Taking into account the emotional information, the procedure is carried out to support the user's willingness to purchase.

[1098] As described above, the server receives input information and emotional information from the user, retrieves a list from a product database, and uses generative AI to provide information. The user looks at the list on their device, selects a product, and simulates the experience of the product through an experiential video. If the user finds a product they like, they can actually purchase it. By taking emotional information into account, customization and approaches can be made to match the user's emotions.

[1099] Example 2

[1100] Next, a description will be given of Example 2. In the following description, the data processing device 12 is referred to as a "server" and the smart device 14 is referred to as a "terminal."

[1101] Current virtual playground applications simply recommend products and games based on preferences and profile information, without considering the user's emotional state. As a result, they are unable to make suggestions that reflect the user's temporary emotions and moods, which makes it difficult to improve satisfaction or increase purchasing motivation. In addition, since detailed information about specific products is not provided, it is difficult for users to deepen their understanding of the products, which can reduce their motivation to purchase. Furthermore, it is difficult for users to imagine themselves actually playing the products, making it difficult for them to make a purchasing decision through the experience.

[1102] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1103] In this invention, the server includes: a means for grasping the emotional state of the user using emotion recognition technology; a means for selecting a toy or game suitable for the user from a product database based on the user's interests, age, and sex and the user's emotional state; a means for generating detailed information or a way of playing the selected toy or game based on a prompt sentence including the user's interests, age, and sex and instructing to generate detailed information or a way of playing the selected toy or game, and a generative AI; a means for presenting the generated detailed information or a way of playing to the user; a means for inputting a prepared video showing the state of playing the toy or game into a video from the user's line of sight while playing the toy or game, and a means for presenting the converted video to the user; a means for presenting to the user a screen including a purchase button for purchasing the toy or game and a purchase form for inputting purchase information necessary for the purchase, and a means for performing an order process based on the purchase information input in the purchase form when the purchase button is pressed. This enables appropriate product suggestions that reflect the user's temporary emotions and moods.

[1104] "User preferences" are objects or categories that a user finds particularly interesting or appealing.

[1105] A "database" is an information accumulation system that systematically organizes various information and data and allows them to be efficiently managed, searched, and retrieved.

[1106] "Generative AI" is an algorithm that applies natural language processing technology to generate new text information based on input data and prompts.

[1107] "Emotion recognition technology" is a technology that analyzes information such as voice, facial expressions, and text to accurately determine a user's emotional state.

[1108] "Profile Information" is a collective term for data relating to a user's individual attributes, such as age, gender, interests, preferences, etc.

[1109] "Product suggestions" are the act of recommending appropriate toys and games based on a user's profile information and emotional information.

[1110] A "simulated experience video" is a simulation video that makes it appear as if the user is actually using the toy or game.

[1111] "Purchase Option" means the choice or process through which a User may actually purchase a suggested Toy or Game.

[1112] "Natural language processing" refers to the techniques and processes that enable computers to understand and generate human language.

[1113] The present invention relates to a system for suggesting optimal toys and games based on a user's emotional state and profile information. This system is implemented by the following specific configuration and operation.

[1114] Hardware and Software Configuration

[1115] Server: A high-performance computer server is used to manage user profile information and emotion information, execute database queries, and run data generation models. The software uses MySQL (registered trademark) as a database management system (DBMS) and Python as a scripting language.

[1116] Terminal: A device through which a user accesses the system, such as a tablet or smartphone. These terminals have built-in cameras and microphones and receive audio and video input from the user. The software uses Google (registered trademark) Cloud Speech-to-Text as the voice recognition technology and Microsoft (registered trademark) Azure (registered trademark) Face API as the face recognition technology.

[1117] Emotion engine: An AI engine for recognizing the user's emotions, analyzing multiple pieces of information such as voice, facial expressions, and text to grasp the user's emotional state. This process uses an API that operates in real time.

[1118] Generative AI: Using a data generation model, it provides users with detailed product information and how to play in natural language. ChatGPT by OpenAI (registered trademark) can be used as this model. An example of a prompt sentence is "Tell me about a dinosaur toy that a 5-year-old boy would like."

[1119] Product Database: A database that stores information about toys and games, and the server executes queries to retrieve data.

[1120] Example of operation

[1121] When a user registers with the system, he or she enters profile information such as preferences, interests, age, gender, etc. from their terminal. This information is sent to the server and stored in a database. After registration is complete, the camera and microphone capture the user's voice and video when the user starts interacting with the system.

[1122] The emotion engine analyzes these data and determines the emotion state of “user having fun.” Meanwhile, the server combines the user’s profile information with the emotion information to perform a query to select appropriate toys and games from the product database.

[1123] For example, if a 5-year-old boy is interested in dinosaur toys, the server will retrieve products from the database that match the criteria, such as "5-year-old boy, dinosaurs, having fun." Next, the generative AI will generate detailed information about the selected product and display it as text on the device. Specifically, the information created will be, "Enjoy the realistic movement and sounds of this Tyrannosaurus toy!"

[1124] Furthermore, for products that the user has shown an interest in, the virtual experience video conversion function will provide a simulation video of what it would be like to play with the toy or game, allowing the user to visually confirm the product's usage experience.

[1125] Finally, the terminal presents the user with purchasing options and connects to an online shopping platform, where the user can simply press the purchase button to purchase the product.

[1126] In this way, the system takes into account the user's emotional state and provides consistent support from suggesting the most suitable toys and games to purchasing them.

[1127] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1128] Step 1:

[1129] Collection of User Information

[1130] Terminal: When a user first accesses the system, they go through a registration process in which they fill in a form with information such as preferences, interests, age, and gender.

[1131] Input: Profile information entered by the user (preferences, interests, age, gender)

[1132] Output: The entered profile information is sent to the server and stored in the database.

[1133] Specific operation: A user enters information into an HTML form displayed on the terminal screen and clicks the submit button.

[1134] Step 2:

[1135] emotion recognition

[1136] Terminal: Uses a camera and microphone to capture the user's voice and video in real time to initiate a dialogue with the system.

[1137] Input: Video and audio data of the user's face

[1138] Output: The captured video and audio data is sent to the emotion engine.

[1139] Specific actions: The user speaks to the device and directs their facial expressions to the camera.

[1140] Emotion Engine: Analyzes captured audio and video to recognize the user's emotional state.

[1141] Input: Video and audio data

[1142] Output: Analyzed emotion information (e.g. happy, excited)

[1143] How it works: Emotion recognition algorithms use voice and facial recognition technology to determine emotional state.

[1144] Step 3:

[1145] Integration of user and emotion information

[1146] Server: Integrates the emotion information sent from the emotion engine with the user's profile information collected in advance.

[1147] Input: User profile information and emotion information

[1148] Output: Consolidated user information

[1149] Specific operation: The server executes a Python script to retrieve profile information and emotion information from the database and integrate them.

[1150] Step 4:

[1151] Get product data

[1152] Server: Based on the integrated user information, selects appropriate toys and games from a product database.

[1153] Input: Integrated user information (age, gender, interests, emotional state)

[1154] Output: A list of toys and games suitable for the user.

[1155] Specific operation: The server executes the query "SELECT FROM product database WHERE age = 5 AND interest = 'dinosaur' AND mood = 'happy'".

[1156] Step 5:

[1157] Generate detailed information

[1158] Generative AI: Uses NLP models to generate detailed information about selected toys and games. Example prompt: "Tell me about a dinosaur toy that a 5-year-old boy would like."

[1159] Input: A list of suitable toys and games, a prompt

[1160] Output: Generated detailed information (e.g. "This T-Rex toy moves and sounds realistic!")

[1161] Specific operation: Generative AI generates detailed product information based on the prompt text

[1162] Step 6:

[1163] Display of proposed products and provision of experience videos

[1164] Terminal: The generated detailed information is displayed to the user, and the play scene is recreated using a virtual experience video conversion function.

[1165] Input: Detailed information and a list of toys and games, and video data prepared in advance for toys and games showing the state of playing with them

[1166] Output: Details displayed to the user, and footage from the user's point of view

[1167] Specific operation: The server inputs a prepared video showing a user playing a toy or game, and converts it into a video from the user's point of view while playing the toy or game. The terminal displays the generated text of detailed information and the video from the user's point of view on the screen. In addition, when converting into an image from the user's point of view, the server may convert video data prepared in advance for a toy or game, which represents the state of playing the toy or game, into an image from the user's point of view, based on a prompt sentence that includes the user's emotions and instructs the conversion to an image from the user's point of view while playing the toy or game, and the generative AI. This makes it possible to convert into an image that takes the user's emotions into consideration. Specifically, by instructing in the prompt sentence to convert into an image with a color tone or tempo that matches the user's emotions, it is possible to convert into an image from the user's point of view that takes the user's emotions into consideration.

[1168] Step 7:

[1169] Present purchasing options

[1170] On your device: Displays options to directly purchase suggested toys and games. Connects to online shopping platforms.

[1171] Input: Proposed product, purchase option data

[1172] Output: A screen including a purchase button for purchasing a toy or game and a purchase form for inputting the necessary purchase information.

[1173] Specific operation: The terminal displays a screen including a purchase button and a purchase form. When the user enters purchase information and clicks the purchase button, the user is redirected to the online store and the order is processed.

[1174] (Application example 2)

[1175] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal".

[1176] In current virtual stores and online shopping, it is difficult to suggest appropriate products and toys to users and to realistically convey the experience of actually using them. In particular, there is a lack of means to provide product suggestions and virtual experiences that reflect the user's emotions and interests, and this poses a challenge in increasing purchasing motivation and satisfaction. To solve this problem, a system is needed that can accurately grasp the user's emotional state and customize product suggestions and experiences based on that information.

[1177] The identification process by the identification processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means. In this invention, the server includes a means for acquiring a list of products or play equipment from an information database based on the user's preferences, interests, age, or sex, a means for providing the user with detailed information or usage methods of the products or play equipment using a natural language processing engine, a means for analyzing the user's emotional state using an emotion recognition engine, and a means for converting the experience of using the play equipment into an image expressed from the user's point of view. This makes it possible to propose products that reflect the user's emotions and interests, and the user can experience the actual feel of use through a virtual experience, which is expected to increase purchasing motivation and satisfaction.

[1178] An "emotion recognition engine" is a device or software that analyzes information such as voice, facial expressions, and text to grasp a user's emotional state.

[1179] A "natural language processing engine" is a technology that uses generative AI models and other techniques to provide users with detailed information about toys and games and how to play them in natural language.

[1180] An "information database" is a database that stores information about various products and play equipment.

[1181] "Based on the user's preferences, interests, age, or gender" means that the most suitable products or play equipment are selected taking into account the user's individual attribute information.

[1182] "A means for converting the experience of using play equipment into an image expressed from the user's perspective" is a technology that virtually recreates the experience of using play equipment or a product and displays the image from the user's perspective.

[1183] The "server" is a central management system that links emotion recognition engines, natural language processing engines, information databases, etc. to provide users with optimal product suggestions and experiences.

[1184] An embodiment of the present invention comprises a system including the following main elements:

[1185] Emotion Recognition Engine

[1186] The server uses an emotion recognition engine to analyze the user's emotional state. The emotion recognition engine analyzes data such as the user's voice, facial expressions, or text messages captured using a camera or microphone. Here, face recognition and facial expression recognition are performed using OpenCV and deep learning models. This makes it possible to grasp the user's real-time emotions.

[1187] Information Database

[1188] The server retrieves a list of appropriate products and play equipment from the information database based on the emotion data obtained from the emotion recognition engine and profile information such as the user's age, gender, interests, etc. The information database stores detailed information about each product and play equipment, recommended age, gender, features, etc.

[1189] Natural Language Processing Engine

[1190] The server uses a natural language processing engine (generative AI model) to generate detailed information and instructions for playing with each product based on a list of products and play equipment retrieved from an information database. This makes it possible to provide information to users in a natural dialogue format. ChatGPT from OpenAI is used as the generative AI model.

[1191] Terminal Display

[1192] The terminal displays a list of the acquired products and playground equipment, as well as detailed information, to the user. It also displays images that express the experience of using the playground equipment from the user's point of view. A smartphone or a head-mounted display (HMD) is used for this. Using a virtual experience image conversion function, it provides an experience that makes the user feel as if they are actually using the product.

[1193] Buying Options

[1194] The terminal provides the user with the option to purchase products and play equipment, allowing the user to purchase the product they like on the spot.

[1195] Examples and prompts

[1196] For example, when a five-year-old boy is using a smartphone or HMD, the camera reads his facial expression and the emotion recognition engine detects that he is smiling. The server is a five-year-old boy, and selects remote control cars and building block sets as product candidates based on the smiling state. The generative AI model generates detailed information about the selected products and how to play with them, explaining them to the user in natural language. The user can enjoy the experience of operating a remote control car in a virtual space with the HMD, and if they like it, they can purchase it on the spot.

[1197] Example prompt:

[1198] What are the features of the following toy? Remote control car

[1199] This makes it possible to suggest products that reflect the user's emotions and interests, and since the user can check out products through a virtual experience before purchasing, it is expected to increase purchasing motivation and satisfaction.

[1200] The flow of the specific process in the application example 2 will be described with reference to FIG.

[1201] Step 1:

[1202] The device uses a camera and microphone to capture the user's voice and facial expressions. Input includes the user's facial image and voice data. This data is transmitted to the server in real time.

[1203] Step 2:

[1204] The server starts an emotion recognition engine to analyze the transmitted voice and facial expression data. The analysis process uses OpenCV and deep learning models. As input, the user's voice and facial image data are used, and as output, the user's emotional state (e.g., happy, sad, surprised, etc.) is obtained.

[1205] Step 3:

[1206] The server queries the information database based on the emotional state obtained from the emotion recognition engine and the user's age, gender, and interests to obtain a list of products and toys. The user's emotional state, age, gender, and interests are used as inputs, and a list of suitable products and toys is obtained as output.

[1207] Step 4:

[1208] The server uses a natural language processing engine (generative AI model) to generate detailed information and usage instructions for each product from the list of acquired products and play equipment. This process uses generative AI such as ChatGPT from OpenAI. Product names and category information are used as input, and the generated detailed information and usage instructions for the product are obtained as output.

[1209] Step 5:

[1210] The server transmits the generated product information to the terminal, which displays it to the user. As input, text data of the product information is used, and as output, the product information is displayed on the user's device.

[1211] Step 6:

[1212] The terminal uses the virtual experience video conversion function to provide a virtual experience of the product selected by the user. In this process, 3D models and computer graphics technology are used. The product information selected by the user is used as input, and the virtual experience video from the user's point of view is obtained as output.

[1213] Step 7:

[1214] After the user enjoys the experience, the terminal offers purchasing options, using the user's selected product information as input and obtaining a display of purchasing options as output.

[1215] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1216] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by making a neural network perform deep learning. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image is input. The data generation model 58 performs inference on the input inference data according to the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1217] In the above embodiment, an example was given in which the specific process was performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the robot 414.

[1218] The emotion identification model 59 as an emotion engine may determine the emotion of the user according to a specific mapping. Specifically, the emotion identification model 59 may determine the emotion of the user according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the emotion of the robot, and the identification processing unit 290 may perform identification processing using the emotion of the robot.

[1219] FIG. 9 is a diagram showing an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive emotions are arranged. The more outside the concentric circles, the more emotions that represent states and actions that arise from a state of mind are arranged. Emotions are a concept that includes emotions and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions that occur in the brain are arranged. On the right side of the concentric circles, emotions that are generally induced by situational judgment are arranged. On the upper and lower sides of the concentric circles, emotions that are generally generated from reactions that occur in the brain and are induced by situational judgment are arranged. In addition, on the upper side of the concentric circles, emotions of "pleasure" are arranged, and on the lower side, emotions of "discomfort" are arranged. In this way, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1220] These emotions are distributed in the 3 o'clock direction of emotion map 400 and usually fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1221] The inside of emotion map 400 represents what is going on inside one's mind, and the outside of emotion map 400 represents behavior, so the further out you go on emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1222] Here, human emotions are based on various balances such as posture and blood sugar level, and when these balances are far from the ideal, it indicates an unpleasant state, and when they are close to the ideal, it indicates a pleasant state. For robots, cars, motorcycles, etc., emotions can also be created based on various balances such as posture and battery level, and when these balances are far from the ideal, it indicates an unpleasant state, and when they are close to the ideal, it indicates a pleasant state. The emotion map may be generated based on, for example, the emotion map of Dr. Mitsuyoshi (Research on speech emotion recognition and emotion brain physiological signal analysis system, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). In the left half of the emotion map, emotions belonging to an area called "reaction" where sensation is dominant are lined up. In addition, in the right half of the emotion map, emotions belonging to an area called "situation" where situation recognition is dominant are lined up.

[1223] The emotion map defines two emotions that promote learning. The first is the negative emotion around the middle of "repentance" or "remorse" on the situation side. In other words, this is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the positive emotion around "desire" on the response side. In other words, this is when the robot has positive feelings such as "I want more" or "I want to know more."

[1224] The emotion identification model 59 inputs the user input to a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the emotion of the user. This neural network is pre-trained based on multiple learning data that are combinations of the user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in Fig. 10. Fig. 10 shows an example in which multiple emotions, "relief," "calm," and "encouraging," have similar emotion values.

[1225] Although the system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, the system according to the present disclosure is not necessarily implemented in a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program that runs on a personal computer, or an application that runs on a smartphone or the like. The method according to the present disclosure may be provided to a user in the form of SaaS (Software as a Service).

[1226] In the above embodiment, an example is given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to input data.

[1227] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable computer-readable non-transitory storage medium such as a Universal Serial Bus (USB) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1228] In addition, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 upon request from the data processing device 12.

[1229] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1230] As the hardware resource for executing the specific process, various processors as shown below can be used. An example of the processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing the specific process by executing software, i.e., a program. Another example of the processor is a dedicated electric circuit, which is a processor having a circuit configuration designed exclusively for executing the specific process, such as a Field-Programmable Gate Array (FPGA), a Programmable Logic Device (PLD), or an Application Specific Integrated Circuit (ASIC). Each processor has a built-in or connected memory, and each processor executes the specific process by using the memory.

[1231] The hardware resource that executes the specific process may be one of these various processors, or may be a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific process may be a single processor.

[1232] As an example of a configuration using one processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a configuration using a processor that realizes the functions of the entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1233] Furthermore, more specifically, the hardware structure of these various processors can be an electric circuit that combines circuit elements such as semiconductor elements. The specific processes described above are merely examples. It goes without saying that unnecessary steps may be deleted, new steps may be added, or the order of processes may be changed without departing from the spirit of the invention.

[1234] The above description and illustrations are detailed descriptions of the parts related to the technology of the present disclosure, and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, function, action, and effect is an example of the configuration, function, action, and effect of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above description and illustrations, within the scope of the gist of the technology of the present disclosure. In addition, in order to avoid confusion and to facilitate understanding of the parts related to the technology of the present disclosure, the above description and illustrations omit explanations of technical common sense that do not require explanation in order to enable the implementation of the technology of the present disclosure.

[1235] All publications, patent applications, and standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or standard was specifically and individually indicated to be incorporated by reference.

[1236] The following is further disclosed regarding the above embodiment.

[1237] (Claim 1) 1. A system for providing a virtual playground application, comprising: means for retrieving a list of toys or games from a product database based on a user's preferences, interests, age, or gender; A means for providing detailed information or a method of playing with a product to the user using a data generation model; A means for converting an image showing a state of playing a toy or game into an image from the user's line of sight; A system including:

[1238] (Claim 2) 2. The system according to claim 1, further comprising a means for displaying the converted image so as to allow the user to simulate an experience of the product selected by the user. (Claim 3) The system of claim 1 or claim 2, further comprising means for providing the user with an option to purchase.

[1239] (Claim 4) 10. The system of claim 1, further comprising means for customizing toy or game suggestions and experiences based on the user's emotions using an emotion engine that recognizes the user's emotions.

[1240] (Claim 5) 2. The system according to claim 1, further comprising means for providing a list of toys and games according to the user's emotions by combining with an emotion engine for recognizing the user's emotions.

[1241] (Claim 6) The system according to claim 1, further comprising a means for customizing the experiential video of the toy or game according to the user's emotions by combining with an emotion engine for recognizing the user's emotions.

[1242] "Example 1" (Claim 1) means for retrieving a list of toys or games from a product database based on a user's preferences, interests, age, or gender; A means for providing detailed information or a method of playing with a product to the user using a generative AI model; A means for converting a state of playing a toy or game into a line of sight of the user; means for transmitting information input by a user to a server; A means for displaying the list acquired by the server from the product database on a terminal; A means for generating a prompt sentence based on a product selected by a user and acquiring detailed information thereof; A system including:

[1243] (Claim 2) 2. The system according to claim 1, further comprising a means for displaying the converted image so as to allow the user to simulate an experience of the product selected by the user.

[1244] (Claim 3) The system of claim 1 or claim 2, further comprising means for providing the user with an option to purchase.

[1245] "Application example 1" (Claim 1) means for retrieving a list of toys or games from a product database based on a user's preferences, interests, age, or gender; A means for providing detailed information or a method of playing with a product to the user using a data generation model; A means for converting a state of playing a toy or game into a line of sight of the user; means for displaying a list of suitable toys and games recommendations to the user; A means for generating detailed information about a toy or game in natural language using generative AI; a means for displaying the generated detailed information and recommendation list on a user's terminal; A system including:

[1246] (Claim 2) 2. The system according to claim 1, further comprising a means for displaying the converted image so as to allow the user to simulate an experience of the product selected by the user.

[1247] (Claim 3) The system of claim 1 further comprising means for providing the user with a purchasing option.

[1248] "Example 2 of combining emotion engines" (Claim 1) means for retrieving a list of toys or games from a product database based on a user's preferences, interests, age, or gender; A means for grasping a user's emotional state using emotion recognition technology; A means for providing detailed information or a method of playing with a product to the user using a data generation model; A means for converting an image showing a state of playing a toy or game into an image from the user's line of sight; A means for optimizing product suggestions by integrating the profile information and emotion information of the user; A system including:

[1249] (Claim 2) 2. The system according to claim 1, further comprising a means for displaying the converted image so as to allow the user to simulate an experience of the product selected by the user.

[1250] (Claim 3) The system of claim 1 further comprising means for providing the user with a purchasing option.

[1251] "Application example 2 when combining emotion engines" (Claim 1) means for retrieving a list of products or playthings from an information database based on a user's preferences, interests, age, or gender; A means for providing detailed information or usage instructions for a product or play equipment to the user using a natural language processing engine; means for analyzing an emotional state of a user using an emotion recognition engine; A means for converting the experience of using the playground equipment into an image expressed from the user's point of view; A system including:

[1252] (Claim 2) 2. The system according to claim 1, further comprising a means for displaying the converted image so as to allow the user to have a virtual experience of the play equipment selected by the user.

[1253] (Claim 3) The system of claim 1 or claim 2, further comprising means for providing the user with an option to purchase. [Explanation of symbols]

[1254] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. A means for grasping a user's emotional state using emotion recognition technology; means for selecting a toy or game suited to the user from a product database based on the user's interests, age, and sex and the user's emotional state; A means for generating detailed information or a method of playing the selected toy or game based on a prompt sentence including the user's interests, age, and gender, and instructing the user to generate detailed information or a method of playing the selected toy or game, and a generative AI; A means for presenting the generated detailed information or playing method to the user; A means for receiving a prepared image showing a state of playing the toy or game and converting the image into an image from the viewpoint of a user playing the toy or game; means for presenting the converted image to the user; A system including:

2. means for presenting to the user a screen including a purchase button for purchasing the toy or game and a purchase form for inputting purchase information required for the purchase; 2. The system according to claim 1, further comprising: a means for carrying out an order process based on the purchase information inputted in said purchase form when said purchase button is pressed.

3. The system of claim 1, wherein the converting means converts the image into one from the user's point of view based on a prompt sentence that includes the user's emotions and instructs the conversion to be made into an image from the user's point of view while playing a toy or game, and on a generative AI.

Images