System

Abstract

A system is provided.SOLUTION: A system comprising: means for providing a list of materials and Traditional Art Object required to create a piece of equipment selected by a user; means for recording and storing a user's progress in real time; means for generating a feedback based on the user's progress; and means for transmitting and displaying the generated feedback to the user's device.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Despite their cultural value, traditional Japanese crafts currently face serious challenges, such as a lack of successors and an aging artisan workforce. Regional and language barriers, in particular, make it difficult for young people and overseas cultural enthusiasts to learn these techniques. Furthermore, a lack of personalization tailored to individual skill levels and languages ​​spoken makes it difficult for users to learn efficiently at home. Furthermore, a lack of real-time support and community interaction is also a problem. [Means for solving the problem]

[0005] To solve the above-mentioned problems, the present invention provides a system that provides a list of materials and tools necessary to create a traditional craft selected by a user. The system also includes a means for recording a user's progress in real time and providing feedback generated using artificial intelligence. The system also provides personalized learning support based on the user's skill level and language, and includes a community interaction function for users. The system also includes a means for saving progress and generating feedback in real time. This allows users to efficiently learn traditional craft techniques from home and pass on these techniques widely across regional and language barriers.

[0006] "User" refers to a person who uses the system, and can be an individual or an organization.

[0007] "Traditional crafts" refer to handicrafts that have particular cultural or historical value and are made by hand.

[0008] "Materials" refers to the raw materials and parts needed to create a traditional craft.

[0009] "Tools" refers to the tools and equipment used to create traditional crafts.

[0010] A "list" refers to a list of materials and tools.

[0011] A "means" refers to a method or device used to achieve a particular purpose.

[0012] "Progress" refers to information indicating the current stage of the user's traditional craft creation process.

[0013] "Recording" refers to the act of saving progress or the saved content.

[0014] "Storage" refers to the act of retaining data or information.

[0015] "Artificial intelligence" is a branch of computer science that refers to algorithms and systems that mimic human intellectual activity.

[0016] "Feedback" refers to evaluation and advice on the user's progress.

[0017] "Terminal" refers to a computer or mobile device through which a user accesses the system.

[0018] "Display" refers to the act of showing information on a terminal screen.

[0019] "Personalized" refers to a state that is customized to suit the individual needs and characteristics of the user.

[0020] "Learning support" refers to the act of providing support for users to acquire new knowledge and skills.

[0021] "Community interaction" refers to activities in which users share information and provide mutual support.

[0022] "Real-time" refers to processing or reaction occurring immediately, without delay. [Brief explanation of the drawings]

[0023] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6]FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0024] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0025] First, the terms used in the following description will be explained.

[0026] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0027] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0028] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0029] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0030] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0031] [First embodiment]

[0032] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0033] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0034] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0035] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0036] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0037] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0038] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0039] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0040] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0041] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0042] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0043] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0044] This invention is a system that supports users in the process of creating traditional Japanese crafts at home in real time. This system consists of three main components: a server, a terminal, and a user. Below, we will explain how to specifically implement this system.

[0045] User Interface (UI) Embodiments

[0046] When a user accesses the platform, they are first shown a login page. After the user enters their login information, that information is sent from the device to the server. The server retrieves the user's profile information from the database, generates a dashboard with appropriate content, and sends it back to the user. The dashboard displays the user's past crafting history and a list of recommended crafts.

[0047] View the list of recommended crafts

[0048] The server generates a list of suitable traditional crafts based on the user's preferences and past history and sends it to the device. The device displays this list on the UI and prompts the user to select the crafts they are interested in. Once the user selects a craft, the information is sent back to the server.

[0049] Providing necessary materials and tools

[0050] The server retrieves from the database a list of materials and tools needed for the selected craft and returns it to the terminal, which displays this information to the user to help them prepare to begin crafting.

[0051] Step-by-step guide and user progress record

[0052] Once the user starts crafting, the device provides a UI that tracks the progress of each step. As the user completes a step, the progress information is reported on the UI and sent to the server.

[0053] Real-time feedback

[0054] The server analyzes the received progress information in real time and uses artificial intelligence to generate feedback, including recommended next steps and technical advice, which is then sent to the device and displayed to the user.

[0055] Personalized learning support

[0056] The server adjusts the program to provide personalized learning support according to each user's skill level and language. It also introduces a multilingual system to provide an easy-to-use environment for users overseas.

[0057] Community interaction features

[0058] The device provides a community forum to encourage interaction between users, where users can share their crafting progress and finished products and receive comments and advice from other users. This feature allows users to deepen their interactions with other craft enthusiasts.

[0059] Additional Support

[0060] If users have any questions or problems during production, they can receive real-time support through the system's Q&A section or chat function. The server will forward these questions to the AI ​​or support team for quick answers.

[0061] In this way, this system supports users in efficiently learning traditional craft techniques from home and passing on these techniques widely across regional and language barriers.

[0062] The processing flow will be explained below.

[0063] Step 1:

[0064] The user accesses the platform's login page and enters their login information (user ID and password).

[0065] Step 2:

[0066] The terminal sends the login information entered by the user to the server.

[0067] Step 3:

[0068] The server receives the login information and retrieves the user's profile information from the database, including past crafting history and preferences (e.g., favorite craft categories).

[0069] Step 4:

[0070] Based on the profile information acquired by the server, a list of recommended traditional crafts is generated and sent to the terminal.

[0071] Step 5:

[0072] The device displays a list of received crafts in the UI and prompts the user to select one.

[0073] Step 6:

[0074] The user selects the craft of interest from the list.

[0075] Step 7:

[0076] The terminal transmits the user's selection information to the server.

[0077] Step 8:

[0078] The server retrieves from the database a list of materials and tools required for the selected craft and transmits the list to the terminal.

[0079] Step 9:

[0080] The list of materials and tools received by the terminal is displayed on the UI, and the user is prompted to confirm the necessary preparations.

[0081] Step 10:

[0082] The user prepares for production and starts making the craft by following the instructions on the screen.

[0083] Step 11:

[0084] The device records the progress of each step and provides a UI that allows the user to report their progress.

[0085] Step 12:

[0086] Each time the user completes a step, they press a "Step Completed" button to report their progress.

[0087] Step 13:

[0088] The terminal sends progress information to the server.

[0089] Step 14:

[0090] The server stores the progress information received and uses artificial intelligence to generate feedback, including recommended next steps and technical advice.

[0091] Step 15:

[0092] The server transmits the generated feedback to the terminal.

[0093] Step 16:

[0094] The device receives feedback and displays it in the UI, informing the user of next steps and areas for improvement.

[0095] Step 17:

[0096] The server will adjust the program to provide personalized learning support according to the user's skill level and language. A multilingual system will also be introduced to provide an environment that is easy for overseas users to use.

[0097] Step 18:

[0098] The device provides a community forum to encourage interaction between users, allowing them to post their production progress and finished products.

[0099] Step 19:

[0100] If users have questions or problems during production, they can get support using the Q&A section or chat function.

[0101] Step 20:

[0102] The server forwards user questions to an artificial intelligence or support team to provide quick answers.

[0103] Step 21:

[0104] The terminal notifies the user of the received answer and provides assistance in solving the problem.

[0105] This series of processes allows users to efficiently learn traditional Japanese crafts from home, making it possible to widely pass on skills beyond regional and language barriers.

[0106] Example 1

[0107] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0108] Conventional traditional craft production support systems have had issues such as users being unable to easily obtain a list of the materials and tools they need, difficulty in recording progress in real time, a lack of immediate feedback, insufficient personalized learning support according to the user's skill level and preferences, and a lack of promotion of interaction between users. To solve these problems, a system with more advanced data processing and real-time feedback functions is needed.

[0109] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0110] In this invention, the server includes means for providing a list of materials and tools required to create a craft item selected by the user, means for recording and saving the user's progress in real time, means for generating feedback based on the user's progress using artificial intelligence, means for sending the generated feedback to the user's device and displaying it, means for acquiring login information when the user accesses the platform and generating a dashboard based on that information, means for generating and displaying a list of recommended items based on the user's preferences and past history, and means for reporting progress information each time the user completes a step and providing recommendations for the next step and technical advice using artificial intelligence, thereby enabling users to efficiently and enjoyably progress with craft creation at home.

[0111] "User" refers to a person who uses the system to create a craft.

[0112] "Crafts" refers to handcrafted items made using traditional techniques.

[0113] "Materials" refers to the materials needed to create a craft item.

[0114] "Tools" refers to the implements and equipment used in creating crafts.

[0115] "List" refers to information in the form of a list that summarizes items of materials and tools.

[0116] "Progress" refers to information indicating how much of each step of craft creation the user has completed.

[0117] "Real-time" refers to processing and display occurring immediately without delay.

[0118] "Artificial intelligence" refers to technology that uses machine learning and data analysis to automatically make judgments and predictions.

[0119] "Feedback" refers to information including evaluations and advice regarding the user's actions and status.

[0120] "Device" refers to a device used by a user, such as a computer or smartphone.

[0121] "Platform" refers to the online environment through which Users access and engage in craft creation.

[0122] "Login Information" refers to the authentication data entered by a User to access a System.

[0123] "Dashboard" refers to an interface that displays a user's profile information and activity history in a centralized manner.

[0124] "Preferences" refers to information based on a user's preferences.

[0125] "History" refers to a record of a user's past activities and behavior.

[0126] "Recommended Items" refers to a list of crafts and materials suggested based on the user's preferences and history.

[0127] A "guide" refers to information that provides instructions for a user to create a craft item.

[0128] "Advice" refers to technical advice provided to users by artificial intelligence.

[0129] "Reporting" refers to the act of notifying the server of the work that the user has done and the progress of that work.

[0130] This invention is a system that supports users in the process of creating traditional Japanese crafts at home in real time. The system mainly consists of three elements: a server, a terminal, and a user.

[0131] First, when a user accesses the platform, a login page appears on the device. The user enters their login information (username and password), which is then sent from the device to the server. The server authenticates the login information and retrieves the user's profile information from the database. As a result, the server generates a dashboard dedicated to the user, which is sent to the device and displayed. The dashboard includes the user's past crafting history and a list of recommended crafts.

[0132] The server then generates a list of suitable traditional crafts based on the user's preferences and past history, using a data analysis algorithm. The generated list is sent to the device and displayed to the user. Once the user selects a craft of interest, the information is sent back to the server. The server retrieves a list of materials and tools needed for the selected craft from the database and returns the information to the device for display.

[0133] Once the user begins crafting, the device provides a UI that records the progress of each step. Each time the user completes a step, the progress information is sent from the device to the server. The server analyzes the received progress information in real time and generates feedback using artificial intelligence (AI). The feedback includes recommendations for next steps and technical advice, which is sent to the device and displayed to the user.

[0134] The server also adjusts the program to provide personalized learning support based on each user's skill level and language. This allows for a multilingual system, making it easier for users overseas to use. For example, it can provide an interface in multiple languages, including English, French, and German.

[0135] Additionally, the device provides a community forum for users to interact with each other. Here, users can share their crafting progress and finished products and receive comments and advice from other users. This promotes communication between craft enthusiasts. Furthermore, through the Q&A section and chat function, users can receive real-time support when they have questions or problems during their crafting. The server forwards these questions to the AI ​​or support team for quick answers.

[0136] For example:

[0137] When a user logs in and tries to create "Edo Kiriko," the device retrieves past production history from the server and displays it to the user. The server then recommends "Edo Kiriko" and displays the necessary materials (glass, cutting tools, etc.). As the user progresses with the production, the device records the progress and sends it to the server. The server uses AI to generate next step guides and advice in real time and provides them to the user.

[0138] Example prompts to input to the generative AI model:

[0139] Please describe in detail the system that provides real-time support to users' progress in creating traditional Japanese crafts, including the following points:

[0140] 1. Processing flow from user login to dashboard display

[0141] 2. How to generate and display a list of recommended crafts

[0142] 3. The process of providing necessary materials and tools

[0143] 4. Step-by-step progress recording and real-time feedback

[0144] 5. Personalized learning support and multilingual support

[0145] 6. Community Interaction Features and Additional Support Details

[0146] Please also provide a detailed description of the data flow and the technologies used.

[0147] In this way, this system supports users in efficiently learning traditional craft techniques from home and passing on these techniques widely across regional and language barriers.

[0148] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0149] Step 1:

[0150] A user accesses the platform and is presented with a login page.

[0151] Specific operation: The user enters their username and password. The entered data (login information) is sent from the terminal to the server.

[0152] Input and Output:

[0153] Input: Username, Password

[0154] Data processing: Encryption of entered login information

[0155] Output: Encrypted login information

[0156] Step 2:

[0157] The server receives the login information and retrieves the user's profile information from a database.

[0158] Specific operation: The server decrypts the received login information and performs authentication. Based on the authentication results, it searches the database for the user's profile information. Based on the obtained data, it generates a dashboard and sends it to the device.

[0159] Input and Output:

[0160] Input: Encrypted login information

[0161] Data calculation: Decryption of login information, verification of user information

[0162] Output: User profile information, generated dashboard

[0163] Step 3:

[0164] The terminal displays the dashboard received from the server to the user.

[0165] Specific operation: The device renders the received HTML data and displays the user's crafting history and a list of recommended crafts in the browser.

[0166] Input and Output:

[0167] Input: Generated dashboard

[0168] Data processing: Rendering HTML data

[0169] Output: The displayed dashboard

[0170] Step 4:

[0171] The server generates a list of suitable traditional crafts based on the user's preferences and past history and sends it to the terminal.

[0172] Specific operation: The server retrieves the user's preference information and past production history from the database, runs the recommendation algorithm, generates a recommendation list in JSON format, and sends it to the device.

[0173] Input and Output:

[0174] Input: User preference information, past history

[0175] Data Computing: Running the recommendation algorithm

[0176] Output: Recommended craft list

[0177] Step 5:

[0178] The terminal displays a list of recommended crafts to the user and prompts the user to select a craft they are interested in. Once the user selects a craft, the information is sent back to the server.

[0179] Specific operation: The device renders the received recommendation list on the UI, and when the user presses the select button, it sends the ID of the selected craft to the server.

[0180] Input and Output:

[0181] Input: Recommended Craft List

[0182] Data processing: UI display of recommendation list

[0183] Output: ID of the selected craft

[0184] Step 6:

[0185] The server retrieves from the database a list of materials and tools required for the selected craft and transmits it to the terminal.

[0186] Specific operation: The server searches the database based on the acquired craft ID, generates a list of required materials and tools, and sends it to the terminal in JSON format.

[0187] Input and Output:

[0188] Input: ID of selected craft

[0189] Data calculation: database search, list generation

[0190] Output: List of materials and tools

[0191] Step 7:

[0192] The terminal displays a list of materials and tools to the user.

[0193] Specific operation: The device renders the received list on the UI and helps the user prepare to start production.

[0194] Input and Output:

[0195] Input: List of materials and tools

[0196] Data processing: List UI display

[0197] Output: List of displayed materials and tools

[0198] Step 8:

[0199] Once the user starts crafting, the device provides a UI that tracks the progress of each step. Each time the user completes a step, the device sends that progress information to the server.

[0200] What it does: The device displays a progress tracker, allows the user to click a button to complete each step, and sends progress information to the server when a step is completed.

[0201] Input and Output:

[0202] Input: User progress information

[0203] Data processing: recording and sending progress information

[0204] Output: Progress information sent to the server

[0205] Step 9:

[0206] The server analyzes the received progress information in real time and generates feedback using artificial intelligence (AI).

[0207] What it does: The server uses AI models to analyze progress data, generate next step recommendations and technical advice, and send the generated feedback to the device.

[0208] Input and Output:

[0209] Input: User progress information

[0210] Data calculation: Analysis using AI models, feedback generation

[0211] Output: Generated feedback

[0212] Step 10:

[0213] The terminal displays the generated feedback to the user.

[0214] Specific operation: The device renders the feedback data received from the server on the UI and displays it to the user.

[0215] Input and Output:

[0216] Input: Generated feedback

[0217] Data processing: UI display of feedback data

[0218] Output: Displayed feedback

[0219] Step 11:

[0220] The server provides personalized learning support based on each user's skill level and language usage, providing a multilingual environment.

[0221] What happens: The server selects appropriate content and feedback based on the user's profile data, and translates it if necessary.

[0222] Input and Output:

[0223] Input: User profile data

[0224] Data Operations: Content Selection and Translation

[0225] Output: Personalized learning assistance

[0226] Step 12:

[0227] The device provides a community forum for users to interact with each other.

[0228] Specific operation: The device has a forum function and displays an interface where users can post and comment. Posts and comments are sent to the server and displayed to other users.

[0229] Input and Output:

[0230] Input: User posts and comments

[0231] Data processing: sending and displaying posts and comments

[0232] Output: Posts and comments that are visible to other users

[0233] Step 13:

[0234] If users have any questions or issues, they can get real-time support through the Q&A section or chat function. The server will forward these questions to the AI ​​or support team for quick answers.

[0235] Specific operation: The device displays a question form or chat window and sends the user's question to the server, which then forwards the question to the appropriate support system, generates an answer, and sends it to the device.

[0236] Input and Output:

[0237] Input: User question

[0238] Data Computation: Question Transfer and Answer Generation

[0239] Output: The answer sent to the user

[0240] (Application example 1)

[0241] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0242] Conventional support systems for traditional craft creation are primarily used at home, making it difficult to provide appropriate real-time support during workshops or craft classes held in brick-and-mortar stores. Furthermore, there is a lack of means to provide detailed support information and feedback for each step of a user's craft creation process in a store, making it difficult for users to create crafts efficiently and with high quality.

[0243] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0244] In this invention, the server includes means for providing a list of materials and tools required to create a traditional craft selected by the user, means for recording and saving the user's progress in real time, means for generating feedback using artificial intelligence based on the user's progress, means for sending the generated feedback to the user's terminal and displaying it, means for guiding the user through the production procedure in real time via a smart device, and means for providing support information for each step as the user produces the traditional craft in the store, thereby enabling the user to produce the traditional craft in the store with high efficiency.

[0245] The "means for providing a list of materials and tools required to create a traditional craft selected by the user" is a system for obtaining from a server specific items of materials and tools required for the traditional craft the user wishes to create and displaying them on the user's terminal.

[0246] "Means for recording and saving the user's progress in real time" refers to a system that has the function of recording the user's progress as they complete each step of production and saving that information on a server.

[0247] "Means for generating feedback using artificial intelligence based on the user's progress" refers to a function that analyzes the user's current production progress and generates feedback such as recommendations for next steps and improvements using an artificial intelligence algorithm.

[0248] "Means for transmitting and displaying the generated feedback to the user's terminal" refers to a system that notifies the user's terminal of feedback information generated by artificial intelligence and displays it in real time.

[0249] "Means for guiding users through the production procedure in real time via a smart device" refers to a system for providing step-by-step guidance through the production procedure via a smart device worn by the user (such as smart glasses or a head-mounted display).

[0250] "Means for providing support information for each step when a user is creating something in a store" refers to a system that provides the support information needed for each step (e.g., how to prepare materials, how to use tools, etc.) in real time when a user is performing a production activity in a physical store.

[0251] This invention is a system that supports users in the process of creating traditional Japanese crafts in real time in a brick-and-mortar store. The system consists of three main components: a server, a terminal (e.g., smart glasses), and a user. Below, we will explain how to specifically implement this system.

[0252] User Interface (UI) Embodiments

[0253] When a user accesses the platform, they are first shown a login page. After the user enters their login information, that information is sent from the device (smart glasses) to the server. The server retrieves the user's profile information from the database, generates a dashboard with appropriate content, and sends it back to the user. The dashboard displays the user's past crafting history and a list of recommended crafts.

[0254] View the list of recommended crafts

[0255] The server generates a list of suitable traditional crafts based on the user's preferences and past history and sends it to the device. The device displays this list on the UI and prompts the user to select the crafts they are interested in. Once the user selects a craft, the information is sent back to the server.

[0256] Providing necessary materials and tools

[0257] The server retrieves from the database a list of materials and tools needed for the selected craft and returns it to the terminal, which displays this information to the user to help them prepare to begin crafting.

[0258] Step-by-step guide and user progress record

[0259] Once the user starts crafting, the device (smart glasses) provides a UI that records the progress of each step. Each time the user completes a step, the progress information is reported on the UI and sent to the server.

[0260] Real-time feedback

[0261] The server analyzes the received progress information in real time and uses artificial intelligence to generate feedback, including recommended next steps and technical advice, which is then sent to the device and displayed to the user.

[0262] In-store guide function

[0263] When users create crafts in the store, they are guided through each step in real time via a device (smart glasses). This guide includes audio guidance and visual instructions to help users complete the work smoothly.

[0264] Specific examples

[0265] For example, suppose a user selects "lacquerware" in a physical store. The server retrieves a list of the necessary materials and tools (lacquerware, wood, brushes, knives, etc.) and displays it on the user's smart glasses. Once the user starts making the product, the smart glasses provide real-time guidance on how to prepare the materials and the work procedures for each step. In addition, each time the user reports their progress, feedback and advice on the next steps is provided.

[0266] Prompt Sentence Examples

[0267] "What should I pay attention to in the next step?"

[0268] "What's the best way to treat sumac?"

[0269] This allows users to create crafts in their physical stores with high efficiency and high quality.

[0270] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0271] Step 1:

[0272] User Login

[0273] Input: User ID and password

[0274] Process: The user accesses the login page and enters their login information. The device sends this information to the server.

[0275] Output: The server authenticates the user and retrieves their profile information.

[0276] Step 2:

[0277] Viewing the Dashboard

[0278] Input: User profile information

[0279] Processing: The server generates a dashboard containing appropriate content based on the profile information, past production history, and preferences.

[0280] Output: The terminal displays the user's dashboard, which includes past crafting history and a list of recommended crafts.

[0281] Step 3:

[0282] Generate and display a list of recommended crafts

[0283] Input: User preferences and past history

[0284] Processing: The server generates a list of suitable traditional crafts based on preferences and past history and sends it to the device.

[0285] Output: A list of recommended traditional crafts will be displayed on the device.

[0286] Step 4:

[0287] Craft Selection

[0288] Input: Recommended Craft List

[0289] Process: The user selects the artifact of interest and sends the information from the device to the server.

[0290] Output: The server receives the selected craft information.

[0291] Step 5:

[0292] Providing a list of materials and tools needed

[0293] Input: Selected craft information

[0294] Processing: The server retrieves the list of materials and tools required for the selected craft from the database and sends it to the terminal.

[0295] Output: A list of ingredients and tools will be displayed on the terminal.

[0296] Step 6:

[0297] Getting started and step-by-step guide

[0298] Input: Materials and tools list, crafting step guide

[0299] Process: The user starts the creation, and the device (smart glasses) provides a UI that records the progress of each step.

[0300] Output: A step-by-step build guide is displayed on the terminal, and the user reports their progress.

[0301] Step 7:

[0302] Tracking progress and generating feedback

[0303] Input: User progress information

[0304] Processing: The server analyzes the received progress information in real time and generates feedback using artificial intelligence.

[0305] Output: The generated feedback information is sent to the terminal and displayed to the user.

[0306] Step 8:

[0307] Real-time guide in physical stores

[0308] Input: User's production progress information

[0309] Processing: The server provides real-time step-by-step guidance through the smart glasses based on the user's progress in the physical store.

[0310] Output: Guidance information for each production step is displayed on the user's smart glasses.

[0311] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0312] This invention is a system that provides real-time support for users to efficiently learn and create traditional Japanese crafts at home. The system incorporates an emotion engine that recognizes the user's emotions and appropriately adjusts feedback based on those emotions. The following describes how to specifically implement this system.

[0313] User Interface (UI) Embodiments

[0314] A user accesses the platform and logs in. The login information is sent from the device to the server, which then retrieves the user's profile information from the database. Based on this information, a list of recommended traditional crafts is generated and sent to the device. The device then displays this list in the UI and prompts the user to make a selection.

[0315] View and select from a list of recommended crafts

[0316] When a user selects a craft, the selection information is sent from the terminal to the server. The server retrieves a list of materials and tools required for the selected craft from a database and sends it to the terminal. The terminal displays this list to the user and supports them in preparing for the craft.

[0317] Production progress record and real-time feedback

[0318] Once the user begins crafting, the device provides a UI that tracks progress at each step. Each time progress is reported, the information is sent to the server, which stores the progress information and uses artificial intelligence to generate feedback. The feedback includes recommendations for next steps and technical advice, which is sent to the device and displayed to the user.

[0319] Introducing the Emotion Engine

[0320] To recognize the user's emotions, the device is equipped with a camera and microphone, which can analyze the user's facial expressions, tone of voice, input text, etc. The recognized emotional information is sent to the server in real time.

[0321] Emotion-based feedback regulation

[0322] The server uses the emotional information sent by the emotion engine to tailor the feedback it generates, for example providing detailed and helpful advice if the user is confused, or concise next step instructions if the user is focused on a task.

[0323] Personalized learning support and community interaction features

[0324] The server will adjust to provide personalized learning support according to the user's skill level and language. It will also introduce a multilingual system to provide an easy-to-use environment for users overseas.

[0325] The device provides a community forum to encourage interaction between users, where users can share their progress and finished products and receive comments and advice from other users.

[0326] Additional Support

[0327] If users have any questions or problems during production, they can get support through the Q&A section or chat function. The server forwards these questions to the AI ​​or support team, who will provide a quick answer. The answer received is then notified to the user via their device.

[0328] In this way, this system is extremely useful for users to thoroughly learn traditional craft techniques at home and to pass on these techniques widely across regional and language barriers. Furthermore, the introduction of an emotion engine provides more effective and personalized feedback, further improving user satisfaction and learning efficiency.

[0329] The processing flow will be explained below.

[0330] Step 1:

[0331] The user accesses the platform and enters their login information (user ID and password).

[0332] Step 2:

[0333] The terminal sends the login information entered by the user to the server.

[0334] Step 3:

[0335] The server receives the login information and retrieves the user's profile information from the database, including past production history and preferences.

[0336] Step 4:

[0337] Based on the profile information acquired by the server, a list of recommended traditional crafts is generated and sent to the terminal.

[0338] Step 5:

[0339] The device displays a list of received crafts in the UI and prompts the user to select one.

[0340] Step 6:

[0341] The user selects the craft of interest from the list.

[0342] Step 7:

[0343] The terminal transmits the user's selection information to the server.

[0344] Step 8:

[0345] The server retrieves from the database a list of materials and tools required for the selected craft and transmits the list to the terminal.

[0346] Step 9:

[0347] The list of materials and tools received by the terminal is displayed on the UI, and the user is prompted to confirm the necessary preparations.

[0348] Step 10:

[0349] The user prepares for production and starts making the craft by following the instructions on the screen.

[0350] Step 11:

[0351] The device records the progress of each step and provides a UI that allows the user to report their progress.

[0352] Step 12:

[0353] Each time the user completes a step, they press a "Step Completed" button to report their progress.

[0354] Step 13:

[0355] The terminal transmits the user's progress information to the server.

[0356] Step 14:

[0357] The device analyzes the user's facial expressions and voice in real time through a camera and microphone, and transmits the emotional information to a server.

[0358] Step 15:

[0359] The server stores the received progress and emotion information and uses artificial intelligence to generate feedback, including recommended next steps and technical advice.

[0360] Step 16:

[0361] The server tailors the feedback based on the user's emotional state, for example adding detailed explanations if the user is confused or providing concise instructions if the user is making good progress.

[0362] Step 17:

[0363] The server transmits the generated feedback to the terminal.

[0364] Step 18:

[0365] The device receives feedback and displays it in the UI, informing the user of next steps and areas for improvement.

[0366] Step 19:

[0367] The server will adjust the program to provide personalized learning support according to the user's skill level and language. A multilingual system will also be introduced to provide an environment that is easy for overseas users to use.

[0368] Step 20:

[0369] The device provides a community forum to encourage interaction between users, allowing them to post their production progress and finished products.

[0370] Step 21:

[0371] If users have questions or problems during production, they can get support using the Q&A section or chat function.

[0372] Step 22:

[0373] The server forwards user questions to an artificial intelligence or support team to provide quick answers.

[0374] Step 23:

[0375] The terminal notifies the user of the received answer and provides assistance in solving the problem.

[0376] This series of processes allows users to efficiently learn traditional Japanese crafts from home, enabling the skills to be passed down widely across regions and language barriers.In addition, the introduction of an emotion engine provides more effective and personalized feedback, further improving user satisfaction and learning efficiency.

[0377] Example 2

[0378] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0379] Previous traditional craft learning systems lacked adequate support for users to efficiently learn and create crafts at home. Furthermore, they lacked a system that recognized users' emotions and technical difficulties in real time and provided appropriate feedback, resulting in problems that reduced learning efficiency and satisfaction. Furthermore, they lacked multilingual support and user-to-user interaction functions, making them insufficiently popular with overseas users.

[0380] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0381] In this invention, the server includes: means for providing information on materials and tools needed to create a craft selected by the user; means for recording and saving the user's progress in real time; means for generating feedback using artificial intelligence based on the user's progress; means for transmitting and displaying the generated feedback to the user's device; and means for recognizing the user's emotions and adjusting the feedback based on those emotions. This allows users to efficiently learn traditional crafts from home and receive personalized feedback, improving learning efficiency and satisfaction. Furthermore, multilingual support and user interaction functions enable the service to be popularized among overseas users.

[0382] "Users" refer to individuals who use this system to learn how to create traditional crafts and actually create them.

[0383] A "terminal" is a device that a user uses to access and use the system, and includes, for example, electronic devices such as computers, tablets, and smartphones.

[0384] "Server" refers to a central computing device that manages user login information, progress, emotional data, etc., and generates and provides necessary feedback.

[0385] "Crafts" are objects that users can learn about and create, and refer to handicrafts such as traditional Japanese crafts.

[0386] "Materials and tool information" refers to a list of the specific materials and tools needed to create the craft selected by the user.

[0387] "Progress" refers to the step a user is currently at in the craft creation process and a record of the tasks they have completed.

[0388] "Artificial intelligence" refers to a computer program or system that generates appropriate feedback based on data such as a user's progress.

[0389] "Feedback" refers to information such as advice, guidance, and next step instructions that a user can use as a reference when creating a craft.

[0390] "Emotion recognition" refers to technology that analyzes a user's facial expressions, tone of voice, and input text to understand their emotional state at that time.

[0391] "Feedback adjustment" refers to changing the content and level of detail of the feedback provided based on the user's emotional information.

[0392] "Personalized learning support" refers to a function that provides learning support optimized according to the user's skill level and language used.

[0393] "Community interaction function" refers to an online platform where users can share their production progress and finished products and receive comments and advice.

[0394] This invention is a system that supports users in the process of efficiently learning and creating traditional crafts at home in real time. The system includes an emotion engine that recognizes the user's emotions and adjusts feedback based on those emotions. The following describes how to specifically implement this system.

[0395] User Interface Embodiments

[0396] A user accesses the platform using a device and logs in. The login information is sent from the device to the server. The server retrieves the user's profile information from the database, generates a list of recommended traditional crafts based on this information, and sends it to the device. The device then displays this list on the UI and prompts the user to make a selection.

[0397] As a concrete example, when a user logs in, a list of traditional Japanese crafts is displayed on the terminal.

[0398] Example prompt: "When the user presses the login button"

[0399] View and select from a list of recommended crafts

[0400] When a user selects a craft, the selection information is sent from the terminal to the server. The server retrieves information on the materials and tools required for the selected craft from a database and sends it to the terminal. The terminal displays this information to the user and supports them in preparing for the craft.

[0401] As a specific example, when a user selects "Japanese paper doll," a list of materials and tools required to create it is displayed on the terminal.

[0402] Example prompt: "Show a list of materials needed for the selected craft."

[0403] Production progress record and real-time feedback

[0404] It provides a UI for users to start crafting and record their progress at each step. Each time progress is reported, the information is sent from the device to the server. The server stores the progress information and generates feedback using a generative AI model. The generated feedback includes recommendations for next steps and technical advice, and is sent to the device and displayed to the user.

[0405] For example, when a user completes step 3, they are given specific instructions and advice on how to proceed to the next step.

[0406] Example prompt: "Generate next step feedback based on your progress."

[0407] Emotion recognition by emotion engine

[0408] The device is equipped with a camera and microphone, which allows it to analyze the user's facial expressions and tone of voice, and this data is sent to a server in real time.

[0409] As a specific example, when a user shows a confused expression, the expression is captured and transmitted to the server as emotion data.

[0410] Example prompt: "Recognize user emotions in real time."

[0411] Adjusting Feedback Based on Emotions

[0412] The server analyzes the received emotional information and generates appropriate feedback: detailed advice if the user is confused, or concise instructions if the user is distracted.

[0413] As a concrete example, confused users are provided with detailed guidelines for next steps.

[0414] Example prompt: "Adjust feedback based on user emotion."

[0415] Personalized learning support and community interaction features

[0416] The server provides personalized learning support according to the user's skill level and language, and a multilingual system is implemented. The terminals provide a community forum to encourage interaction between users. Users can share their progress and finished products here and receive comments and advice from other users.

[0417] As a concrete example, users can post their creations to the community and receive feedback.

[0418] Example prompt: "The ability to share progress and finished products."

[0419] Additional Support

[0420] If users have any questions or problems during production, they can get support through the Q&A section or chat function. The server forwards these questions to the AI ​​or support team, who will provide a quick answer. The answer received is then notified to the user via their device.

[0421] For example, a user can enter a question and within a few minutes the AI ​​will provide a solution.

[0422] Example prompt: "Submit support questions and get answers in real time."

[0423] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0424] Step 1: Submit login information

[0425] A user accesses the login screen using a terminal and enters a username and password. The terminal sends this login information to the server. When the login information is sent to the server, the server accesses the database and performs authentication. If the authentication is successful, the server obtains the user's profile information and sends it to the terminal.

[0426] Input: Username, Password

[0427] Output: User profile information

[0428] Specific operation:

[0429] The user enters their username and password in the login form and clicks the Login button.

[0430] The device sends the login information to the server.

[0431] Step 2: Obtain profile information and provide a list of crafts

[0432] The server generates a list of recommended traditional crafts based on the profile information of the successfully authenticated user. The server then sends this list to the device. The device then displays the received craft list on a user interface and prompts the user to make a selection.

[0433] Input: User profile information

[0434] Output: Recommended craft list

[0435] Specific operation:

[0436] The server retrieves the user's profile information from the database.

[0437] The server generates a list of crafts based on the profile information and sends it to the terminal.

[0438] Step 3: Select a craft and get material information

[0439] The user selects the craft they want to create from the craft list. The device sends this selection information to the server. The server retrieves information about the materials and tools required for the selected craft from the database and sends it to the device. The device displays this information to the user.

[0440] Input: Craft selection information

[0441] Output: List of materials and tools needed

[0442] Specific operation:

[0443] The user selects "Japanese paper doll" from the crafts list.

[0444] The terminal transmits the selection information to the server.

[0445] Step 4: Record your progress and submit

[0446] The user starts creating a craft and records the progress of each step. The device sends the user's progress information in real time to the server, which stores the received progress information and uses it to generate feedback about the next step.

[0447] Input: User progress information

[0448] Output: Feedback for next steps

[0449] Specific operation:

[0450] The user completes production step 3 and records their progress on the device.

[0451] The terminal sends progress information to the server.

[0452] Step 5: Generate and display feedback as you progress

[0453] The server uses the generative AI model to generate appropriate feedback based on the received progress information. The generated feedback includes recommendations for next steps and technical advice. The server then sends the generated feedback to the device, which then displays it to the user.

[0454] Input: User progress information

[0455] Output: Feedback

[0456] Specific operation:

[0457] The server analyzes the progress information and generates feedback on next steps.

[0458] The device displays the feedback to the user.

[0459] Step 6: Collect and send emotion recognition data

[0460] The device is equipped with a camera and microphone to analyze the user's facial expressions and tone of voice, and the recognized emotional information is sent to a server in real time.

[0461] Input: User's facial expression data, voice data

[0462] Output: Emotional information

[0463] Specific operation:

[0464] The device's camera captures the user's facial expressions and analyzes the emotional data.

[0465] The device transmits the emotion information to the server.

[0466] Step 7: Adjusting Emotional Feedback

[0467] The server analyzes the received emotional information and adjusts the generated feedback content, providing detailed advice if the user is confused or simple instructions if the user is focused. The adjusted feedback is then sent to the device, which displays it to the user.

[0468] Input: Emotion information, progress information

[0469] Output: Regulated Feedback

[0470] Specific operation:

[0471] The server analyzes the emotional information and adjusts the feedback.

[0472] The device will display adjusted feedback.

[0473] Step 8: Providing personalized assistance and community features

[0474] The server provides support tailored to the user's skill level and language, and the terminal provides a community forum to encourage interaction between users, where they can share their progress and finished products and receive feedback.

[0475] Input: User profile information, skill level, preferred language

[0476] Output: personalized assistance, community posts

[0477] Specific operation:

[0478] Users post their creations to the community and receive comments from other users.

[0479] The server provides support according to the user's technical level.

[0480] Step 9: Providing real-time support

[0481] If users have questions or problems during production, they can get support through the Q&A section or chat function. The server forwards these questions to the AI ​​or support team, which generates a quick answer. The answer is then sent to the user via their device.

[0482] Input: User question

[0483] Output:Answer

[0484] Specific operation:

[0485] The user types their question into the chat box.

[0486] The server generates an answer using AI and sends it to the device.

[0487] (Application example 2)

[0488] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0489] In modern factories, when humans and robots work together, it is difficult for the robot to provide appropriate feedback based on the worker's emotions and progress. Systems are needed that can instantly recognize when a worker becomes confused or stressed and adjust the work environment and support provided. Furthermore, existing systems lack the ability to provide personalized support that adapts to the worker's progress and emotions, preventing improvements in work efficiency and satisfaction.

[0490] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for providing a list of materials and tools necessary to create a traditional craft selected by the user; means for recording and saving the user's progress in real time; means for generating feedback using artificial intelligence based on the user's progress; means for sending the generated feedback to the user's terminal and displaying it; and means for recognizing emotions using a camera and microphone and adjusting the feedback content based on the emotions. This makes it possible to provide appropriate feedback according to the worker's emotions and progress in collaborative work between humans and robots in a factory, thereby improving work efficiency and satisfaction.

[0491] A "terminal" is a device used by a user, and includes a computer, smartphone, tablet, etc.

[0492] "Server" means a computer system that manages and processes information, records user data and progress, and generates and sends feedback.

[0493] "Artificial intelligence" refers to algorithms and systems that mimic human intelligence, analyze data, make decisions, and use that technology to generate feedback.

[0494] "Means for recognizing emotions" refers to technology that uses input devices such as cameras and microphones to analyze a user's facial expressions and voice and determine their emotional state.

[0495] "Feedback" refers to information that provides appropriate advice or instructions based on the user's progress and emotions.

[0496] "Personalized learning support" refers to a system that provides individualized learning support tailored to the user's skill level and language.

[0497] "Community interaction function" refers to a system function that allows users to share information and communicate with each other.

[0498] "Traditional crafts" refer to handicrafts made using traditional Japanese techniques and methods.

[0499] This invention is a system for supporting collaborative work between humans and robots in factories, and aims to appropriately adjust feedback based on the user's emotions and progress. Specific embodiments of this system will be described below.

[0500] System Overview

[0501] The server provides a list of materials and tools required based on the task selected by the user, records and saves the user's progress in real time, and uses artificial intelligence to generate feedback based on the user's progress, which is then sent to the user's device for display.

[0502] A key part of the system is that it uses a camera and microphone to recognize the user's emotions and adjusts the feedback content based on those emotions.

[0503] Hardware and software used

[0504] For hardware, we use a camera (webcam), microphone, and speaker. For software, we use the following:

[0505] OpenCV: Image processing

[0506] DeepFace: Emotion recognition

[0507] SpeechRecognition: Speech recognition

[0508] gTTS: speech generation

[0509] Data processing and calculation methods

[0510] Acquiring camera data

[0511] The server uses a camera to capture the user's facial image in real time, which then initiates the process of recognizing emotions from the user's facial expressions. Specifically, the server processes the image using OpenCV and analyzes emotions using DeepFace.

[0512] Speech recognition

[0513] The server uses a microphone to capture the user's voice, converts the voice data into text, and uses the SpeechRecognition library to recognize the voice and analyze what the user is saying.

[0514] Generate feedback

[0515] The server generates appropriate feedback based on the emotion and speech recognition results. Specifically, if the user is confused, it provides detailed explanations and assistance, and if the user is satisfied, it sends an appropriate message of encouragement. The feedback message is converted into speech using gTTS and played back through the speaker.

[0516] Specific examples and prompts for the generative AI model

[0517] Specific examples

[0518] If the worker looks confused, the server generates a voice message saying, "Are you OK? May I help you?" and plays it through the speaker. If the user wants to know how to use a new tool while working, the server responds to the prompt, "Please tell me how to use that tool," by providing detailed instructions and advice.

[0519] Prompt Sentence Examples

[0520] "Turn the following text into speech: Are you OK? Can I help you?"

[0521] In this way, this system can recognize the user's emotions in real time and provide optimal feedback in response, thereby improving work efficiency and user satisfaction.In addition, by providing personalized learning support and community interaction functions, it also contributes to improving users' skills and forming communities.

[0522] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0523] Step 1:

[0524] The server uses a camera to capture images of the user's face in real time. The input data obtained from the camera is an image of the user's face. This image data is then processed using OpenCV and converted into an analyzable format.

[0525] Step 2:

[0526] The server uses DeepFace to analyze the facial video data obtained in step 1 and recognize the user's emotions. As a result of the analysis, the main emotions (e.g., confusion, joy) are output. This analysis result is stored in the server and used to generate feedback in the next step.

[0527] Step 3:

[0528] The server uses a microphone to capture the user's voice. The input data is the user's voice. This voice data is then recognized using SpeechRecognition and converted into text data. The converted text data is used to generate feedback in the next step.

[0529] Step 4:

[0530] The server generates appropriate feedback content based on the emotion data acquired in step 2 and the text data acquired in step 3. Specifically, it uses a generative AI model to create feedback text that corresponds to the user's condition and request. For example, if the user is confused, it generates the message "Are you okay? May I help you?"

[0531] Step 5:

[0532] The server uses gTTS to convert the generated feedback text into speech. The input data is the feedback text obtained in step 4, and the output data is an audio file. The server plays this audio file through a speaker to provide feedback to the user.

[0533] Step 6:

[0534] The device records the user's progress in real time and sends the data to the server. The input data is progress information based on the user's operations and actions, and the output data is saved to the server.

[0535] Step 7:

[0536] The server integrates and manages the collected progress data and emotional data, and dynamically adjusts the feedback content, thereby personalizing the user experience and providing optimal support at all times.

[0537] In this way, throughout each step, the server generates and provides feedback in real time according to the user's emotions and progress, realizing a system that improves work efficiency and user satisfaction.

[0538] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0539] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0540] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0541] [Second embodiment]

[0542] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0543] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0544] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0545] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0546] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0547] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0548] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0549] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0550] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0551] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0552] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0553] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0554] This invention is a system that supports users in the process of creating traditional Japanese crafts at home in real time. This system consists of three main components: a server, a terminal, and a user. Below, we will explain how to specifically implement this system.

[0555] User Interface (UI) Embodiments

[0556] When a user accesses the platform, they are first shown a login page. After the user enters their login information, that information is sent from the device to the server. The server retrieves the user's profile information from the database, generates a dashboard with appropriate content, and sends it back to the user. The dashboard displays the user's past crafting history and a list of recommended crafts.

[0557] View the list of recommended crafts

[0558] The server generates a list of suitable traditional crafts based on the user's preferences and past history and sends it to the device. The device displays this list on the UI and prompts the user to select the crafts they are interested in. Once the user selects a craft, the information is sent back to the server.

[0559] Providing necessary materials and tools

[0560] The server retrieves from the database a list of materials and tools needed for the selected craft and returns it to the terminal, which displays this information to the user to help them prepare to begin crafting.

[0561] Step-by-step guide and user progress record

[0562] Once the user starts crafting, the device provides a UI that tracks the progress of each step. As the user completes a step, the progress information is reported on the UI and sent to the server.

[0563] Real-time feedback

[0564] The server analyzes the received progress information in real time and uses artificial intelligence to generate feedback, including recommended next steps and technical advice, which is then sent to the device and displayed to the user.

[0565] Personalized learning support

[0566] The server adjusts the program to provide personalized learning support according to each user's skill level and language. It also introduces a multilingual system to provide an easy-to-use environment for users overseas.

[0567] Community interaction features

[0568] The device provides a community forum to encourage interaction between users, where users can share their crafting progress and finished products and receive comments and advice from other users. This feature allows users to deepen their interactions with other craft enthusiasts.

[0569] Additional Support

[0570] If users have any questions or problems during production, they can receive real-time support through the system's Q&A section or chat function. The server will forward these questions to the AI ​​or support team for quick answers.

[0571] In this way, this system supports users in efficiently learning traditional craft techniques from home and passing on these techniques widely across regional and language barriers.

[0572] The processing flow will be explained below.

[0573] Step 1:

[0574] The user accesses the platform's login page and enters their login information (user ID and password).

[0575] Step 2:

[0576] The terminal sends the login information entered by the user to the server.

[0577] Step 3:

[0578] The server receives the login information and retrieves the user's profile information from the database, including past crafting history and preferences (e.g., favorite craft categories).

[0579] Step 4:

[0580] Based on the profile information acquired by the server, a list of recommended traditional crafts is generated and sent to the terminal.

[0581] Step 5:

[0582] The device displays a list of received crafts in the UI and prompts the user to select one.

[0583] Step 6:

[0584] The user selects the craft of interest from the list.

[0585] Step 7:

[0586] The terminal transmits the user's selection information to the server.

[0587] Step 8:

[0588] The server retrieves from the database a list of materials and tools required for the selected craft and transmits the list to the terminal.

[0589] Step 9:

[0590] The list of materials and tools received by the terminal is displayed on the UI, and the user is prompted to confirm the necessary preparations.

[0591] Step 10:

[0592] The user prepares for production and starts making the craft by following the instructions on the screen.

[0593] Step 11:

[0594] The device records the progress of each step and provides a UI that allows the user to report their progress.

[0595] Step 12:

[0596] Each time the user completes a step, they press a "Step Completed" button to report their progress.

[0597] Step 13:

[0598] The terminal sends progress information to the server.

[0599] Step 14:

[0600] The server stores the progress information received and uses artificial intelligence to generate feedback, including recommended next steps and technical advice.

[0601] Step 15:

[0602] The server transmits the generated feedback to the terminal.

[0603] Step 16:

[0604] The device receives feedback and displays it in the UI, informing the user of next steps and areas for improvement.

[0605] Step 17:

[0606] The server will adjust the program to provide personalized learning support according to the user's skill level and language. A multilingual system will also be introduced to provide an environment that is easy for overseas users to use.

[0607] Step 18:

[0608] The device provides a community forum to encourage interaction between users, allowing them to post their production progress and finished products.

[0609] Step 19:

[0610] If users have questions or problems during production, they can get support using the Q&A section or chat function.

[0611] Step 20:

[0612] The server forwards user questions to an artificial intelligence or support team to provide quick answers.

[0613] Step 21:

[0614] The terminal notifies the user of the received answer and provides assistance in solving the problem.

[0615] This series of processes allows users to efficiently learn traditional Japanese crafts from home, making it possible to widely pass on skills beyond regional and language barriers.

[0616] Example 1

[0617] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0618] Conventional traditional craft production support systems have had issues such as users being unable to easily obtain a list of the materials and tools they need, difficulty in recording progress in real time, a lack of immediate feedback, insufficient personalized learning support according to the user's skill level and preferences, and a lack of promotion of interaction between users. To solve these problems, a system with more advanced data processing and real-time feedback functions is needed.

[0619] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0620] In this invention, the server includes means for providing a list of materials and tools required to create a craft item selected by the user, means for recording and saving the user's progress in real time, means for generating feedback based on the user's progress using artificial intelligence, means for sending the generated feedback to the user's device and displaying it, means for acquiring login information when the user accesses the platform and generating a dashboard based on that information, means for generating and displaying a list of recommended items based on the user's preferences and past history, and means for reporting progress information each time the user completes a step and providing recommendations for the next step and technical advice using artificial intelligence, thereby enabling users to efficiently and enjoyably progress with craft creation at home.

[0621] "User" refers to a person who uses the system to create a craft.

[0622] "Crafts" refers to handcrafted items made using traditional techniques.

[0623] "Materials" refers to the materials needed to create a craft item.

[0624] "Tools" refers to the implements and equipment used in creating crafts.

[0625] "List" refers to information in the form of a list that summarizes items of materials and tools.

[0626] "Progress" refers to information indicating how much of each step of craft creation the user has completed.

[0627] "Real-time" refers to processing and display occurring immediately without delay.

[0628] "Artificial intelligence" refers to technology that uses machine learning and data analysis to automatically make judgments and predictions.

[0629] "Feedback" refers to information including evaluations and advice regarding the user's actions and status.

[0630] "Device" refers to a device used by a user, such as a computer or smartphone.

[0631] "Platform" refers to the online environment through which Users access and engage in craft creation.

[0632] "Login Information" refers to the authentication data entered by a User to access a System.

[0633] "Dashboard" refers to an interface that displays a user's profile information and activity history in a centralized manner.

[0634] "Preferences" refers to information based on a user's preferences.

[0635] "History" refers to a record of a user's past activities and behavior.

[0636] "Recommended Items" refers to a list of crafts and materials suggested based on the user's preferences and history.

[0637] A "guide" refers to information that provides instructions for a user to create a craft item.

[0638] "Advice" refers to technical advice provided to users by artificial intelligence.

[0639] "Reporting" refers to the act of notifying the server of the work that the user has done and the progress of that work.

[0640] This invention is a system that supports users in the process of creating traditional Japanese crafts at home in real time. The system mainly consists of three elements: a server, a terminal, and a user.

[0641] First, when a user accesses the platform, a login page appears on the device. The user enters their login information (username and password), which is then sent from the device to the server. The server authenticates the login information and retrieves the user's profile information from the database. As a result, the server generates a dashboard dedicated to the user, which is sent to the device and displayed. The dashboard includes the user's past crafting history and a list of recommended crafts.

[0642] The server then generates a list of suitable traditional crafts based on the user's preferences and past history, using a data analysis algorithm. The generated list is sent to the device and displayed to the user. Once the user selects a craft of interest, the information is sent back to the server. The server retrieves a list of materials and tools needed for the selected craft from the database and returns the information to the device for display.

[0643] Once the user begins crafting, the device provides a UI that records the progress of each step. Each time the user completes a step, the progress information is sent from the device to the server. The server analyzes the received progress information in real time and generates feedback using artificial intelligence (AI). The feedback includes recommendations for next steps and technical advice, which is sent to the device and displayed to the user.

[0644] The server also adjusts the program to provide personalized learning support based on each user's skill level and language. This allows for a multilingual system, making it easier for users overseas to use. For example, it can provide an interface in multiple languages, including English, French, and German.

[0645] Additionally, the device provides a community forum for users to interact with each other. Here, users can share their crafting progress and finished products and receive comments and advice from other users. This promotes communication between craft enthusiasts. Furthermore, through the Q&A section and chat function, users can receive real-time support when they have questions or problems during their crafting. The server forwards these questions to the AI ​​or support team for quick answers.

[0646] For example:

[0647] When a user logs in and tries to create "Edo Kiriko," the device retrieves past production history from the server and displays it to the user. The server then recommends "Edo Kiriko" and displays the necessary materials (glass, cutting tools, etc.). As the user progresses with the production, the device records the progress and sends it to the server. The server uses AI to generate next step guides and advice in real time and provides them to the user.

[0648] Example prompts to input to the generative AI model:

[0649] Please describe in detail the system that provides real-time support to users' progress in creating traditional Japanese crafts, including the following points:

[0650] 1. Processing flow from user login to dashboard display

[0651] 2. How to generate and display a list of recommended crafts

[0652] 3. The process of providing necessary materials and tools

[0653] 4. Step-by-step progress recording and real-time feedback

[0654] 5. Personalized learning support and multilingual support

[0655] 6. Community Interaction Features and Additional Support Details

[0656] Please also provide a detailed description of the data flow and the technologies used.

[0657] In this way, this system supports users in efficiently learning traditional craft techniques from home and passing on these techniques widely across regional and language barriers.

[0658] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0659] Step 1:

[0660] A user accesses the platform and is presented with a login page.

[0661] Specific operation: The user enters their username and password. The entered data (login information) is sent from the terminal to the server.

[0662] Input and Output:

[0663] Input: Username, Password

[0664] Data processing: Encryption of entered login information

[0665] Output: Encrypted login information

[0666] Step 2:

[0667] The server receives the login information and retrieves the user's profile information from a database.

[0668] Specific operation: The server decrypts the received login information and performs authentication. Based on the authentication results, it searches the database for the user's profile information. Based on the obtained data, it generates a dashboard and sends it to the device.

[0669] Input and Output:

[0670] Input: Encrypted login information

[0671] Data calculation: Decryption of login information, verification of user information

[0672] Output: User profile information, generated dashboard

[0673] Step 3:

[0674] The terminal displays the dashboard received from the server to the user.

[0675] Specific operation: The device renders the received HTML data and displays the user's crafting history and a list of recommended crafts in the browser.

[0676] Input and Output:

[0677] Input: Generated dashboard

[0678] Data processing: Rendering HTML data

[0679] Output: The displayed dashboard

[0680] Step 4:

[0681] The server generates a list of suitable traditional crafts based on the user's preferences and past history and sends it to the terminal.

[0682] Specific operation: The server retrieves the user's preference information and past production history from the database, runs the recommendation algorithm, generates a recommendation list in JSON format, and sends it to the device.

[0683] Input and Output:

[0684] Input: User preference information, past history

[0685] Data Computing: Running the recommendation algorithm

[0686] Output: Recommended craft list

[0687] Step 5:

[0688] The terminal displays a list of recommended crafts to the user and prompts the user to select a craft they are interested in. Once the user selects a craft, the information is sent back to the server.

[0689] Specific operation: The device renders the received recommendation list on the UI, and when the user presses the select button, it sends the ID of the selected craft to the server.

[0690] Input and Output:

[0691] Input: Recommended Craft List

[0692] Data processing: UI display of recommendation list

[0693] Output: ID of the selected craft

[0694] Step 6:

[0695] The server retrieves from the database a list of materials and tools required for the selected craft and transmits it to the terminal.

[0696] Specific operation: The server searches the database based on the acquired craft ID, generates a list of required materials and tools, and sends it to the terminal in JSON format.

[0697] Input and Output:

[0698] Input: ID of selected craft

[0699] Data calculation: database search, list generation

[0700] Output: List of materials and tools

[0701] Step 7:

[0702] The terminal displays a list of materials and tools to the user.

[0703] Specific operation: The device renders the received list on the UI and helps the user prepare to start production.

[0704] Input and Output:

[0705] Input: List of materials and tools

[0706] Data processing: List UI display

[0707] Output: List of displayed materials and tools

[0708] Step 8:

[0709] Once the user starts crafting, the device provides a UI that tracks the progress of each step. Each time the user completes a step, the device sends that progress information to the server.

[0710] What it does: The device displays a progress tracker, allows the user to click a button to complete each step, and sends progress information to the server when a step is completed.

[0711] Input and Output:

[0712] Input: User progress information

[0713] Data processing: recording and sending progress information

[0714] Output: Progress information sent to the server

[0715] Step 9:

[0716] The server analyzes the received progress information in real time and generates feedback using artificial intelligence (AI).

[0717] What it does: The server uses AI models to analyze progress data, generate next step recommendations and technical advice, and send the generated feedback to the device.

[0718] Input and Output:

[0719] Input: User progress information

[0720] Data calculation: Analysis using AI models, feedback generation

[0721] Output: Generated feedback

[0722] Step 10:

[0723] The terminal displays the generated feedback to the user.

[0724] Specific operation: The device renders the feedback data received from the server on the UI and displays it to the user.

[0725] Input and Output:

[0726] Input: Generated feedback

[0727] Data processing: UI display of feedback data

[0728] Output: Displayed feedback

[0729] Step 11:

[0730] The server provides personalized learning support based on each user's skill level and language usage, providing a multilingual environment.

[0731] What happens: The server selects appropriate content and feedback based on the user's profile data, and translates it if necessary.

[0732] Input and Output:

[0733] Input: User profile data

[0734] Data Operations: Content Selection and Translation

[0735] Output: Personalized learning assistance

[0736] Step 12:

[0737] The device provides a community forum for users to interact with each other.

[0738] Specific operation: The device has a forum function and displays an interface where users can post and comment. Posts and comments are sent to the server and displayed to other users.

[0739] Input and Output:

[0740] Input: User posts and comments

[0741] Data processing: sending and displaying posts and comments

[0742] Output: Posts and comments that are visible to other users

[0743] Step 13:

[0744] If users have any questions or issues, they can get real-time support through the Q&A section or chat function. The server will forward these questions to the AI ​​or support team for quick answers.

[0745] Specific operation: The device displays a question form or chat window and sends the user's question to the server, which then forwards the question to the appropriate support system, generates an answer, and sends it to the device.

[0746] Input and Output:

[0747] Input: User question

[0748] Data Computation: Question Transfer and Answer Generation

[0749] Output: The answer sent to the user

[0750] (Application example 1)

[0751] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0752] Conventional support systems for traditional craft creation are primarily used at home, making it difficult to provide appropriate real-time support during workshops or craft classes held in brick-and-mortar stores. Furthermore, there is a lack of means to provide detailed support information and feedback for each step of a user's craft creation process in a store, making it difficult for users to create crafts efficiently and with high quality.

[0753] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0754] In this invention, the server includes means for providing a list of materials and tools required to create a traditional craft selected by the user, means for recording and saving the user's progress in real time, means for generating feedback using artificial intelligence based on the user's progress, means for sending the generated feedback to the user's terminal and displaying it, means for guiding the user through the production procedure in real time via a smart device, and means for providing support information for each step as the user produces the traditional craft in the store, thereby enabling the user to produce the traditional craft in the store with high efficiency.

[0755] The "means for providing a list of materials and tools required to create a traditional craft selected by the user" is a system for obtaining from a server specific items of materials and tools required for the traditional craft the user wishes to create and displaying them on the user's terminal.

[0756] "Means for recording and saving the user's progress in real time" refers to a system that has the function of recording the user's progress as they complete each step of production and saving that information on a server.

[0757] "Means for generating feedback using artificial intelligence based on the user's progress" refers to a function that analyzes the user's current production progress and generates feedback such as recommendations for next steps and improvements using an artificial intelligence algorithm.

[0758] "Means for transmitting and displaying the generated feedback to the user's terminal" refers to a system that notifies the user's terminal of feedback information generated by artificial intelligence and displays it in real time.

[0759] "Means for guiding users through the production procedure in real time via a smart device" refers to a system for providing step-by-step guidance through the production procedure via a smart device worn by the user (such as smart glasses or a head-mounted display).

[0760] "Means for providing support information for each step when a user is creating something in a store" refers to a system that provides the support information needed for each step (e.g., how to prepare materials, how to use tools, etc.) in real time when a user is performing a production activity in a physical store.

[0761] This invention is a system that supports users in the process of creating traditional Japanese crafts in real time in a brick-and-mortar store. The system consists of three main components: a server, a terminal (e.g., smart glasses), and a user. Below, we will explain how to specifically implement this system.

[0762] User Interface (UI) Embodiments

[0763] When a user accesses the platform, they are first shown a login page. After the user enters their login information, that information is sent from the device (smart glasses) to the server. The server retrieves the user's profile information from the database, generates a dashboard with appropriate content, and sends it back to the user. The dashboard displays the user's past crafting history and a list of recommended crafts.

[0764] View the list of recommended crafts

[0765] The server generates a list of suitable traditional crafts based on the user's preferences and past history and sends it to the device. The device displays this list on the UI and prompts the user to select the crafts they are interested in. Once the user selects a craft, the information is sent back to the server.

[0766] Providing necessary materials and tools

[0767] The server retrieves from the database a list of materials and tools needed for the selected craft and returns it to the terminal, which displays this information to the user to help them prepare to begin crafting.

[0768] Step-by-step guide and user progress record

[0769] Once the user starts crafting, the device (smart glasses) provides a UI that records the progress of each step. Each time the user completes a step, the progress information is reported on the UI and sent to the server.

[0770] Real-time feedback

[0771] The server analyzes the received progress information in real time and uses artificial intelligence to generate feedback, including recommended next steps and technical advice, which is then sent to the device and displayed to the user.

[0772] In-store guide function

[0773] When users create crafts in the store, they are guided through each step in real time via a device (smart glasses). This guide includes audio guidance and visual instructions to help users complete the work smoothly.

[0774] Specific examples

[0775] For example, suppose a user selects "lacquerware" in a physical store. The server retrieves a list of the necessary materials and tools (lacquerware, wood, brushes, knives, etc.) and displays it on the user's smart glasses. Once the user starts making the product, the smart glasses provide real-time guidance on how to prepare the materials and the work procedures for each step. In addition, each time the user reports their progress, feedback and advice on the next steps is provided.

[0776] Prompt Sentence Examples

[0777] "What should I pay attention to in the next step?"

[0778] "What's the best way to treat sumac?"

[0779] This allows users to create crafts in their physical stores with high efficiency and high quality.

[0780] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0781] Step 1:

[0782] User Login

[0783] Input: User ID and password

[0784] Process: The user accesses the login page and enters their login information. The device sends this information to the server.

[0785] Output: The server authenticates the user and retrieves their profile information.

[0786] Step 2:

[0787] Viewing the Dashboard

[0788] Input: User profile information

[0789] Processing: The server generates a dashboard containing appropriate content based on the profile information, past production history, and preferences.

[0790] Output: The terminal displays the user's dashboard, which includes past crafting history and a list of recommended crafts.

[0791] Step 3:

[0792] Generate and display a list of recommended crafts

[0793] Input: User preferences and past history

[0794] Processing: The server generates a list of suitable traditional crafts based on preferences and past history and sends it to the device.

[0795] Output: A list of recommended traditional crafts will be displayed on the device.

[0796] Step 4:

[0797] Craft Selection

[0798] Input: Recommended Craft List

[0799] Process: The user selects the artifact of interest and sends the information from the device to the server.

[0800] Output: The server receives the selected craft information.

[0801] Step 5:

[0802] Providing a list of materials and tools needed

[0803] Input: Selected craft information

[0804] Processing: The server retrieves the list of materials and tools required for the selected craft from the database and sends it to the terminal.

[0805] Output: A list of ingredients and tools will be displayed on the terminal.

[0806] Step 6:

[0807] Getting started and step-by-step guide

[0808] Input: Materials and tools list, crafting step guide

[0809] Process: The user starts the creation, and the device (smart glasses) provides a UI that records the progress of each step.

[0810] Output: A step-by-step build guide is displayed on the terminal, and the user reports their progress.

[0811] Step 7:

[0812] Tracking progress and generating feedback

[0813] Input: User progress information

[0814] Processing: The server analyzes the received progress information in real time and generates feedback using artificial intelligence.

[0815] Output: The generated feedback information is sent to the terminal and displayed to the user.

[0816] Step 8:

[0817] Real-time guide in physical stores

[0818] Input: User's production progress information

[0819] Processing: The server provides real-time step-by-step guidance through the smart glasses based on the user's progress in the physical store.

[0820] Output: Guidance information for each production step is displayed on the user's smart glasses.

[0821] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0822] This invention is a system that provides real-time support for users to efficiently learn and create traditional Japanese crafts at home. The system incorporates an emotion engine that recognizes the user's emotions and appropriately adjusts feedback based on those emotions. The following describes how to specifically implement this system.

[0823] User Interface (UI) Embodiments

[0824] A user accesses the platform and logs in. The login information is sent from the device to the server, which then retrieves the user's profile information from the database. Based on this information, a list of recommended traditional crafts is generated and sent to the device. The device then displays this list in the UI and prompts the user to make a selection.

[0825] View and select from a list of recommended crafts

[0826] When a user selects a craft, the selection information is sent from the terminal to the server. The server retrieves a list of materials and tools required for the selected craft from a database and sends it to the terminal. The terminal displays this list to the user and supports them in preparing for the craft.

[0827] Production progress record and real-time feedback

[0828] Once the user begins crafting, the device provides a UI that tracks progress at each step. Each time progress is reported, the information is sent to the server, which stores the progress information and uses artificial intelligence to generate feedback. The feedback includes recommendations for next steps and technical advice, which is sent to the device and displayed to the user.

[0829] Introducing the Emotion Engine

[0830] To recognize the user's emotions, the device is equipped with a camera and microphone, which can analyze the user's facial expressions, tone of voice, input text, etc. The recognized emotional information is sent to the server in real time.

[0831] Emotion-based feedback regulation

[0832] The server uses the emotional information sent by the emotion engine to tailor the feedback it generates, for example providing detailed and helpful advice if the user is confused, or concise next step instructions if the user is focused on a task.

[0833] Personalized learning support and community interaction features

[0834] The server will adjust to provide personalized learning support according to the user's skill level and language. It will also introduce a multilingual system to provide an easy-to-use environment for users overseas.

[0835] The device provides a community forum to encourage interaction between users, where users can share their progress and finished products and receive comments and advice from other users.

[0836] Additional Support

[0837] If users have any questions or problems during production, they can get support through the Q&A section or chat function. The server forwards these questions to the AI ​​or support team, who will provide a quick answer. The answer received is then notified to the user via their device.

[0838] In this way, this system is extremely useful for users to thoroughly learn traditional craft techniques at home and to pass on these techniques widely across regional and language barriers. Furthermore, the introduction of an emotion engine provides more effective and personalized feedback, further improving user satisfaction and learning efficiency.

[0839] The processing flow will be explained below.

[0840] Step 1:

[0841] The user accesses the platform and enters their login information (user ID and password).

[0842] Step 2:

[0843] The terminal sends the login information entered by the user to the server.

[0844] Step 3:

[0845] The server receives the login information and retrieves the user's profile information from the database, including past production history and preferences.

[0846] Step 4:

[0847] Based on the profile information acquired by the server, a list of recommended traditional crafts is generated and sent to the terminal.

[0848] Step 5:

[0849] The device displays a list of received crafts in the UI and prompts the user to select one.

[0850] Step 6:

[0851] The user selects the craft of interest from the list.

[0852] Step 7:

[0853] The terminal transmits the user's selection information to the server.

[0854] Step 8:

[0855] The server retrieves from the database a list of materials and tools required for the selected craft and transmits the list to the terminal.

[0856] Step 9:

[0857] The list of materials and tools received by the terminal is displayed on the UI, and the user is prompted to confirm the necessary preparations.

[0858] Step 10:

[0859] The user prepares for production and starts making the craft by following the instructions on the screen.

[0860] Step 11:

[0861] The device records the progress of each step and provides a UI that allows the user to report their progress.

[0862] Step 12:

[0863] Each time the user completes a step, they press a "Step Completed" button to report their progress.

[0864] Step 13:

[0865] The terminal transmits the user's progress information to the server.

[0866] Step 14:

[0867] The device analyzes the user's facial expressions and voice in real time through a camera and microphone, and transmits the emotional information to a server.

[0868] Step 15:

[0869] The server stores the received progress and emotion information and uses artificial intelligence to generate feedback, including recommended next steps and technical advice.

[0870] Step 16:

[0871] The server tailors the feedback based on the user's emotional state, for example adding detailed explanations if the user is confused or providing concise instructions if the user is making good progress.

[0872] Step 17:

[0873] The server transmits the generated feedback to the terminal.

[0874] Step 18:

[0875] The device receives feedback and displays it in the UI, informing the user of next steps and areas for improvement.

[0876] Step 19:

[0877] The server will adjust the program to provide personalized learning support according to the user's skill level and language. A multilingual system will also be introduced to provide an environment that is easy for overseas users to use.

[0878] Step 20:

[0879] The device provides a community forum to encourage interaction between users, allowing them to post their production progress and finished products.

[0880] Step 21:

[0881] If users have questions or problems during production, they can get support using the Q&A section or chat function.

[0882] Step 22:

[0883] The server forwards user questions to an artificial intelligence or support team to provide quick answers.

[0884] Step 23:

[0885] The terminal notifies the user of the received answer and provides assistance in solving the problem.

[0886] This series of processes allows users to efficiently learn traditional Japanese crafts from home, enabling the skills to be passed down widely across regions and language barriers.In addition, the introduction of an emotion engine provides more effective and personalized feedback, further improving user satisfaction and learning efficiency.

[0887] Example 2

[0888] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0889] Previous traditional craft learning systems lacked adequate support for users to efficiently learn and create crafts at home. Furthermore, they lacked a system that recognized users' emotions and technical difficulties in real time and provided appropriate feedback, resulting in problems that reduced learning efficiency and satisfaction. Furthermore, they lacked multilingual support and user-to-user interaction functions, making them insufficiently popular with overseas users.

[0890] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0891] In this invention, the server includes: means for providing information on materials and tools needed to create a craft selected by the user; means for recording and saving the user's progress in real time; means for generating feedback using artificial intelligence based on the user's progress; means for transmitting and displaying the generated feedback to the user's device; and means for recognizing the user's emotions and adjusting the feedback based on those emotions. This allows users to efficiently learn traditional crafts from home and receive personalized feedback, improving learning efficiency and satisfaction. Furthermore, multilingual support and user interaction functions enable the service to be popularized among overseas users.

[0892] "Users" refer to individuals who use this system to learn how to create traditional crafts and actually create them.

[0893] A "terminal" is a device that a user uses to access and use the system, and includes, for example, electronic devices such as computers, tablets, and smartphones.

[0894] "Server" refers to a central computing device that manages user login information, progress, emotional data, etc., and generates and provides necessary feedback.

[0895] "Crafts" are objects that users can learn about and create, and refer to handicrafts such as traditional Japanese crafts.

[0896] "Materials and tool information" refers to a list of the specific materials and tools needed to create the craft selected by the user.

[0897] "Progress" refers to the step a user is currently at in the craft creation process and a record of the tasks they have completed.

[0898] "Artificial intelligence" refers to a computer program or system that generates appropriate feedback based on data such as a user's progress.

[0899] "Feedback" refers to information such as advice, guidance, and next step instructions that a user can use as a reference when creating a craft.

[0900] "Emotion recognition" refers to technology that analyzes a user's facial expressions, tone of voice, and input text to understand their emotional state at that time.

[0901] "Feedback adjustment" refers to changing the content and level of detail of the feedback provided based on the user's emotional information.

[0902] "Personalized learning support" refers to a function that provides learning support optimized according to the user's skill level and language used.

[0903] "Community interaction function" refers to an online platform where users can share their production progress and finished products and receive comments and advice.

[0904] This invention is a system that supports users in the process of efficiently learning and creating traditional crafts at home in real time. The system includes an emotion engine that recognizes the user's emotions and adjusts feedback based on those emotions. The following describes how to specifically implement this system.

[0905] User Interface Embodiments

[0906] A user accesses the platform using a device and logs in. The login information is sent from the device to the server. The server retrieves the user's profile information from the database, generates a list of recommended traditional crafts based on this information, and sends it to the device. The device then displays this list on the UI and prompts the user to make a selection.

[0907] As a concrete example, when a user logs in, a list of traditional Japanese crafts is displayed on the terminal.

[0908] Example prompt: "When the user presses the login button"

[0909] View and select from a list of recommended crafts

[0910] When a user selects a craft, the selection information is sent from the terminal to the server. The server retrieves information on the materials and tools required for the selected craft from a database and sends it to the terminal. The terminal displays this information to the user and supports them in preparing for the craft.

[0911] As a specific example, when a user selects "Japanese paper doll," a list of materials and tools required to create it is displayed on the terminal.

[0912] Example prompt: "Show a list of materials needed for the selected craft."

[0913] Production progress record and real-time feedback

[0914] It provides a UI for users to start crafting and record their progress at each step. Each time progress is reported, the information is sent from the device to the server. The server stores the progress information and generates feedback using a generative AI model. The generated feedback includes recommendations for next steps and technical advice, and is sent to the device and displayed to the user.

[0915] For example, when a user completes step 3, they are given specific instructions and advice on how to proceed to the next step.

[0916] Example prompt: "Generate next step feedback based on your progress."

[0917] Emotion recognition by emotion engine

[0918] The device is equipped with a camera and microphone, which allows it to analyze the user's facial expressions and tone of voice, and this data is sent to a server in real time.

[0919] As a specific example, when a user shows a confused expression, the expression is captured and transmitted to the server as emotion data.

[0920] Example prompt: "Recognize user emotions in real time."

[0921] Adjusting Feedback Based on Emotions

[0922] The server analyzes the received emotional information and generates appropriate feedback: detailed advice if the user is confused, or concise instructions if the user is distracted.

[0923] As a concrete example, confused users are provided with detailed guidelines for next steps.

[0924] Example prompt: "Adjust feedback based on user emotion."

[0925] Personalized learning support and community interaction features

[0926] The server provides personalized learning support according to the user's skill level and language, and a multilingual system is implemented. The terminals provide a community forum to encourage interaction between users. Users can share their progress and finished products here and receive comments and advice from other users.

[0927] As a concrete example, users can post their creations to the community and receive feedback.

[0928] Example prompt: "The ability to share progress and finished products."

[0929] Additional Support

[0930] If users have any questions or problems during production, they can get support through the Q&A section or chat function. The server forwards these questions to the AI ​​or support team, who will provide a quick answer. The answer received is then notified to the user via their device.

[0931] For example, a user can enter a question and within a few minutes the AI ​​will provide a solution.

[0932] Example prompt: "Submit support questions and get answers in real time."

[0933] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0934] Step 1: Submit login information

[0935] A user accesses the login screen using a terminal and enters a username and password. The terminal sends this login information to the server. When the login information is sent to the server, the server accesses the database and performs authentication. If the authentication is successful, the server obtains the user's profile information and sends it to the terminal.

[0936] Input: Username, Password

[0937] Output: User profile information

[0938] Specific operation:

[0939] The user enters their username and password in the login form and clicks the Login button.

[0940] The device sends the login information to the server.

[0941] Step 2: Obtain profile information and provide a list of crafts

[0942] The server generates a list of recommended traditional crafts based on the profile information of the successfully authenticated user. The server then sends this list to the device. The device then displays the received craft list on a user interface and prompts the user to make a selection.

[0943] Input: User profile information

[0944] Output: Recommended craft list

[0945] Specific operation:

[0946] The server retrieves the user's profile information from the database.

[0947] The server generates a list of crafts based on the profile information and sends it to the terminal.

[0948] Step 3: Select a craft and get material information

[0949] The user selects the craft they want to create from the craft list. The device sends this selection information to the server. The server retrieves information about the materials and tools required for the selected craft from the database and sends it to the device. The device displays this information to the user.

[0950] Input: Craft selection information

[0951] Output: List of materials and tools needed

[0952] Specific operation:

[0953] The user selects "Japanese paper doll" from the crafts list.

[0954] The terminal transmits the selection information to the server.

[0955] Step 4: Record your progress and submit

[0956] The user starts creating a craft and records the progress of each step. The device sends the user's progress information in real time to the server, which stores the received progress information and uses it to generate feedback about the next step.

[0957] Input: User progress information

[0958] Output: Feedback for next steps

[0959] Specific operation:

[0960] The user completes production step 3 and records their progress on the device.

[0961] The terminal sends progress information to the server.

[0962] Step 5: Generate and display feedback as you progress

[0963] The server uses the generative AI model to generate appropriate feedback based on the received progress information. The generated feedback includes recommendations for next steps and technical advice. The server then sends the generated feedback to the device, which then displays it to the user.

[0964] Input: User progress information

[0965] Output: Feedback

[0966] Specific operation:

[0967] The server analyzes the progress information and generates feedback on next steps.

[0968] The device displays the feedback to the user.

[0969] Step 6: Collect and send emotion recognition data

[0970] The device is equipped with a camera and microphone to analyze the user's facial expressions and tone of voice, and the recognized emotional information is sent to a server in real time.

[0971] Input: User's facial expression data, voice data

[0972] Output: Emotional information

[0973] Specific operation:

[0974] The device's camera captures the user's facial expressions and analyzes the emotional data.

[0975] The device transmits the emotion information to the server.

[0976] Step 7: Adjusting Emotional Feedback

[0977] The server analyzes the received emotional information and adjusts the generated feedback content, providing detailed advice if the user is confused or simple instructions if the user is focused. The adjusted feedback is then sent to the device, which displays it to the user.

[0978] Input: Emotion information, progress information

[0979] Output: Regulated Feedback

[0980] Specific operation:

[0981] The server analyzes the emotional information and adjusts the feedback.

[0982] The device will display adjusted feedback.

[0983] Step 8: Providing personalized assistance and community features

[0984] The server provides support tailored to the user's skill level and language, and the terminal provides a community forum to encourage interaction between users, where they can share their progress and finished products and receive feedback.

[0985] Input: User profile information, skill level, preferred language

[0986] Output: personalized assistance, community posts

[0987] Specific operation:

[0988] Users post their creations to the community and receive comments from other users.

[0989] The server provides support according to the user's technical level.

[0990] Step 9: Providing real-time support

[0991] If users have questions or problems during production, they can get support through the Q&A section or chat function. The server forwards these questions to the AI ​​or support team, which generates a quick answer. The answer is then sent to the user via their device.

[0992] Input: User question

[0993] Output:Answer

[0994] Specific operation:

[0995] The user types their question into the chat box.

[0996] The server generates an answer using AI and sends it to the device.

[0997] (Application example 2)

[0998] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0999] In modern factories, when humans and robots work together, it is difficult for the robot to provide appropriate feedback based on the worker's emotions and progress. Systems are needed that can instantly recognize when a worker becomes confused or stressed and adjust the work environment and support provided. Furthermore, existing systems lack the ability to provide personalized support that adapts to the worker's progress and emotions, preventing improvements in work efficiency and satisfaction.

[1000] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for providing a list of materials and tools necessary to create a traditional craft selected by the user; means for recording and saving the user's progress in real time; means for generating feedback using artificial intelligence based on the user's progress; means for sending the generated feedback to the user's terminal and displaying it; and means for recognizing emotions using a camera and microphone and adjusting the feedback content based on the emotions. This makes it possible to provide appropriate feedback according to the worker's emotions and progress in collaborative work between humans and robots in a factory, thereby improving work efficiency and satisfaction.

[1001] A "terminal" is a device used by a user, and includes a computer, smartphone, tablet, etc.

[1002] "Server" means a computer system that manages and processes information, records user data and progress, and generates and sends feedback.

[1003] "Artificial intelligence" refers to algorithms and systems that mimic human intelligence, analyze data, make decisions, and use that technology to generate feedback.

[1004] "Means for recognizing emotions" refers to technology that uses input devices such as cameras and microphones to analyze a user's facial expressions and voice and determine their emotional state.

[1005] "Feedback" refers to information that provides appropriate advice or instructions based on the user's progress and emotions.

[1006] "Personalized learning support" refers to a system that provides individualized learning support tailored to the user's skill level and language.

[1007] "Community interaction function" refers to a system function that allows users to share information and communicate with each other.

[1008] "Traditional crafts" refer to handicrafts made using traditional Japanese techniques and methods.

[1009] This invention is a system for supporting collaborative work between humans and robots in factories, and aims to appropriately adjust feedback based on the user's emotions and progress. Specific embodiments of this system will be described below.

[1010] System Overview

[1011] The server provides a list of materials and tools required based on the task selected by the user, records and saves the user's progress in real time, and uses artificial intelligence to generate feedback based on the user's progress, which is then sent to the user's device for display.

[1012] A key part of the system is that it uses a camera and microphone to recognize the user's emotions and adjusts the feedback content based on those emotions.

[1013] Hardware and software used

[1014] For hardware, we use a camera (webcam), microphone, and speaker. For software, we use the following:

[1015] OpenCV: Image processing

[1016] DeepFace: Emotion recognition

[1017] SpeechRecognition: Speech recognition

[1018] gTTS: speech generation

[1019] Data processing and calculation methods

[1020] Acquiring camera data

[1021] The server uses a camera to capture the user's facial image in real time, which then initiates the process of recognizing emotions from the user's facial expressions. Specifically, the server processes the image using OpenCV and analyzes emotions using DeepFace.

[1022] Speech recognition

[1023] The server uses a microphone to capture the user's voice, converts the voice data into text, and uses the SpeechRecognition library to recognize the voice and analyze what the user is saying.

[1024] Generate feedback

[1025] The server generates appropriate feedback based on the emotion and speech recognition results. Specifically, if the user is confused, it provides detailed explanations and assistance, and if the user is satisfied, it sends an appropriate message of encouragement. The feedback message is converted into speech using gTTS and played back through the speaker.

[1026] Specific examples and prompts for the generative AI model

[1027] Specific examples

[1028] If the worker looks confused, the server generates a voice message saying, "Are you OK? May I help you?" and plays it through the speaker. If the user wants to know how to use a new tool while working, the server responds to the prompt, "Please tell me how to use that tool," by providing detailed instructions and advice.

[1029] Prompt Sentence Examples

[1030] "Turn the following text into speech: Are you OK? Can I help you?"

[1031] In this way, this system can recognize the user's emotions in real time and provide optimal feedback in response, thereby improving work efficiency and user satisfaction.In addition, by providing personalized learning support and community interaction functions, it also contributes to improving users' skills and forming communities.

[1032] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1033] Step 1:

[1034] The server uses a camera to capture images of the user's face in real time. The input data obtained from the camera is an image of the user's face. This image data is then processed using OpenCV and converted into an analyzable format.

[1035] Step 2:

[1036] The server uses DeepFace to analyze the facial video data obtained in step 1 and recognize the user's emotions. As a result of the analysis, the main emotions (e.g., confusion, joy) are output. This analysis result is stored in the server and used to generate feedback in the next step.

[1037] Step 3:

[1038] The server uses a microphone to capture the user's voice. The input data is the user's voice. This voice data is then recognized using SpeechRecognition and converted into text data. The converted text data is used to generate feedback in the next step.

[1039] Step 4:

[1040] The server generates appropriate feedback content based on the emotion data acquired in step 2 and the text data acquired in step 3. Specifically, it uses a generative AI model to create feedback text that corresponds to the user's condition and request. For example, if the user is confused, it generates the message "Are you okay? May I help you?"

[1041] Step 5:

[1042] The server uses gTTS to convert the generated feedback text into speech. The input data is the feedback text obtained in step 4, and the output data is an audio file. The server plays this audio file through a speaker to provide feedback to the user.

[1043] Step 6:

[1044] The device records the user's progress in real time and sends the data to the server. The input data is progress information based on the user's operations and actions, and the output data is saved to the server.

[1045] Step 7:

[1046] The server integrates and manages the collected progress data and emotional data, and dynamically adjusts the feedback content, thereby personalizing the user experience and providing optimal support at all times.

[1047] In this way, throughout each step, the server generates and provides feedback in real time according to the user's emotions and progress, realizing a system that improves work efficiency and user satisfaction.

[1048] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1049] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1050] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1051] [Third embodiment]

[1052] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1053] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1054] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1055] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1056] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1057] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1058] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1059] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1060] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1061] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1062] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1063] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1064] This invention is a system that supports users in the process of creating traditional Japanese crafts at home in real time. This system consists of three main components: a server, a terminal, and a user. Below, we will explain how to specifically implement this system.

[1065] User Interface (UI) Embodiments

[1066] When a user accesses the platform, they are first shown a login page. After the user enters their login information, that information is sent from the device to the server. The server retrieves the user's profile information from the database, generates a dashboard with appropriate content, and sends it back to the user. The dashboard displays the user's past crafting history and a list of recommended crafts.

[1067] View the list of recommended crafts

[1068] The server generates a list of suitable traditional crafts based on the user's preferences and past history and sends it to the device. The device displays this list on the UI and prompts the user to select the crafts they are interested in. Once the user selects a craft, the information is sent back to the server.

[1069] Providing necessary materials and tools

[1070] The server retrieves from the database a list of materials and tools needed for the selected craft and returns it to the terminal, which displays this information to the user to help them prepare to begin crafting.

[1071] Step-by-step guide and user progress record

[1072] Once the user starts crafting, the device provides a UI that tracks the progress of each step. As the user completes a step, the progress information is reported on the UI and sent to the server.

[1073] Real-time feedback

[1074] The server analyzes the received progress information in real time and uses artificial intelligence to generate feedback, including recommended next steps and technical advice, which is then sent to the device and displayed to the user.

[1075] Personalized learning support

[1076] The server adjusts the program to provide personalized learning support according to each user's skill level and language. It also introduces a multilingual system to provide an easy-to-use environment for users overseas.

[1077] Community interaction features

[1078] The device provides a community forum to encourage interaction between users, where users can share their crafting progress and finished products and receive comments and advice from other users. This feature allows users to deepen their interactions with other craft enthusiasts.

[1079] Additional Support

[1080] If users have any questions or problems during production, they can receive real-time support through the system's Q&A section or chat function. The server will forward these questions to the AI ​​or support team for quick answers.

[1081] In this way, this system supports users in efficiently learning traditional craft techniques from home and passing on these techniques widely across regional and language barriers.

[1082] The processing flow will be explained below.

[1083] Step 1:

[1084] The user accesses the platform's login page and enters their login information (user ID and password).

[1085] Step 2:

[1086] The terminal sends the login information entered by the user to the server.

[1087] Step 3:

[1088] The server receives the login information and retrieves the user's profile information from the database, including past crafting history and preferences (e.g., favorite craft categories).

[1089] Step 4:

[1090] Based on the profile information acquired by the server, a list of recommended traditional crafts is generated and sent to the terminal.

[1091] Step 5:

[1092] The device displays a list of received crafts in the UI and prompts the user to select one.

[1093] Step 6:

[1094] The user selects the craft of interest from the list.

[1095] Step 7:

[1096] The terminal transmits the user's selection information to the server.

[1097] Step 8:

[1098] The server retrieves from the database a list of materials and tools required for the selected craft and transmits the list to the terminal.

[1099] Step 9:

[1100] The list of materials and tools received by the terminal is displayed on the UI, and the user is prompted to confirm the necessary preparations.

[1101] Step 10:

[1102] The user prepares for production and starts making the craft by following the instructions on the screen.

[1103] Step 11:

[1104] The device records the progress of each step and provides a UI that allows the user to report their progress.

[1105] Step 12:

[1106] Each time the user completes a step, they press a "Step Completed" button to report their progress.

[1107] Step 13:

[1108] The terminal sends progress information to the server.

[1109] Step 14:

[1110] The server stores the progress information received and uses artificial intelligence to generate feedback, including recommended next steps and technical advice.

[1111] Step 15:

[1112] The server transmits the generated feedback to the terminal.

[1113] Step 16:

[1114] The device receives feedback and displays it in the UI, informing the user of next steps and areas for improvement.

[1115] Step 17:

[1116] The server will adjust the program to provide personalized learning support according to the user's skill level and language. A multilingual system will also be introduced to provide an environment that is easy for overseas users to use.

[1117] Step 18:

[1118] The device provides a community forum to encourage interaction between users, allowing them to post their production progress and finished products.

[1119] Step 19:

[1120] If users have questions or problems during production, they can get support using the Q&A section or chat function.

[1121] Step 20:

[1122] The server forwards user questions to an artificial intelligence or support team to provide quick answers.

[1123] Step 21:

[1124] The terminal notifies the user of the received answer and provides assistance in solving the problem.

[1125] This series of processes allows users to efficiently learn traditional Japanese crafts from home, making it possible to widely pass on skills beyond regional and language barriers.

[1126] Example 1

[1127] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1128] Conventional traditional craft production support systems have had issues such as users being unable to easily obtain a list of the materials and tools they need, difficulty in recording progress in real time, a lack of immediate feedback, insufficient personalized learning support according to the user's skill level and preferences, and a lack of promotion of interaction between users. To solve these problems, a system with more advanced data processing and real-time feedback functions is needed.

[1129] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1130] In this invention, the server includes means for providing a list of materials and tools required to create a craft item selected by the user, means for recording and saving the user's progress in real time, means for generating feedback based on the user's progress using artificial intelligence, means for sending the generated feedback to the user's device and displaying it, means for acquiring login information when the user accesses the platform and generating a dashboard based on that information, means for generating and displaying a list of recommended items based on the user's preferences and past history, and means for reporting progress information each time the user completes a step and providing recommendations for the next step and technical advice using artificial intelligence, thereby enabling users to efficiently and enjoyably progress with craft creation at home.

[1131] "User" refers to a person who uses the system to create a craft.

[1132] "Crafts" refers to handcrafted items made using traditional techniques.

[1133] "Materials" refers to the materials needed to create a craft item.

[1134] "Tools" refers to the implements and equipment used in creating crafts.

[1135] "List" refers to information in the form of a list that summarizes items of materials and tools.

[1136] "Progress" refers to information indicating how much of each step of craft creation the user has completed.

[1137] "Real-time" refers to processing and display occurring immediately without delay.

[1138] "Artificial intelligence" refers to technology that uses machine learning and data analysis to automatically make judgments and predictions.

[1139] "Feedback" refers to information including evaluations and advice regarding the user's actions and status.

[1140] "Device" refers to a device used by a user, such as a computer or smartphone.

[1141] "Platform" refers to the online environment through which Users access and engage in craft creation.

[1142] "Login Information" refers to the authentication data entered by a User to access a System.

[1143] "Dashboard" refers to an interface that displays a user's profile information and activity history in a centralized manner.

[1144] "Preferences" refers to information based on a user's preferences.

[1145] "History" refers to a record of a user's past activities and behavior.

[1146] "Recommended Items" refers to a list of crafts and materials suggested based on the user's preferences and history.

[1147] A "guide" refers to information that provides instructions for a user to create a craft item.

[1148] "Advice" refers to technical advice provided to users by artificial intelligence.

[1149] "Reporting" refers to the act of notifying the server of the work that the user has done and the progress of that work.

[1150] This invention is a system that supports users in the process of creating traditional Japanese crafts at home in real time. The system mainly consists of three elements: a server, a terminal, and a user.

[1151] First, when a user accesses the platform, a login page appears on the device. The user enters their login information (username and password), which is then sent from the device to the server. The server authenticates the login information and retrieves the user's profile information from the database. As a result, the server generates a dashboard dedicated to the user, which is sent to the device and displayed. The dashboard includes the user's past crafting history and a list of recommended crafts.

[1152] The server then generates a list of suitable traditional crafts based on the user's preferences and past history, using a data analysis algorithm. The generated list is sent to the device and displayed to the user. Once the user selects a craft of interest, the information is sent back to the server. The server retrieves a list of materials and tools needed for the selected craft from the database and returns the information to the device for display.

[1153] Once the user begins crafting, the device provides a UI that records the progress of each step. Each time the user completes a step, the progress information is sent from the device to the server. The server analyzes the received progress information in real time and generates feedback using artificial intelligence (AI). The feedback includes recommendations for next steps and technical advice, which is sent to the device and displayed to the user.

[1154] The server also adjusts the program to provide personalized learning support based on each user's skill level and language. This allows for a multilingual system, making it easier for users overseas to use. For example, it can provide an interface in multiple languages, including English, French, and German.

[1155] Additionally, the device provides a community forum for users to interact with each other. Here, users can share their crafting progress and finished products and receive comments and advice from other users. This promotes communication between craft enthusiasts. Furthermore, through the Q&A section and chat function, users can receive real-time support when they have questions or problems during their crafting. The server forwards these questions to the AI ​​or support team for quick answers.

[1156] For example:

[1157] When a user logs in and tries to create "Edo Kiriko," the device retrieves past production history from the server and displays it to the user. The server then recommends "Edo Kiriko" and displays the necessary materials (glass, cutting tools, etc.). As the user progresses with the production, the device records the progress and sends it to the server. The server uses AI to generate next step guides and advice in real time and provides them to the user.

[1158] Example prompts to input to the generative AI model:

[1159] Please describe in detail the system that provides real-time support to users' progress in creating traditional Japanese crafts, including the following points:

[1160] 1. Processing flow from user login to dashboard display

[1161] 2. How to generate and display a list of recommended crafts

[1162] 3. The process of providing necessary materials and tools

[1163] 4. Step-by-step progress recording and real-time feedback

[1164] 5. Personalized learning support and multilingual support

[1165] 6. Community Interaction Features and Additional Support Details

[1166] Please also provide a detailed description of the data flow and the technologies used.

[1167] In this way, this system supports users in efficiently learning traditional craft techniques from home and passing on these techniques widely across regional and language barriers.

[1168] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1169] Step 1:

[1170] A user accesses the platform and is presented with a login page.

[1171] Specific operation: The user enters their username and password. The entered data (login information) is sent from the terminal to the server.

[1172] Input and Output:

[1173] Input: Username, Password

[1174] Data processing: Encryption of entered login information

[1175] Output: Encrypted login information

[1176] Step 2:

[1177] The server receives the login information and retrieves the user's profile information from a database.

[1178] Specific operation: The server decrypts the received login information and performs authentication. Based on the authentication results, it searches the database for the user's profile information. Based on the obtained data, it generates a dashboard and sends it to the device.

[1179] Input and Output:

[1180] Input: Encrypted login information

[1181] Data calculation: Decryption of login information, verification of user information

[1182] Output: User profile information, generated dashboard

[1183] Step 3:

[1184] The terminal displays the dashboard received from the server to the user.

[1185] Specific operation: The device renders the received HTML data and displays the user's crafting history and a list of recommended crafts in the browser.

[1186] Input and Output:

[1187] Input: Generated dashboard

[1188] Data processing: Rendering HTML data

[1189] Output: The displayed dashboard

[1190] Step 4:

[1191] The server generates a list of suitable traditional crafts based on the user's preferences and past history and sends it to the terminal.

[1192] Specific operation: The server retrieves the user's preference information and past production history from the database, runs the recommendation algorithm, generates a recommendation list in JSON format, and sends it to the device.

[1193] Input and Output:

[1194] Input: User preference information, past history

[1195] Data Computing: Running the recommendation algorithm

[1196] Output: Recommended craft list

[1197] Step 5:

[1198] The terminal displays a list of recommended crafts to the user and prompts the user to select a craft they are interested in. Once the user selects a craft, the information is sent back to the server.

[1199] Specific operation: The device renders the received recommendation list on the UI, and when the user presses the select button, it sends the ID of the selected craft to the server.

[1200] Input and Output:

[1201] Input: Recommended Craft List

[1202] Data processing: UI display of recommendation list

[1203] Output: ID of the selected craft

[1204] Step 6:

[1205] The server retrieves from the database a list of materials and tools required for the selected craft and transmits it to the terminal.

[1206] Specific operation: The server searches the database based on the acquired craft ID, generates a list of required materials and tools, and sends it to the terminal in JSON format.

[1207] Input and Output:

[1208] Input: ID of selected craft

[1209] Data calculation: database search, list generation

[1210] Output: List of materials and tools

[1211] Step 7:

[1212] The terminal displays a list of materials and tools to the user.

[1213] Specific operation: The device renders the received list on the UI and helps the user prepare to start production.

[1214] Input and Output:

[1215] Input: List of materials and tools

[1216] Data processing: List UI display

[1217] Output: List of displayed materials and tools

[1218] Step 8:

[1219] Once the user starts crafting, the device provides a UI that tracks the progress of each step. Each time the user completes a step, the device sends that progress information to the server.

[1220] What it does: The device displays a progress tracker, allows the user to click a button to complete each step, and sends progress information to the server when a step is completed.

[1221] Input and Output:

[1222] Input: User progress information

[1223] Data processing: recording and sending progress information

[1224] Output: Progress information sent to the server

[1225] Step 9:

[1226] The server analyzes the received progress information in real time and generates feedback using artificial intelligence (AI).

[1227] What it does: The server uses AI models to analyze progress data, generate next step recommendations and technical advice, and send the generated feedback to the device.

[1228] Input and Output:

[1229] Input: User progress information

[1230] Data calculation: Analysis using AI models, feedback generation

[1231] Output: Generated feedback

[1232] Step 10:

[1233] The terminal displays the generated feedback to the user.

[1234] Specific operation: The device renders the feedback data received from the server on the UI and displays it to the user.

[1235] Input and Output:

[1236] Input: Generated feedback

[1237] Data processing: UI display of feedback data

[1238] Output: Displayed feedback

[1239] Step 11:

[1240] The server provides personalized learning support based on each user's skill level and language usage, providing a multilingual environment.

[1241] What happens: The server selects appropriate content and feedback based on the user's profile data, and translates it if necessary.

[1242] Input and Output:

[1243] Input: User profile data

[1244] Data Operations: Content Selection and Translation

[1245] Output: Personalized learning assistance

[1246] Step 12:

[1247] The device provides a community forum for users to interact with each other.

[1248] Specific operation: The device has a forum function and displays an interface where users can post and comment. Posts and comments are sent to the server and displayed to other users.

[1249] Input and Output:

[1250] Input: User posts and comments

[1251] Data processing: sending and displaying posts and comments

[1252] Output: Posts and comments that are visible to other users

[1253] Step 13:

[1254] If users have any questions or issues, they can get real-time support through the Q&A section or chat function. The server will forward these questions to the AI ​​or support team for quick answers.

[1255] Specific operation: The device displays a question form or chat window and sends the user's question to the server, which then forwards the question to the appropriate support system, generates an answer, and sends it to the device.

[1256] Input and Output:

[1257] Input: User question

[1258] Data Computation: Question Transfer and Answer Generation

[1259] Output: The answer sent to the user

[1260] (Application example 1)

[1261] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1262] Conventional support systems for traditional craft creation are primarily used at home, making it difficult to provide appropriate real-time support during workshops or craft classes held in brick-and-mortar stores. Furthermore, there is a lack of means to provide detailed support information and feedback for each step of a user's craft creation process in a store, making it difficult for users to create crafts efficiently and with high quality.

[1263] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1264] In this invention, the server includes means for providing a list of materials and tools required to create a traditional craft selected by the user, means for recording and saving the user's progress in real time, means for generating feedback using artificial intelligence based on the user's progress, means for sending the generated feedback to the user's terminal and displaying it, means for guiding the user through the production procedure in real time via a smart device, and means for providing support information for each step as the user produces the traditional craft in the store, thereby enabling the user to produce the traditional craft in the store with high efficiency.

[1265] The "means for providing a list of materials and tools required to create a traditional craft selected by the user" is a system for obtaining from a server specific items of materials and tools required for the traditional craft the user wishes to create and displaying them on the user's terminal.

[1266] "Means for recording and saving the user's progress in real time" refers to a system that has the function of recording the user's progress as they complete each step of production and saving that information on a server.

[1267] "Means for generating feedback using artificial intelligence based on the user's progress" refers to a function that analyzes the user's current production progress and generates feedback such as recommendations for next steps and improvements using an artificial intelligence algorithm.

[1268] "Means for transmitting and displaying the generated feedback to the user's terminal" refers to a system that notifies the user's terminal of feedback information generated by artificial intelligence and displays it in real time.

[1269] "Means for guiding users through the production procedure in real time via a smart device" refers to a system for providing step-by-step guidance through the production procedure via a smart device worn by the user (such as smart glasses or a head-mounted display).

[1270] "Means for providing support information for each step when a user is creating something in a store" refers to a system that provides the support information needed for each step (e.g., how to prepare materials, how to use tools, etc.) in real time when a user is performing a production activity in a physical store.

[1271] This invention is a system that supports users in the process of creating traditional Japanese crafts in real time in a brick-and-mortar store. The system consists of three main components: a server, a terminal (e.g., smart glasses), and a user. Below, we will explain how to specifically implement this system.

[1272] User Interface (UI) Embodiments

[1273] When a user accesses the platform, they are first shown a login page. After the user enters their login information, that information is sent from the device (smart glasses) to the server. The server retrieves the user's profile information from the database, generates a dashboard with appropriate content, and sends it back to the user. The dashboard displays the user's past crafting history and a list of recommended crafts.

[1274] View the list of recommended crafts

[1275] The server generates a list of suitable traditional crafts based on the user's preferences and past history and sends it to the device. The device displays this list on the UI and prompts the user to select the crafts they are interested in. Once the user selects a craft, the information is sent back to the server.

[1276] Providing necessary materials and tools

[1277] The server retrieves from the database a list of materials and tools needed for the selected craft and returns it to the terminal, which displays this information to the user to help them prepare to begin crafting.

[1278] Step-by-step guide and user progress record

[1279] Once the user starts crafting, the device (smart glasses) provides a UI that records the progress of each step. Each time the user completes a step, the progress information is reported on the UI and sent to the server.

[1280] Real-time feedback

[1281] The server analyzes the received progress information in real time and uses artificial intelligence to generate feedback, including recommended next steps and technical advice, which is then sent to the device and displayed to the user.

[1282] In-store guide function

[1283] When users create crafts in the store, they are guided through each step in real time via a device (smart glasses). This guide includes audio guidance and visual instructions to help users complete the work smoothly.

[1284] Specific examples

[1285] For example, suppose a user selects "lacquerware" in a physical store. The server retrieves a list of the necessary materials and tools (lacquerware, wood, brushes, knives, etc.) and displays it on the user's smart glasses. Once the user starts making the product, the smart glasses provide real-time guidance on how to prepare the materials and the work procedures for each step. In addition, each time the user reports their progress, feedback and advice on the next steps is provided.

[1286] Prompt Sentence Examples

[1287] "What should I pay attention to in the next step?"

[1288] "What's the best way to treat sumac?"

[1289] This allows users to create crafts in their physical stores with high efficiency and high quality.

[1290] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1291] Step 1:

[1292] User Login

[1293] Input: User ID and password

[1294] Process: The user accesses the login page and enters their login information. The device sends this information to the server.

[1295] Output: The server authenticates the user and retrieves their profile information.

[1296] Step 2:

[1297] Viewing the Dashboard

[1298] Input: User profile information

[1299] Processing: The server generates a dashboard containing appropriate content based on the profile information, past production history, and preferences.

[1300] Output: The terminal displays the user's dashboard, which includes past crafting history and a list of recommended crafts.

[1301] Step 3:

[1302] Generate and display a list of recommended crafts

[1303] Input: User preferences and past history

[1304] Processing: The server generates a list of suitable traditional crafts based on preferences and past history and sends it to the device.

[1305] Output: A list of recommended traditional crafts will be displayed on the device.

[1306] Step 4:

[1307] Craft Selection

[1308] Input: Recommended Craft List

[1309] Process: The user selects the artifact of interest and sends the information from the device to the server.

[1310] Output: The server receives the selected craft information.

[1311] Step 5:

[1312] Providing a list of materials and tools needed

[1313] Input: Selected craft information

[1314] Processing: The server retrieves the list of materials and tools required for the selected craft from the database and sends it to the terminal.

[1315] Output: A list of ingredients and tools will be displayed on the terminal.

[1316] Step 6:

[1317] Getting started and step-by-step guide

[1318] Input: Materials and tools list, crafting step guide

[1319] Process: The user starts the creation, and the device (smart glasses) provides a UI that records the progress of each step.

[1320] Output: A step-by-step build guide is displayed on the terminal, and the user reports their progress.

[1321] Step 7:

[1322] Tracking progress and generating feedback

[1323] Input: User progress information

[1324] Processing: The server analyzes the received progress information in real time and generates feedback using artificial intelligence.

[1325] Output: The generated feedback information is sent to the terminal and displayed to the user.

[1326] Step 8:

[1327] Real-time guide in physical stores

[1328] Input: User's production progress information

[1329] Processing: The server provides real-time step-by-step guidance through the smart glasses based on the user's progress in the physical store.

[1330] Output: Guidance information for each production step is displayed on the user's smart glasses.

[1331] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1332] This invention is a system that provides real-time support for users to efficiently learn and create traditional Japanese crafts at home. The system incorporates an emotion engine that recognizes the user's emotions and appropriately adjusts feedback based on those emotions. The following describes how to specifically implement this system.

[1333] User Interface (UI) Embodiments

[1334] A user accesses the platform and logs in. The login information is sent from the device to the server, which then retrieves the user's profile information from the database. Based on this information, a list of recommended traditional crafts is generated and sent to the device. The device then displays this list in the UI and prompts the user to make a selection.

[1335] View and select from a list of recommended crafts

[1336] When a user selects a craft, the selection information is sent from the terminal to the server. The server retrieves a list of materials and tools required for the selected craft from a database and sends it to the terminal. The terminal displays this list to the user and supports them in preparing for the craft.

[1337] Production progress record and real-time feedback

[1338] Once the user begins crafting, the device provides a UI that tracks progress at each step. Each time progress is reported, the information is sent to the server, which stores the progress information and uses artificial intelligence to generate feedback. The feedback includes recommendations for next steps and technical advice, which is sent to the device and displayed to the user.

[1339] Introducing the Emotion Engine

[1340] To recognize the user's emotions, the device is equipped with a camera and microphone, which can analyze the user's facial expressions, tone of voice, input text, etc. The recognized emotional information is sent to the server in real time.

[1341] Emotion-based feedback regulation

[1342] The server uses the emotional information sent by the emotion engine to tailor the feedback it generates, for example providing detailed and helpful advice if the user is confused, or concise next step instructions if the user is focused on a task.

[1343] Personalized learning support and community interaction features

[1344] The server will adjust to provide personalized learning support according to the user's skill level and language. It will also introduce a multilingual system to provide an easy-to-use environment for users overseas.

[1345] The device provides a community forum to encourage interaction between users, where users can share their progress and finished products and receive comments and advice from other users.

[1346] Additional Support

[1347] If users have any questions or problems during production, they can get support through the Q&A section or chat function. The server forwards these questions to the AI ​​or support team, who will provide a quick answer. The answer received is then notified to the user via their device.

[1348] In this way, this system is extremely useful for users to thoroughly learn traditional craft techniques at home and to pass on these techniques widely across regional and language barriers. Furthermore, the introduction of an emotion engine provides more effective and personalized feedback, further improving user satisfaction and learning efficiency.

[1349] The processing flow will be explained below.

[1350] Step 1:

[1351] The user accesses the platform and enters their login information (user ID and password).

[1352] Step 2:

[1353] The terminal sends the login information entered by the user to the server.

[1354] Step 3:

[1355] The server receives the login information and retrieves the user's profile information from the database, including past production history and preferences.

[1356] Step 4:

[1357] Based on the profile information acquired by the server, a list of recommended traditional crafts is generated and sent to the terminal.

[1358] Step 5:

[1359] The device displays a list of received crafts in the UI and prompts the user to select one.

[1360] Step 6:

[1361] The user selects the craft of interest from the list.

[1362] Step 7:

[1363] The terminal transmits the user's selection information to the server.

[1364] Step 8:

[1365] The server retrieves from the database a list of materials and tools required for the selected craft and transmits the list to the terminal.

[1366] Step 9:

[1367] The list of materials and tools received by the terminal is displayed on the UI, and the user is prompted to confirm the necessary preparations.

[1368] Step 10:

[1369] The user prepares for production and starts making the craft by following the instructions on the screen.

[1370] Step 11:

[1371] The device records the progress of each step and provides a UI that allows the user to report their progress.

[1372] Step 12:

[1373] Each time the user completes a step, they press a "Step Completed" button to report their progress.

[1374] Step 13:

[1375] The terminal transmits the user's progress information to the server.

[1376] Step 14:

[1377] The device analyzes the user's facial expressions and voice in real time through a camera and microphone, and transmits the emotional information to a server.

[1378] Step 15:

[1379] The server stores the received progress and emotion information and uses artificial intelligence to generate feedback, including recommended next steps and technical advice.

[1380] Step 16:

[1381] The server tailors the feedback based on the user's emotional state, for example adding detailed explanations if the user is confused or providing concise instructions if the user is making good progress.

[1382] Step 17:

[1383] The server transmits the generated feedback to the terminal.

[1384] Step 18:

[1385] The device receives feedback and displays it in the UI, informing the user of next steps and areas for improvement.

[1386] Step 19:

[1387] The server will adjust the program to provide personalized learning support according to the user's skill level and language. A multilingual system will also be introduced to provide an environment that is easy for overseas users to use.

[1388] Step 20:

[1389] The device provides a community forum to encourage interaction between users, allowing them to post their production progress and finished products.

[1390] Step 21:

[1391] If users have questions or problems during production, they can get support using the Q&A section or chat function.

[1392] Step 22:

[1393] The server forwards user questions to an artificial intelligence or support team to provide quick answers.

[1394] Step 23:

[1395] The terminal notifies the user of the received answer and provides assistance in solving the problem.

[1396] This series of processes allows users to efficiently learn traditional Japanese crafts from home, enabling the skills to be passed down widely across regions and language barriers.In addition, the introduction of an emotion engine provides more effective and personalized feedback, further improving user satisfaction and learning efficiency.

[1397] Example 2

[1398] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1399] Previous traditional craft learning systems lacked adequate support for users to efficiently learn and create crafts at home. Furthermore, they lacked a system that recognized users' emotions and technical difficulties in real time and provided appropriate feedback, resulting in problems that reduced learning efficiency and satisfaction. Furthermore, they lacked multilingual support and user-to-user interaction functions, making them insufficiently popular with overseas users.

[1400] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1401] In this invention, the server includes: means for providing information on materials and tools needed to create a craft selected by the user; means for recording and saving the user's progress in real time; means for generating feedback using artificial intelligence based on the user's progress; means for transmitting and displaying the generated feedback to the user's device; and means for recognizing the user's emotions and adjusting the feedback based on those emotions. This allows users to efficiently learn traditional crafts from home and receive personalized feedback, improving learning efficiency and satisfaction. Furthermore, multilingual support and user interaction functions enable the service to be popularized among overseas users.

[1402] "Users" refer to individuals who use this system to learn how to create traditional crafts and actually create them.

[1403] A "terminal" is a device that a user uses to access and use the system, and includes, for example, electronic devices such as computers, tablets, and smartphones.

[1404] "Server" refers to a central computing device that manages user login information, progress, emotional data, etc., and generates and provides necessary feedback.

[1405] "Crafts" are objects that users can learn about and create, and refer to handicrafts such as traditional Japanese crafts.

[1406] "Materials and tool information" refers to a list of the specific materials and tools needed to create the craft selected by the user.

[1407] "Progress" refers to the step a user is currently at in the craft creation process and a record of the tasks they have completed.

[1408] "Artificial intelligence" refers to a computer program or system that generates appropriate feedback based on data such as a user's progress.

[1409] "Feedback" refers to information such as advice, guidance, and next step instructions that a user can use as a reference when creating a craft.

[1410] "Emotion recognition" refers to technology that analyzes a user's facial expressions, tone of voice, and input text to understand their emotional state at that time.

[1411] "Feedback adjustment" refers to changing the content and level of detail of the feedback provided based on the user's emotional information.

[1412] "Personalized learning support" refers to a function that provides learning support optimized according to the user's skill level and language used.

[1413] "Community interaction function" refers to an online platform where users can share their production progress and finished products and receive comments and advice.

[1414] This invention is a system that supports users in the process of efficiently learning and creating traditional crafts at home in real time. The system includes an emotion engine that recognizes the user's emotions and adjusts feedback based on those emotions. The following describes how to specifically implement this system.

[1415] User Interface Embodiments

[1416] A user accesses the platform using a device and logs in. The login information is sent from the device to the server. The server retrieves the user's profile information from the database, generates a list of recommended traditional crafts based on this information, and sends it to the device. The device then displays this list on the UI and prompts the user to make a selection.

[1417] As a concrete example, when a user logs in, a list of traditional Japanese crafts is displayed on the terminal.

[1418] Example prompt: "When the user presses the login button"

[1419] View and select from a list of recommended crafts

[1420] When a user selects a craft, the selection information is sent from the terminal to the server. The server retrieves information on the materials and tools required for the selected craft from a database and sends it to the terminal. The terminal displays this information to the user and supports them in preparing for the craft.

[1421] As a specific example, when a user selects "Japanese paper doll," a list of materials and tools required to create it is displayed on the terminal.

[1422] Example prompt: "Show a list of materials needed for the selected craft."

[1423] Production progress record and real-time feedback

[1424] It provides a UI for users to start crafting and record their progress at each step. Each time progress is reported, the information is sent from the device to the server. The server stores the progress information and generates feedback using a generative AI model. The generated feedback includes recommendations for next steps and technical advice, and is sent to the device and displayed to the user.

[1425] For example, when a user completes step 3, they are given specific instructions and advice on how to proceed to the next step.

[1426] Example prompt: "Generate next step feedback based on your progress."

[1427] Emotion recognition by emotion engine

[1428] The device is equipped with a camera and microphone, which allows it to analyze the user's facial expressions and tone of voice, and this data is sent to a server in real time.

[1429] As a specific example, when a user shows a confused expression, the expression is captured and transmitted to the server as emotion data.

[1430] Example prompt: "Recognize user emotions in real time."

[1431] Adjusting Feedback Based on Emotions

[1432] The server analyzes the received emotional information and generates appropriate feedback: detailed advice if the user is confused, or concise instructions if the user is distracted.

[1433] As a concrete example, confused users are provided with detailed guidelines for next steps.

[1434] Example prompt: "Adjust feedback based on user emotion."

[1435] Personalized learning support and community interaction features

[1436] The server provides personalized learning support according to the user's skill level and language, and a multilingual system is implemented. The terminals provide a community forum to encourage interaction between users. Users can share their progress and finished products here and receive comments and advice from other users.

[1437] As a concrete example, users can post their creations to the community and receive feedback.

[1438] Example prompt: "The ability to share progress and finished products."

[1439] Additional Support

[1440] If users have any questions or problems during production, they can get support through the Q&A section or chat function. The server forwards these questions to the AI ​​or support team, who will provide a quick answer. The answer received is then notified to the user via their device.

[1441] For example, a user can enter a question and within a few minutes the AI ​​will provide a solution.

[1442] Example prompt: "Submit support questions and get answers in real time."

[1443] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1444] Step 1: Submit login information

[1445] A user accesses the login screen using a terminal and enters a username and password. The terminal sends this login information to the server. When the login information is sent to the server, the server accesses the database and performs authentication. If the authentication is successful, the server obtains the user's profile information and sends it to the terminal.

[1446] Input: Username, Password

[1447] Output: User profile information

[1448] Specific operation:

[1449] The user enters their username and password in the login form and clicks the Login button.

[1450] The device sends the login information to the server.

[1451] Step 2: Obtain profile information and provide a list of crafts

[1452] The server generates a list of recommended traditional crafts based on the profile information of the successfully authenticated user. The server then sends this list to the device. The device then displays the received craft list on a user interface and prompts the user to make a selection.

[1453] Input: User profile information

[1454] Output: Recommended craft list

[1455] Specific operation:

[1456] The server retrieves the user's profile information from the database.

[1457] The server generates a list of crafts based on the profile information and sends it to the terminal.

[1458] Step 3: Select a craft and get material information

[1459] The user selects the craft they want to create from the craft list. The device sends this selection information to the server. The server retrieves information about the materials and tools required for the selected craft from the database and sends it to the device. The device displays this information to the user.

[1460] Input: Craft selection information

[1461] Output: List of materials and tools needed

[1462] Specific operation:

[1463] The user selects "Japanese paper doll" from the crafts list.

[1464] The terminal transmits the selection information to the server.

[1465] Step 4: Record your progress and submit

[1466] The user starts creating a craft and records the progress of each step. The device sends the user's progress information in real time to the server, which stores the received progress information and uses it to generate feedback about the next step.

[1467] Input: User progress information

[1468] Output: Feedback for next steps

[1469] Specific operation:

[1470] The user completes production step 3 and records their progress on the device.

[1471] The terminal sends progress information to the server.

[1472] Step 5: Generate and display feedback as you progress

[1473] The server uses the generative AI model to generate appropriate feedback based on the received progress information. The generated feedback includes recommendations for next steps and technical advice. The server then sends the generated feedback to the device, which then displays it to the user.

[1474] Input: User progress information

[1475] Output: Feedback

[1476] Specific operation:

[1477] The server analyzes the progress information and generates feedback on next steps.

[1478] The device displays the feedback to the user.

[1479] Step 6: Collect and send emotion recognition data

[1480] The device is equipped with a camera and microphone to analyze the user's facial expressions and tone of voice, and the recognized emotional information is sent to a server in real time.

[1481] Input: User's facial expression data, voice data

[1482] Output: Emotional information

[1483] Specific operation:

[1484] The device's camera captures the user's facial expressions and analyzes the emotional data.

[1485] The device transmits the emotion information to the server.

[1486] Step 7: Adjusting Emotional Feedback

[1487] The server analyzes the received emotional information and adjusts the generated feedback content, providing detailed advice if the user is confused or simple instructions if the user is focused. The adjusted feedback is then sent to the device, which displays it to the user.

[1488] Input: Emotion information, progress information

[1489] Output: Regulated Feedback

[1490] Specific operation:

[1491] The server analyzes the emotional information and adjusts the feedback.

[1492] The device will display adjusted feedback.

[1493] Step 8: Providing personalized assistance and community features

[1494] The server provides support tailored to the user's skill level and language, and the terminal provides a community forum to encourage interaction between users, where they can share their progress and finished products and receive feedback.

[1495] Input: User profile information, skill level, preferred language

[1496] Output: personalized assistance, community posts

[1497] Specific operation:

[1498] Users post their creations to the community and receive comments from other users.

[1499] The server provides support according to the user's technical level.

[1500] Step 9: Providing real-time support

[1501] If users have questions or problems during production, they can get support through the Q&A section or chat function. The server forwards these questions to the AI ​​or support team, which generates a quick answer. The answer is then sent to the user via their device.

[1502] Input: User question

[1503] Output:Answer

[1504] Specific operation:

[1505] The user types their question into the chat box.

[1506] The server generates an answer using AI and sends it to the device.

[1507] (Application example 2)

[1508] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1509] In modern factories, when humans and robots work together, it is difficult for the robot to provide appropriate feedback based on the worker's emotions and progress. Systems are needed that can instantly recognize when a worker becomes confused or stressed and adjust the work environment and support provided. Furthermore, existing systems lack the ability to provide personalized support that adapts to the worker's progress and emotions, preventing improvements in work efficiency and satisfaction.

[1510] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for providing a list of materials and tools necessary to create a traditional craft selected by the user; means for recording and saving the user's progress in real time; means for generating feedback using artificial intelligence based on the user's progress; means for sending the generated feedback to the user's terminal and displaying it; and means for recognizing emotions using a camera and microphone and adjusting the feedback content based on the emotions. This makes it possible to provide appropriate feedback according to the worker's emotions and progress in collaborative work between humans and robots in a factory, thereby improving work efficiency and satisfaction.

[1511] A "terminal" is a device used by a user, and includes a computer, smartphone, tablet, etc.

[1512] "Server" means a computer system that manages and processes information, records user data and progress, and generates and sends feedback.

[1513] "Artificial intelligence" refers to algorithms and systems that mimic human intelligence, analyze data, make decisions, and use that technology to generate feedback.

[1514] "Means for recognizing emotions" refers to technology that uses input devices such as cameras and microphones to analyze a user's facial expressions and voice and determine their emotional state.

[1515] "Feedback" refers to information that provides appropriate advice or instructions based on the user's progress and emotions.

[1516] "Personalized learning support" refers to a system that provides individualized learning support tailored to the user's skill level and language.

[1517] "Community interaction function" refers to a system function that allows users to share information and communicate with each other.

[1518] "Traditional crafts" refer to handicrafts made using traditional Japanese techniques and methods.

[1519] This invention is a system for supporting collaborative work between humans and robots in factories, and aims to appropriately adjust feedback based on the user's emotions and progress. Specific embodiments of this system will be described below.

[1520] System Overview

[1521] The server provides a list of materials and tools required based on the task selected by the user, records and saves the user's progress in real time, and uses artificial intelligence to generate feedback based on the user's progress, which is then sent to the user's device for display.

[1522] A key part of the system is that it uses a camera and microphone to recognize the user's emotions and adjusts the feedback content based on those emotions.

[1523] Hardware and software used

[1524] For hardware, we use a camera (webcam), microphone, and speaker. For software, we use the following:

[1525] OpenCV: Image processing

[1526] DeepFace: Emotion recognition

[1527] SpeechRecognition: Speech recognition

[1528] gTTS: speech generation

[1529] Data processing and calculation methods

[1530] Acquiring camera data

[1531] The server uses a camera to capture the user's facial image in real time, which then initiates the process of recognizing emotions from the user's facial expressions. Specifically, the server processes the image using OpenCV and analyzes emotions using DeepFace.

[1532] Speech recognition

[1533] The server uses a microphone to capture the user's voice, converts the voice data into text, and uses the SpeechRecognition library to recognize the voice and analyze what the user is saying.

[1534] Generate feedback

[1535] The server generates appropriate feedback based on the emotion and speech recognition results. Specifically, if the user is confused, it provides detailed explanations and assistance, and if the user is satisfied, it sends an appropriate message of encouragement. The feedback message is converted into speech using gTTS and played back through the speaker.

[1536] Specific examples and prompts for the generative AI model

[1537] Specific examples

[1538] If the worker looks confused, the server generates a voice message saying, "Are you OK? May I help you?" and plays it through the speaker. If the user wants to know how to use a new tool while working, the server responds to the prompt, "Please tell me how to use that tool," by providing detailed instructions and advice.

[1539] Prompt Sentence Examples

[1540] "Turn the following text into speech: Are you OK? Can I help you?"

[1541] In this way, this system can recognize the user's emotions in real time and provide optimal feedback in response, thereby improving work efficiency and user satisfaction.In addition, by providing personalized learning support and community interaction functions, it also contributes to improving users' skills and forming communities.

[1542] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1543] Step 1:

[1544] The server uses a camera to capture images of the user's face in real time. The input data obtained from the camera is an image of the user's face. This image data is then processed using OpenCV and converted into an analyzable format.

[1545] Step 2:

[1546] The server uses DeepFace to analyze the facial video data obtained in step 1 and recognize the user's emotions. As a result of the analysis, the main emotions (e.g., confusion, joy) are output. This analysis result is stored in the server and used to generate feedback in the next step.

[1547] Step 3:

[1548] The server uses a microphone to capture the user's voice. The input data is the user's voice. This voice data is then recognized using SpeechRecognition and converted into text data. The converted text data is used to generate feedback in the next step.

[1549] Step 4:

[1550] The server generates appropriate feedback content based on the emotion data acquired in step 2 and the text data acquired in step 3. Specifically, it uses a generative AI model to create feedback text that corresponds to the user's condition and request. For example, if the user is confused, it generates the message "Are you okay? May I help you?"

[1551] Step 5:

[1552] The server uses gTTS to convert the generated feedback text into speech. The input data is the feedback text obtained in step 4, and the output data is an audio file. The server plays this audio file through a speaker to provide feedback to the user.

[1553] Step 6:

[1554] The device records the user's progress in real time and sends the data to the server. The input data is progress information based on the user's operations and actions, and the output data is saved to the server.

[1555] Step 7:

[1556] The server integrates and manages the collected progress data and emotional data, and dynamically adjusts the feedback content, thereby personalizing the user experience and providing optimal support at all times.

[1557] In this way, throughout each step, the server generates and provides feedback in real time according to the user's emotions and progress, realizing a system that improves work efficiency and user satisfaction.

[1558] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1559] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1560] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1561] [Fourth embodiment]

[1562] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1563] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1564] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1565] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1566] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1567] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1568] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1569] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1570] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1571] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1572] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1573] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1574] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1575] This invention is a system that supports users in the process of creating traditional Japanese crafts at home in real time. This system consists of three main components: a server, a terminal, and a user. Below, we will explain how to specifically implement this system.

[1576] User Interface (UI) Embodiments

[1577] When a user accesses the platform, they are first shown a login page. After the user enters their login information, that information is sent from the device to the server. The server retrieves the user's profile information from the database, generates a dashboard with appropriate content, and sends it back to the user. The dashboard displays the user's past crafting history and a list of recommended crafts.

[1578] View the list of recommended crafts

[1579] The server generates a list of suitable traditional crafts based on the user's preferences and past history and sends it to the device. The device displays this list on the UI and prompts the user to select the crafts they are interested in. Once the user selects a craft, the information is sent back to the server.

[1580] Providing necessary materials and tools

[1581] The server retrieves from the database a list of materials and tools needed for the selected craft and returns it to the terminal, which displays this information to the user to help them prepare to begin crafting.

[1582] Step-by-step guide and user progress record

[1583] Once the user starts crafting, the device provides a UI that tracks the progress of each step. As the user completes a step, the progress information is reported on the UI and sent to the server.

[1584] Real-time feedback

[1585] The server analyzes the received progress information in real time and uses artificial intelligence to generate feedback, including recommended next steps and technical advice, which is then sent to the device and displayed to the user.

[1586] Personalized learning support

[1587] The server adjusts the program to provide personalized learning support according to each user's skill level and language. It also introduces a multilingual system to provide an easy-to-use environment for users overseas.

[1588] Community interaction features

[1589] The device provides a community forum to encourage interaction between users, where users can share their crafting progress and finished products and receive comments and advice from other users. This feature allows users to deepen their interactions with other craft enthusiasts.

[1590] Additional Support

[1591] If users have any questions or problems during production, they can receive real-time support through the system's Q&A section or chat function. The server will forward these questions to the AI ​​or support team for quick answers.

[1592] In this way, this system supports users in efficiently learning traditional craft techniques from home and passing on these techniques widely across regional and language barriers.

[1593] The processing flow will be explained below.

[1594] Step 1:

[1595] The user accesses the platform's login page and enters their login information (user ID and password).

[1596] Step 2:

[1597] The terminal sends the login information entered by the user to the server.

[1598] Step 3:

[1599] The server receives the login information and retrieves the user's profile information from the database, including past crafting history and preferences (e.g., favorite craft categories).

[1600] Step 4:

[1601] Based on the profile information acquired by the server, a list of recommended traditional crafts is generated and sent to the terminal.

[1602] Step 5:

[1603] The device displays a list of received crafts in the UI and prompts the user to select one.

[1604] Step 6:

[1605] The user selects the craft of interest from the list.

[1606] Step 7:

[1607] The terminal transmits the user's selection information to the server.

[1608] Step 8:

[1609] The server retrieves from the database a list of materials and tools required for the selected craft and transmits the list to the terminal.

[1610] Step 9:

[1611] The list of materials and tools received by the terminal is displayed on the UI, and the user is prompted to confirm the necessary preparations.

[1612] Step 10:

[1613] The user prepares for production and starts making the craft by following the instructions on the screen.

[1614] Step 11:

[1615] The device records the progress of each step and provides a UI that allows the user to report their progress.

[1616] Step 12:

[1617] Each time the user completes a step, they press a "Step Completed" button to report their progress.

[1618] Step 13:

[1619] The terminal sends progress information to the server.

[1620] Step 14:

[1621] The server stores the progress information received and uses artificial intelligence to generate feedback, including recommended next steps and technical advice.

[1622] Step 15:

[1623] The server transmits the generated feedback to the terminal.

[1624] Step 16:

[1625] The device receives feedback and displays it in the UI, informing the user of next steps and areas for improvement.

[1626] Step 17:

[1627] The server will adjust the program to provide personalized learning support according to the user's skill level and language. A multilingual system will also be introduced to provide an environment that is easy for overseas users to use.

[1628] Step 18:

[1629] The device provides a community forum to encourage interaction between users, allowing them to post their production progress and finished products.

[1630] Step 19:

[1631] If users have questions or problems during production, they can get support using the Q&A section or chat function.

[1632] Step 20:

[1633] The server forwards user questions to an artificial intelligence or support team to provide quick answers.

[1634] Step 21:

[1635] The terminal notifies the user of the received answer and provides assistance in solving the problem.

[1636] This series of processes allows users to efficiently learn traditional Japanese crafts from home, making it possible to widely pass on skills beyond regional and language barriers.

[1637] Example 1

[1638] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1639] Conventional traditional craft production support systems have had issues such as users being unable to easily obtain a list of the materials and tools they need, difficulty in recording progress in real time, a lack of immediate feedback, insufficient personalized learning support according to the user's skill level and preferences, and a lack of promotion of interaction between users. To solve these problems, a system with more advanced data processing and real-time feedback functions is needed.

[1640] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1641] In this invention, the server includes means for providing a list of materials and tools required to create a craft item selected by the user, means for recording and saving the user's progress in real time, means for generating feedback based on the user's progress using artificial intelligence, means for sending the generated feedback to the user's device and displaying it, means for acquiring login information when the user accesses the platform and generating a dashboard based on that information, means for generating and displaying a list of recommended items based on the user's preferences and past history, and means for reporting progress information each time the user completes a step and providing recommendations for the next step and technical advice using artificial intelligence, thereby enabling users to efficiently and enjoyably progress with craft creation at home.

[1642] "User" refers to a person who uses the system to create a craft.

[1643] "Crafts" refers to handcrafted items made using traditional techniques.

[1644] "Materials" refers to the materials needed to create a craft item.

[1645] "Tools" refers to the implements and equipment used in creating crafts.

[1646] "List" refers to information in the form of a list that summarizes items of materials and tools.

[1647] "Progress" refers to information indicating how much of each step of craft creation the user has completed.

[1648] "Real-time" refers to processing and display occurring immediately without delay.

[1649] "Artificial intelligence" refers to technology that uses machine learning and data analysis to automatically make judgments and predictions.

[1650] "Feedback" refers to information including evaluations and advice regarding the user's actions and status.

[1651] "Device" refers to a device used by a user, such as a computer or smartphone.

[1652] "Platform" refers to the online environment through which Users access and engage in craft creation.

[1653] "Login Information" refers to the authentication data entered by a User to access a System.

[1654] "Dashboard" refers to an interface that displays a user's profile information and activity history in a centralized manner.

[1655] "Preferences" refers to information based on a user's preferences.

[1656] "History" refers to a record of a user's past activities and behavior.

[1657] "Recommended Items" refers to a list of crafts and materials suggested based on the user's preferences and history.

[1658] A "guide" refers to information that provides instructions for a user to create a craft item.

[1659] "Advice" refers to technical advice provided to users by artificial intelligence.

[1660] "Reporting" refers to the act of notifying the server of the work that the user has done and the progress of that work.

[1661] This invention is a system that supports users in the process of creating traditional Japanese crafts at home in real time. The system mainly consists of three elements: a server, a terminal, and a user.

[1662] First, when a user accesses the platform, a login page appears on the device. The user enters their login information (username and password), which is then sent from the device to the server. The server authenticates the login information and retrieves the user's profile information from the database. As a result, the server generates a dashboard dedicated to the user, which is sent to the device and displayed. The dashboard includes the user's past crafting history and a list of recommended crafts.

[1663] The server then generates a list of suitable traditional crafts based on the user's preferences and past history, using a data analysis algorithm. The generated list is sent to the device and displayed to the user. Once the user selects a craft of interest, the information is sent back to the server. The server retrieves a list of materials and tools needed for the selected craft from the database and returns the information to the device for display.

[1664] Once the user begins crafting, the device provides a UI that records the progress of each step. Each time the user completes a step, the progress information is sent from the device to the server. The server analyzes the received progress information in real time and generates feedback using artificial intelligence (AI). The feedback includes recommendations for next steps and technical advice, which is sent to the device and displayed to the user.

[1665] The server also adjusts the program to provide personalized learning support based on each user's skill level and language. This allows for a multilingual system, making it easier for users overseas to use. For example, it can provide an interface in multiple languages, including English, French, and German.

[1666] Additionally, the device provides a community forum for users to interact with each other. Here, users can share their crafting progress and finished products and receive comments and advice from other users. This promotes communication between craft enthusiasts. Furthermore, through the Q&A section and chat function, users can receive real-time support when they have questions or problems during their crafting. The server forwards these questions to the AI ​​or support team for quick answers.

[1667] For example:

[1668] When a user logs in and tries to create "Edo Kiriko," the device retrieves past production history from the server and displays it to the user. The server then recommends "Edo Kiriko" and displays the necessary materials (glass, cutting tools, etc.). As the user progresses with the production, the device records the progress and sends it to the server. The server uses AI to generate next step guides and advice in real time and provides them to the user.

[1669] Example prompts to input to the generative AI model:

[1670] Please describe in detail the system that provides real-time support to users' progress in creating traditional Japanese crafts, including the following points:

[1671] 1. Processing flow from user login to dashboard display

[1672] 2. How to generate and display a list of recommended crafts

[1673] 3. The process of providing necessary materials and tools

[1674] 4. Step-by-step progress recording and real-time feedback

[1675] 5. Personalized learning support and multilingual support

[1676] 6. Community Interaction Features and Additional Support Details

[1677] Please also provide a detailed description of the data flow and the technologies used.

[1678] In this way, this system supports users in efficiently learning traditional craft techniques from home and passing on these techniques widely across regional and language barriers.

[1679] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1680] Step 1:

[1681] A user accesses the platform and is presented with a login page.

[1682] Specific operation: The user enters their username and password. The entered data (login information) is sent from the terminal to the server.

[1683] Input and Output:

[1684] Input: Username, Password

[1685] Data processing: Encryption of entered login information

[1686] Output: Encrypted login information

[1687] Step 2:

[1688] The server receives the login information and retrieves the user's profile information from a database.

[1689] Specific operation: The server decrypts the received login information and performs authentication. Based on the authentication results, it searches the database for the user's profile information. Based on the obtained data, it generates a dashboard and sends it to the device.

[1690] Input and Output:

[1691] Input: Encrypted login information

[1692] Data calculation: Decryption of login information, verification of user information

[1693] Output: User profile information, generated dashboard

[1694] Step 3:

[1695] The terminal displays the dashboard received from the server to the user.

[1696] Specific operation: The device renders the received HTML data and displays the user's crafting history and a list of recommended crafts in the browser.

[1697] Input and Output:

[1698] Input: Generated dashboard

[1699] Data processing: Rendering HTML data

[1700] Output: The displayed dashboard

[1701] Step 4:

[1702] The server generates a list of suitable traditional crafts based on the user's preferences and past history and sends it to the terminal.

[1703] Specific operation: The server retrieves the user's preference information and past production history from the database, runs the recommendation algorithm, generates a recommendation list in JSON format, and sends it to the device.

[1704] Input and Output:

[1705] Input: User preference information, past history

[1706] Data Computing: Running the recommendation algorithm

[1707] Output: Recommended craft list

[1708] Step 5:

[1709] The terminal displays a list of recommended crafts to the user and prompts the user to select a craft they are interested in. Once the user selects a craft, the information is sent back to the server.

[1710] Specific operation: The device renders the received recommendation list on the UI, and when the user presses the select button, it sends the ID of the selected craft to the server.

[1711] Input and Output:

[1712] Input: Recommended Craft List

[1713] Data processing: UI display of recommendation list

[1714] Output: ID of the selected craft

[1715] Step 6:

[1716] The server retrieves from the database a list of materials and tools required for the selected craft and transmits it to the terminal.

[1717] Specific operation: The server searches the database based on the acquired craft ID, generates a list of required materials and tools, and sends it to the terminal in JSON format.

[1718] Input and Output:

[1719] Input: ID of selected craft

[1720] Data calculation: database search, list generation

[1721] Output: List of materials and tools

[1722] Step 7:

[1723] The terminal displays a list of materials and tools to the user.

[1724] Specific operation: The device renders the received list on the UI and helps the user prepare to start production.

[1725] Input and Output:

[1726] Input: List of materials and tools

[1727] Data processing: List UI display

[1728] Output: List of displayed materials and tools

[1729] Step 8:

[1730] Once the user starts crafting, the device provides a UI that tracks the progress of each step. Each time the user completes a step, the device sends that progress information to the server.

[1731] What it does: The device displays a progress tracker, allows the user to click a button to complete each step, and sends progress information to the server when a step is completed.

[1732] Input and Output:

[1733] Input: User progress information

[1734] Data processing: recording and sending progress information

[1735] Output: Progress information sent to the server

[1736] Step 9:

[1737] The server analyzes the received progress information in real time and generates feedback using artificial intelligence (AI).

[1738] What it does: The server uses AI models to analyze progress data, generate next step recommendations and technical advice, and send the generated feedback to the device.

[1739] Input and Output:

[1740] Input: User progress information

[1741] Data calculation: Analysis using AI models, feedback generation

[1742] Output: Generated feedback

[1743] Step 10:

[1744] The terminal displays the generated feedback to the user.

[1745] Specific operation: The device renders the feedback data received from the server on the UI and displays it to the user.

[1746] Input and Output:

[1747] Input: Generated feedback

[1748] Data processing: UI display of feedback data

[1749] Output: Displayed feedback

[1750] Step 11:

[1751] The server provides personalized learning support based on each user's skill level and language usage, providing a multilingual environment.

[1752] What happens: The server selects appropriate content and feedback based on the user's profile data, and translates it if necessary.

[1753] Input and Output:

[1754] Input: User profile data

[1755] Data Operations: Content Selection and Translation

[1756] Output: Personalized learning assistance

[1757] Step 12:

[1758] The device provides a community forum for users to interact with each other.

[1759] Specific operation: The device has a forum function and displays an interface where users can post and comment. Posts and comments are sent to the server and displayed to other users.

[1760] Input and Output:

[1761] Input: User posts and comments

[1762] Data processing: sending and displaying posts and comments

[1763] Output: Posts and comments that are visible to other users

[1764] Step 13:

[1765] If users have any questions or issues, they can get real-time support through the Q&A section or chat function. The server will forward these questions to the AI ​​or support team for quick answers.

[1766] Specific operation: The device displays a question form or chat window and sends the user's question to the server, which then forwards the question to the appropriate support system, generates an answer, and sends it to the device.

[1767] Input and Output:

[1768] Input: User question

[1769] Data Computation: Question Transfer and Answer Generation

[1770] Output: The answer sent to the user

[1771] (Application example 1)

[1772] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1773] Conventional support systems for traditional craft creation are primarily used at home, making it difficult to provide appropriate real-time support during workshops or craft classes held in brick-and-mortar stores. Furthermore, there is a lack of means to provide detailed support information and feedback for each step of a user's craft creation process in a store, making it difficult for users to create crafts efficiently and with high quality.

[1774] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1775] In this invention, the server includes means for providing a list of materials and tools required to create a traditional craft selected by the user, means for recording and saving the user's progress in real time, means for generating feedback using artificial intelligence based on the user's progress, means for sending the generated feedback to the user's terminal and displaying it, means for guiding the user through the production procedure in real time via a smart device, and means for providing support information for each step as the user produces the traditional craft in the store, thereby enabling the user to produce the traditional craft in the store with high efficiency.

[1776] The "means for providing a list of materials and tools required to create a traditional craft selected by the user" is a system for obtaining from a server specific items of materials and tools required for the traditional craft the user wishes to create and displaying them on the user's terminal.

[1777] "Means for recording and saving the user's progress in real time" refers to a system that has the function of recording the user's progress as they complete each step of production and saving that information on a server.

[1778] "Means for generating feedback using artificial intelligence based on the user's progress" refers to a function that analyzes the user's current production progress and generates feedback such as recommendations for next steps and improvements using an artificial intelligence algorithm.

[1779] "Means for transmitting and displaying the generated feedback to the user's terminal" refers to a system that notifies the user's terminal of feedback information generated by artificial intelligence and displays it in real time.

[1780] "Means for guiding users through the production procedure in real time via a smart device" refers to a system for providing step-by-step guidance through the production procedure via a smart device worn by the user (such as smart glasses or a head-mounted display).

[1781] "Means for providing support information for each step when a user is creating something in a store" refers to a system that provides the support information needed for each step (e.g., how to prepare materials, how to use tools, etc.) in real time when a user is performing a production activity in a physical store.

[1782] This invention is a system that supports users in the process of creating traditional Japanese crafts in real time in a brick-and-mortar store. The system consists of three main components: a server, a terminal (e.g., smart glasses), and a user. Below, we will explain how to specifically implement this system.

[1783] User Interface (UI) Embodiments

[1784] When a user accesses the platform, they are first shown a login page. After the user enters their login information, that information is sent from the device (smart glasses) to the server. The server retrieves the user's profile information from the database, generates a dashboard with appropriate content, and sends it back to the user. The dashboard displays the user's past crafting history and a list of recommended crafts.

[1785] View the list of recommended crafts

[1786] The server generates a list of suitable traditional crafts based on the user's preferences and past history and sends it to the device. The device displays this list on the UI and prompts the user to select the crafts they are interested in. Once the user selects a craft, the information is sent back to the server.

[1787] Providing necessary materials and tools

[1788] The server retrieves from the database a list of materials and tools needed for the selected craft and returns it to the terminal, which displays this information to the user to help them prepare to begin crafting.

[1789] Step-by-step guide and user progress record

[1790] Once the user starts crafting, the device (smart glasses) provides a UI that records the progress of each step. Each time the user completes a step, the progress information is reported on the UI and sent to the server.

[1791] Real-time feedback

[1792] The server analyzes the received progress information in real time and uses artificial intelligence to generate feedback, including recommended next steps and technical advice, which is then sent to the device and displayed to the user.

[1793] In-store guide function

[1794] When users create crafts in the store, they are guided through each step in real time via a device (smart glasses). This guide includes audio guidance and visual instructions to help users complete the work smoothly.

[1795] Specific examples

[1796] For example, suppose a user selects "lacquerware" in a physical store. The server retrieves a list of the necessary materials and tools (lacquerware, wood, brushes, knives, etc.) and displays it on the user's smart glasses. Once the user starts making the product, the smart glasses provide real-time guidance on how to prepare the materials and the work procedures for each step. In addition, each time the user reports their progress, feedback and advice on the next steps is provided.

[1797] Prompt Sentence Examples

[1798] "What should I pay attention to in the next step?"

[1799] "What's the best way to treat sumac?"

[1800] This allows users to create crafts in their physical stores with high efficiency and high quality.

[1801] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1802] Step 1:

[1803] User Login

[1804] Input: User ID and password

[1805] Process: The user accesses the login page and enters their login information. The device sends this information to the server.

[1806] Output: The server authenticates the user and retrieves their profile information.

[1807] Step 2:

[1808] Viewing the Dashboard

[1809] Input: User profile information

[1810] Processing: The server generates a dashboard containing appropriate content based on the profile information, past production history, and preferences.

[1811] Output: The terminal displays the user's dashboard, which includes past crafting history and a list of recommended crafts.

[1812] Step 3:

[1813] Generate and display a list of recommended crafts

[1814] Input: User preferences and past history

[1815] Processing: The server generates a list of suitable traditional crafts based on preferences and past history and sends it to the device.

[1816] Output: A list of recommended traditional crafts will be displayed on the device.

[1817] Step 4:

[1818] Craft Selection

[1819] Input: Recommended Craft List

[1820] Process: The user selects the artifact of interest and sends the information from the device to the server.

[1821] Output: The server receives the selected craft information.

[1822] Step 5:

[1823] Providing a list of materials and tools needed

[1824] Input: Selected craft information

[1825] Processing: The server retrieves the list of materials and tools required for the selected craft from the database and sends it to the terminal.

[1826] Output: A list of ingredients and tools will be displayed on the terminal.

[1827] Step 6:

[1828] Getting started and step-by-step guide

[1829] Input: Materials and tools list, crafting step guide

[1830] Process: The user starts the creation, and the device (smart glasses) provides a UI that records the progress of each step.

[1831] Output: A step-by-step build guide is displayed on the terminal, and the user reports their progress.

[1832] Step 7:

[1833] Tracking progress and generating feedback

[1834] Input: User progress information

[1835] Processing: The server analyzes the received progress information in real time and generates feedback using artificial intelligence.

[1836] Output: The generated feedback information is sent to the terminal and displayed to the user.

[1837] Step 8:

[1838] Real-time guide in physical stores

[1839] Input: User's production progress information

[1840] Processing: The server provides real-time step-by-step guidance through the smart glasses based on the user's progress in the physical store.

[1841] Output: Guidance information for each production step is displayed on the user's smart glasses.

[1842] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1843] This invention is a system that provides real-time support for users to efficiently learn and create traditional Japanese crafts at home. The system incorporates an emotion engine that recognizes the user's emotions and appropriately adjusts feedback based on those emotions. The following describes how to specifically implement this system.

[1844] User Interface (UI) Embodiments

[1845] A user accesses the platform and logs in. The login information is sent from the device to the server, which then retrieves the user's profile information from the database. Based on this information, a list of recommended traditional crafts is generated and sent to the device. The device then displays this list in the UI and prompts the user to make a selection.

[1846] View and select from a list of recommended crafts

[1847] When a user selects a craft, the selection information is sent from the terminal to the server. The server retrieves a list of materials and tools required for the selected craft from a database and sends it to the terminal. The terminal displays this list to the user and supports them in preparing for the craft.

[1848] Production progress record and real-time feedback

[1849] Once the user begins crafting, the device provides a UI that tracks progress at each step. Each time progress is reported, the information is sent to the server, which stores the progress information and uses artificial intelligence to generate feedback. The feedback includes recommendations for next steps and technical advice, which is sent to the device and displayed to the user.

[1850] Introducing the Emotion Engine

[1851] To recognize the user's emotions, the device is equipped with a camera and microphone, which can analyze the user's facial expressions, tone of voice, input text, etc. The recognized emotional information is sent to the server in real time.

[1852] Emotion-based feedback regulation

[1853] The server uses the emotional information sent by the emotion engine to tailor the feedback it generates, for example providing detailed and helpful advice if the user is confused, or concise next step instructions if the user is focused on a task.

[1854] Personalized learning support and community interaction features

[1855] The server will adjust to provide personalized learning support according to the user's skill level and language. It will also introduce a multilingual system to provide an easy-to-use environment for users overseas.

[1856] The device provides a community forum to encourage interaction between users, where users can share their progress and finished products and receive comments and advice from other users.

[1857] Additional Support

[1858] If users have any questions or problems during production, they can get support through the Q&A section or chat function. The server forwards these questions to the AI ​​or support team, who will provide a quick answer. The answer received is then notified to the user via their device.

[1859] In this way, this system is extremely useful for users to thoroughly learn traditional craft techniques at home and to pass on these techniques widely across regional and language barriers. Furthermore, the introduction of an emotion engine provides more effective and personalized feedback, further improving user satisfaction and learning efficiency.

[1860] The processing flow will be explained below.

[1861] Step 1:

[1862] The user accesses the platform and enters their login information (user ID and password).

[1863] Step 2:

[1864] The terminal sends the login information entered by the user to the server.

[1865] Step 3:

[1866] The server receives the login information and retrieves the user's profile information from the database, including past production history and preferences.

[1867] Step 4:

[1868] Based on the profile information acquired by the server, a list of recommended traditional crafts is generated and sent to the terminal.

[1869] Step 5:

[1870] The device displays a list of received crafts in the UI and prompts the user to select one.

[1871] Step 6:

[1872] The user selects the craft of interest from the list.

[1873] Step 7:

[1874] The terminal transmits the user's selection information to the server.

[1875] Step 8:

[1876] The server retrieves from the database a list of materials and tools required for the selected craft and transmits the list to the terminal.

[1877] Step 9:

[1878] The list of materials and tools received by the terminal is displayed on the UI, and the user is prompted to confirm the necessary preparations.

[1879] Step 10:

[1880] The user prepares for production and starts making the craft by following the instructions on the screen.

[1881] Step 11:

[1882] The device records the progress of each step and provides a UI that allows the user to report their progress.

[1883] Step 12:

[1884] Each time the user completes a step, they press a "Step Completed" button to report their progress.

[1885] Step 13:

[1886] The terminal transmits the user's progress information to the server.

[1887] Step 14:

[1888] The device analyzes the user's facial expressions and voice in real time through a camera and microphone, and transmits the emotional information to a server.

[1889] Step 15:

[1890] The server stores the received progress and emotion information and uses artificial intelligence to generate feedback, including recommended next steps and technical advice.

[1891] Step 16:

[1892] The server tailors the feedback based on the user's emotional state, for example adding detailed explanations if the user is confused or providing concise instructions if the user is making good progress.

[1893] Step 17:

[1894] The server transmits the generated feedback to the terminal.

[1895] Step 18:

[1896] The device receives feedback and displays it in the UI, informing the user of next steps and areas for improvement.

[1897] Step 19:

[1898] The server will adjust the program to provide personalized learning support according to the user's skill level and language. A multilingual system will also be introduced to provide an environment that is easy for overseas users to use.

[1899] Step 20:

[1900] The device provides a community forum to encourage interaction between users, allowing them to post their production progress and finished products.

[1901] Step 21:

[1902] If users have questions or problems during production, they can get support using the Q&A section or chat function.

[1903] Step 22:

[1904] The server forwards user questions to an artificial intelligence or support team to provide quick answers.

[1905] Step 23:

[1906] The terminal notifies the user of the received answer and provides assistance in solving the problem.

[1907] This series of processes allows users to efficiently learn traditional Japanese crafts from home, enabling the skills to be passed down widely across regions and language barriers.In addition, the introduction of an emotion engine provides more effective and personalized feedback, further improving user satisfaction and learning efficiency.

[1908] Example 2

[1909] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1910] Previous traditional craft learning systems lacked adequate support for users to efficiently learn and create crafts at home. Furthermore, they lacked a system that recognized users' emotions and technical difficulties in real time and provided appropriate feedback, resulting in problems that reduced learning efficiency and satisfaction. Furthermore, they lacked multilingual support and user-to-user interaction functions, making them insufficiently popular with overseas users.

[1911] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1912] In this invention, the server includes: means for providing information on materials and tools needed to create a craft selected by the user; means for recording and saving the user's progress in real time; means for generating feedback using artificial intelligence based on the user's progress; means for transmitting and displaying the generated feedback to the user's device; and means for recognizing the user's emotions and adjusting the feedback based on those emotions. This allows users to efficiently learn traditional crafts from home and receive personalized feedback, improving learning efficiency and satisfaction. Furthermore, multilingual support and user interaction functions enable the service to be popularized among overseas users.

[1913] "Users" refer to individuals who use this system to learn how to create traditional crafts and actually create them.

[1914] A "terminal" is a device that a user uses to access and use the system, and includes, for example, electronic devices such as computers, tablets, and smartphones.

[1915] "Server" refers to a central computing device that manages user login information, progress, emotional data, etc., and generates and provides necessary feedback.

[1916] "Crafts" are objects that users can learn about and create, and refer to handicrafts such as traditional Japanese crafts.

[1917] "Materials and tool information" refers to a list of the specific materials and tools needed to create the craft selected by the user.

[1918] "Progress" refers to the step a user is currently at in the craft creation process and a record of the tasks they have completed.

[1919] "Artificial intelligence" refers to a computer program or system that generates appropriate feedback based on data such as a user's progress.

[1920] "Feedback" refers to information such as advice, guidance, and next step instructions that a user can use as a reference when creating a craft.

[1921] "Emotion recognition" refers to technology that analyzes a user's facial expressions, tone of voice, and input text to understand their emotional state at that time.

[1922] "Feedback adjustment" refers to changing the content and level of detail of the feedback provided based on the user's emotional information.

[1923] "Personalized learning support" refers to a function that provides learning support optimized according to the user's skill level and language used.

[1924] "Community interaction function" refers to an online platform where users can share their production progress and finished products and receive comments and advice.

[1925] This invention is a system that supports users in the process of efficiently learning and creating traditional crafts at home in real time. The system includes an emotion engine that recognizes the user's emotions and adjusts feedback based on those emotions. The following describes how to specifically implement this system.

[1926] User Interface Embodiments

[1927] A user accesses the platform using a device and logs in. The login information is sent from the device to the server. The server retrieves the user's profile information from the database, generates a list of recommended traditional crafts based on this information, and sends it to the device. The device then displays this list on the UI and prompts the user to make a selection.

[1928] As a concrete example, when a user logs in, a list of traditional Japanese crafts is displayed on the terminal.

[1929] Example prompt: "When the user presses the login button"

[1930] View and select from a list of recommended crafts

[1931] When a user selects a craft, the selection information is sent from the terminal to the server. The server retrieves information on the materials and tools required for the selected craft from a database and sends it to the terminal. The terminal displays this information to the user and supports them in preparing for the craft.

[1932] As a specific example, when a user selects "Japanese paper doll," a list of materials and tools required to create it is displayed on the terminal.

[1933] Example prompt: "Show a list of materials needed for the selected craft."

[1934] Production progress record and real-time feedback

[1935] It provides a UI for users to start crafting and record their progress at each step. Each time progress is reported, the information is sent from the device to the server. The server stores the progress information and generates feedback using a generative AI model. The generated feedback includes recommendations for next steps and technical advice, and is sent to the device and displayed to the user.

[1936] For example, when a user completes step 3, they are given specific instructions and advice on how to proceed to the next step.

[1937] Example prompt: "Generate next step feedback based on your progress."

[1938] Emotion recognition by emotion engine

[1939] The device is equipped with a camera and microphone, which allows it to analyze the user's facial expressions and tone of voice, and this data is sent to a server in real time.

[1940] As a specific example, when a user shows a confused expression, the expression is captured and transmitted to the server as emotion data.

[1941] Example prompt: "Recognize user emotions in real time."

[1942] Adjusting Feedback Based on Emotions

[1943] The server analyzes the received emotional information and generates appropriate feedback: detailed advice if the user is confused, or concise instructions if the user is distracted.

[1944] As a concrete example, confused users are provided with detailed guidelines for next steps.

[1945] Example prompt: "Adjust feedback based on user emotion."

[1946] Personalized learning support and community interaction features

[1947] The server provides personalized learning support according to the user's skill level and language, and a multilingual system is implemented. The terminals provide a community forum to encourage interaction between users. Users can share their progress and finished products here and receive comments and advice from other users.

[1948] As a concrete example, users can post their creations to the community and receive feedback.

[1949] Example prompt: "The ability to share progress and finished products."

[1950] Additional Support

[1951] If users have any questions or problems during production, they can get support through the Q&A section or chat function. The server forwards these questions to the AI ​​or support team, who will provide a quick answer. The answer received is then notified to the user via their device.

[1952] For example, a user can enter a question and within a few minutes the AI ​​will provide a solution.

[1953] Example prompt: "Submit support questions and get answers in real time."

[1954] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1955] Step 1: Submit login information

[1956] A user accesses the login screen using a terminal and enters a username and password. The terminal sends this login information to the server. When the login information is sent to the server, the server accesses the database and performs authentication. If the authentication is successful, the server obtains the user's profile information and sends it to the terminal.

[1957] Input: Username, Password

[1958] Output: User profile information

[1959] Specific operation:

[1960] The user enters their username and password in the login form and clicks the Login button.

[1961] The device sends the login information to the server.

[1962] Step 2: Obtain profile information and provide a list of crafts

[1963] The server generates a list of recommended traditional crafts based on the profile information of the successfully authenticated user. The server then sends this list to the device. The device then displays the received craft list on a user interface and prompts the user to make a selection.

[1964] Input: User profile information

[1965] Output: Recommended craft list

[1966] Specific operation:

[1967] The server retrieves the user's profile information from the database.

[1968] The server generates a list of crafts based on the profile information and sends it to the terminal.

[1969] Step 3: Select a craft and get material information

[1970] The user selects the craft they want to create from the craft list. The device sends this selection information to the server. The server retrieves information about the materials and tools required for the selected craft from the database and sends it to the device. The device displays this information to the user.

[1971] Input: Craft selection information

[1972] Output: List of materials and tools needed

[1973] Specific operation:

[1974] The user selects "Japanese paper doll" from the crafts list.

[1975] The terminal transmits the selection information to the server.

[1976] Step 4: Record your progress and submit

[1977] The user starts creating a craft and records the progress of each step. The device sends the user's progress information in real time to the server, which stores the received progress information and uses it to generate feedback about the next step.

[1978] Input: User progress information

[1979] Output: Feedback for next steps

[1980] Specific operation:

[1981] The user completes production step 3 and records their progress on the device.

[1982] The terminal sends progress information to the server.

[1983] Step 5: Generate and display feedback as you progress

[1984] The server uses the generative AI model to generate appropriate feedback based on the received progress information. The generated feedback includes recommendations for next steps and technical advice. The server then sends the generated feedback to the device, which then displays it to the user.

[1985] Input: User progress information

[1986] Output: Feedback

[1987] Specific operation:

[1988] The server analyzes the progress information and generates feedback on next steps.

[1989] The device displays the feedback to the user.

[1990] Step 6: Collect and send emotion recognition data

[1991] The device is equipped with a camera and microphone to analyze the user's facial expressions and tone of voice, and the recognized emotional information is sent to a server in real time.

[1992] Input: User's facial expression data, voice data

[1993] Output: Emotional information

[1994] Specific operation:

[1995] The device's camera captures the user's facial expressions and analyzes the emotional data.

[1996] The device transmits the emotion information to the server.

[1997] Step 7: Adjusting Emotional Feedback

[1998] The server analyzes the received emotional information and adjusts the generated feedback content, providing detailed advice if the user is confused or simple instructions if the user is focused. The adjusted feedback is then sent to the device, which displays it to the user.

[1999] Input: Emotion information, progress information

[2000] Output: Regulated Feedback

[2001] Specific operation:

[2002] The server analyzes the emotional information and adjusts the feedback.

[2003] The device will display adjusted feedback.

[2004] Step 8: Providing personalized assistance and community features

[2005] The server provides support tailored to the user's skill level and language, and the terminal provides a community forum to encourage interaction between users, where they can share their progress and finished products and receive feedback.

[2006] Input: User profile information, skill level, preferred language

[2007] Output: personalized assistance, community posts

[2008] Specific operation:

[2009] Users post their creations to the community and receive comments from other users.

[2010] The server provides support according to the user's technical level.

[2011] Step 9: Providing real-time support

[2012] If users have questions or problems during production, they can get support through the Q&A section or chat function. The server forwards these questions to the AI ​​or support team, which generates a quick answer. The answer is then sent to the user via their device.

[2013] Input: User question

[2014] Output:Answer

[2015] Specific operation:

[2016] The user types their question into the chat box.

[2017] The server generates an answer using AI and sends it to the device.

[2018] (Application example 2)

[2019] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2020] In modern factories, when humans and robots work together, it is difficult for the robot to provide appropriate feedback based on the worker's emotions and progress. Systems are needed that can instantly recognize when a worker becomes confused or stressed and adjust the work environment and support provided. Furthermore, existing systems lack the ability to provide personalized support that adapts to the worker's progress and emotions, preventing improvements in work efficiency and satisfaction.

[2021] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for providing a list of materials and tools necessary to create a traditional craft selected by the user; means for recording and saving the user's progress in real time; means for generating feedback using artificial intelligence based on the user's progress; means for sending the generated feedback to the user's terminal and displaying it; and means for recognizing emotions using a camera and microphone and adjusting the feedback content based on the emotions. This makes it possible to provide appropriate feedback according to the worker's emotions and progress in collaborative work between humans and robots in a factory, thereby improving work efficiency and satisfaction.

[2022] A "terminal" is a device used by a user, and includes a computer, smartphone, tablet, etc.

[2023] "Server" means a computer system that manages and processes information, records user data and progress, and generates and sends feedback.

[2024] "Artificial intelligence" refers to algorithms and systems that mimic human intelligence, analyze data, make decisions, and use that technology to generate feedback.

[2025] "Means for recognizing emotions" refers to technology that uses input devices such as cameras and microphones to analyze a user's facial expressions and voice and determine their emotional state.

[2026] "Feedback" refers to information that provides appropriate advice or instructions based on the user's progress and emotions.

[2027] "Personalized learning support" refers to a system that provides individualized learning support tailored to the user's skill level and language.

[2028] "Community interaction function" refers to a system function that allows users to share information and communicate with each other.

[2029] "Traditional crafts" refer to handicrafts made using traditional Japanese techniques and methods.

[2030] This invention is a system for supporting collaborative work between humans and robots in factories, and aims to appropriately adjust feedback based on the user's emotions and progress. Specific embodiments of this system will be described below.

[2031] System Overview

[2032] The server provides a list of materials and tools required based on the task selected by the user, records and saves the user's progress in real time, and uses artificial intelligence to generate feedback based on the user's progress, which is then sent to the user's device for display.

[2033] A key part of the system is that it uses a camera and microphone to recognize the user's emotions and adjusts the feedback content based on those emotions.

[2034] Hardware and software used

[2035] For hardware, we use a camera (webcam), microphone, and speaker. For software, we use the following:

[2036] OpenCV: Image processing

[2037] DeepFace: Emotion recognition

[2038] SpeechRecognition: Speech recognition

[2039] gTTS: speech generation

[2040] Data processing and calculation methods

[2041] Acquiring camera data

[2042] The server uses a camera to capture the user's facial image in real time, which then initiates the process of recognizing emotions from the user's facial expressions. Specifically, the server processes the image using OpenCV and analyzes emotions using DeepFace.

[2043] Speech recognition

[2044] The server uses a microphone to capture the user's voice, converts the voice data into text, and uses the SpeechRecognition library to recognize the voice and analyze what the user is saying.

[2045] Generate feedback

[2046] The server generates appropriate feedback based on the emotion and speech recognition results. Specifically, if the user is confused, it provides detailed explanations and assistance, and if the user is satisfied, it sends an appropriate message of encouragement. The feedback message is converted into speech using gTTS and played back through the speaker.

[2047] Specific examples and prompts for the generative AI model

[2048] Specific examples

[2049] If the worker looks confused, the server generates a voice message saying, "Are you OK? May I help you?" and plays it through the speaker. If the user wants to know how to use a new tool while working, the server responds to the prompt, "Please tell me how to use that tool," by providing detailed instructions and advice.

[2050] Prompt Sentence Examples

[2051] "Turn the following text into speech: Are you OK? Can I help you?"

[2052] In this way, this system can recognize the user's emotions in real time and provide optimal feedback in response, thereby improving work efficiency and user satisfaction.In addition, by providing personalized learning support and community interaction functions, it also contributes to improving users' skills and forming communities.

[2053] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2054] Step 1:

[2055] The server uses a camera to capture images of the user's face in real time. The input data obtained from the camera is an image of the user's face. This image data is then processed using OpenCV and converted into an analyzable format.

[2056] Step 2:

[2057] The server uses DeepFace to analyze the facial video data obtained in step 1 and recognize the user's emotions. As a result of the analysis, the main emotions (e.g., confusion, joy) are output. This analysis result is stored in the server and used to generate feedback in the next step.

[2058] Step 3:

[2059] The server uses a microphone to capture the user's voice. The input data is the user's voice. This voice data is then recognized using SpeechRecognition and converted into text data. The converted text data is used to generate feedback in the next step.

[2060] Step 4:

[2061] The server generates appropriate feedback content based on the emotion data acquired in step 2 and the text data acquired in step 3. Specifically, it uses a generative AI model to create feedback text that corresponds to the user's condition and request. For example, if the user is confused, it generates the message "Are you okay? May I help you?"

[2062] Step 5:

[2063] The server uses gTTS to convert the generated feedback text into speech. The input data is the feedback text obtained in step 4, and the output data is an audio file. The server plays this audio file through a speaker to provide feedback to the user.

[2064] Step 6:

[2065] The device records the user's progress in real time and sends the data to the server. The input data is progress information based on the user's operations and actions, and the output data is saved to the server.

[2066] Step 7:

[2067] The server integrates and manages the collected progress data and emotional data, and dynamically adjusts the feedback content, thereby personalizing the user experience and providing optimal support at all times.

[2068] In this way, throughout each step, the server generates and provides feedback in real time according to the user's emotions and progress, realizing a system that improves work efficiency and user satisfaction.

[2069] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[2070] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2071] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[2072] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[2073] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[2074] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[2075] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[2076] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[2077] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[2078] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[2079] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[2080] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[2081] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[2082] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[2083] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[2084] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[2085] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[2086] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[2087] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[2088] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[2089] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[2090] The following is further disclosed regarding the above embodiment.

[2091] (Claim 1)

[2092] means for providing a list of materials and tools required to create a traditional craft selected by the user;

[2093] a means for recording and storing the user's progress in real time;

[2094] means for generating feedback based on the user's progress using artificial intelligence;

[2095] means for transmitting the generated feedback to a user's terminal and displaying the feedback;

[2096] A system including:

[2097] (Claim 2)

[2098] 2. The system according to claim 1, which provides personalized learning support according to the user's skill level and language.

[2099] (Claim 3)

[2100] 10. The system according to claim 1, further comprising a function for user community interaction.

[2101] "Example 1"

[2102] (Claim 1)

[2103] means for providing a list of materials and tools required to create a craft item selected by the user;

[2104] a means for recording and storing the user's progress in real time;

[2105] means for generating feedback based on the user's progress using artificial intelligence;

[2106] means for transmitting and displaying the generated feedback to a user device;

[2107] A means of obtaining login information when a user accesses the platform and generating a dashboard based on that information;

[2108] means for generating and displaying a list of recommended items based on the user's preferences and past history;

[2109] A means to report progress as the user completes each step and use artificial intelligence to provide next step recommendations and technical advice;

[2110] A system including:

[2111] (Claim 2)

[2112] 10. The system according to claim 1, which provides personalized learning support according to the user's skill level and language used, and provides a multilingual environment.

[2113] (Claim 3)

[2114] 10. The system of claim 1, which provides a forum for promoting interaction between users, allowing users to share their progress and work.

[2115] "Application Example 1"

[2116] (Claim 1)

[2117] means for providing a list of materials and tools required to create a traditional craft selected by the user;

[2118] a means for recording and storing the user's progress in real time;

[2119] means for generating feedback based on the user's progress using artificial intelligence;

[2120] means for transmitting the generated feedback to a user's terminal and displaying the feedback;

[2121] A means for guiding a user through the production procedure in real time via a smart device;

[2122] A means for providing step-by-step support information when a user makes a product in a store;

[2123] A system including:

[2124] (Claim 2)

[2125] 2. The system according to claim 1, which provides personalized learning support according to the user's skill level and language.

[2126] (Claim 3)

[2127] 10. The system according to claim 1, further comprising a function for user community interaction.

[2128] "Example 2: Combining Emotion Engines"

[2129] (Claim 1)

[2130] means for providing information on materials and tools required to create a craft item selected by the user;

[2131] a means for recording and storing the user's progress in real time;

[2132] means for generating feedback based on the user's progress using artificial intelligence;

[2133] means for transmitting the generated feedback to a user's terminal and displaying the feedback;

[2134] means for recognizing a user's emotion and adjusting feedback based on the emotion;

[2135] A system including:

[2136] (Claim 2)

[2137] 2. The system according to claim 1, which provides personalized learning support according to the user's skill level and language.

[2138] (Claim 3)

[2139] 10. The system according to claim 1, further comprising a function for user community interaction.

[2140] "Application example 2 when combining emotion engines"

[2141] (Claim 1)

[2142] means for providing a list of materials and tools required to create a traditional craft selected by the user;

[2143] a means for recording and storing the user's progress in real time;

[2144] means for generating feedback based on the user's progress using artificial intelligence;

[2145] means for transmitting the generated feedback to a user's terminal and displaying the feedback;

[2146] A means of recognizing emotions using a camera or microphone and adjusting the feedback content based on those emotions;

[2147] A system including:

[2148] (Claim 2)

[2149] 2. The system according to claim 1, which provides personalized learning support according to the user's skill level and language.

[2150] (Claim 3)

[2151] 10. The system according to claim 1, further comprising a function for user community interaction. [Explanation of symbols]

[2152] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. means for providing a list of materials and tools required to create a traditional craft selected by the user; a means for recording and storing the user's progress in real time; means for generating feedback based on the user's progress using artificial intelligence; means for transmitting the generated feedback to a user's terminal and displaying the feedback; A system including:

2. 10. The system according to claim 1, wherein the system provides personalized learning support according to the user's skill level and language.

3. The system according to claim 1, further comprising a function for community interaction between users.

Images