System

Abstract

A system is provided.SOLUTION: A system comprising: means for receiving school timetable information input by a user terminal; means for predicting gear needed based on a particular day of the week or season; means for generating a list of gear needed; means for converting the generated list into audio data; means for transmitting the audio data to the user terminal; and means for playing the transmitted audio data at the user terminal.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] For new first graders, checking the next day's school schedule while preparing the necessary belongings can be a difficult task, and parents often need help and confirmation. This is especially true for families with working parents, who are busy with school commutes and extracurricular activities, which can cause additional stress. This invention aims to help parents and children start their new lives with peace of mind and prevent them from forgetting to prepare their belongings. [Means for solving the problem]

[0005] The present invention provides a system including means for receiving school timetable information entered by a user terminal, means for predicting what items will be needed based on specific days of the week and seasons, means for generating a list of the necessary items, means for converting the generated list into voice data, means for transmitting the voice data to the user terminal, and means for playing the transmitted voice data on the user terminal. This system is expected to facilitate the daily lives of parents and children by enabling parents and children to prepare everything they need for the next day without missing anything and by appropriately reminding them of what items will be needed based on specific days of the week and seasons.

[0006] A "user terminal" is an electronic device that allows a user to input information and perform operations such as display and audio playback.

[0007] "Timetable information" is information that indicates the schedule of daily class subjects and activities at school.

[0008] "Day of the week" is information that indicates each day of the week, and refers to a specific day such as Monday, Tuesday, or Wednesday.

[0009] "Season" is information that indicates the time of year divided into spring, summer, autumn, and winter.

[0010] "Necessary items" are items such as teaching materials and equipment needed to participate in classes and activities.

[0011] "Prediction means" is a function that estimates what you will need to bring based on timetable information and specific conditions.

[0012] The "means for generating a list" is a function that allows you to display the necessary items in a list format.

[0013] The "means for converting into voice data" is a function for converting text information into voice information.

[0014] The "transmitting means" is a communication function for transmitting the generated belongings list to the user terminal.

[0015] The "reproducing means" is a function for allowing the user to listen to the transmitted audio data. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0017] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0020] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0021] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0022] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0024] [First embodiment]

[0025] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0026] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0029] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0032] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0036] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0037] This invention is a system that uses a user terminal to assist in preparing school supplies. It generates a list of necessary supplies based on the next day's schedule and provides voice reminders, reducing the burden on children and their parents.

[0038] System implementation

[0039] Users launch a dedicated application using a user device such as a smartphone or tablet. First, a parent or child logs in to the app and sends authentication information to the server. The server receives this authentication information and authenticates the user. If authentication is successful, the user is taken to the home screen.

[0040] Entering timetable information

[0041] The user (parent or child) then enters information about the child's school into the application, including the child's name, class, and tomorrow's schedule. This data is sent to the server, where it is stored in a database.

[0042] Generate a packing list

[0043] The server analyzes the received timetable information and identifies the teaching materials and items required for each subject. It also predicts additional items to bring based on the day of the week and the season. For example, it determines that sneakers are needed on Fridays and jackets are needed in spring. This information is combined to generate the final packing list.

[0044] Generating and transmitting audio data

[0045] The inventory list is then converted into audio data using a text-to-speech (TTS) API, which the server then sends to the user's device via HTTP or HTTPS.

[0046] Reminders on user devices

[0047] The user (child) opens the application again before the next day and requests, by voice or touch, "Tell me what to prepare for tomorrow." The user's device plays the saved voice data and announces the list of items to bring. For example, the user might be reminded, "Tomorrow's items to bring are textbooks (Japanese and math), arts and crafts kit, gym clothes, and sneakers."

[0048] Application Examples

[0049] This system can be used to provide not only reminders for inventory lists, but also medication reminders and childcare reminders. For example, an elderly person can be set to be reminded to take their medicine at 6 p.m. In this case, the server generates audio data at the set time and sends it to the device. The device then plays the audio data at the set time, reminding the user to "take their medicine at 6 p.m."

[0050] Through the detailed steps described above, the present invention provides effective support for users and their guardians to prepare all necessary belongings and start their new life with peace of mind.

[0051] The processing flow will be explained below.

[0052] Step 1:

[0053] The user starts the application on their smartphone or tablet and enters their user ID and password on the login screen, which is then sent to the server.

[0054] Step 2:

[0055] The server receives the user's authentication information and checks it against a database. If authentication is successful, the login is approved and a signal is sent to the user's device to display the home screen.

[0056] Step 3:

[0057] The user enters their child's information (name, class, next day's schedule) on the home screen. The entered information is sent to the server and stored in a database.

[0058] Step 4:

[0059] The server analyzes the received timetable information. First, it identifies the teaching materials and items required for each subject. For this analysis, it uses the standard list of items to bring for each subject in the timetable (e.g., Japanese textbook, notebook, pencil case).

[0060] Step 5:

[0061] The server predicts what additional items to bring based on the day of the week, season, and weather information obtained from the weather API. For example, if it's Friday, sneakers are added to the list, and if it's a rainy day, rain gear is added.

[0062] Step 6:

[0063] The server generates a list of required items and converts it into audio data using a Text-to-Speech (TTS) API, which is then stored on the server.

[0064] Step 7:

[0065] The server sends the voice data to the user's device. The communication protocol is HTTP or HTTPS. The sent voice data is saved on the user's device.

[0066] Step 8:

[0067] To check the preparations for the next day, the user launches the application again and says "Check tomorrow's preparation list" by voice input or touch operation. The user's device detects this and plays the saved voice data.

[0068] Step 9:

[0069] The user device plays the audio data and notifies the user of the list of items to bring, allowing the user (child) to check what to bring the next day and prepare appropriately.

[0070] Step 10:

[0071] The user device can extend the reminder function to notify users of medication reminders at specific times. For example, it can play a voice reminder to an elderly person saying, "Take your medicine at 6 p.m."

[0072] The above is the specific processing flow of this system.

[0073] Example 1

[0074] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0075] In today's busy households, it is difficult to properly prepare school supplies. Especially when children prepare their own school supplies, forgetting or failing to prepare frequently causes considerable stress for both children and their parents. Furthermore, it is difficult to accurately track what school supplies should be prepared for different days of the week and seasons, so a system that automates this process and efficiently manages them is needed.

[0076] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0077] In this invention, the server includes a means for receiving timetable information of the educational institution input by the user terminal, a means for predicting items needed based on a specific day of the week or season, and a means for generating a list of needed items, thereby enabling children and their parents to efficiently prepare their belongings and prevent them from forgetting their belongings.

[0078] "User terminal" refers to a portable information device for inputting and receiving timetable information and inventory lists for educational institutions.

[0079] "Educational institutions" refers to schools, cram schools, and other educational facilities.

[0080] "Timetable information" refers to schedule information for classes and activities at an educational institution, and is a list of subjects and activities that take place on specific days and at specific times.

[0081] "Items" refers to teaching materials and tools necessary for lessons and activities.

[0082] "Predictive means" refers to the ability of a device or program to automatically determine what items are needed based on specific days of the week or seasons.

[0083] The "means for generating a list" refers to the function of a device or program that organizes predicted items and creates a list in a list format.

[0084] "Means for converting into audio data" refers to text-to-speech technology that converts the generated item list into natural audio.

[0085] The "means for transmitting voice data" refers to a communication technology for transmitting the generated voice data to a user terminal.

[0086] "Means for playing" refers to the function of a device or program for playing back audio data received on a user terminal.

[0087] "Text reminder" refers to a format that notifies you of your belongings using text information rather than audio data.

[0088] This invention is a system that uses user terminals to assist educational institutions in preparing belongings. It generates a list of necessary items based on the next day's schedule and reminds students of this list via voice, thereby reducing the burden on students and their guardians.

[0089] First, the user launches a dedicated application using a user device such as a smartphone or tablet. The user (guardian or beneficiary) enters their user ID and password on the login screen and sends them to the server. The sent authentication information is received by the server, which then collates it with information in a database to authenticate the user. If authentication is successful, the user is taken to the home screen.

[0090] Next, the user enters the recipient's name, class, and the next day's schedule information into a dedicated form in the application. This information is formatted by the user's terminal and sent to the server, which receives the information and stores it in a database.

[0091] Based on the submitted timetable information, the server identifies the items required for each subject. It then analyzes the day of the week and seasonal information to predict additional items. For example, it determines that sneakers are needed on Fridays and jackets are needed in spring. This information is then combined to generate the final item list.

[0092] The generated inventory list is converted into voice data using a text-to-speech (TTS) software API, and the server transmits the converted voice data to the user's device using the HTTP or HTTPS protocol.

[0093] The user (beneficiary) opens the application again before the next day and requests, by voice input or touch operation, "Tell me what to prepare for tomorrow." The user device plays the saved voice data and reminds the user, "Tomorrow's items to bring are textbooks (Japanese and math), arts and crafts kit, gym clothes, and sneakers."

[0094] As a specific example of how this works, for example, a list of belongings generated based on Monday's timetable would include "Japanese textbook, math notebook, gym clothes." If it's Friday, "sneakers" would be added. If it's spring, "jacket" would be added.

[0095] By using the following prompts to the generative AI model, you can elicit specific explanations of the system's processing flow.

[0096] Example prompt:

[0097] "Please explain in detail how this system reminds users of their belongings. For example, please explain in detail the process by which the server analyzes the timetable information and generates the belongings list."

[0098] Through the above process, the present invention provides effective support for the user and guardian to prepare all necessary belongings and start their new life with peace of mind.

[0099] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0100] Step 1: User launches application and logs in

[0101] The user launches the dedicated application using the user device. After launching, the user (guardian or beneficiary) enters their user ID and password on the login screen and taps the login button. The user device sends the entered authentication information to the server. The server compares the received authentication information with information in the database and authenticates the user. If authentication is successful, the server sends a success message and the user device transitions to the home screen.

[0102] Input: User ID, Password

[0103] Data processing: User ID and password verification

[0104] Output: Authentication success message

[0105] Specific operation: When the user taps the login button, authentication information is sent, the server authenticates it, and returns a success message.

[0106] Step 2: Enter your school and timetable information

[0107] The user (parent or beneficiary) enters the beneficiary's name, class, and next day's timetable information into a dedicated form in the application. The user's device formats the entered information and sends it to the server. The server receives this information and stores it in a database.

[0108] Input: Beneficiary's name, class, next day's timetable information

[0109] Data processing: Formatting and sending information

[0110] Output: Acknowledgement of receipt of information

[0111] Specific operation: When a user enters information into a dedicated form and taps the submit button, the information is sent to the server and stored in a database.

[0112] Step 3: Generate a packing list

[0113] The server analyzes the received timetable information and identifies the items needed for each subject stored in the database. It also analyzes information about the day of the week and the season to predict additional items. It then combines this information to generate the final packing list.

[0114] Input: Timetable information, day of the week information, seasonal information

[0115] Data calculation: Identifying items and predicting additional items

[0116] Output: Final inventory list

[0117] Specific operation: The server analyzes the timetable information, adds items according to the day of the week and season, and creates a list of items to bring.

[0118] Step 4: Generate and send audio data

[0119] The server converts the generated inventory list into audio data using a text-to-speech software (TTS) API, and sends the converted audio data to the user's device using HTTP or HTTPS protocol.

[0120] Input: Final packing list

[0121] Data processing: conversion to audio data

[0122] Output: Audio data

[0123] Specific operation: The server sends the inventory list to the TTS API, and the generated voice data is sent to the user's terminal.

[0124] Step 5: Remind users on their devices

[0125] The user (beneficiary) opens the application again before the next day and requests, by voice input or touch operation, "Tell me what to prepare for tomorrow." The user device plays the saved voice data and reminds the user, "Tomorrow's items to bring are textbooks (Japanese and math), arts and crafts kit, gym clothes, and sneakers."

[0126] Input: Voice or touch

[0127] Data processing: processing requests and playing audio data

[0128] Output: Voice reminder

[0129] Specific operation: When the user voice-inputs "Tell me how to prepare for tomorrow" or taps a button, the device plays the voice data as a reminder.

[0130] (Application example 1)

[0131] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0132] When shopping in a physical store, it is difficult for users to efficiently find the items they need to purchase, resulting in unnecessary time and effort. Furthermore, especially in large stores, it is not uncommon for users to miss or have difficulty finding items that they need to purchase based on their shopping list. Furthermore, there is a lack of a reminder function to ensure users don't forget to purchase the items they need. Therefore, a system that solves this problem is needed.

[0133] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0134] In this invention, the server includes means for receiving school timetable information input by a user terminal, means for predicting items needed based on specific days of the week and seasons, means for generating a list of items needed, means for converting the generated list into voice data, means for transmitting the voice data to the user terminal, means for playing the transmitted voice data on the user terminal, and means for guiding the location of items included in the list based on location information of each department in the store, thereby enabling users to shop efficiently in a physical store and purchase all necessary items.

[0135] A "user terminal" is a device such as a smartphone, tablet, or smart glasses that is directly operated by a user to input and receive information.

[0136] "School timetable information" is data that includes information about the times and contents of classes that students will take the next day and on specific days, as well as the teaching materials and items they will need to take with them.

[0137] "Day-of-the-week and season-based items" refers to items or clothing needed on specific days of the week or during specific seasons, such as sneakers needed on Fridays or a jacket needed in the spring.

[0138] The "list of necessary items" is a list of all items needed for the next day or a specific day, generated based on the user's input information and predicted day of the week and season information.

[0139] The "means for converting into audio data" is a technique for converting the inventory list in text format into audio format and generating an audio file or audio stream that can be listened to by the user.

[0140] The "means for transmitting voice data to a user terminal" refers to a communication means for transmitting voice data generated by the server to a user terminal via the Internet.

[0141] "Means for reproducing audio data at a user terminal" refers to a technique for reproducing audio data received by a user terminal and providing it to the user as auditory information.

[0142] "Location information for each department within the store" is information about where each item is displayed within the physical store, and is data indicating in which department or section a particular item is located.

[0143] "Item location guidance" refers to technology that guides users to specific locations or departments within a store to help them efficiently find items on their shopping list.

[0144] The present invention is a system for supporting efficient shopping in a physical store using a user terminal. Specific embodiments of this system will be described below.

[0145] System implementation

[0146] A user launches a dedicated shopping assistant application using a user device such as a smartphone or smart glasses. First, the user logs in to the application and sends authentication information to the server. The server receives this authentication information and performs user authentication. If authentication is successful, the user is taken to the application's home screen.

[0147] Entering a shopping list

[0148] The user enters the items they want to include in their shopping list into the application, and this data is sent to the server where it is stored in a database.

[0149] Shopping list location guide

[0150] The server analyzes the received shopping list information and identifies which department in the store each item is located in. Based on the store location information, it generates a list to guide the user to the location of the required items. The list information includes the location and department details of each item.

[0151] Generating and transmitting audio data

[0152] The generated shopping list and location information is converted into audio data using a Text-to-Speech (TTS) API, and the server generates this audio data and sends it to the user's device using HTTP or HTTPS protocols.

[0153] Reminders and guidance on user devices

[0154] The user uses the application in the store and requests, by voice or touch, "What item should I buy next?" The user's device plays the saved voice data and provides a voice prompt of the items on the list and their location. For example, the user might be reminded, "The next item to buy is milk. It's in the refrigerated section."

[0155] Specific processing

[0156] Hardware: Smartphones, smart glasses

[0157] Software: Dedicated shopping assistant application, server-side authentication system, database, Text-to-Speech (TTS) API

[0158] Data processing flow:

[0159] 1. The user logs in to the app and sends their credentials to the server.

[0160] 2. The server authenticates the user and receives and stores the shopping list data.

[0161] 3. The server analyzes the information in the list and determines the location of the item within the store.

[0162] 4. Based on the location information, the list is converted into audio data and sent to the user's device.

[0163] 5. The user terminal plays back the received voice data and provides the user with a voice guide of the next item to be purchased and its location.

[0164] Specific examples

[0165] For example, a user logs in to the application and enters "milk," "bread," and "detergent" as a list of items to be purchased next. The server uses this information to identify which section of the physical store each item is located in, and generates a voice reminder based on the location information of the refrigerated goods section, bakery section, and daily necessities section. This reminder is played on the user's device, supporting the user in efficient shopping.

[0166] Prompt Sentence Examples

[0167] Develop an application that helps users shop efficiently in physical stores. The main features are shopping list creation and voice reminders. The reminders guide users through their shopping by guiding them to the next item to be purchased and its location.

[0168] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0169] Step 1:

[0170] The user launches the application and enters authentication information on the login screen. The authentication information includes the user ID and password. The server authenticates the user based on this information. The server compares the authentication information with records in the database, and if there is a match, returns a status of successful authentication. The user's device receives a notification of successful authentication and moves to the home screen.

[0171] Step 2:

[0172] The user inputs a shopping list into the application. This list contains the names of multiple items the user plans to purchase. The input data is sent to the server as form data. The server receives this data and saves the shopping list in a database. Input: Shopping list (item names). Output: Shopping list saved in the database.

[0173] Step 3:

[0174] The server analyzes the shopping list information and identifies the location information for each item in the store. It references the location information in the database to determine which section or department each item is located in. Based on this data, it generates a list containing location information for each item. Input: Saved shopping list. Output: List containing location information for the store.

[0175] Step 4:

[0176] The server converts the list generated by the server into audio data using a Text-to-Speech (TTS) API. The list information in text format is input to the API, which generates an audio file or audio stream. Input: List containing in-store location information. Output: Audio data.

[0177] Step 5:

[0178] The voice data is sent to the user terminal using the HTTP or HTTPS protocol. The server uses a communication means to send the generated voice data to the user terminal. Input: Voice data. Output: Voice data sent to the user terminal.

[0179] Step 6:

[0180] The user reopens the application in the store and requests, by voice or touch, "Tell me what item I'll buy next." The user device receives this request and plays the saved voice data. Input: User request. Output: Played voice data (reminder).

[0181] Step 7:

[0182] The user device plays the audio data and provides audible instructions about the next item in the list and its location. For example, "The next item to purchase is milk. It's in the refrigerated section." Input: The stored audio data. Output: Audio instructions provided to the user.

[0183] keyword:

[0184] Generative AI model, prompt sentence

[0185] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0186] This invention is a system that uses a user terminal to help users prepare school supplies. It generates a list of necessary supplies based on the next day's schedule and reminds users of the list by voice and text. In addition, by combining it with an emotion engine, the content of the reminder can be flexibly adjusted according to the user's emotions.

[0187] System implementation

[0188] First, the user launches a dedicated application using a user device such as a smartphone or tablet. The user (parent or child) enters authentication information on the login screen and sends it to the server. The server authenticates the user using the received information, and if authentication is successful, the home screen is displayed on the user device.

[0189] Entering timetable information

[0190] The user enters basic information about their child (name, class), the next day's schedule, and any special notes (lessons, health, etc.) on the home screen. This information is sent to the server and stored in a database.

[0191] Generate a packing list

[0192] The server identifies the materials and supplies needed for each subject based on the timetable information. It also predicts what additional items students need to bring based on the day of the week, season, and weather information obtained from a weather API. For example, if it's Friday, sneakers are added to the list, and if it's a rainy day, rain gear is added.

[0193] Use of emotion engine

[0194] Furthermore, the app will introduce an emotion engine that analyzes the user's voice input and facial expressions. The camera captures the user's voice tone and facial expressions when they input information into the app, and the emotion engine analyzes them. Based on the analysis results, if the user is feeling anxious or stressed, the reminder content will be made more detailed or an encouraging message will be added. Conversely, if the user is feeling positive, the app will send reminders that emphasize successful experiences.

[0195] Generating and transmitting audio data

[0196] The list of necessary items is converted into voice data using a text-to-speech software (TTS) API, and the server sends the generated voice data to the user's device and stores it on the user's device.

[0197] Reminders on user devices

[0198] To check what to prepare for the next day, the user launches the application again and uses voice input or touch operations to say, "Tell me what to prepare for tomorrow." The user's device detects this and plays back the saved voice data. For example, the user may be reminded, "Tomorrow's items to bring are textbooks (Japanese and math), arts and crafts kit, gym clothes, and sneakers."

[0199] Application Examples

[0200] This system can be applied not only to reminders of inventory lists, but also to medication reminders and childcare reminders. For example, it can be set to notify an elderly person, "Please take your medicine at 6 p.m." In this case, the emotion engine can provide more detailed explanations or add encouraging messages to anxious users.

[0201] Through the detailed steps described above, the present invention provides effective support for users to prepare all their belongings and live their daily lives with peace of mind.The introduction of an emotion engine enables flexible reminders that respond to the emotions of each individual user.

[0202] The processing flow will be explained below.

[0203] Step 1:

[0204] The user starts the application on their smartphone or tablet and enters their user ID and password on the login screen. The entered authentication information is sent to the server.

[0205] Step 2:

[0206] The server receives the user's authentication information and checks it against a database. If authentication is successful, the login is approved and a signal is sent to the user's device to display the home screen.

[0207] Step 3:

[0208] The user enters their child's information (name, class, next day's schedule) on the home screen. The entered information is sent to the server and stored in a database.

[0209] Step 4:

[0210] The server analyzes the received timetable information and identifies the materials and items required for each subject. For example, it determines that a Japanese language class requires a Japanese language textbook, notebook, and pencil case.

[0211] Step 5:

[0212] The server predicts additional items to bring based on the day of the week, season, and weather information obtained from the weather API. For example, if it's Friday, it will add "sneakers," and if rain is expected, it will add "rain gear."

[0213] Step 6:

[0214] The server generates the required inventory list, which is done using the generated text list.

[0215] Step 7:

[0216] The inventory list is converted into audio data using a text-to-speech (TTS) software API specified by the server, and the converted audio data is stored on the server.

[0217] Step 8:

[0218] The server sends the audio data to the user's device using the HTTP or HTTPS protocol, and the audio data is stored on the user's device.

[0219] Step 9:

[0220] To check the preparations for the next day, the user launches the application again and says "Check tomorrow's preparation list" by voice input or touch operation. The user's device detects this and plays the saved voice data.

[0221] Step 10:

[0222] The user device plays the audio data and notifies the user of the list of items to bring. For example, it may remind the user that "Tomorrow's items to bring are textbooks (Japanese and math), arts and crafts kit, gym clothes, and sneakers."

[0223] Step 11:

[0224] When using the emotion engine, when a user inputs information into the app, the device's camera and microphone capture the user's facial expressions and tone of voice, and this emotional data is sent to the server.

[0225] Step 12:

[0226] The emotion engine on the server analyzes the user's emotional data. If the user is feeling anxious or stressed, the system will adjust the reminder content by providing a detailed explanation of the item to bring. Conversely, if the user is expressing positive emotions, the system will send reminders that emphasize successful experiences.

[0227] Step 13:

[0228] The server converts the adjusted reminder content back into audio data and transmits it to the user terminal, which plays back the audio data and notifies the user of the adjusted reminder content.

[0229] The above is the specific processing flow when combining the emotion engine of this system.

[0230] Example 2

[0231] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0232] Conventional luggage preparation support systems have the function of generating a luggage list based on the user's input timetable information, season, and day of the week, but they lack the ability to provide reminders that take into account the user's emotional state. As a result, luggage preparation may not be carried out efficiently, and the user's psychological support may be insufficient. Furthermore, since voice reminders alone do not provide enough visual information, a more effective notification method is desired.

[0233] The specification process by the specification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for receiving school timetable information input by a user terminal, a means for predicting necessary belongings based on specific days of the week and seasons, and a means for adjusting reminder content using an emotion engine that analyzes the user's voice input and facial expressions. This makes it possible to provide flexible reminders that take into account the user's emotional state, allowing the user to prepare belongings more efficiently and with peace of mind.

[0234] A "user terminal" is an electronic device, such as a smartphone, tablet, or PC, that a user uses to input information or receive notifications.

[0235] "Timetable information" is information that indicates the schedule of school lessons and activities, including subjects, days of the week, and time periods.

[0236] "Items required for specific days of the week or seasons" are items that students are required to bring depending on the content of the class, weather, or season. For example, sneakers are required on days when there is a physical education class, and rain gear is required on days when rain is expected.

[0237] A "list of necessary belongings" is a list of items that a user should bring with them the next day or on a specific day.

[0238] "Audio Data" means text information converted into a format that can be played as sound, and is generated using a Text-to-Speech (TTS) API.

[0239] An "emotion engine" refers to software or hardware that has the functionality to analyze a user's tone of voice and facial expressions and assess their emotional state.

[0240] "Adjusting reminder content" refers to changing the content and format of notifications and messages provided based on the user's emotional state. For example, a user who is feeling anxious can receive additional detailed explanations or encouraging messages.

[0241] "Text reminders" refer to a method of providing notifications and instructions to users through text information that can be read as words. These can be displayed visually on the screen of the user's device.

[0242] The present invention is a system that uses a user terminal to assist in preparing school supplies, and is implemented by the following procedure. First, the user launches a dedicated application on the user terminal, such as a smartphone or tablet. The user enters authentication information on the login screen and sends it to the server. The server authenticates the user using the received information, and if authentication is successful, displays the home screen on the user terminal.

[0243] Entering timetable information

[0244] The user enters basic information about their child (name, class), the next day's schedule, and any special notes (lessons, health, etc.) on the home screen. This information is sent from the user's device to the server and stored in the server's database.

[0245] Generate a packing list

[0246] The server needs to identify the necessary materials and belongings for each subject based on the timetable information. It also predicts what additional items to bring based on the day of the week, season, and weather information obtained from a weather API. For example, if it's Friday, sneakers are added to the list, and if it's a rainy day, rain gear is added.

[0247] Use of emotion engine

[0248] An emotion engine is introduced to analyze the user's voice input and facial expressions. When the user inputs voice or facial expressions into the app, this is captured using a camera and microphone and sent to a server. The emotion engine on the server analyzes this and adjusts the reminder content based on the analysis results. For example, if the user is feeling anxious or stressed, the reminder content can be made more detailed or an encouraging message can be added. Conversely, if the user is feeling positive, a reminder that emphasizes successful experiences can be sent.

[0249] Generating and transmitting audio data

[0250] The inventory list is converted into audio data using a text-to-speech (TTS) software API, and the server sends the generated audio data to the user's device and stores it on the user's device.

[0251] Reminders on user devices

[0252] To check what to prepare for the next day, the user launches the application again and uses voice input or touch operations to say, "Tell me what to prepare for tomorrow." The user's device detects this and plays back the saved voice data. For example, the user might be reminded, "Tomorrow's items to bring are textbooks (Japanese and math), arts and crafts kit, gym clothes, and sneakers."

[0253] Application Examples

[0254] This system can be applied not only to reminders of inventory lists, but also to medication reminders and childcare reminders. For example, it can be set to notify an elderly person, "Please take your medicine at 6 p.m." In this case, the emotion engine can provide more detailed explanations or add encouraging messages to anxious users.

[0255] Specific examples

[0256] For example, if the next day's schedule looks like this:

[0257] Day of the week: Friday

[0258] Subjects: Japanese, arithmetic, art, physical education

[0259] Special note: Plans to join soccer club after school

[0260] Weather information: Rain forecast

[0261] In this case, the server adds the following items to the list:

[0262] Japanese language textbook

[0263] Mathematics textbook

[0264] Art and craft set

[0265] gym clothes

[0266] athletic shoes

[0267] Soccer practice equipment

[0268] rain gear

[0269] Example of input prompt for generative AI model

[0270] "Please provide the next day's schedule information and weather forecast in the following format, and generate a list of items you need to bring. Also, please generate a reminder message based on the user's emotions.

[0271] (Timetable information)

[0272] Day of the week: Friday

[0273] Subjects: Japanese, arithmetic, art, physical education

[0274] Special note: Plans to join soccer club after school

[0275] (weather forecast)

[0276] Weather: Rain

[0277] (User sentiment)

[0278] Voice Tone: A little anxious

[0279] Expression: Not calm

[0280] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0281] Step 1:

[0282] The user launches the app and authenticates

[0283] The user launches a dedicated application on their smartphone or tablet. When the login screen appears, the user enters their email address and password and taps the "Login" button. This input information is sent to the server. The server compares the received authentication information with the database, and if authentication is successful, the server sends a successful authentication response to the user's device, and the home screen is displayed on the user's device. The input is the email address and password, and the output is the user's home screen.

[0284] Step 2:

[0285] The user enters the next day's schedule information and special notes

[0286] The user opens a form on the home screen to enter the child's name, class, next day's schedule information, and any special notes. When the user enters this information and taps the "Submit" button, the entered data is sent to the server. The server stores the received information in a database. The input at this time is the child's name, class, schedule information, and special notes, and the output is the information stored in the database.

[0287] Step 3:

[0288] Server generates inventory list

[0289] The server retrieves timetable information stored in the database and identifies the necessary items for each subject. In doing so, the server also takes into account the day of the week, season, and weather information retrieved from the weather API. For example, if there is a physical education class on Friday and rain is expected, sneakers and rain gear will be added to the list of items to bring. The input is the timetable information, season information, and weather information from the database, and the output is a list of necessary items to bring.

[0290] Step 4:

[0291] Use of emotion engine

[0292] When a user inputs voice or facial expressions into the app, the device's camera and microphone capture this and send the data to the server. The server's emotion engine analyzes this and evaluates the user's emotional state. Based on the analysis results, the server adjusts the reminder content. For example, if the user is feeling anxious, it may add a detailed explanation or an encouraging message. The input data at this time is audio and video data, and the output is the adjusted reminder content.

[0293] Step 5:

[0294] Convert your inventory list into audio data and send it

[0295] The server converts the final inventory list into voice data using a text-to-speech (TTS) API. This generated voice data is sent to the user's device and stored on the device. The input is the inventory list, and the output is voice data.

[0296] Step 6:

[0297] Reminders on user devices

[0298] The user launches the app again to check the next day's preparations, and issues instructions by voice input or touch operation, such as "Tell me what to prepare for tomorrow." The device detects this and plays back the saved voice data. For example, the device may be reminded, "Tomorrow's items to bring are textbooks (Japanese and math), arts and crafts kit, gym clothes, and sneakers." The input is the user's voice input or touch operation, and the output is the voice playback data.

[0299] (Application example 2)

[0300] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0301] Previous school baggage preparation support systems were limited to generating a baggage list based on the next day's schedule, and were unable to flexibly respond to the user's emotional state or specific circumstances. They also lacked appropriate reminders for items to be prepared on the factory production line, and lacked emotional support for users. This led to missed items and emotional stress.

[0302] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0303] In this invention, the server includes means for receiving school timetable information entered by a user terminal, means for predicting necessary items based on a specific day of the week or season, means for generating a list of necessary items, means for converting the generated list into voice data, means for transmitting the voice data to the user terminal, means for playing the transmitted voice data on the user terminal, means for identifying necessary items based on the schedule obtained from the server, means for analyzing the user's emotional state using an emotion analysis engine, and means for adjusting reminder messages according to the user's emotional state. This enables flexible reminders according to the user's emotional state, prevents users from forgetting to bring or prepare items, and provides emotional support.

[0304] "School timetable information" is information necessary for students to participate in classes and extracurricular activities, and indicates the dates, times, and locations of specific subjects and activities.

[0305] "Day-of-the-week and season-based supplies" are items that are needed for specific activities or lessons depending on the day of the week or the season.

[0306] A "packing list" is a list of items that a user needs to bring with them the next day or on a specific day.

[0307] "Audio data" refers to data obtained by converting text-format information into audio, and is used by the user to receive information aurally.

[0308] A "user terminal" is an electronic device that is personally owned by a user, such as a smartphone, tablet, or smart glasses.

[0309] A "server" is a central device that receives information from multiple user terminals, processes the information, and returns appropriate data.

[0310] An "emotion analysis engine" is a program that analyzes the user's emotional state from voice input and facial expressions, and allows the system to respond based on the results.

[0311] A "remind message" refers to a notification message that provides the user with necessary information or encourages them to take action.

[0312] "Schedules acquired from a server" refers to data on pre-set plans or schedules, such as factory production lines or school timetables, acquired by a server.

[0313] The present invention relates to a system for supporting the preparation of belongings for school or a production line using a user terminal. Specific embodiments will be described below.

[0314] First, the user launches a dedicated application using a user device such as a smartphone or smart glasses. The user (parent or worker) enters authentication information on the login screen and sends it to the server. The server authenticates the user using the received information, and if authentication is successful, the home screen is displayed on the user device.

[0315] Next, the user enters basic information (name, class or production line), the next day's timetable and production schedule, and any special notes (lessons or specific processes) on the home screen. This information is sent to the server and stored in a database.

[0316] The server identifies the materials and parts needed for each subject and process based on timetable information or production schedules. It also predicts what additional items to bring based on the day of the week, season, and weather information obtained from a weather API. For example, if it's Friday, sneakers are added to the list, and if it's a rainy day, rain gear is added.

[0317] Furthermore, a pre-installed emotion analysis engine is introduced. The emotion engine analyzes the user's voice input and facial expressions. Based on the analysis results, if the user is feeling anxious or stressed, the reminder content will be made more detailed or an encouraging message will be added. Conversely, if the user is feeling positive, a reminder that emphasizes successful experiences will be sent.

[0318] The list of necessary items is converted into voice data using a text-to-speech software (TTS) API, and the server sends the generated voice data to the user's device and stores it on the user's device.

[0319] To check what to prepare for the next day, the user launches the application again and uses voice input or touch operations to say, "Tell me what to prepare for tomorrow." The user's device detects this and plays back the saved voice data. For example, the user might be reminded, "Tomorrow's items to bring are textbooks (Japanese and math), arts and crafts kit, gym clothes, and sneakers."

[0320] The same can be applied to factory production lines. When a worker asks, "What should I prepare for tomorrow?", a voice message will be sent reminding them of the tools and parts they will need based on the next day's production schedule. For example, they might be reminded, "Tomorrow's items to bring are a wrench and a screwdriver, and since it's a rainy day, don't forget to bring rain gear."

[0321] Hardware and software:

[0322] Hardware: smart glasses (e.g. Google Glass), smartphones, servers

[0323] Software: EmotionEngine (emotion analysis engine), TTSAPI (text-to-speech software), Weather API (weather information acquisition)

[0324] Examples:

[0325] Example 1: A production line worker asks, "What should I prepare for tomorrow?" and is reminded, "Tomorrow, bring a wrench, a screwdriver, and don't forget your rain gear because it's a rainy day."

[0326] Example 2: When a student asks, "What should I prepare for tomorrow?", they are reminded, "Tomorrow, bring your textbooks (Japanese and math), arts and crafts kit, gym clothes, and sneakers."

[0327] Example prompt sentence:

[0328] "Tell me what to bring tomorrow. Add a feature that will remind me by voice what tools I need, like wrenches, screwdrivers, etc."

[0329] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0330] Step 1:

[0331] The user enters their credentials on the login screen.

[0332] Input: Username, Password

[0333] Data processing: The user terminal encrypts the entered authentication information and sends it to the server.

[0334] Output: The server receives the authentication information and checks it against its database.

[0335] Specific operation: The server checks the authentication information in the database, and if authentication is successful, displays the home screen on the user's device.

[0336] Step 2:

[0337] The user enters timetable information and special notes on the home screen.

[0338] Input: Timetable information, special notes (lessons, specific processes, etc.)

[0339] Data processing: The user terminal formats the entered information and sends it to the server.

[0340] Output: The server stores the received information in a database.

[0341] Specific operation: The server uses the timetable information and special notes stored in the database to create the schedule for the next day.

[0342] Step 3:

[0343] The server generates a list of necessary items to bring based on the timetable information.

[0344] Input: Timetable information, special notes

[0345] Data processing: The server lists the necessary materials and parts for each subject and process. It also uses a weather API to obtain weather information and predicts what additional items to bring.

[0346] Output: List of necessary items

[0347] Specific actions: For example, if it's Friday and you have exercise, add sneakers to the list, or if it's a rainy day, add rain gear to the list.

[0348] Step 4:

[0349] The server converts the inventory list into audio data.

[0350] Input: inventory list

[0351] Data processing: The server uses a text-to-speech (TTS) API to convert the inventory list into audio data.

[0352] Output: Audio data

[0353] Specific operation: Input the inventory list text into the TTS API and generate a corresponding audio file.

[0354] Step 5:

[0355] The server transmits the voice data to the user terminal.

[0356] Input: Audio data

[0357] Data processing: The server sends the generated voice data to the user terminal.

[0358] Output: Audio data stored on the user's device

[0359] Specific operation: The server transfers the voice data to the user device, and the user device stores it in its internal storage.

[0360] Step 6:

[0361] The user requests a packing list reminder via voice or touch.

[0362] Input: Voice input, touch operation

[0363] Data processing: The user device recognizes voice input and touch operations and sends a reminder request to the server.

[0364] Output: Remind request

[0365] Specific operation: When the device receives a reminder request, it plays the saved audio data.

[0366] Step 7:

[0367] The server uses an emotion analysis engine to analyze the user's emotional state.

[0368] Input: User's voice input, facial expression data

[0369] Data processing: The emotion analysis engine analyzes the user's tone of voice and facial expressions to identify their emotional state.

[0370] Output: User's emotional state

[0371] Specific behavior: For example, if the user is feeling anxious, the engine generates an "encouraging message."

[0372] Step 8:

[0373] The user terminal plays back the audio data.

[0374] Input: Voice data, emotion analysis results

[0375] Data processing: The audio data is adjusted according to the emotion and played back to the user.

[0376] Output: Play reminder message

[0377] Specific operation: For example, the reminder message "Tomorrow, bring a wrench, a screwdriver, and rain gear" is played.

[0378] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0379] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0380] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0381] [Second embodiment]

[0382] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0383] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0384] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0385] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0386] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0387] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0388] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0389] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0390] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0391] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0392] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0393] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0394] This invention is a system that uses a user terminal to assist in preparing school supplies. It generates a list of necessary supplies based on the next day's schedule and provides voice reminders, reducing the burden on children and their parents.

[0395] System implementation

[0396] Users launch a dedicated application using a user device such as a smartphone or tablet. First, a parent or child logs in to the app and sends authentication information to the server. The server receives this authentication information and authenticates the user. If authentication is successful, the user is taken to the home screen.

[0397] Entering timetable information

[0398] The user (parent or child) then enters information about the child's school into the application, including the child's name, class, and tomorrow's schedule. This data is sent to the server, where it is stored in a database.

[0399] Generate a packing list

[0400] The server analyzes the received timetable information and identifies the teaching materials and items required for each subject. It also predicts additional items to bring based on the day of the week and the season. For example, it determines that sneakers are needed on Fridays and jackets are needed in spring. This information is combined to generate the final packing list.

[0401] Generating and transmitting audio data

[0402] The inventory list is then converted into audio data using a text-to-speech (TTS) API, which the server then sends to the user's device via HTTP or HTTPS.

[0403] Reminders on user devices

[0404] The user (child) opens the application again before the next day and requests, by voice or touch, "Tell me what to prepare for tomorrow." The user's device plays the saved voice data and announces the list of items to bring. For example, the user might be reminded, "Tomorrow's items to bring are textbooks (Japanese and math), arts and crafts kit, gym clothes, and sneakers."

[0405] Application Examples

[0406] This system can be used to provide not only reminders for inventory lists, but also medication reminders and childcare reminders. For example, an elderly person can be set to be reminded to take their medicine at 6 p.m. In this case, the server generates audio data at the set time and sends it to the device. The device then plays the audio data at the set time, reminding the user to "take their medicine at 6 p.m."

[0407] Through the detailed steps described above, the present invention provides effective support for users and their guardians to prepare all necessary belongings and start their new life with peace of mind.

[0408] The processing flow will be explained below.

[0409] Step 1:

[0410] The user starts the application on their smartphone or tablet and enters their user ID and password on the login screen, which is then sent to the server.

[0411] Step 2:

[0412] The server receives the user's authentication information and checks it against a database. If authentication is successful, the login is approved and a signal is sent to the user's device to display the home screen.

[0413] Step 3:

[0414] The user enters their child's information (name, class, next day's schedule) on the home screen. The entered information is sent to the server and stored in a database.

[0415] Step 4:

[0416] The server analyzes the received timetable information. First, it identifies the teaching materials and items required for each subject. For this analysis, it uses the standard list of items to bring for each subject in the timetable (e.g., Japanese textbook, notebook, pencil case).

[0417] Step 5:

[0418] The server predicts what additional items to bring based on the day of the week, season, and weather information obtained from the weather API. For example, if it's Friday, sneakers are added to the list, and if it's a rainy day, rain gear is added.

[0419] Step 6:

[0420] The server generates a list of required items and converts it into audio data using a Text-to-Speech (TTS) API, which is then stored on the server.

[0421] Step 7:

[0422] The server sends the voice data to the user's device. The communication protocol is HTTP or HTTPS. The sent voice data is saved on the user's device.

[0423] Step 8:

[0424] To check the preparations for the next day, the user launches the application again and says "Check tomorrow's preparation list" by voice input or touch operation. The user's device detects this and plays the saved voice data.

[0425] Step 9:

[0426] The user device plays the audio data and notifies the user of the list of items to bring, allowing the user (child) to check what to bring the next day and prepare appropriately.

[0427] Step 10:

[0428] The user device can extend the reminder function to notify users of medication reminders at specific times. For example, it can play a voice reminder to an elderly person saying, "Take your medicine at 6 p.m."

[0429] The above is the specific processing flow of this system.

[0430] Example 1

[0431] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0432] In today's busy households, it is difficult to properly prepare school supplies. Especially when children prepare their own school supplies, forgetting or failing to prepare frequently causes considerable stress for both children and their parents. Furthermore, it is difficult to accurately track what school supplies should be prepared for different days of the week and seasons, so a system that automates this process and efficiently manages them is needed.

[0433] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0434] In this invention, the server includes a means for receiving timetable information of the educational institution input by the user terminal, a means for predicting items needed based on a specific day of the week or season, and a means for generating a list of needed items, thereby enabling children and their parents to efficiently prepare their belongings and prevent them from forgetting their belongings.

[0435] "User terminal" refers to a portable information device for inputting and receiving timetable information and inventory lists for educational institutions.

[0436] "Educational institutions" refers to schools, cram schools, and other educational facilities.

[0437] "Timetable information" refers to schedule information for classes and activities at an educational institution, and is a list of subjects and activities that take place on specific days and at specific times.

[0438] "Items" refers to teaching materials and tools necessary for lessons and activities.

[0439] "Predictive means" refers to the ability of a device or program to automatically determine what items are needed based on specific days of the week or seasons.

[0440] The "means for generating a list" refers to the function of a device or program that organizes predicted items and creates a list in a list format.

[0441] "Means for converting into audio data" refers to text-to-speech technology that converts the generated item list into natural audio.

[0442] The "means for transmitting voice data" refers to a communication technology for transmitting the generated voice data to a user terminal.

[0443] "Means for playing" refers to the function of a device or program for playing back audio data received on a user terminal.

[0444] "Text reminder" refers to a format that notifies you of your belongings using text information rather than audio data.

[0445] This invention is a system that uses user terminals to assist educational institutions in preparing belongings. It generates a list of necessary items based on the next day's schedule and reminds students of this list via voice, thereby reducing the burden on students and their guardians.

[0446] First, the user launches a dedicated application using a user device such as a smartphone or tablet. The user (guardian or beneficiary) enters their user ID and password on the login screen and sends them to the server. The sent authentication information is received by the server, which then collates it with information in a database to authenticate the user. If authentication is successful, the user is taken to the home screen.

[0447] Next, the user enters the recipient's name, class, and the next day's schedule information into a dedicated form in the application. This information is formatted by the user's terminal and sent to the server, which receives the information and stores it in a database.

[0448] Based on the submitted timetable information, the server identifies the items required for each subject. It then analyzes the day of the week and seasonal information to predict additional items. For example, it determines that sneakers are needed on Fridays and jackets are needed in spring. This information is then combined to generate the final item list.

[0449] The generated inventory list is converted into voice data using a text-to-speech (TTS) software API, and the server transmits the converted voice data to the user's device using the HTTP or HTTPS protocol.

[0450] The user (beneficiary) opens the application again before the next day and requests, by voice input or touch operation, "Tell me what to prepare for tomorrow." The user device plays the saved voice data and reminds the user, "Tomorrow's items to bring are textbooks (Japanese and math), arts and crafts kit, gym clothes, and sneakers."

[0451] As a specific example of how this works, for example, a list of belongings generated based on Monday's timetable would include "Japanese textbook, math notebook, gym clothes." If it's Friday, "sneakers" would be added. If it's spring, "jacket" would be added.

[0452] By using the following prompts to the generative AI model, you can elicit specific explanations of the system's processing flow.

[0453] Example prompt:

[0454] "Please explain in detail how this system reminds users of their belongings. For example, please explain in detail the process by which the server analyzes the timetable information and generates the belongings list."

[0455] Through the above process, the present invention provides effective support for the user and guardian to prepare all necessary belongings and start their new life with peace of mind.

[0456] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0457] Step 1: User launches application and logs in

[0458] The user launches the dedicated application using the user device. After launching, the user (guardian or beneficiary) enters their user ID and password on the login screen and taps the login button. The user device sends the entered authentication information to the server. The server compares the received authentication information with information in the database and authenticates the user. If authentication is successful, the server sends a success message and the user device transitions to the home screen.

[0459] Input: User ID, Password

[0460] Data processing: User ID and password verification

[0461] Output: Authentication success message

[0462] Specific operation: When the user taps the login button, authentication information is sent, the server authenticates it, and returns a success message.

[0463] Step 2: Enter your school and timetable information

[0464] The user (parent or beneficiary) enters the beneficiary's name, class, and next day's timetable information into a dedicated form in the application. The user's device formats the entered information and sends it to the server. The server receives this information and stores it in a database.

[0465] Input: Beneficiary's name, class, next day's timetable information

[0466] Data processing: Formatting and sending information

[0467] Output: Acknowledgement of receipt of information

[0468] Specific operation: When a user enters information into a dedicated form and taps the submit button, the information is sent to the server and stored in a database.

[0469] Step 3: Generate a packing list

[0470] The server analyzes the received timetable information and identifies the items needed for each subject stored in the database. It also analyzes information about the day of the week and the season to predict additional items. It then combines this information to generate the final packing list.

[0471] Input: Timetable information, day of the week information, seasonal information

[0472] Data calculation: Identifying items and predicting additional items

[0473] Output: Final inventory list

[0474] Specific operation: The server analyzes the timetable information, adds items according to the day of the week and season, and creates a list of items to bring.

[0475] Step 4: Generate and send audio data

[0476] The server converts the generated inventory list into audio data using a text-to-speech software (TTS) API, and sends the converted audio data to the user's device using HTTP or HTTPS protocol.

[0477] Input: Final packing list

[0478] Data processing: conversion to audio data

[0479] Output: Audio data

[0480] Specific operation: The server sends the inventory list to the TTS API, and the generated voice data is sent to the user's terminal.

[0481] Step 5: Remind users on their devices

[0482] The user (beneficiary) opens the application again before the next day and requests, by voice input or touch operation, "Tell me what to prepare for tomorrow." The user device plays the saved voice data and reminds the user, "Tomorrow's items to bring are textbooks (Japanese and math), arts and crafts kit, gym clothes, and sneakers."

[0483] Input: Voice or touch

[0484] Data processing: processing requests and playing audio data

[0485] Output: Voice reminder

[0486] Specific operation: When the user voice-inputs "Tell me how to prepare for tomorrow" or taps a button, the device plays the voice data as a reminder.

[0487] (Application example 1)

[0488] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0489] When shopping in a physical store, it is difficult for users to efficiently find the items they need to purchase, resulting in unnecessary time and effort. Furthermore, especially in large stores, it is not uncommon for users to miss or have difficulty finding items that they need to purchase based on their shopping list. Furthermore, there is a lack of a reminder function to ensure users don't forget to purchase the items they need. Therefore, a system that solves this problem is needed.

[0490] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0491] In this invention, the server includes means for receiving school timetable information input by a user terminal, means for predicting items needed based on specific days of the week and seasons, means for generating a list of items needed, means for converting the generated list into voice data, means for transmitting the voice data to the user terminal, means for playing the transmitted voice data on the user terminal, and means for guiding the location of items included in the list based on location information of each department in the store, thereby enabling users to shop efficiently in a physical store and purchase all necessary items.

[0492] A "user terminal" is a device such as a smartphone, tablet, or smart glasses that is directly operated by a user to input and receive information.

[0493] "School timetable information" is data that includes information about the times and contents of classes that students will take the next day and on specific days, as well as the teaching materials and items they will need to take with them.

[0494] "Day-of-the-week and season-based items" refers to items or clothing needed on specific days of the week or during specific seasons, such as sneakers needed on Fridays or a jacket needed in the spring.

[0495] The "list of necessary items" is a list of all items needed for the next day or a specific day, generated based on the user's input information and predicted day of the week and season information.

[0496] The "means for converting into audio data" is a technique for converting the inventory list in text format into audio format and generating an audio file or audio stream that can be listened to by the user.

[0497] The "means for transmitting voice data to a user terminal" refers to a communication means for transmitting voice data generated by the server to a user terminal via the Internet.

[0498] "Means for reproducing audio data at a user terminal" refers to a technique for reproducing audio data received by a user terminal and providing it to the user as auditory information.

[0499] "Location information for each department within the store" is information about where each item is displayed within the physical store, and is data indicating in which department or section a particular item is located.

[0500] "Item location guidance" refers to technology that guides users to specific locations or departments within a store to help them efficiently find items on their shopping list.

[0501] The present invention is a system for supporting efficient shopping in a physical store using a user terminal. Specific embodiments of this system will be described below.

[0502] System implementation

[0503] A user launches a dedicated shopping assistant application using a user device such as a smartphone or smart glasses. First, the user logs in to the application and sends authentication information to the server. The server receives this authentication information and performs user authentication. If authentication is successful, the user is taken to the application's home screen.

[0504] Entering a shopping list

[0505] The user enters the items they want to include in their shopping list into the application, and this data is sent to the server where it is stored in a database.

[0506] Shopping list location guide

[0507] The server analyzes the received shopping list information and identifies which department in the store each item is located in. Based on the store location information, it generates a list to guide the user to the location of the required items. The list information includes the location and department details of each item.

[0508] Generating and transmitting audio data

[0509] The generated shopping list and location information is converted into audio data using a Text-to-Speech (TTS) API, and the server generates this audio data and sends it to the user's device using HTTP or HTTPS protocols.

[0510] Reminders and guidance on user devices

[0511] The user uses the application in the store and requests, by voice or touch, "What item should I buy next?" The user's device plays the saved voice data and provides a voice prompt of the items on the list and their location. For example, the user might be reminded, "The next item to buy is milk. It's in the refrigerated section."

[0512] Specific processing

[0513] Hardware: Smartphones, smart glasses

[0514] Software: Dedicated shopping assistant application, server-side authentication system, database, Text-to-Speech (TTS) API

[0515] Data processing flow:

[0516] 1. The user logs in to the app and sends their credentials to the server.

[0517] 2. The server authenticates the user and receives and stores the shopping list data.

[0518] 3. The server analyzes the information in the list and determines the location of the item within the store.

[0519] 4. Based on the location information, the list is converted into audio data and sent to the user's device.

[0520] 5. The user terminal plays back the received voice data and provides the user with a voice guide of the next item to be purchased and its location.

[0521] Specific examples

[0522] For example, a user logs in to the application and enters "milk," "bread," and "detergent" as a list of items to be purchased next. The server uses this information to identify which section of the physical store each item is located in, and generates a voice reminder based on the location information of the refrigerated goods section, bakery section, and daily necessities section. This reminder is played on the user's device, supporting the user in efficient shopping.

[0523] Prompt Sentence Examples

[0524] Develop an application that helps users shop efficiently in physical stores. The main features are shopping list creation and voice reminders. The reminders guide users through their shopping by guiding them to the next item to be purchased and its location.

[0525] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0526] Step 1:

[0527] The user launches the application and enters authentication information on the login screen. The authentication information includes the user ID and password. The server authenticates the user based on this information. The server compares the authentication information with records in the database, and if there is a match, returns a status of successful authentication. The user's device receives a notification of successful authentication and moves to the home screen.

[0528] Step 2:

[0529] The user inputs a shopping list into the application. This list contains the names of multiple items the user plans to purchase. The input data is sent to the server as form data. The server receives this data and saves the shopping list in a database. Input: Shopping list (item names). Output: Shopping list saved in the database.

[0530] Step 3:

[0531] The server analyzes the shopping list information and identifies the location information for each item in the store. It references the location information in the database to determine which section or department each item is located in. Based on this data, it generates a list containing location information for each item. Input: Saved shopping list. Output: List containing location information for the store.

[0532] Step 4:

[0533] The server converts the list generated by the server into audio data using a Text-to-Speech (TTS) API. The list information in text format is input to the API, which generates an audio file or audio stream. Input: List containing in-store location information. Output: Audio data.

[0534] Step 5:

[0535] The voice data is sent to the user terminal using the HTTP or HTTPS protocol. The server uses a communication means to send the generated voice data to the user terminal. Input: Voice data. Output: Voice data sent to the user terminal.

[0536] Step 6:

[0537] The user reopens the application in the store and requests, by voice or touch, "Tell me what item I'll buy next." The user device receives this request and plays the saved voice data. Input: User request. Output: Played voice data (reminder).

[0538] Step 7:

[0539] The user device plays the audio data and provides audible instructions about the next item in the list and its location. For example, "The next item to purchase is milk. It's in the refrigerated section." Input: The stored audio data. Output: Audio instructions provided to the user.

[0540] keyword:

[0541] Generative AI model, prompt sentence

[0542] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0543] This invention is a system that uses a user terminal to help users prepare school supplies. It generates a list of necessary supplies based on the next day's schedule and reminds users of the list by voice and text. In addition, by combining it with an emotion engine, the content of the reminder can be flexibly adjusted according to the user's emotions.

[0544] System implementation

[0545] First, the user launches a dedicated application using a user device such as a smartphone or tablet. The user (parent or child) enters authentication information on the login screen and sends it to the server. The server authenticates the user using the received information, and if authentication is successful, the home screen is displayed on the user device.

[0546] Entering timetable information

[0547] The user enters basic information about their child (name, class), the next day's schedule, and any special notes (lessons, health, etc.) on the home screen. This information is sent to the server and stored in a database.

[0548] Generate a packing list

[0549] The server identifies the materials and supplies needed for each subject based on the timetable information. It also predicts what additional items students need to bring based on the day of the week, season, and weather information obtained from a weather API. For example, if it's Friday, sneakers are added to the list, and if it's a rainy day, rain gear is added.

[0550] Use of emotion engine

[0551] Furthermore, the app will introduce an emotion engine that analyzes the user's voice input and facial expressions. The camera captures the user's voice tone and facial expressions when they input information into the app, and the emotion engine analyzes them. Based on the analysis results, if the user is feeling anxious or stressed, the reminder content will be made more detailed or an encouraging message will be added. Conversely, if the user is feeling positive, the app will send reminders that emphasize successful experiences.

[0552] Generating and transmitting audio data

[0553] The list of necessary items is converted into voice data using a text-to-speech software (TTS) API, and the server sends the generated voice data to the user's device and stores it on the user's device.

[0554] Reminders on user devices

[0555] To check what to prepare for the next day, the user launches the application again and uses voice input or touch operations to say, "Tell me what to prepare for tomorrow." The user's device detects this and plays back the saved voice data. For example, the user may be reminded, "Tomorrow's items to bring are textbooks (Japanese and math), arts and crafts kit, gym clothes, and sneakers."

[0556] Application Examples

[0557] This system can be applied not only to reminders of inventory lists, but also to medication reminders and childcare reminders. For example, it can be set to notify an elderly person, "Please take your medicine at 6 p.m." In this case, the emotion engine can provide more detailed explanations or add encouraging messages to anxious users.

[0558] Through the detailed steps described above, the present invention provides effective support for users to prepare all their belongings and live their daily lives with peace of mind.The introduction of an emotion engine enables flexible reminders that respond to the emotions of each individual user.

[0559] The processing flow will be explained below.

[0560] Step 1:

[0561] The user starts the application on their smartphone or tablet and enters their user ID and password on the login screen. The entered authentication information is sent to the server.

[0562] Step 2:

[0563] The server receives the user's authentication information and checks it against a database. If authentication is successful, the login is approved and a signal is sent to the user's device to display the home screen.

[0564] Step 3:

[0565] The user enters their child's information (name, class, next day's schedule) on the home screen. The entered information is sent to the server and stored in a database.

[0566] Step 4:

[0567] The server analyzes the received timetable information and identifies the materials and items required for each subject. For example, it determines that a Japanese language class requires a Japanese language textbook, notebook, and pencil case.

[0568] Step 5:

[0569] The server predicts additional items to bring based on the day of the week, season, and weather information obtained from the weather API. For example, if it's Friday, it will add "sneakers," and if rain is expected, it will add "rain gear."

[0570] Step 6:

[0571] The server generates the required inventory list, which is done using the generated text list.

[0572] Step 7:

[0573] The inventory list is converted into audio data using a text-to-speech (TTS) software API specified by the server, and the converted audio data is stored on the server.

[0574] Step 8:

[0575] The server sends the audio data to the user's device using the HTTP or HTTPS protocol, and the audio data is stored on the user's device.

[0576] Step 9:

[0577] To check the preparations for the next day, the user launches the application again and says "Check tomorrow's preparation list" by voice input or touch operation. The user's device detects this and plays the saved voice data.

[0578] Step 10:

[0579] The user device plays the audio data and notifies the user of the list of items to bring. For example, it may remind the user that "Tomorrow's items to bring are textbooks (Japanese and math), arts and crafts kit, gym clothes, and sneakers."

[0580] Step 11:

[0581] When using the emotion engine, when a user inputs information into the app, the device's camera and microphone capture the user's facial expressions and tone of voice, and this emotional data is sent to the server.

[0582] Step 12:

[0583] The emotion engine on the server analyzes the user's emotional data. If the user is feeling anxious or stressed, the system will adjust the reminder content by providing a detailed explanation of the item to bring. Conversely, if the user is expressing positive emotions, the system will send reminders that emphasize successful experiences.

[0584] Step 13:

[0585] The server converts the adjusted reminder content back into audio data and transmits it to the user terminal, which plays back the audio data and notifies the user of the adjusted reminder content.

[0586] The above is the specific processing flow when combining the emotion engine of this system.

[0587] Example 2

[0588] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0589] Conventional luggage preparation support systems have the function of generating a luggage list based on the user's input timetable information, season, and day of the week, but they lack the ability to provide reminders that take into account the user's emotional state. As a result, luggage preparation may not be carried out efficiently, and the user's psychological support may be insufficient. Furthermore, since voice reminders alone do not provide enough visual information, a more effective notification method is desired.

[0590] The specification process by the specification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for receiving school timetable information input by a user terminal, a means for predicting necessary belongings based on specific days of the week and seasons, and a means for adjusting reminder content using an emotion engine that analyzes the user's voice input and facial expressions. This makes it possible to provide flexible reminders that take into account the user's emotional state, allowing the user to prepare belongings more efficiently and with peace of mind.

[0591] A "user terminal" is an electronic device, such as a smartphone, tablet, or PC, that a user uses to input information or receive notifications.

[0592] "Timetable information" is information that indicates the schedule of school lessons and activities, including subjects, days of the week, and time periods.

[0593] "Items required for specific days of the week or seasons" are items that students are required to bring depending on the content of the class, weather, or season. For example, sneakers are required on days when there is a physical education class, and rain gear is required on days when rain is expected.

[0594] A "list of necessary belongings" is a list of items that a user should bring with them the next day or on a specific day.

[0595] "Audio Data" means text information converted into a format that can be played as sound, and is generated using a Text-to-Speech (TTS) API.

[0596] An "emotion engine" refers to software or hardware that has the functionality to analyze a user's tone of voice and facial expressions and assess their emotional state.

[0597] "Adjusting reminder content" refers to changing the content and format of notifications and messages provided based on the user's emotional state. For example, a user who is feeling anxious can receive additional detailed explanations or encouraging messages.

[0598] "Text reminders" refer to a method of providing notifications and instructions to users through text information that can be read as words. These can be displayed visually on the screen of the user's device.

[0599] The present invention is a system that uses a user terminal to assist in preparing school supplies, and is implemented by the following procedure. First, the user launches a dedicated application on the user terminal, such as a smartphone or tablet. The user enters authentication information on the login screen and sends it to the server. The server authenticates the user using the received information, and if authentication is successful, displays the home screen on the user terminal.

[0600] Entering timetable information

[0601] The user enters basic information about their child (name, class), the next day's schedule, and any special notes (lessons, health, etc.) on the home screen. This information is sent from the user's device to the server and stored in the server's database.

[0602] Generate a packing list

[0603] The server needs to identify the necessary materials and belongings for each subject based on the timetable information. It also predicts what additional items to bring based on the day of the week, season, and weather information obtained from a weather API. For example, if it's Friday, sneakers are added to the list, and if it's a rainy day, rain gear is added.

[0604] Use of emotion engine

[0605] An emotion engine is introduced to analyze the user's voice input and facial expressions. When the user inputs voice or facial expressions into the app, this is captured using a camera and microphone and sent to a server. The emotion engine on the server analyzes this and adjusts the reminder content based on the analysis results. For example, if the user is feeling anxious or stressed, the reminder content can be made more detailed or an encouraging message can be added. Conversely, if the user is feeling positive, a reminder that emphasizes successful experiences can be sent.

[0606] Generating and transmitting audio data

[0607] The inventory list is converted into audio data using a text-to-speech (TTS) software API, and the server sends the generated audio data to the user's device and stores it on the user's device.

[0608] Reminders on user devices

[0609] To check what to prepare for the next day, the user launches the application again and uses voice input or touch operations to say, "Tell me what to prepare for tomorrow." The user's device detects this and plays back the saved voice data. For example, the user might be reminded, "Tomorrow's items to bring are textbooks (Japanese and math), arts and crafts kit, gym clothes, and sneakers."

[0610] Application Examples

[0611] This system can be applied not only to reminders of inventory lists, but also to medication reminders and childcare reminders. For example, it can be set to notify an elderly person, "Please take your medicine at 6 p.m." In this case, the emotion engine can provide more detailed explanations or add encouraging messages to anxious users.

[0612] Specific examples

[0613] For example, if the next day's schedule looks like this:

[0614] Day of the week: Friday

[0615] Subjects: Japanese, arithmetic, art, physical education

[0616] Special note: Plans to join soccer club after school

[0617] Weather information: Rain forecast

[0618] In this case, the server adds the following items to the list:

[0619] Japanese language textbook

[0620] Mathematics textbook

[0621] Art and craft set

[0622] gym clothes

[0623] athletic shoes

[0624] Soccer practice equipment

[0625] rain gear

[0626] Example of input prompt for generative AI model

[0627] "Please provide the next day's schedule information and weather forecast in the following format, and generate a list of items you need to bring. Also, please generate a reminder message based on the user's emotions.

[0628] (Timetable information)

[0629] Day of the week: Friday

[0630] Subjects: Japanese, arithmetic, art, physical education

[0631] Special note: Plans to join soccer club after school

[0632] (weather forecast)

[0633] Weather: Rain

[0634] (User sentiment)

[0635] Voice Tone: A little anxious

[0636] Expression: Not calm

[0637] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0638] Step 1:

[0639] The user launches the app and authenticates

[0640] The user launches a dedicated application on their smartphone or tablet. When the login screen appears, the user enters their email address and password and taps the "Login" button. This input information is sent to the server. The server compares the received authentication information with the database, and if authentication is successful, the server sends a successful authentication response to the user's device, and the home screen is displayed on the user's device. The input is the email address and password, and the output is the user's home screen.

[0641] Step 2:

[0642] The user enters the next day's schedule information and special notes

[0643] The user opens a form on the home screen to enter the child's name, class, next day's schedule information, and any special notes. When the user enters this information and taps the "Submit" button, the entered data is sent to the server. The server stores the received information in a database. The input at this time is the child's name, class, schedule information, and special notes, and the output is the information stored in the database.

[0644] Step 3:

[0645] Server generates inventory list

[0646] The server retrieves timetable information stored in the database and identifies the necessary items for each subject. In doing so, the server also takes into account the day of the week, season, and weather information retrieved from the weather API. For example, if there is a physical education class on Friday and rain is expected, sneakers and rain gear will be added to the list of items to bring. The input is the timetable information, season information, and weather information from the database, and the output is a list of necessary items to bring.

[0647] Step 4:

[0648] Use of emotion engine

[0649] When a user inputs voice or facial expressions into the app, the device's camera and microphone capture this and send the data to the server. The server's emotion engine analyzes this and evaluates the user's emotional state. Based on the analysis results, the server adjusts the reminder content. For example, if the user is feeling anxious, it may add a detailed explanation or an encouraging message. The input data at this time is audio and video data, and the output is the adjusted reminder content.

[0650] Step 5:

[0651] Convert your inventory list into audio data and send it

[0652] The server converts the final inventory list into voice data using a text-to-speech (TTS) API. This generated voice data is sent to the user's device and stored on the device. The input is the inventory list, and the output is voice data.

[0653] Step 6:

[0654] Reminders on user devices

[0655] The user launches the app again to check the next day's preparations, and issues instructions by voice input or touch operation, such as "Tell me what to prepare for tomorrow." The device detects this and plays back the saved voice data. For example, the device may be reminded, "Tomorrow's items to bring are textbooks (Japanese and math), arts and crafts kit, gym clothes, and sneakers." The input is the user's voice input or touch operation, and the output is the voice playback data.

[0656] (Application example 2)

[0657] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0658] Previous school baggage preparation support systems were limited to generating a baggage list based on the next day's schedule, and were unable to flexibly respond to the user's emotional state or specific circumstances. They also lacked appropriate reminders for items to be prepared on the factory production line, and lacked emotional support for users. This led to missed items and emotional stress.

[0659] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0660] In this invention, the server includes means for receiving school timetable information entered by a user terminal, means for predicting necessary items based on a specific day of the week or season, means for generating a list of necessary items, means for converting the generated list into voice data, means for transmitting the voice data to the user terminal, means for playing the transmitted voice data on the user terminal, means for identifying necessary items based on the schedule obtained from the server, means for analyzing the user's emotional state using an emotion analysis engine, and means for adjusting reminder messages according to the user's emotional state. This enables flexible reminders according to the user's emotional state, prevents users from forgetting to bring or prepare items, and provides emotional support.

[0661] "School timetable information" is information necessary for students to participate in classes and extracurricular activities, and indicates the dates, times, and locations of specific subjects and activities.

[0662] "Day-of-the-week and season-based supplies" are items that are needed for specific activities or lessons depending on the day of the week or the season.

[0663] A "packing list" is a list of items that a user needs to bring with them the next day or on a specific day.

[0664] "Audio data" refers to data obtained by converting text-format information into audio, and is used by the user to receive information aurally.

[0665] A "user terminal" is an electronic device that is personally owned by a user, such as a smartphone, tablet, or smart glasses.

[0666] A "server" is a central device that receives information from multiple user terminals, processes the information, and returns appropriate data.

[0667] An "emotion analysis engine" is a program that analyzes the user's emotional state from voice input and facial expressions, and allows the system to respond based on the results.

[0668] A "remind message" refers to a notification message that provides the user with necessary information or encourages them to take action.

[0669] "Schedules acquired from a server" refers to data on pre-set plans or schedules, such as factory production lines or school timetables, acquired by a server.

[0670] The present invention relates to a system for supporting the preparation of belongings for school or a production line using a user terminal. Specific embodiments will be described below.

[0671] First, the user launches a dedicated application using a user device such as a smartphone or smart glasses. The user (parent or worker) enters authentication information on the login screen and sends it to the server. The server authenticates the user using the received information, and if authentication is successful, the home screen is displayed on the user device.

[0672] Next, the user enters basic information (name, class or production line), the next day's timetable and production schedule, and any special notes (lessons or specific processes) on the home screen. This information is sent to the server and stored in a database.

[0673] The server identifies the materials and parts needed for each subject and process based on timetable information or production schedules. It also predicts what additional items to bring based on the day of the week, season, and weather information obtained from a weather API. For example, if it's Friday, sneakers are added to the list, and if it's a rainy day, rain gear is added.

[0674] Furthermore, a pre-installed emotion analysis engine is introduced. The emotion engine analyzes the user's voice input and facial expressions. Based on the analysis results, if the user is feeling anxious or stressed, the reminder content will be made more detailed or an encouraging message will be added. Conversely, if the user is feeling positive, a reminder that emphasizes successful experiences will be sent.

[0675] The list of necessary items is converted into voice data using a text-to-speech software (TTS) API, and the server sends the generated voice data to the user's device and stores it on the user's device.

[0676] To check what to prepare for the next day, the user launches the application again and uses voice input or touch operations to say, "Tell me what to prepare for tomorrow." The user's device detects this and plays back the saved voice data. For example, the user might be reminded, "Tomorrow's items to bring are textbooks (Japanese and math), arts and crafts kit, gym clothes, and sneakers."

[0677] The same can be applied to factory production lines. When a worker asks, "What should I prepare for tomorrow?", a voice message will be sent reminding them of the tools and parts they will need based on the next day's production schedule. For example, they might be reminded, "Tomorrow's items to bring are a wrench and a screwdriver, and since it's a rainy day, don't forget to bring rain gear."

[0678] Hardware and software:

[0679] Hardware: smart glasses (e.g. Google Glass), smartphones, servers

[0680] Software: EmotionEngine (emotion analysis engine), TTSAPI (text-to-speech software), Weather API (weather information acquisition)

[0681] Examples:

[0682] Example 1: A production line worker asks, "What should I prepare for tomorrow?" and is reminded, "Tomorrow, bring a wrench, a screwdriver, and don't forget your rain gear because it's a rainy day."

[0683] Example 2: When a student asks, "What should I prepare for tomorrow?", they are reminded, "Tomorrow, bring your textbooks (Japanese and math), arts and crafts kit, gym clothes, and sneakers."

[0684] Example prompt sentence:

[0685] "Tell me what to bring tomorrow. Add a feature that will remind me by voice what tools I need, like wrenches, screwdrivers, etc."

[0686] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0687] Step 1:

[0688] The user enters their credentials on the login screen.

[0689] Input: Username, Password

[0690] Data processing: The user terminal encrypts the entered authentication information and sends it to the server.

[0691] Output: The server receives the authentication information and checks it against its database.

[0692] Specific operation: The server checks the authentication information in the database, and if authentication is successful, displays the home screen on the user's device.

[0693] Step 2:

[0694] The user enters timetable information and special notes on the home screen.

[0695] Input: Timetable information, special notes (lessons, specific processes, etc.)

[0696] Data processing: The user terminal formats the entered information and sends it to the server.

[0697] Output: The server stores the received information in a database.

[0698] Specific operation: The server uses the timetable information and special notes stored in the database to create the schedule for the next day.

[0699] Step 3:

[0700] The server generates a list of necessary items to bring based on the timetable information.

[0701] Input: Timetable information, special notes

[0702] Data processing: The server lists the necessary materials and parts for each subject and process. It also uses a weather API to obtain weather information and predicts what additional items to bring.

[0703] Output: List of necessary items

[0704] Specific actions: For example, if it's Friday and you have exercise, add sneakers to the list, or if it's a rainy day, add rain gear to the list.

[0705] Step 4:

[0706] The server converts the inventory list into audio data.

[0707] Input: inventory list

[0708] Data processing: The server uses a text-to-speech (TTS) API to convert the inventory list into audio data.

[0709] Output: Audio data

[0710] Specific operation: Input the inventory list text into the TTS API and generate a corresponding audio file.

[0711] Step 5:

[0712] The server transmits the voice data to the user terminal.

[0713] Input: Audio data

[0714] Data processing: The server sends the generated voice data to the user terminal.

[0715] Output: Audio data stored on the user's device

[0716] Specific operation: The server transfers the voice data to the user device, and the user device stores it in its internal storage.

[0717] Step 6:

[0718] The user requests a packing list reminder via voice or touch.

[0719] Input: Voice input, touch operation

[0720] Data processing: The user device recognizes voice input and touch operations and sends a reminder request to the server.

[0721] Output: Remind request

[0722] Specific operation: When the device receives a reminder request, it plays the saved audio data.

[0723] Step 7:

[0724] The server uses an emotion analysis engine to analyze the user's emotional state.

[0725] Input: User's voice input, facial expression data

[0726] Data processing: The emotion analysis engine analyzes the user's tone of voice and facial expressions to identify their emotional state.

[0727] Output: User's emotional state

[0728] Specific behavior: For example, if the user is feeling anxious, the engine generates an "encouraging message."

[0729] Step 8:

[0730] The user terminal plays back the audio data.

[0731] Input: Voice data, emotion analysis results

[0732] Data processing: The audio data is adjusted according to the emotion and played back to the user.

[0733] Output: Play reminder message

[0734] Specific operation: For example, the reminder message "Tomorrow, bring a wrench, a screwdriver, and rain gear" is played.

[0735] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0736] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0737] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0738] [Third embodiment]

[0739] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0740] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0741] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0742] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0743] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0744] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0745] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0746] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0747] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0748] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0749] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0750] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0751] This invention is a system that uses a user terminal to assist in preparing school supplies. It generates a list of necessary supplies based on the next day's schedule and provides voice reminders, reducing the burden on children and their parents.

[0752] System implementation

[0753] Users launch a dedicated application using a user device such as a smartphone or tablet. First, a parent or child logs in to the app and sends authentication information to the server. The server receives this authentication information and authenticates the user. If authentication is successful, the user is taken to the home screen.

[0754] Entering timetable information

[0755] The user (parent or child) then enters information about the child's school into the application, including the child's name, class, and tomorrow's schedule. This data is sent to the server, where it is stored in a database.

[0756] Generate a packing list

[0757] The server analyzes the received timetable information and identifies the teaching materials and items required for each subject. It also predicts additional items to bring based on the day of the week and the season. For example, it determines that sneakers are needed on Fridays and jackets are needed in spring. This information is combined to generate the final packing list.

[0758] Generating and transmitting audio data

[0759] The inventory list is then converted into audio data using a text-to-speech (TTS) API, which the server then sends to the user's device via HTTP or HTTPS.

[0760] Reminders on user devices

[0761] The user (child) opens the application again before the next day and requests, by voice or touch, "Tell me what to prepare for tomorrow." The user's device plays the saved voice data and announces the list of items to bring. For example, the user might be reminded, "Tomorrow's items to bring are textbooks (Japanese and math), arts and crafts kit, gym clothes, and sneakers."

[0762] Application Examples

[0763] This system can be used to provide not only reminders for inventory lists, but also medication reminders and childcare reminders. For example, an elderly person can be set to be reminded to take their medicine at 6 p.m. In this case, the server generates audio data at the set time and sends it to the device. The device then plays the audio data at the set time, reminding the user to "take their medicine at 6 p.m."

[0764] Through the detailed steps described above, the present invention provides effective support for users and their guardians to prepare all necessary belongings and start their new life with peace of mind.

[0765] The processing flow will be explained below.

[0766] Step 1:

[0767] The user starts the application on their smartphone or tablet and enters their user ID and password on the login screen, which is then sent to the server.

[0768] Step 2:

[0769] The server receives the user's authentication information and checks it against a database. If authentication is successful, the login is approved and a signal is sent to the user's device to display the home screen.

[0770] Step 3:

[0771] The user enters their child's information (name, class, next day's schedule) on the home screen. The entered information is sent to the server and stored in a database.

[0772] Step 4:

[0773] The server analyzes the received timetable information. First, it identifies the teaching materials and items required for each subject. For this analysis, it uses the standard list of items to bring for each subject in the timetable (e.g., Japanese textbook, notebook, pencil case).

[0774] Step 5:

[0775] The server predicts what additional items to bring based on the day of the week, season, and weather information obtained from the weather API. For example, if it's Friday, sneakers are added to the list, and if it's a rainy day, rain gear is added.

[0776] Step 6:

[0777] The server generates a list of required items and converts it into audio data using a Text-to-Speech (TTS) API, which is then stored on the server.

[0778] Step 7:

[0779] The server sends the voice data to the user's device. The communication protocol is HTTP or HTTPS. The sent voice data is saved on the user's device.

[0780] Step 8:

[0781] To check the preparations for the next day, the user launches the application again and says "Check tomorrow's preparation list" by voice input or touch operation. The user's device detects this and plays the saved voice data.

[0782] Step 9:

[0783] The user device plays the audio data and notifies the user of the list of items to bring, allowing the user (child) to check what to bring the next day and prepare appropriately.

[0784] Step 10:

[0785] The user device can extend the reminder function to notify users of medication reminders at specific times. For example, it can play a voice reminder to an elderly person saying, "Take your medicine at 6 p.m."

[0786] The above is the specific processing flow of this system.

[0787] Example 1

[0788] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0789] In today's busy households, it is difficult to properly prepare school supplies. Especially when children prepare their own school supplies, forgetting or failing to prepare frequently causes considerable stress for both children and their parents. Furthermore, it is difficult to accurately track what school supplies should be prepared for different days of the week and seasons, so a system that automates this process and efficiently manages them is needed.

[0790] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0791] In this invention, the server includes a means for receiving timetable information of the educational institution input by the user terminal, a means for predicting items needed based on a specific day of the week or season, and a means for generating a list of needed items, thereby enabling children and their parents to efficiently prepare their belongings and prevent them from forgetting their belongings.

[0792] "User terminal" refers to a portable information device for inputting and receiving timetable information and inventory lists for educational institutions.

[0793] "Educational institutions" refers to schools, cram schools, and other educational facilities.

[0794] "Timetable information" refers to schedule information for classes and activities at an educational institution, and is a list of subjects and activities that take place on specific days and at specific times.

[0795] "Items" refers to teaching materials and tools necessary for lessons and activities.

[0796] "Predictive means" refers to the ability of a device or program to automatically determine what items are needed based on specific days of the week or seasons.

[0797] The "means for generating a list" refers to the function of a device or program that organizes predicted items and creates a list in a list format.

[0798] "Means for converting into audio data" refers to text-to-speech technology that converts the generated item list into natural audio.

[0799] The "means for transmitting voice data" refers to a communication technology for transmitting the generated voice data to a user terminal.

[0800] "Means for playing" refers to the function of a device or program for playing back audio data received on a user terminal.

[0801] "Text reminder" refers to a format that notifies you of your belongings using text information rather than audio data.

[0802] This invention is a system that uses user terminals to assist educational institutions in preparing belongings. It generates a list of necessary items based on the next day's schedule and reminds students of this list via voice, thereby reducing the burden on students and their guardians.

[0803] First, the user launches a dedicated application using a user device such as a smartphone or tablet. The user (guardian or beneficiary) enters their user ID and password on the login screen and sends them to the server. The sent authentication information is received by the server, which then collates it with information in a database to authenticate the user. If authentication is successful, the user is taken to the home screen.

[0804] Next, the user enters the recipient's name, class, and the next day's schedule information into a dedicated form in the application. This information is formatted by the user's terminal and sent to the server, which receives the information and stores it in a database.

[0805] Based on the submitted timetable information, the server identifies the items required for each subject. It then analyzes the day of the week and seasonal information to predict additional items. For example, it determines that sneakers are needed on Fridays and jackets are needed in spring. This information is then combined to generate the final item list.

[0806] The generated inventory list is converted into voice data using a text-to-speech (TTS) software API, and the server transmits the converted voice data to the user's device using the HTTP or HTTPS protocol.

[0807] The user (beneficiary) opens the application again before the next day and requests, by voice input or touch operation, "Tell me what to prepare for tomorrow." The user device plays the saved voice data and reminds the user, "Tomorrow's items to bring are textbooks (Japanese and math), arts and crafts kit, gym clothes, and sneakers."

[0808] As a specific example of how this works, for example, a list of belongings generated based on Monday's timetable would include "Japanese textbook, math notebook, gym clothes." If it's Friday, "sneakers" would be added. If it's spring, "jacket" would be added.

[0809] By using the following prompts to the generative AI model, you can elicit specific explanations of the system's processing flow.

[0810] Example prompt:

[0811] "Please explain in detail how this system reminds users of their belongings. For example, please explain in detail the process by which the server analyzes the timetable information and generates the belongings list."

[0812] Through the above process, the present invention provides effective support for the user and guardian to prepare all necessary belongings and start their new life with peace of mind.

[0813] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0814] Step 1: User launches application and logs in

[0815] The user launches the dedicated application using the user device. After launching, the user (guardian or beneficiary) enters their user ID and password on the login screen and taps the login button. The user device sends the entered authentication information to the server. The server compares the received authentication information with information in the database and authenticates the user. If authentication is successful, the server sends a success message and the user device transitions to the home screen.

[0816] Input: User ID, Password

[0817] Data processing: User ID and password verification

[0818] Output: Authentication success message

[0819] Specific operation: When the user taps the login button, authentication information is sent, the server authenticates it, and returns a success message.

[0820] Step 2: Enter your school and timetable information

[0821] The user (parent or beneficiary) enters the beneficiary's name, class, and next day's timetable information into a dedicated form in the application. The user's device formats the entered information and sends it to the server. The server receives this information and stores it in a database.

[0822] Input: Beneficiary's name, class, next day's timetable information

[0823] Data processing: Formatting and sending information

[0824] Output: Acknowledgement of receipt of information

[0825] Specific operation: When a user enters information into a dedicated form and taps the submit button, the information is sent to the server and stored in a database.

[0826] Step 3: Generate a packing list

[0827] The server analyzes the received timetable information and identifies the items needed for each subject stored in the database. It also analyzes information about the day of the week and the season to predict additional items. It then combines this information to generate the final packing list.

[0828] Input: Timetable information, day of the week information, seasonal information

[0829] Data calculation: Identifying items and predicting additional items

[0830] Output: Final inventory list

[0831] Specific operation: The server analyzes the timetable information, adds items according to the day of the week and season, and creates a list of items to bring.

[0832] Step 4: Generate and send audio data

[0833] The server converts the generated inventory list into audio data using a text-to-speech software (TTS) API, and sends the converted audio data to the user's device using HTTP or HTTPS protocol.

[0834] Input: Final packing list

[0835] Data processing: conversion to audio data

[0836] Output: Audio data

[0837] Specific operation: The server sends the inventory list to the TTS API, and the generated voice data is sent to the user's terminal.

[0838] Step 5: Remind users on their devices

[0839] The user (beneficiary) opens the application again before the next day and requests, by voice input or touch operation, "Tell me what to prepare for tomorrow." The user device plays the saved voice data and reminds the user, "Tomorrow's items to bring are textbooks (Japanese and math), arts and crafts kit, gym clothes, and sneakers."

[0840] Input: Voice or touch

[0841] Data processing: processing requests and playing audio data

[0842] Output: Voice reminder

[0843] Specific operation: When the user voice-inputs "Tell me how to prepare for tomorrow" or taps a button, the device plays the voice data as a reminder.

[0844] (Application example 1)

[0845] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0846] When shopping in a physical store, it is difficult for users to efficiently find the items they need to purchase, resulting in unnecessary time and effort. Furthermore, especially in large stores, it is not uncommon for users to miss or have difficulty finding items that they need to purchase based on their shopping list. Furthermore, there is a lack of a reminder function to ensure users don't forget to purchase the items they need. Therefore, a system that solves this problem is needed.

[0847] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0848] In this invention, the server includes means for receiving school timetable information input by a user terminal, means for predicting items needed based on specific days of the week and seasons, means for generating a list of items needed, means for converting the generated list into voice data, means for transmitting the voice data to the user terminal, means for playing the transmitted voice data on the user terminal, and means for guiding the location of items included in the list based on location information of each department in the store, thereby enabling users to shop efficiently in a physical store and purchase all necessary items.

[0849] A "user terminal" is a device such as a smartphone, tablet, or smart glasses that is directly operated by a user to input and receive information.

[0850] "School timetable information" is data that includes information about the times and contents of classes that students will take the next day and on specific days, as well as the teaching materials and items they will need to take with them.

[0851] "Day-of-the-week and season-based items" refers to items or clothing needed on specific days of the week or during specific seasons, such as sneakers needed on Fridays or a jacket needed in the spring.

[0852] The "list of necessary items" is a list of all items needed for the next day or a specific day, generated based on the user's input information and predicted day of the week and season information.

[0853] The "means for converting into audio data" is a technique for converting the inventory list in text format into audio format and generating an audio file or audio stream that can be listened to by the user.

[0854] The "means for transmitting voice data to a user terminal" refers to a communication means for transmitting voice data generated by the server to a user terminal via the Internet.

[0855] "Means for reproducing audio data at a user terminal" refers to a technique for reproducing audio data received by a user terminal and providing it to the user as auditory information.

[0856] "Location information for each department within the store" is information about where each item is displayed within the physical store, and is data indicating in which department or section a particular item is located.

[0857] "Item location guidance" refers to technology that guides users to specific locations or departments within a store to help them efficiently find items on their shopping list.

[0858] The present invention is a system for supporting efficient shopping in a physical store using a user terminal. Specific embodiments of this system will be described below.

[0859] System implementation

[0860] A user launches a dedicated shopping assistant application using a user device such as a smartphone or smart glasses. First, the user logs in to the application and sends authentication information to the server. The server receives this authentication information and performs user authentication. If authentication is successful, the user is taken to the application's home screen.

[0861] Entering a shopping list

[0862] The user enters the items they want to include in their shopping list into the application, and this data is sent to the server where it is stored in a database.

[0863] Shopping list location guide

[0864] The server analyzes the received shopping list information and identifies which department in the store each item is located in. Based on the store location information, it generates a list to guide the user to the location of the required items. The list information includes the location and department details of each item.

[0865] Generating and transmitting audio data

[0866] The generated shopping list and location information is converted into audio data using a Text-to-Speech (TTS) API, and the server generates this audio data and sends it to the user's device using HTTP or HTTPS protocols.

[0867] Reminders and guidance on user devices

[0868] The user uses the application in the store and requests, by voice or touch, "What item should I buy next?" The user's device plays the saved voice data and provides a voice prompt of the items on the list and their location. For example, the user might be reminded, "The next item to buy is milk. It's in the refrigerated section."

[0869] Specific processing

[0870] Hardware: Smartphones, smart glasses

[0871] Software: Dedicated shopping assistant application, server-side authentication system, database, Text-to-Speech (TTS) API

[0872] Data processing flow:

[0873] 1. The user logs in to the app and sends their credentials to the server.

[0874] 2. The server authenticates the user and receives and stores the shopping list data.

[0875] 3. The server analyzes the information in the list and determines the location of the item within the store.

[0876] 4. Based on the location information, the list is converted into audio data and sent to the user's device.

[0877] 5. The user terminal plays back the received voice data and provides the user with a voice guide of the next item to be purchased and its location.

[0878] Specific examples

[0879] For example, a user logs in to the application and enters "milk," "bread," and "detergent" as a list of items to be purchased next. The server uses this information to identify which section of the physical store each item is located in, and generates a voice reminder based on the location information of the refrigerated goods section, bakery section, and daily necessities section. This reminder is played on the user's device, supporting the user in efficient shopping.

[0880] Prompt Sentence Examples

[0881] Develop an application that helps users shop efficiently in physical stores. The main features are shopping list creation and voice reminders. The reminders guide users through their shopping by guiding them to the next item to be purchased and its location.

[0882] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0883] Step 1:

[0884] The user launches the application and enters authentication information on the login screen. The authentication information includes the user ID and password. The server authenticates the user based on this information. The server compares the authentication information with records in the database, and if there is a match, returns a status of successful authentication. The user's device receives a notification of successful authentication and moves to the home screen.

[0885] Step 2:

[0886] The user inputs a shopping list into the application. This list contains the names of multiple items the user plans to purchase. The input data is sent to the server as form data. The server receives this data and saves the shopping list in a database. Input: Shopping list (item names). Output: Shopping list saved in the database.

[0887] Step 3:

[0888] The server analyzes the shopping list information and identifies the location information for each item in the store. It references the location information in the database to determine which section or department each item is located in. Based on this data, it generates a list containing location information for each item. Input: Saved shopping list. Output: List containing location information for the store.

[0889] Step 4:

[0890] The server converts the list generated by the server into audio data using a Text-to-Speech (TTS) API. The list information in text format is input to the API, which generates an audio file or audio stream. Input: List containing in-store location information. Output: Audio data.

[0891] Step 5:

[0892] The voice data is sent to the user terminal using the HTTP or HTTPS protocol. The server uses a communication means to send the generated voice data to the user terminal. Input: Voice data. Output: Voice data sent to the user terminal.

[0893] Step 6:

[0894] The user reopens the application in the store and requests, by voice or touch, "Tell me what item I'll buy next." The user device receives this request and plays the saved voice data. Input: User request. Output: Played voice data (reminder).

[0895] Step 7:

[0896] The user device plays the audio data and provides audible instructions about the next item in the list and its location. For example, "The next item to purchase is milk. It's in the refrigerated section." Input: The stored audio data. Output: Audio instructions provided to the user.

[0897] keyword:

[0898] Generative AI model, prompt sentence

[0899] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0900] This invention is a system that uses a user terminal to help users prepare school supplies. It generates a list of necessary supplies based on the next day's schedule and reminds users of the list by voice and text. In addition, by combining it with an emotion engine, the content of the reminder can be flexibly adjusted according to the user's emotions.

[0901] System implementation

[0902] First, the user launches a dedicated application using a user device such as a smartphone or tablet. The user (parent or child) enters authentication information on the login screen and sends it to the server. The server authenticates the user using the received information, and if authentication is successful, the home screen is displayed on the user device.

[0903] Entering timetable information

[0904] The user enters basic information about their child (name, class), the next day's schedule, and any special notes (lessons, health, etc.) on the home screen. This information is sent to the server and stored in a database.

[0905] Generate a packing list

[0906] The server identifies the materials and supplies needed for each subject based on the timetable information. It also predicts what additional items students need to bring based on the day of the week, season, and weather information obtained from a weather API. For example, if it's Friday, sneakers are added to the list, and if it's a rainy day, rain gear is added.

[0907] Use of emotion engine

[0908] Furthermore, the app will introduce an emotion engine that analyzes the user's voice input and facial expressions. The camera captures the user's voice tone and facial expressions when they input information into the app, and the emotion engine analyzes them. Based on the analysis results, if the user is feeling anxious or stressed, the reminder content will be made more detailed or an encouraging message will be added. Conversely, if the user is feeling positive, the app will send reminders that emphasize successful experiences.

[0909] Generating and transmitting audio data

[0910] The list of necessary items is converted into voice data using a text-to-speech software (TTS) API, and the server sends the generated voice data to the user's device and stores it on the user's device.

[0911] Reminders on user devices

[0912] To check what to prepare for the next day, the user launches the application again and uses voice input or touch operations to say, "Tell me what to prepare for tomorrow." The user's device detects this and plays back the saved voice data. For example, the user may be reminded, "Tomorrow's items to bring are textbooks (Japanese and math), arts and crafts kit, gym clothes, and sneakers."

[0913] Application Examples

[0914] This system can be applied not only to reminders of inventory lists, but also to medication reminders and childcare reminders. For example, it can be set to notify an elderly person, "Please take your medicine at 6 p.m." In this case, the emotion engine can provide more detailed explanations or add encouraging messages to anxious users.

[0915] Through the detailed steps described above, the present invention provides effective support for users to prepare all their belongings and live their daily lives with peace of mind.The introduction of an emotion engine enables flexible reminders that respond to the emotions of each individual user.

[0916] The processing flow will be explained below.

[0917] Step 1:

[0918] The user starts the application on their smartphone or tablet and enters their user ID and password on the login screen. The entered authentication information is sent to the server.

[0919] Step 2:

[0920] The server receives the user's authentication information and checks it against a database. If authentication is successful, the login is approved and a signal is sent to the user's device to display the home screen.

[0921] Step 3:

[0922] The user enters their child's information (name, class, next day's schedule) on the home screen. The entered information is sent to the server and stored in a database.

[0923] Step 4:

[0924] The server analyzes the received timetable information and identifies the materials and items required for each subject. For example, it determines that a Japanese language class requires a Japanese language textbook, notebook, and pencil case.

[0925] Step 5:

[0926] The server predicts additional items to bring based on the day of the week, season, and weather information obtained from the weather API. For example, if it's Friday, it will add "sneakers," and if rain is expected, it will add "rain gear."

[0927] Step 6:

[0928] The server generates the required inventory list, which is done using the generated text list.

[0929] Step 7:

[0930] The inventory list is converted into audio data using a text-to-speech (TTS) software API specified by the server, and the converted audio data is stored on the server.

[0931] Step 8:

[0932] The server sends the audio data to the user's device using the HTTP or HTTPS protocol, and the audio data is stored on the user's device.

[0933] Step 9:

[0934] To check the preparations for the next day, the user launches the application again and says "Check tomorrow's preparation list" by voice input or touch operation. The user's device detects this and plays the saved voice data.

[0935] Step 10:

[0936] The user device plays the audio data and notifies the user of the list of items to bring. For example, it may remind the user that "Tomorrow's items to bring are textbooks (Japanese and math), arts and crafts kit, gym clothes, and sneakers."

[0937] Step 11:

[0938] When using the emotion engine, when a user inputs information into the app, the device's camera and microphone capture the user's facial expressions and tone of voice, and this emotional data is sent to the server.

[0939] Step 12:

[0940] The emotion engine on the server analyzes the user's emotional data. If the user is feeling anxious or stressed, the system will adjust the reminder content by providing a detailed explanation of the item to bring. Conversely, if the user is expressing positive emotions, the system will send reminders that emphasize successful experiences.

[0941] Step 13:

[0942] The server converts the adjusted reminder content back into audio data and transmits it to the user terminal, which plays back the audio data and notifies the user of the adjusted reminder content.

[0943] The above is the specific processing flow when combining the emotion engine of this system.

[0944] Example 2

[0945] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0946] Conventional luggage preparation support systems have the function of generating a luggage list based on the user's input timetable information, season, and day of the week, but they lack the ability to provide reminders that take into account the user's emotional state. As a result, luggage preparation may not be carried out efficiently, and the user's psychological support may be insufficient. Furthermore, since voice reminders alone do not provide enough visual information, a more effective notification method is desired.

[0947] The specification process by the specification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for receiving school timetable information input by a user terminal, a means for predicting necessary belongings based on specific days of the week and seasons, and a means for adjusting reminder content using an emotion engine that analyzes the user's voice input and facial expressions. This makes it possible to provide flexible reminders that take into account the user's emotional state, allowing the user to prepare belongings more efficiently and with peace of mind.

[0948] A "user terminal" is an electronic device, such as a smartphone, tablet, or PC, that a user uses to input information or receive notifications.

[0949] "Timetable information" is information that indicates the schedule of school lessons and activities, including subjects, days of the week, and time periods.

[0950] "Items required for specific days of the week or seasons" are items that students are required to bring depending on the content of the class, weather, or season. For example, sneakers are required on days when there is a physical education class, and rain gear is required on days when rain is expected.

[0951] A "list of necessary belongings" is a list of items that a user should bring with them the next day or on a specific day.

[0952] "Audio Data" means text information converted into a format that can be played as sound, and is generated using a Text-to-Speech (TTS) API.

[0953] An "emotion engine" refers to software or hardware that has the functionality to analyze a user's tone of voice and facial expressions and assess their emotional state.

[0954] "Adjusting reminder content" refers to changing the content and format of notifications and messages provided based on the user's emotional state. For example, a user who is feeling anxious can receive additional detailed explanations or encouraging messages.

[0955] "Text reminders" refer to a method of providing notifications and instructions to users through text information that can be read as words. These can be displayed visually on the screen of the user's device.

[0956] The present invention is a system that uses a user terminal to assist in preparing school supplies, and is implemented by the following procedure. First, the user launches a dedicated application on the user terminal, such as a smartphone or tablet. The user enters authentication information on the login screen and sends it to the server. The server authenticates the user using the received information, and if authentication is successful, displays the home screen on the user terminal.

[0957] Entering timetable information

[0958] The user enters basic information about their child (name, class), the next day's schedule, and any special notes (lessons, health, etc.) on the home screen. This information is sent from the user's device to the server and stored in the server's database.

[0959] Generate a packing list

[0960] The server needs to identify the necessary materials and belongings for each subject based on the timetable information. It also predicts what additional items to bring based on the day of the week, season, and weather information obtained from a weather API. For example, if it's Friday, sneakers are added to the list, and if it's a rainy day, rain gear is added.

[0961] Use of emotion engine

[0962] An emotion engine is introduced to analyze the user's voice input and facial expressions. When the user inputs voice or facial expressions into the app, this is captured using a camera and microphone and sent to a server. The emotion engine on the server analyzes this and adjusts the reminder content based on the analysis results. For example, if the user is feeling anxious or stressed, the reminder content can be made more detailed or an encouraging message can be added. Conversely, if the user is feeling positive, a reminder that emphasizes successful experiences can be sent.

[0963] Generating and transmitting audio data

[0964] The inventory list is converted into audio data using a text-to-speech (TTS) software API, and the server sends the generated audio data to the user's device and stores it on the user's device.

[0965] Reminders on user devices

[0966] To check what to prepare for the next day, the user launches the application again and uses voice input or touch operations to say, "Tell me what to prepare for tomorrow." The user's device detects this and plays back the saved voice data. For example, the user might be reminded, "Tomorrow's items to bring are textbooks (Japanese and math), arts and crafts kit, gym clothes, and sneakers."

[0967] Application Examples

[0968] This system can be applied not only to reminders of inventory lists, but also to medication reminders and childcare reminders. For example, it can be set to notify an elderly person, "Please take your medicine at 6 p.m." In this case, the emotion engine can provide more detailed explanations or add encouraging messages to anxious users.

[0969] Specific examples

[0970] For example, if the next day's schedule looks like this:

[0971] Day of the week: Friday

[0972] Subjects: Japanese, arithmetic, art, physical education

[0973] Special note: Plans to join soccer club after school

[0974] Weather information: Rain forecast

[0975] In this case, the server adds the following items to the list:

[0976] Japanese language textbook

[0977] Mathematics textbook

[0978] Art and craft set

[0979] gym clothes

[0980] athletic shoes

[0981] Soccer practice equipment

[0982] rain gear

[0983] Example of input prompt for generative AI model

[0984] "Please provide the next day's schedule information and weather forecast in the following format, and generate a list of items you need to bring. Also, please generate a reminder message based on the user's emotions.

[0985] (Timetable information)

[0986] Day of the week: Friday

[0987] Subjects: Japanese, arithmetic, art, physical education

[0988] Special note: Plans to join soccer club after school

[0989] (weather forecast)

[0990] Weather: Rain

[0991] (User sentiment)

[0992] Voice Tone: A little anxious

[0993] Expression: Not calm

[0994] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0995] Step 1:

[0996] The user launches the app and authenticates

[0997] The user launches a dedicated application on their smartphone or tablet. When the login screen appears, the user enters their email address and password and taps the "Login" button. This input information is sent to the server. The server compares the received authentication information with the database, and if authentication is successful, the server sends a successful authentication response to the user's device, and the home screen is displayed on the user's device. The input is the email address and password, and the output is the user's home screen.

[0998] Step 2:

[0999] The user enters the next day's schedule information and special notes

[1000] The user opens a form on the home screen to enter the child's name, class, next day's schedule information, and any special notes. When the user enters this information and taps the "Submit" button, the entered data is sent to the server. The server stores the received information in a database. The input at this time is the child's name, class, schedule information, and special notes, and the output is the information stored in the database.

[1001] Step 3:

[1002] Server generates inventory list

[1003] The server retrieves timetable information stored in the database and identifies the necessary items for each subject. In doing so, the server also takes into account the day of the week, season, and weather information retrieved from the weather API. For example, if there is a physical education class on Friday and rain is expected, sneakers and rain gear will be added to the list of items to bring. The input is the timetable information, season information, and weather information from the database, and the output is a list of necessary items to bring.

[1004] Step 4:

[1005] Use of emotion engine

[1006] When a user inputs voice or facial expressions into the app, the device's camera and microphone capture this and send the data to the server. The server's emotion engine analyzes this and evaluates the user's emotional state. Based on the analysis results, the server adjusts the reminder content. For example, if the user is feeling anxious, it may add a detailed explanation or an encouraging message. The input data at this time is audio and video data, and the output is the adjusted reminder content.

[1007] Step 5:

[1008] Convert your inventory list into audio data and send it

[1009] The server converts the final inventory list into voice data using a text-to-speech (TTS) API. This generated voice data is sent to the user's device and stored on the device. The input is the inventory list, and the output is voice data.

[1010] Step 6:

[1011] Reminders on user devices

[1012] The user launches the app again to check the next day's preparations, and issues instructions by voice input or touch operation, such as "Tell me what to prepare for tomorrow." The device detects this and plays back the saved voice data. For example, the device may be reminded, "Tomorrow's items to bring are textbooks (Japanese and math), arts and crafts kit, gym clothes, and sneakers." The input is the user's voice input or touch operation, and the output is the voice playback data.

[1013] (Application example 2)

[1014] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1015] Previous school baggage preparation support systems were limited to generating a baggage list based on the next day's schedule, and were unable to flexibly respond to the user's emotional state or specific circumstances. They also lacked appropriate reminders for items to be prepared on the factory production line, and lacked emotional support for users. This led to missed items and emotional stress.

[1016] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1017] In this invention, the server includes means for receiving school timetable information entered by a user terminal, means for predicting necessary items based on a specific day of the week or season, means for generating a list of necessary items, means for converting the generated list into voice data, means for transmitting the voice data to the user terminal, means for playing the transmitted voice data on the user terminal, means for identifying necessary items based on the schedule obtained from the server, means for analyzing the user's emotional state using an emotion analysis engine, and means for adjusting reminder messages according to the user's emotional state. This enables flexible reminders according to the user's emotional state, prevents users from forgetting to bring or prepare items, and provides emotional support.

[1018] "School timetable information" is information necessary for students to participate in classes and extracurricular activities, and indicates the dates, times, and locations of specific subjects and activities.

[1019] "Day-of-the-week and season-based supplies" are items that are needed for specific activities or lessons depending on the day of the week or the season.

[1020] A "packing list" is a list of items that a user needs to bring with them the next day or on a specific day.

[1021] "Audio data" refers to data obtained by converting text-format information into audio, and is used by the user to receive information aurally.

[1022] A "user terminal" is an electronic device that is personally owned by a user, such as a smartphone, tablet, or smart glasses.

[1023] A "server" is a central device that receives information from multiple user terminals, processes the information, and returns appropriate data.

[1024] An "emotion analysis engine" is a program that analyzes the user's emotional state from voice input and facial expressions, and allows the system to respond based on the results.

[1025] A "remind message" refers to a notification message that provides the user with necessary information or encourages them to take action.

[1026] "Schedules acquired from a server" refers to data on pre-set plans or schedules, such as factory production lines or school timetables, acquired by a server.

[1027] The present invention relates to a system for supporting the preparation of belongings for school or a production line using a user terminal. Specific embodiments will be described below.

[1028] First, the user launches a dedicated application using a user device such as a smartphone or smart glasses. The user (parent or worker) enters authentication information on the login screen and sends it to the server. The server authenticates the user using the received information, and if authentication is successful, the home screen is displayed on the user device.

[1029] Next, the user enters basic information (name, class or production line), the next day's timetable and production schedule, and any special notes (lessons or specific processes) on the home screen. This information is sent to the server and stored in a database.

[1030] The server identifies the materials and parts needed for each subject and process based on timetable information or production schedules. It also predicts what additional items to bring based on the day of the week, season, and weather information obtained from a weather API. For example, if it's Friday, sneakers are added to the list, and if it's a rainy day, rain gear is added.

[1031] Furthermore, a pre-installed emotion analysis engine is introduced. The emotion engine analyzes the user's voice input and facial expressions. Based on the analysis results, if the user is feeling anxious or stressed, the reminder content will be made more detailed or an encouraging message will be added. Conversely, if the user is feeling positive, a reminder that emphasizes successful experiences will be sent.

[1032] The list of necessary items is converted into voice data using a text-to-speech software (TTS) API, and the server sends the generated voice data to the user's device and stores it on the user's device.

[1033] To check what to prepare for the next day, the user launches the application again and uses voice input or touch operations to say, "Tell me what to prepare for tomorrow." The user's device detects this and plays back the saved voice data. For example, the user might be reminded, "Tomorrow's items to bring are textbooks (Japanese and math), arts and crafts kit, gym clothes, and sneakers."

[1034] The same can be applied to factory production lines. When a worker asks, "What should I prepare for tomorrow?", a voice message will be sent reminding them of the tools and parts they will need based on the next day's production schedule. For example, they might be reminded, "Tomorrow's items to bring are a wrench and a screwdriver, and since it's a rainy day, don't forget to bring rain gear."

[1035] Hardware and software:

[1036] Hardware: smart glasses (e.g. Google Glass), smartphones, servers

[1037] Software: EmotionEngine (emotion analysis engine), TTSAPI (text-to-speech software), Weather API (weather information acquisition)

[1038] Examples:

[1039] Example 1: A production line worker asks, "What should I prepare for tomorrow?" and is reminded, "Tomorrow, bring a wrench, a screwdriver, and don't forget your rain gear because it's a rainy day."

[1040] Example 2: When a student asks, "What should I prepare for tomorrow?", they are reminded, "Tomorrow, bring your textbooks (Japanese and math), arts and crafts kit, gym clothes, and sneakers."

[1041] Example prompt sentence:

[1042] "Tell me what to bring tomorrow. Add a feature that will remind me by voice what tools I need, like wrenches, screwdrivers, etc."

[1043] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1044] Step 1:

[1045] The user enters their credentials on the login screen.

[1046] Input: Username, Password

[1047] Data processing: The user terminal encrypts the entered authentication information and sends it to the server.

[1048] Output: The server receives the authentication information and checks it against its database.

[1049] Specific operation: The server checks the authentication information in the database, and if authentication is successful, displays the home screen on the user's device.

[1050] Step 2:

[1051] The user enters timetable information and special notes on the home screen.

[1052] Input: Timetable information, special notes (lessons, specific processes, etc.)

[1053] Data processing: The user terminal formats the entered information and sends it to the server.

[1054] Output: The server stores the received information in a database.

[1055] Specific operation: The server uses the timetable information and special notes stored in the database to create the schedule for the next day.

[1056] Step 3:

[1057] The server generates a list of necessary items to bring based on the timetable information.

[1058] Input: Timetable information, special notes

[1059] Data processing: The server lists the necessary materials and parts for each subject and process. It also uses a weather API to obtain weather information and predicts what additional items to bring.

[1060] Output: List of necessary items

[1061] Specific actions: For example, if it's Friday and you have exercise, add sneakers to the list, or if it's a rainy day, add rain gear to the list.

[1062] Step 4:

[1063] The server converts the inventory list into audio data.

[1064] Input: inventory list

[1065] Data processing: The server uses a text-to-speech (TTS) API to convert the inventory list into audio data.

[1066] Output: Audio data

[1067] Specific operation: Input the inventory list text into the TTS API and generate a corresponding audio file.

[1068] Step 5:

[1069] The server transmits the voice data to the user terminal.

[1070] Input: Audio data

[1071] Data processing: The server sends the generated voice data to the user terminal.

[1072] Output: Audio data stored on the user's device

[1073] Specific operation: The server transfers the voice data to the user device, and the user device stores it in its internal storage.

[1074] Step 6:

[1075] The user requests a packing list reminder via voice or touch.

[1076] Input: Voice input, touch operation

[1077] Data processing: The user device recognizes voice input and touch operations and sends a reminder request to the server.

[1078] Output: Remind request

[1079] Specific operation: When the device receives a reminder request, it plays the saved audio data.

[1080] Step 7:

[1081] The server uses an emotion analysis engine to analyze the user's emotional state.

[1082] Input: User's voice input, facial expression data

[1083] Data processing: The emotion analysis engine analyzes the user's tone of voice and facial expressions to identify their emotional state.

[1084] Output: User's emotional state

[1085] Specific behavior: For example, if the user is feeling anxious, the engine generates an "encouraging message."

[1086] Step 8:

[1087] The user terminal plays back the audio data.

[1088] Input: Voice data, emotion analysis results

[1089] Data processing: The audio data is adjusted according to the emotion and played back to the user.

[1090] Output: Play reminder message

[1091] Specific operation: For example, the reminder message "Tomorrow, bring a wrench, a screwdriver, and rain gear" is played.

[1092] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1093] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1094] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1095] [Fourth embodiment]

[1096] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1097] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1098] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1099] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1100] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1101] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1102] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1103] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1104] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1105] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1106] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1107] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1108] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1109] This invention is a system that uses a user terminal to assist in preparing school supplies. It generates a list of necessary supplies based on the next day's schedule and provides voice reminders, reducing the burden on children and their parents.

[1110] System implementation

[1111] Users launch a dedicated application using a user device such as a smartphone or tablet. First, a parent or child logs in to the app and sends authentication information to the server. The server receives this authentication information and authenticates the user. If authentication is successful, the user is taken to the home screen.

[1112] Entering timetable information

[1113] The user (parent or child) then enters information about the child's school into the application, including the child's name, class, and tomorrow's schedule. This data is sent to the server, where it is stored in a database.

[1114] Generate a packing list

[1115] The server analyzes the received timetable information and identifies the teaching materials and items required for each subject. It also predicts additional items to bring based on the day of the week and the season. For example, it determines that sneakers are needed on Fridays and jackets are needed in spring. This information is combined to generate the final packing list.

[1116] Generating and transmitting audio data

[1117] The inventory list is then converted into audio data using a text-to-speech (TTS) API, which the server then sends to the user's device via HTTP or HTTPS.

[1118] Reminders on user devices

[1119] The user (child) opens the application again before the next day and requests, by voice or touch, "Tell me what to prepare for tomorrow." The user's device plays the saved voice data and announces the list of items to bring. For example, the user might be reminded, "Tomorrow's items to bring are textbooks (Japanese and math), arts and crafts kit, gym clothes, and sneakers."

[1120] Application Examples

[1121] This system can be used to provide not only reminders for inventory lists, but also medication reminders and childcare reminders. For example, an elderly person can be set to be reminded to take their medicine at 6 p.m. In this case, the server generates audio data at the set time and sends it to the device. The device then plays the audio data at the set time, reminding the user to "take their medicine at 6 p.m."

[1122] Through the detailed steps described above, the present invention provides effective support for users and their guardians to prepare all necessary belongings and start their new life with peace of mind.

[1123] The processing flow will be explained below.

[1124] Step 1:

[1125] The user starts the application on their smartphone or tablet and enters their user ID and password on the login screen, which is then sent to the server.

[1126] Step 2:

[1127] The server receives the user's authentication information and checks it against a database. If authentication is successful, the login is approved and a signal is sent to the user's device to display the home screen.

[1128] Step 3:

[1129] The user enters their child's information (name, class, next day's schedule) on the home screen. The entered information is sent to the server and stored in a database.

[1130] Step 4:

[1131] The server analyzes the received timetable information. First, it identifies the teaching materials and items required for each subject. For this analysis, it uses the standard list of items to bring for each subject in the timetable (e.g., Japanese textbook, notebook, pencil case).

[1132] Step 5:

[1133] The server predicts what additional items to bring based on the day of the week, season, and weather information obtained from the weather API. For example, if it's Friday, sneakers are added to the list, and if it's a rainy day, rain gear is added.

[1134] Step 6:

[1135] The server generates a list of required items and converts it into audio data using a Text-to-Speech (TTS) API, which is then stored on the server.

[1136] Step 7:

[1137] The server sends the voice data to the user's device. The communication protocol is HTTP or HTTPS. The sent voice data is saved on the user's device.

[1138] Step 8:

[1139] To check the preparations for the next day, the user launches the application again and says "Check tomorrow's preparation list" by voice input or touch operation. The user's device detects this and plays the saved voice data.

[1140] Step 9:

[1141] The user device plays the audio data and notifies the user of the list of items to bring, allowing the user (child) to check what to bring the next day and prepare appropriately.

[1142] Step 10:

[1143] The user device can extend the reminder function to notify users of medication reminders at specific times. For example, it can play a voice reminder to an elderly person saying, "Take your medicine at 6 p.m."

[1144] The above is the specific processing flow of this system.

[1145] Example 1

[1146] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1147] In today's busy households, it is difficult to properly prepare school supplies. Especially when children prepare their own school supplies, forgetting or failing to prepare frequently causes considerable stress for both children and their parents. Furthermore, it is difficult to accurately track what school supplies should be prepared for different days of the week and seasons, so a system that automates this process and efficiently manages them is needed.

[1148] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1149] In this invention, the server includes a means for receiving timetable information of the educational institution input by the user terminal, a means for predicting items needed based on a specific day of the week or season, and a means for generating a list of needed items, thereby enabling children and their parents to efficiently prepare their belongings and prevent them from forgetting their belongings.

[1150] "User terminal" refers to a portable information device for inputting and receiving timetable information and inventory lists for educational institutions.

[1151] "Educational institutions" refers to schools, cram schools, and other educational facilities.

[1152] "Timetable information" refers to schedule information for classes and activities at an educational institution, and is a list of subjects and activities that take place on specific days and at specific times.

[1153] "Items" refers to teaching materials and tools necessary for lessons and activities.

[1154] "Predictive means" refers to the ability of a device or program to automatically determine what items are needed based on specific days of the week or seasons.

[1155] The "means for generating a list" refers to the function of a device or program that organizes predicted items and creates a list in a list format.

[1156] "Means for converting into audio data" refers to text-to-speech technology that converts the generated item list into natural audio.

[1157] The "means for transmitting voice data" refers to a communication technology for transmitting the generated voice data to a user terminal.

[1158] "Means for playing" refers to the function of a device or program for playing back audio data received on a user terminal.

[1159] "Text reminder" refers to a format that notifies you of your belongings using text information rather than audio data.

[1160] This invention is a system that uses user terminals to assist educational institutions in preparing belongings. It generates a list of necessary items based on the next day's schedule and reminds students of this list via voice, thereby reducing the burden on students and their guardians.

[1161] First, the user launches a dedicated application using a user device such as a smartphone or tablet. The user (guardian or beneficiary) enters their user ID and password on the login screen and sends them to the server. The sent authentication information is received by the server, which then collates it with information in a database to authenticate the user. If authentication is successful, the user is taken to the home screen.

[1162] Next, the user enters the recipient's name, class, and the next day's schedule information into a dedicated form in the application. This information is formatted by the user's terminal and sent to the server, which receives the information and stores it in a database.

[1163] Based on the submitted timetable information, the server identifies the items required for each subject. It then analyzes the day of the week and seasonal information to predict additional items. For example, it determines that sneakers are needed on Fridays and jackets are needed in spring. This information is then combined to generate the final item list.

[1164] The generated inventory list is converted into voice data using a text-to-speech (TTS) software API, and the server transmits the converted voice data to the user's device using the HTTP or HTTPS protocol.

[1165] The user (beneficiary) opens the application again before the next day and requests, by voice input or touch operation, "Tell me what to prepare for tomorrow." The user device plays the saved voice data and reminds the user, "Tomorrow's items to bring are textbooks (Japanese and math), arts and crafts kit, gym clothes, and sneakers."

[1166] As a specific example of how this works, for example, a list of belongings generated based on Monday's timetable would include "Japanese textbook, math notebook, gym clothes." If it's Friday, "sneakers" would be added. If it's spring, "jacket" would be added.

[1167] By using the following prompts to the generative AI model, you can elicit specific explanations of the system's processing flow.

[1168] Example prompt:

[1169] "Please explain in detail how this system reminds users of their belongings. For example, please explain in detail the process by which the server analyzes the timetable information and generates the belongings list."

[1170] Through the above process, the present invention provides effective support for the user and guardian to prepare all necessary belongings and start their new life with peace of mind.

[1171] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1172] Step 1: User launches application and logs in

[1173] The user launches the dedicated application using the user device. After launching, the user (guardian or beneficiary) enters their user ID and password on the login screen and taps the login button. The user device sends the entered authentication information to the server. The server compares the received authentication information with information in the database and authenticates the user. If authentication is successful, the server sends a success message and the user device transitions to the home screen.

[1174] Input: User ID, Password

[1175] Data processing: User ID and password verification

[1176] Output: Authentication success message

[1177] Specific operation: When the user taps the login button, authentication information is sent, the server authenticates it, and returns a success message.

[1178] Step 2: Enter your school and timetable information

[1179] The user (parent or beneficiary) enters the beneficiary's name, class, and next day's timetable information into a dedicated form in the application. The user's device formats the entered information and sends it to the server. The server receives this information and stores it in a database.

[1180] Input: Beneficiary's name, class, next day's timetable information

[1181] Data processing: Formatting and sending information

[1182] Output: Acknowledgement of receipt of information

[1183] Specific operation: When a user enters information into a dedicated form and taps the submit button, the information is sent to the server and stored in a database.

[1184] Step 3: Generate a packing list

[1185] The server analyzes the received timetable information and identifies the items needed for each subject stored in the database. It also analyzes information about the day of the week and the season to predict additional items. It then combines this information to generate the final packing list.

[1186] Input: Timetable information, day of the week information, seasonal information

[1187] Data calculation: Identifying items and predicting additional items

[1188] Output: Final inventory list

[1189] Specific operation: The server analyzes the timetable information, adds items according to the day of the week and season, and creates a list of items to bring.

[1190] Step 4: Generate and send audio data

[1191] The server converts the generated inventory list into audio data using a text-to-speech software (TTS) API, and sends the converted audio data to the user's device using HTTP or HTTPS protocol.

[1192] Input: Final packing list

[1193] Data processing: conversion to audio data

[1194] Output: Audio data

[1195] Specific operation: The server sends the inventory list to the TTS API, and the generated voice data is sent to the user's terminal.

[1196] Step 5: Remind users on their devices

[1197] The user (beneficiary) opens the application again before the next day and requests, by voice input or touch operation, "Tell me what to prepare for tomorrow." The user device plays the saved voice data and reminds the user, "Tomorrow's items to bring are textbooks (Japanese and math), arts and crafts kit, gym clothes, and sneakers."

[1198] Input: Voice or touch

[1199] Data processing: processing requests and playing audio data

[1200] Output: Voice reminder

[1201] Specific operation: When the user voice-inputs "Tell me how to prepare for tomorrow" or taps a button, the device plays the voice data as a reminder.

[1202] (Application example 1)

[1203] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1204] When shopping in a physical store, it is difficult for users to efficiently find the items they need to purchase, resulting in unnecessary time and effort. Furthermore, especially in large stores, it is not uncommon for users to miss or have difficulty finding items that they need to purchase based on their shopping list. Furthermore, there is a lack of a reminder function to ensure users don't forget to purchase the items they need. Therefore, a system that solves this problem is needed.

[1205] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1206] In this invention, the server includes means for receiving school timetable information input by a user terminal, means for predicting items needed based on specific days of the week and seasons, means for generating a list of items needed, means for converting the generated list into voice data, means for transmitting the voice data to the user terminal, means for playing the transmitted voice data on the user terminal, and means for guiding the location of items included in the list based on location information of each department in the store, thereby enabling users to shop efficiently in a physical store and purchase all necessary items.

[1207] A "user terminal" is a device such as a smartphone, tablet, or smart glasses that is directly operated by a user to input and receive information.

[1208] "School timetable information" is data that includes information about the times and contents of classes that students will take the next day and on specific days, as well as the teaching materials and items they will need to take with them.

[1209] "Day-of-the-week and season-based items" refers to items or clothing needed on specific days of the week or during specific seasons, such as sneakers needed on Fridays or a jacket needed in the spring.

[1210] The "list of necessary items" is a list of all items needed for the next day or a specific day, generated based on the user's input information and predicted day of the week and season information.

[1211] The "means for converting into audio data" is a technique for converting the inventory list in text format into audio format and generating an audio file or audio stream that can be listened to by the user.

[1212] The "means for transmitting voice data to a user terminal" refers to a communication means for transmitting voice data generated by the server to a user terminal via the Internet.

[1213] "Means for reproducing audio data at a user terminal" refers to a technique for reproducing audio data received by a user terminal and providing it to the user as auditory information.

[1214] "Location information for each department within the store" is information about where each item is displayed within the physical store, and is data indicating in which department or section a particular item is located.

[1215] "Item location guidance" refers to technology that guides users to specific locations or departments within a store to help them efficiently find items on their shopping list.

[1216] The present invention is a system for supporting efficient shopping in a physical store using a user terminal. Specific embodiments of this system will be described below.

[1217] System implementation

[1218] A user launches a dedicated shopping assistant application using a user device such as a smartphone or smart glasses. First, the user logs in to the application and sends authentication information to the server. The server receives this authentication information and performs user authentication. If authentication is successful, the user is taken to the application's home screen.

[1219] Entering a shopping list

[1220] The user enters the items they want to include in their shopping list into the application, and this data is sent to the server where it is stored in a database.

[1221] Shopping list location guide

[1222] The server analyzes the received shopping list information and identifies which department in the store each item is located in. Based on the store location information, it generates a list to guide the user to the location of the required items. The list information includes the location and department details of each item.

[1223] Generating and transmitting audio data

[1224] The generated shopping list and location information is converted into audio data using a Text-to-Speech (TTS) API, and the server generates this audio data and sends it to the user's device using HTTP or HTTPS protocols.

[1225] Reminders and guidance on user devices

[1226] The user uses the application in the store and requests, by voice or touch, "What item should I buy next?" The user's device plays the saved voice data and provides a voice prompt of the items on the list and their location. For example, the user might be reminded, "The next item to buy is milk. It's in the refrigerated section."

[1227] Specific processing

[1228] Hardware: Smartphones, smart glasses

[1229] Software: Dedicated shopping assistant application, server-side authentication system, database, Text-to-Speech (TTS) API

[1230] Data processing flow:

[1231] 1. The user logs in to the app and sends their credentials to the server.

[1232] 2. The server authenticates the user and receives and stores the shopping list data.

[1233] 3. The server analyzes the information in the list and determines the location of the item within the store.

[1234] 4. Based on the location information, the list is converted into audio data and sent to the user's device.

[1235] 5. The user terminal plays back the received voice data and provides the user with a voice guide of the next item to be purchased and its location.

[1236] Specific examples

[1237] For example, a user logs in to the application and enters "milk," "bread," and "detergent" as a list of items to be purchased next. The server uses this information to identify which section of the physical store each item is located in, and generates a voice reminder based on the location information of the refrigerated goods section, bakery section, and daily necessities section. This reminder is played on the user's device, supporting the user in efficient shopping.

[1238] Prompt Sentence Examples

[1239] Develop an application that helps users shop efficiently in physical stores. The main features are shopping list creation and voice reminders. The reminders guide users through their shopping by guiding them to the next item to be purchased and its location.

[1240] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1241] Step 1:

[1242] The user launches the application and enters authentication information on the login screen. The authentication information includes the user ID and password. The server authenticates the user based on this information. The server compares the authentication information with records in the database, and if there is a match, returns a status of successful authentication. The user's device receives a notification of successful authentication and moves to the home screen.

[1243] Step 2:

[1244] The user inputs a shopping list into the application. This list contains the names of multiple items the user plans to purchase. The input data is sent to the server as form data. The server receives this data and saves the shopping list in a database. Input: Shopping list (item names). Output: Shopping list saved in the database.

[1245] Step 3:

[1246] The server analyzes the shopping list information and identifies the location information for each item in the store. It references the location information in the database to determine which section or department each item is located in. Based on this data, it generates a list containing location information for each item. Input: Saved shopping list. Output: List containing location information for the store.

[1247] Step 4:

[1248] The server converts the list generated by the server into audio data using a Text-to-Speech (TTS) API. The list information in text format is input to the API, which generates an audio file or audio stream. Input: List containing in-store location information. Output: Audio data.

[1249] Step 5:

[1250] The voice data is sent to the user terminal using the HTTP or HTTPS protocol. The server uses a communication means to send the generated voice data to the user terminal. Input: Voice data. Output: Voice data sent to the user terminal.

[1251] Step 6:

[1252] The user reopens the application in the store and requests, by voice or touch, "Tell me what item I'll buy next." The user device receives this request and plays the saved voice data. Input: User request. Output: Played voice data (reminder).

[1253] Step 7:

[1254] The user device plays the audio data and provides audible instructions about the next item in the list and its location. For example, "The next item to purchase is milk. It's in the refrigerated section." Input: The stored audio data. Output: Audio instructions provided to the user.

[1255] keyword:

[1256] Generative AI model, prompt sentence

[1257] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1258] This invention is a system that uses a user terminal to help users prepare school supplies. It generates a list of necessary supplies based on the next day's schedule and reminds users of the list by voice and text. In addition, by combining it with an emotion engine, the content of the reminder can be flexibly adjusted according to the user's emotions.

[1259] System implementation

[1260] First, the user launches a dedicated application using a user device such as a smartphone or tablet. The user (parent or child) enters authentication information on the login screen and sends it to the server. The server authenticates the user using the received information, and if authentication is successful, the home screen is displayed on the user device.

[1261] Entering timetable information

[1262] The user enters basic information about their child (name, class), the next day's schedule, and any special notes (lessons, health, etc.) on the home screen. This information is sent to the server and stored in a database.

[1263] Generate a packing list

[1264] The server identifies the materials and supplies needed for each subject based on the timetable information. It also predicts what additional items students need to bring based on the day of the week, season, and weather information obtained from a weather API. For example, if it's Friday, sneakers are added to the list, and if it's a rainy day, rain gear is added.

[1265] Use of emotion engine

[1266] Furthermore, the app will introduce an emotion engine that analyzes the user's voice input and facial expressions. The camera captures the user's voice tone and facial expressions when they input information into the app, and the emotion engine analyzes them. Based on the analysis results, if the user is feeling anxious or stressed, the reminder content will be made more detailed or an encouraging message will be added. Conversely, if the user is feeling positive, the app will send reminders that emphasize successful experiences.

[1267] Generating and transmitting audio data

[1268] The list of necessary items is converted into voice data using a text-to-speech software (TTS) API, and the server sends the generated voice data to the user's device and stores it on the user's device.

[1269] Reminders on user devices

[1270] To check what to prepare for the next day, the user launches the application again and uses voice input or touch operations to say, "Tell me what to prepare for tomorrow." The user's device detects this and plays back the saved voice data. For example, the user may be reminded, "Tomorrow's items to bring are textbooks (Japanese and math), arts and crafts kit, gym clothes, and sneakers."

[1271] Application Examples

[1272] This system can be applied not only to reminders of inventory lists, but also to medication reminders and childcare reminders. For example, it can be set to notify an elderly person, "Please take your medicine at 6 p.m." In this case, the emotion engine can provide more detailed explanations or add encouraging messages to anxious users.

[1273] Through the detailed steps described above, the present invention provides effective support for users to prepare all their belongings and live their daily lives with peace of mind.The introduction of an emotion engine enables flexible reminders that respond to the emotions of each individual user.

[1274] The processing flow will be explained below.

[1275] Step 1:

[1276] The user starts the application on their smartphone or tablet and enters their user ID and password on the login screen. The entered authentication information is sent to the server.

[1277] Step 2:

[1278] The server receives the user's authentication information and checks it against a database. If authentication is successful, the login is approved and a signal is sent to the user's device to display the home screen.

[1279] Step 3:

[1280] The user enters their child's information (name, class, next day's schedule) on the home screen. The entered information is sent to the server and stored in a database.

[1281] Step 4:

[1282] The server analyzes the received timetable information and identifies the materials and items required for each subject. For example, it determines that a Japanese language class requires a Japanese language textbook, notebook, and pencil case.

[1283] Step 5:

[1284] The server predicts additional items to bring based on the day of the week, season, and weather information obtained from the weather API. For example, if it's Friday, it will add "sneakers," and if rain is expected, it will add "rain gear."

[1285] Step 6:

[1286] The server generates the required inventory list, which is done using the generated text list.

[1287] Step 7:

[1288] The inventory list is converted into audio data using a text-to-speech (TTS) software API specified by the server, and the converted audio data is stored on the server.

[1289] Step 8:

[1290] The server sends the audio data to the user's device using the HTTP or HTTPS protocol, and the audio data is stored on the user's device.

[1291] Step 9:

[1292] To check the preparations for the next day, the user launches the application again and says "Check tomorrow's preparation list" by voice input or touch operation. The user's device detects this and plays the saved voice data.

[1293] Step 10:

[1294] The user device plays the audio data and notifies the user of the list of items to bring. For example, it may remind the user that "Tomorrow's items to bring are textbooks (Japanese and math), arts and crafts kit, gym clothes, and sneakers."

[1295] Step 11:

[1296] When using the emotion engine, when a user inputs information into the app, the device's camera and microphone capture the user's facial expressions and tone of voice, and this emotional data is sent to the server.

[1297] Step 12:

[1298] The emotion engine on the server analyzes the user's emotional data. If the user is feeling anxious or stressed, the system will adjust the reminder content by providing a detailed explanation of the item to bring. Conversely, if the user is expressing positive emotions, the system will send reminders that emphasize successful experiences.

[1299] Step 13:

[1300] The server converts the adjusted reminder content back into audio data and transmits it to the user terminal, which plays back the audio data and notifies the user of the adjusted reminder content.

[1301] The above is the specific processing flow when combining the emotion engine of this system.

[1302] Example 2

[1303] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1304] Conventional luggage preparation support systems have the function of generating a luggage list based on the user's input timetable information, season, and day of the week, but they lack the ability to provide reminders that take into account the user's emotional state. As a result, luggage preparation may not be carried out efficiently, and the user's psychological support may be insufficient. Furthermore, since voice reminders alone do not provide enough visual information, a more effective notification method is desired.

[1305] The specification process by the specification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for receiving school timetable information input by a user terminal, a means for predicting necessary belongings based on specific days of the week and seasons, and a means for adjusting reminder content using an emotion engine that analyzes the user's voice input and facial expressions. This makes it possible to provide flexible reminders that take into account the user's emotional state, allowing the user to prepare belongings more efficiently and with peace of mind.

[1306] A "user terminal" is an electronic device, such as a smartphone, tablet, or PC, that a user uses to input information or receive notifications.

[1307] "Timetable information" is information that indicates the schedule of school lessons and activities, including subjects, days of the week, and time periods.

[1308] "Items required for specific days of the week or seasons" are items that students are required to bring depending on the content of the class, weather, or season. For example, sneakers are required on days when there is a physical education class, and rain gear is required on days when rain is expected.

[1309] A "list of necessary belongings" is a list of items that a user should bring with them the next day or on a specific day.

[1310] "Audio Data" means text information converted into a format that can be played as sound, and is generated using a Text-to-Speech (TTS) API.

[1311] An "emotion engine" refers to software or hardware that has the functionality to analyze a user's tone of voice and facial expressions and assess their emotional state.

[1312] "Adjusting reminder content" refers to changing the content and format of notifications and messages provided based on the user's emotional state. For example, a user who is feeling anxious can receive additional detailed explanations or encouraging messages.

[1313] "Text reminders" refer to a method of providing notifications and instructions to users through text information that can be read as words. These can be displayed visually on the screen of the user's device.

[1314] The present invention is a system that uses a user terminal to assist in preparing school supplies, and is implemented by the following procedure. First, the user launches a dedicated application on the user terminal, such as a smartphone or tablet. The user enters authentication information on the login screen and sends it to the server. The server authenticates the user using the received information, and if authentication is successful, displays the home screen on the user terminal.

[1315] Entering timetable information

[1316] The user enters basic information about their child (name, class), the next day's schedule, and any special notes (lessons, health, etc.) on the home screen. This information is sent from the user's device to the server and stored in the server's database.

[1317] Generate a packing list

[1318] The server needs to identify the necessary materials and belongings for each subject based on the timetable information. It also predicts what additional items to bring based on the day of the week, season, and weather information obtained from a weather API. For example, if it's Friday, sneakers are added to the list, and if it's a rainy day, rain gear is added.

[1319] Use of emotion engine

[1320] An emotion engine is introduced to analyze the user's voice input and facial expressions. When the user inputs voice or facial expressions into the app, this is captured using a camera and microphone and sent to a server. The emotion engine on the server analyzes this and adjusts the reminder content based on the analysis results. For example, if the user is feeling anxious or stressed, the reminder content can be made more detailed or an encouraging message can be added. Conversely, if the user is feeling positive, a reminder that emphasizes successful experiences can be sent.

[1321] Generating and transmitting audio data

[1322] The inventory list is converted into audio data using a text-to-speech (TTS) software API, and the server sends the generated audio data to the user's device and stores it on the user's device.

[1323] Reminders on user devices

[1324] To check what to prepare for the next day, the user launches the application again and uses voice input or touch operations to say, "Tell me what to prepare for tomorrow." The user's device detects this and plays back the saved voice data. For example, the user might be reminded, "Tomorrow's items to bring are textbooks (Japanese and math), arts and crafts kit, gym clothes, and sneakers."

[1325] Application Examples

[1326] This system can be applied not only to reminders of inventory lists, but also to medication reminders and childcare reminders. For example, it can be set to notify an elderly person, "Please take your medicine at 6 p.m." In this case, the emotion engine can provide more detailed explanations or add encouraging messages to anxious users.

[1327] Specific examples

[1328] For example, if the next day's schedule looks like this:

[1329] Day of the week: Friday

[1330] Subjects: Japanese, arithmetic, art, physical education

[1331] Special note: Plans to join soccer club after school

[1332] Weather information: Rain forecast

[1333] In this case, the server adds the following items to the list:

[1334] Japanese language textbook

[1335] Mathematics textbook

[1336] Art and craft set

[1337] gym clothes

[1338] athletic shoes

[1339] Soccer practice equipment

[1340] rain gear

[1341] Example of input prompt for generative AI model

[1342] "Please provide the next day's schedule information and weather forecast in the following format, and generate a list of items you need to bring. Also, please generate a reminder message based on the user's emotions.

[1343] (Timetable information)

[1344] Day of the week: Friday

[1345] Subjects: Japanese, arithmetic, art, physical education

[1346] Special note: Plans to join soccer club after school

[1347] (weather forecast)

[1348] Weather: Rain

[1349] (User sentiment)

[1350] Voice Tone: A little anxious

[1351] Expression: Not calm

[1352] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1353] Step 1:

[1354] The user launches the app and authenticates

[1355] The user launches a dedicated application on their smartphone or tablet. When the login screen appears, the user enters their email address and password and taps the "Login" button. This input information is sent to the server. The server compares the received authentication information with the database, and if authentication is successful, the server sends a successful authentication response to the user's device, and the home screen is displayed on the user's device. The input is the email address and password, and the output is the user's home screen.

[1356] Step 2:

[1357] The user enters the next day's schedule information and special notes

[1358] The user opens a form on the home screen to enter the child's name, class, next day's schedule information, and any special notes. When the user enters this information and taps the "Submit" button, the entered data is sent to the server. The server stores the received information in a database. The input at this time is the child's name, class, schedule information, and special notes, and the output is the information stored in the database.

[1359] Step 3:

[1360] Server generates inventory list

[1361] The server retrieves timetable information stored in the database and identifies the necessary items for each subject. In doing so, the server also takes into account the day of the week, season, and weather information retrieved from the weather API. For example, if there is a physical education class on Friday and rain is expected, sneakers and rain gear will be added to the list of items to bring. The input is the timetable information, season information, and weather information from the database, and the output is a list of necessary items to bring.

[1362] Step 4:

[1363] Use of emotion engine

[1364] When a user inputs voice or facial expressions into the app, the device's camera and microphone capture this and send the data to the server. The server's emotion engine analyzes this and evaluates the user's emotional state. Based on the analysis results, the server adjusts the reminder content. For example, if the user is feeling anxious, it may add a detailed explanation or an encouraging message. The input data at this time is audio and video data, and the output is the adjusted reminder content.

[1365] Step 5:

[1366] Convert your inventory list into audio data and send it

[1367] The server converts the final inventory list into voice data using a text-to-speech (TTS) API. This generated voice data is sent to the user's device and stored on the device. The input is the inventory list, and the output is voice data.

[1368] Step 6:

[1369] Reminders on user devices

[1370] The user launches the app again to check the next day's preparations, and issues instructions by voice input or touch operation, such as "Tell me what to prepare for tomorrow." The device detects this and plays back the saved voice data. For example, the device may be reminded, "Tomorrow's items to bring are textbooks (Japanese and math), arts and crafts kit, gym clothes, and sneakers." The input is the user's voice input or touch operation, and the output is the voice playback data.

[1371] (Application example 2)

[1372] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1373] Previous school baggage preparation support systems were limited to generating a baggage list based on the next day's schedule, and were unable to flexibly respond to the user's emotional state or specific circumstances. They also lacked appropriate reminders for items to be prepared on the factory production line, and lacked emotional support for users. This led to missed items and emotional stress.

[1374] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1375] In this invention, the server includes means for receiving school timetable information entered by a user terminal, means for predicting necessary items based on a specific day of the week or season, means for generating a list of necessary items, means for converting the generated list into voice data, means for transmitting the voice data to the user terminal, means for playing the transmitted voice data on the user terminal, means for identifying necessary items based on the schedule obtained from the server, means for analyzing the user's emotional state using an emotion analysis engine, and means for adjusting reminder messages according to the user's emotional state. This enables flexible reminders according to the user's emotional state, prevents users from forgetting to bring or prepare items, and provides emotional support.

[1376] "School timetable information" is information necessary for students to participate in classes and extracurricular activities, and indicates the dates, times, and locations of specific subjects and activities.

[1377] "Day-of-the-week and season-based supplies" are items that are needed for specific activities or lessons depending on the day of the week or the season.

[1378] A "packing list" is a list of items that a user needs to bring with them the next day or on a specific day.

[1379] "Audio data" refers to data obtained by converting text-format information into audio, and is used by the user to receive information aurally.

[1380] A "user terminal" is an electronic device that is personally owned by a user, such as a smartphone, tablet, or smart glasses.

[1381] A "server" is a central device that receives information from multiple user terminals, processes the information, and returns appropriate data.

[1382] An "emotion analysis engine" is a program that analyzes the user's emotional state from voice input and facial expressions, and allows the system to respond based on the results.

[1383] A "remind message" refers to a notification message that provides the user with necessary information or encourages them to take action.

[1384] "Schedules acquired from a server" refers to data on pre-set plans or schedules, such as factory production lines or school timetables, acquired by a server.

[1385] The present invention relates to a system for supporting the preparation of belongings for school or a production line using a user terminal. Specific embodiments will be described below.

[1386] First, the user launches a dedicated application using a user device such as a smartphone or smart glasses. The user (parent or worker) enters authentication information on the login screen and sends it to the server. The server authenticates the user using the received information, and if authentication is successful, the home screen is displayed on the user device.

[1387] Next, the user enters basic information (name, class or production line), the next day's timetable and production schedule, and any special notes (lessons or specific processes) on the home screen. This information is sent to the server and stored in a database.

[1388] The server identifies the materials and parts needed for each subject and process based on timetable information or production schedules. It also predicts what additional items to bring based on the day of the week, season, and weather information obtained from a weather API. For example, if it's Friday, sneakers are added to the list, and if it's a rainy day, rain gear is added.

[1389] Furthermore, a pre-installed emotion analysis engine is introduced. The emotion engine analyzes the user's voice input and facial expressions. Based on the analysis results, if the user is feeling anxious or stressed, the reminder content will be made more detailed or an encouraging message will be added. Conversely, if the user is feeling positive, a reminder that emphasizes successful experiences will be sent.

[1390] The list of necessary items is converted into voice data using a text-to-speech software (TTS) API, and the server sends the generated voice data to the user's device and stores it on the user's device.

[1391] To check what to prepare for the next day, the user launches the application again and uses voice input or touch operations to say, "Tell me what to prepare for tomorrow." The user's device detects this and plays back the saved voice data. For example, the user might be reminded, "Tomorrow's items to bring are textbooks (Japanese and math), arts and crafts kit, gym clothes, and sneakers."

[1392] The same can be applied to factory production lines. When a worker asks, "What should I prepare for tomorrow?", a voice message will be sent reminding them of the tools and parts they will need based on the next day's production schedule. For example, they might be reminded, "Tomorrow's items to bring are a wrench and a screwdriver, and since it's a rainy day, don't forget to bring rain gear."

[1393] Hardware and software:

[1394] Hardware: smart glasses (e.g. Google Glass), smartphones, servers

[1395] Software: EmotionEngine (emotion analysis engine), TTSAPI (text-to-speech software), Weather API (weather information acquisition)

[1396] Examples:

[1397] Example 1: A production line worker asks, "What should I prepare for tomorrow?" and is reminded, "Tomorrow, bring a wrench, a screwdriver, and don't forget your rain gear because it's a rainy day."

[1398] Example 2: When a student asks, "What should I prepare for tomorrow?", they are reminded, "Tomorrow, bring your textbooks (Japanese and math), arts and crafts kit, gym clothes, and sneakers."

[1399] Example prompt sentence:

[1400] "Tell me what to bring tomorrow. Add a feature that will remind me by voice what tools I need, like wrenches, screwdrivers, etc."

[1401] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1402] Step 1:

[1403] The user enters their credentials on the login screen.

[1404] Input: Username, Password

[1405] Data processing: The user terminal encrypts the entered authentication information and sends it to the server.

[1406] Output: The server receives the authentication information and checks it against its database.

[1407] Specific operation: The server checks the authentication information in the database, and if authentication is successful, displays the home screen on the user's device.

[1408] Step 2:

[1409] The user enters timetable information and special notes on the home screen.

[1410] Input: Timetable information, special notes (lessons, specific processes, etc.)

[1411] Data processing: The user terminal formats the entered information and sends it to the server.

[1412] Output: The server stores the received information in a database.

[1413] Specific operation: The server uses the timetable information and special notes stored in the database to create the schedule for the next day.

[1414] Step 3:

[1415] The server generates a list of necessary items to bring based on the timetable information.

[1416] Input: Timetable information, special notes

[1417] Data processing: The server lists the necessary materials and parts for each subject and process. It also uses a weather API to obtain weather information and predicts what additional items to bring.

[1418] Output: List of necessary items

[1419] Specific actions: For example, if it's Friday and you have exercise, add sneakers to the list, or if it's a rainy day, add rain gear to the list.

[1420] Step 4:

[1421] The server converts the inventory list into audio data.

[1422] Input: inventory list

[1423] Data processing: The server uses a text-to-speech (TTS) API to convert the inventory list into audio data.

[1424] Output: Audio data

[1425] Specific operation: Input the inventory list text into the TTS API and generate a corresponding audio file.

[1426] Step 5:

[1427] The server transmits the voice data to the user terminal.

[1428] Input: Audio data

[1429] Data processing: The server sends the generated voice data to the user terminal.

[1430] Output: Audio data stored on the user's device

[1431] Specific operation: The server transfers the voice data to the user device, and the user device stores it in its internal storage.

[1432] Step 6:

[1433] The user requests a packing list reminder via voice or touch.

[1434] Input: Voice input, touch operation

[1435] Data processing: The user device recognizes voice input and touch operations and sends a reminder request to the server.

[1436] Output: Remind request

[1437] Specific operation: When the device receives a reminder request, it plays the saved audio data.

[1438] Step 7:

[1439] The server uses an emotion analysis engine to analyze the user's emotional state.

[1440] Input: User's voice input, facial expression data

[1441] Data processing: The emotion analysis engine analyzes the user's tone of voice and facial expressions to identify their emotional state.

[1442] Output: User's emotional state

[1443] Specific behavior: For example, if the user is feeling anxious, the engine generates an "encouraging message."

[1444] Step 8:

[1445] The user terminal plays back the audio data.

[1446] Input: Voice data, emotion analysis results

[1447] Data processing: The audio data is adjusted according to the emotion and played back to the user.

[1448] Output: Play reminder message

[1449] Specific operation: For example, the reminder message "Tomorrow, bring a wrench, a screwdriver, and rain gear" is played.

[1450] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1451] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1452] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1453] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1454] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1455] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1456] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1457] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1458] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1459] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1460] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1461] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1462] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1463] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1464] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1465] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1466] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1467] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1468] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1469] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1470] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1471] The following is further disclosed regarding the above embodiment.

[1472] (Claim 1)

[1473] means for receiving school timetable information input by a user terminal;

[1474] A way to predict what you will need based on a particular day of the week or season;

[1475] A means for generating a list of necessary items;

[1476] means for converting the generated list into audio data;

[1477] means for transmitting voice data to a user terminal;

[1478] means for playing the transmitted audio data on the user terminal;

[1479] A system including:

[1480] (Claim 2)

[1481] 10. The system of claim 1, wherein items corresponding to specific days of the week or seasons are added to the list based on user input information.

[1482] (Claim 3)

[1483] 2. The system according to claim 1, wherein a reminder in text format is sent to the user terminal and displayed in addition to the voice data.

[1484] "Example 1"

[1485] (Claim 1)

[1486] means for receiving timetable information of an educational institution inputted by a user terminal;

[1487] A means of predicting what items are needed based on specific days of the week or seasons;

[1488] means for generating a list of required items;

[1489] means for converting the generated list into audio data;

[1490] means for transmitting voice data to a user terminal;

[1491] means for playing the transmitted audio data on the user terminal;

[1492] A system including:

[1493] (Claim 2)

[1494] The system according to claim 1, wherein items corresponding to specific days of the week or seasons are added to the list based on information input by the user.

[1495] (Claim 3)

[1496] 2. The system according to claim 1, wherein a reminder in text format is sent to the user terminal and displayed in addition to the voice data.

[1497] "Application Example 1"

[1498] (Claim 1)

[1499] means for receiving school timetable information input by a user terminal;

[1500] A way to predict what you will need based on a particular day of the week or season;

[1501] A means for generating a list of necessary items;

[1502] means for converting the generated list into audio data;

[1503] means for transmitting voice data to a user terminal;

[1504] means for playing the transmitted audio data on the user terminal;

[1505] a means for providing guidance on the location of the items included in the list based on the location information of each department in the store;

[1506] A system including:

[1507] (Claim 2)

[1508] 10. The system of claim 1, wherein items corresponding to specific days of the week or seasons are added to the list based on user input information.

[1509] (Claim 3)

[1510] 2. The system according to claim 1, wherein a reminder in text format is sent to the user terminal and displayed in addition to the voice data.

[1511] (Claim 4)

[1512] 10. The system of claim 1, further comprising means for utilizing location information to identify a user's location and providing location guidance for an item based on the location.

[1513] "Example 2: Combining Emotion Engines"

[1514] (Claim 1)

[1515] means for receiving school timetable information input by a user terminal;

[1516] A way to predict what you will need based on a particular day of the week or season;

[1517] A means for generating a list of necessary items;

[1518] means for converting the generated list into audio data;

[1519] means for transmitting voice data to a user terminal;

[1520] means for playing the transmitted audio data on the user terminal;

[1521] a means for adjusting the reminder content by utilizing an emotion engine that analyzes the user's voice input and facial expressions;

[1522] A system including:

[1523] (Claim 2)

[1524] 10. The system of claim 1, wherein items corresponding to specific days of the week or seasons are added to the list based on user input information.

[1525] (Claim 3)

[1526] 2. The system according to claim 1, wherein a reminder in text format is sent to the user terminal and displayed in addition to the voice data.

[1527] "Application example 2 when combining emotion engines"

[1528] (Claim 1)

[1529] means for receiving school timetable information input by a user terminal;

[1530] A way to predict what you will need based on a particular day of the week or season;

[1531] A means for generating a list of necessary items;

[1532] means for converting the generated list into audio data;

[1533] means for transmitting voice data to a user terminal;

[1534] means for playing the transmitted audio data on the user terminal;

[1535] means for identifying required items based on a schedule obtained from the server;

[1536] means for analyzing the emotional state of a user by an emotion analysis engine;

[1537] means for adjusting the reminder message according to the user's emotional state;

[1538] A system including:

[1539] (Claim 2)

[1540] 10. The system of claim 1, wherein the system generates a packing list that incorporates weather information in addition to items needed based on specific days of the week and seasons.

[1541] (Claim 3)

[1542] 2. The system according to claim 1, wherein a reminder in text format is sent to the user terminal and displayed in addition to the voice data. [Explanation of symbols]

[1543] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. means for receiving school timetable information input by a user terminal; A way to predict what you will need based on a particular day of the week or season; A means for generating a list of necessary items; means for converting the generated list into audio data; means for transmitting voice data to a user terminal; means for playing the transmitted audio data on the user terminal; A system including:

2. The system of claim 1, wherein the system adds items corresponding to specific days of the week or seasons to the list based on information input by the user.

3. 2. The system according to claim 1, wherein a reminder in text format is sent to the user terminal and displayed in addition to the voice data.

Images