system

Abstract

We provide the system. [Solution] A means of setting a daily schedule based on user input on a mobile information terminal, A server device provides means for saving and analyzing the aforementioned lifestyle schedule, A means for transmitting notification data generated by the server device to the mobile information terminal and notifying the user, Means for transmitting the user's response to the server device via the mobile information terminal, A system that includes this.

Description

Technical Field

[0001] The technology of this disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Modern children are easily attracted to portable information terminals and have the problem of being difficult to manage time. In particular, by being engrossed in games and video viewing, they cannot maintain their daily life rhythms such as homework, bathing, and going to bed, and parental intervention is often required. Such a situation increases the burden on parents and hinders the improvement of children's autonomous time management ability.

Means for Solving the Problems

[0005] This invention provides a system that supports children's daily routines using a mobile device. Specifically, the system sets a daily schedule entered by the user on the mobile device, and this data is stored and managed on a server device. The server device uses text-generating artificial intelligence and speech-generating artificial intelligence to generate notifications based on the schedule. The generated notifications are delivered to the user via the mobile device, and the user responds to the notification. This response is sent to the server device, and by analyzing the response results, it is possible to adjust the timing of the next notification. As a result, children can naturally become more aware of time management, reducing the burden on parents.

[0006] A "portable information terminal" is an information processing device that a user can carry with them, and includes smartphones and tablets.

[0007] A "daily schedule" is a set of actions and plans that should be performed at specific times in daily life, and is information used to manage the user's routine.

[0008] A "server device" is a computer system that acquires, manages, and processes data via a network, providing necessary services in response to requests from client mobile devices.

[0009] "Text generation artificial intelligence" is a technology that uses natural language processing techniques to generate appropriate text messages based on specific inputs or situations.

[0010] "Speech-generating artificial intelligence" is a technology that converts text data into speech data and outputs it as natural-sounding speech.

[0011] "Notification data" refers to information, including text and audio data, generated by a server device and transmitted to a mobile device in order to inform the user of specific information.

[0012] "Response data" refers to information that indicates the choices and actions taken by the user in response to a notification, and is data transmitted from the mobile device so that the server device can analyze it. [Brief explanation of the drawing]

[0013] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] This is a sequence diagram showing the processing flow of the data processing system in Example 2, which incorporates an emotion engine. [Figure 14]It is a sequence diagram showing the processing flow of a data processing system in Application Example 2 when a sentiment engine is combined.

Embodiments for Carrying Out the Invention

[0014] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0015] First, the terms used in the following description will be explained.

[0016] In the following embodiments, a numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0017] In the following embodiments, a numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0018] In the following embodiments, a numbered storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, etc.

[0019] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0020] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0021] [First Embodiment]

[0022] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0023] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0024] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0025] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0026] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0027] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0028] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0029] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0030] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0031] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0032] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0033] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0034] The system according to the present invention supports a child's daily routine via a mobile information terminal and is realized through the cooperation of three parties: a server device, a mobile information terminal, and a user. A specific embodiment of this system is described below.

[0035] First, the user sets their daily schedule using an application installed on their mobile device. The user can set routines for activities such as homework, bathing, and bedtime, and customize them as needed. The set schedule is then sent from the mobile device to a server device and stored in a database.

[0036] The server device manages notification timing for each user based on the received schedule information. Text-generating artificial intelligence creates user-friendly notification messages based on the user's schedule. Furthermore, speech-generating artificial intelligence converts the text into speech format, making it easier to recognize by ear.

[0037] As the set time approaches, the server device sends the generated notification data to the mobile device. The mobile device displays the received notification on its screen and also prompts the user to confirm their actions via voice. For example, a message such as "It's 8 PM. Have you finished your homework?" might be sent.

[0038] After the user confirms the notification, they respond by tapping an option on their mobile device. This selection data is then sent back to the server device, where it is recorded and analyzed. For example, if the user answers "No," the server device will take appropriate action, such as reviewing the timing of the next notification, to help the user manage their time.

[0039] Therefore, the system based on the present invention enables users to habitualize their behavior, improve their time management skills, and consequently reduce parental intervention.

[0040] The following describes the processing flow.

[0041] Step 1:

[0042] The user operates their mobile device to launch the application and open the daily schedule settings screen. The user enters their time schedule, including homework, bath time, and bedtime, and saves the settings.

[0043] Step 2:

[0044] The terminal sends the schedule data entered by the user to the server. The transmitted data is organized by user and stored in a database.

[0045] Step 3:

[0046] The server calculates the time to send a notification based on the received schedule and determines the content of the notification to be sent at that time.

[0047] Step 4:

[0048] The server generates notification messages using text generation artificial intelligence. For example, a message like "It's 8 PM. Have you finished your homework?" might be created.

[0049] Step 5:

[0050] The server uses speech-generating artificial intelligence to convert the generated text messages into audio data. This is to make the audio data more naturally recognizable to the user.

[0051] Step 6:

[0052] As the specified time approaches, the server sends the generated notification data to the device. The device then processes the received data immediately.

[0053] Step 7:

[0054] The device displays a notification on the screen and plays it aloud for the user. This allows the user to confirm the notification content both visually and aurally.

[0055] Step 8:

[0056] The user selects a response to the notification. Options such as "Yes" or "No" are presented, and the user taps the appropriate option.

[0057] Step 9:

[0058] The terminal sends the user's response back to the server. This response data is used for subsequent processing.

[0059] Step 10:

[0060] The server analyzes the response data it receives to understand user behavior patterns. Based on the analysis results, it plans to adjust the timing and content of future notifications.

[0061] (Example 1)

[0062] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0063] In modern society, carefully managing children's daily routines and promoting habit formation is crucial for busy parents. However, a lack of timely notifications and feedback hinders the development of children's time management skills. To address this issue, it is necessary to support children's behavior through appropriately tuned notification systems and response analysis.

[0064] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0065] In this invention, the server includes means for setting an action plan based on information input in a mobile information terminal, means for storing and analyzing the action plan in an information processing device, and means for transmitting notification information generated by the information processing device to the mobile information terminal and notifying the user. This makes it possible to manage the user's daily rhythm and promote actions at appropriate times.

[0066] A "portable information terminal" refers to a device that a user can carry around and use for inputting and displaying information.

[0067] "Information input" refers to the act of a user entering data via a mobile device.

[0068] An "action plan" refers to a schedule or plan for managing and implementing the user's daily routine.

[0069] An "information processing device" refers to a device that receives, stores, analyzes, and generates notification information for data.

[0070] "Notification information" refers to messages and audio data generated for users based on action plans.

[0071] "Text generation artificial intelligence" refers to an algorithm that automatically generates text data.

[0072] "Speech-generating artificial intelligence" refers to an algorithm that converts text data into speech.

[0073] "User" refers to an individual who implements an action plan and receives notifications via a mobile device.

[0074] "Response information" refers to data obtained when a user responds to a notification using their mobile device.

[0075] "Analysis results" refer to the results of the analysis derived by the information processing device based on the user's response information.

[0076] The system of the present invention consists of the cooperation of three parties: an information processing device, a mobile information terminal, and a user. Specifically, the user sets their own action plan using an application installed on the mobile information terminal. This action plan includes specific activities such as homework and bedtime. The set action plan is transmitted from the mobile information terminal to the information processing device.

[0077] The information processing device stores and analyzes received action plans in a database. This analysis includes a process of optimization using a generative AI model based on past behavioral patterns and current action plans. Furthermore, the information processing device utilizes text-generating artificial intelligence to generate user-friendly notification information. This notification information is then converted into speech by speech-generating artificial intelligence, providing users with visual and auditory instructions.

[0078] When a set time has elapsed, the information processing device sends a notification to the mobile device. The mobile device displays the notification on its screen and prompts the user to take action via voice. For example, a specific notification message such as "It's 8 PM. Have you finished your homework?" is sent.

[0079] The user checks the notification and responds using their mobile device. The user's response is sent to the information processing device and recorded in the database. The information processing device analyzes the response information and adjusts the next notification time based on the results.

[0080] As a concrete example, the following prompt can be used to input into the generation AI model: "Generate a notification message to check if the user has finished their homework by 8 PM." In this way, the system manages the user's daily routine and prompts them to take action at the appropriate time.

[0081] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0082] Step 1:

[0083] Users set their activity plans using an application on their mobile device. Specifically, they input their schedules, such as homework and bedtime, on the screen. The entered data is converted into an internal data format and temporarily stored on the device. This prepares the user's specific schedule as digital data.

[0084] Step 2:

[0085] The terminal transmits the user-configured action plan data to the information processing device. For transmission, the terminal uses a network connection and securely encrypts the data, packaging it into packets. During this process, the action plan data is properly delivered to the server and prepared for reception.

[0086] Step 3:

[0087] The server analyzes the received action plan data and stores it in the database. To maintain data integrity, the server performs transaction processing and optimizes the storage process. Based on the input data, it accumulates user history and uses it for later analysis.

[0088] Step 4:

[0089] The server uses a generated AI model to determine the timing of the next notification, based on the action plan recorded in the database. The server inputs prompts into the AI ​​model, referencing past data, to calculate the optimal notification time. This process creates a plan for the next notification.

[0090] Step 5:

[0091] The server uses text generation artificial intelligence to generate notification information for the user. Specifically, it generates a message prompting the user, such as, "Have you finished your homework yet?" This information is formatted as text data and prepared for subsequent processing.

[0092] Step 6:

[0093] The server then uses speech-generating artificial intelligence to convert the generated text messages into audio data. The server selects the optimal audio format and outputs an audio file suitable for playback on the device.

[0094] Step 7:

[0095] The server sends generated notification information (text and audio) to the terminal based on the determined notification time. The server manages the sending timing and executes the process according to the user's schedule.

[0096] Step 8:

[0097] The terminal displays notification information received from the server on its screen and outputs it as audio. The terminal plays the audio through its speaker, while the screen displays response options that the user can select.

[0098] Step 9:

[0099] The user responds by tapping the options displayed on the device. The selected option is recorded on the device as a digital signal. This allows for the collection of user feedback.

[0100] Step 10:

[0101] The terminal sends the user's response to the server. The terminal forms an encrypted data packet and promptly transmits it to the server. This input data is then passed to the server for the next feedback cycle.

[0102] Step 11:

[0103] The server analyzes the received response data and adjusts the timing of the next notification. Based on the response data, it performs analysis using a generated AI model and calculates adjustment results that match the operating pattern. This enables continuous notification optimization.

[0104] (Application Example 1)

[0105] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0106] In modern consumer behavior, controlling purchasing behavior and managing budgets remains a challenge, especially for younger generations. There is a need for effective methods to help young people develop healthy spending habits while minimizing parental intervention. There is a growing need for systems that can provide timely and appropriate advice, rather than simply issuing notifications or suggesting reductions.

[0107] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0108] In this invention, the server includes means for setting information about consumption activities based on user input in a mobile information terminal, means for storing and analyzing the information about consumption activities in the server device, and means for predicting consumption behavior using an AI model generated by the server device and generating presentation information based on that prediction. This makes it possible to provide optimal advice and warnings in real time according to the user's consumption behavior.

[0109] A "portable information terminal" refers to an electronic communication device owned and carried by an individual, such as a smartphone or tablet.

[0110] "User input" refers to the act of a user providing information to a system via a mobile device, including touch operations and voice input.

[0111] "Information related to consumer activity" refers to data related to users' daily purchasing behavior and budgets.

[0112] A "server device" is a central management device that processes and stores data on a network and coordinates with other devices.

[0113] A "generative AI model" refers to a set of algorithms built to perform specific tasks using machine learning and artificial intelligence technologies.

[0114] "Notification data" refers to messages or alerts sent to a mobile device to inform the user of specific information.

[0115] "Presented information" refers to advice and warnings provided to users in relation to their consumer activities.

[0116] "Real-time optimal advice" refers to the most appropriate guidance and instructions that are generated and provided instantly based on the user's actions and circumstances.

[0117] This invention utilizes a mobile information terminal, a server device, and a generative AI model to construct a management system related to consumer activity. The mobile information terminal provides a means for the user to input information about their consumer activity. This input information is transmitted to the server device in real time.

[0118] The server centrally stores received consumer activity information and uses a generated AI model for analysis. The AI ​​model analyzes the user's spending trends and predicted purchasing behavior patterns, and generates the most appropriate information regarding future consumer behavior. This allows the server to generate user-friendly notification data regarding budget management and spending pace. For example, it can send messages that include specific suggestions such as, "You're likely to exceed your budget this month. If your next purchase isn't urgent, let's wait until next month."

[0119] In this way, users can more effectively manage their own consumption habits, enabling the establishment of reliable consumption habits without excessive parental intervention. An example of a specific prompt message would be, "Predict the timing of the next purchase based on the child's purchase history and generate friendly notifications. Provide messages that support purchasing behavior."

[0120] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0121] Step 1:

[0122] The user operates a mobile device and inputs information about their purchasing activities. Here, the user enters the products they plan to purchase and their budget into the device, and the entered data is sent directly to the server.

[0123] Step 2:

[0124] The server temporarily records consumer activity information received from mobile devices and supplies it as input data to a generating AI model. This AI model analyzes the user's purchasing behavior based on past consumer data and performs calculations to predict future consumer behavior.

[0125] Step 3:

[0126] The generative AI model generates notification data, including an optimal spending plan, based on predicted purchasing behavior. Here, the AI ​​uses the analysis results to create prompt sentences, which in turn generate user-friendly notification messages. For example, a message like, "You're likely to exceed your budget. Let's reconsider your next purchase," might be generated.

[0127] Step 4:

[0128] The server sends the generated notification data to the mobile device. At this time, the server converts the text obtained from the generating AI model into a format that is easy for the user to receive, such as converting it into an audio format, before sending it.

[0129] Step 5:

[0130] The mobile device displays notifications received from the server to the user and informs the user of their spending status via voice. The user can also respond to notifications using options on the device.

[0131] Step 6:

[0132] The user's response data is sent back to the server, which stores this data and uses it as information for subsequent analysis and notification data generation. Based on user feedback, the timing of the next notification may also be adjusted.

[0133] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0134] This invention provides a system that combines a mobile information terminal and a server device to support the user's daily routine while responding to the user's emotional state using an emotion engine. In this embodiment, the mobile information terminal is equipped with an interface for receiving schedule information and emotional state from the user as input.

[0135] Users set their daily schedules on their mobile devices, which helps manage their planned daily rhythms. During schedule setting, the device analyzes the user's emotional state using an emotion engine based on their voice and text. This emotional information is transmitted from the mobile device to a server device, influencing subsequent processes.

[0136] The server device stores the user's daily schedule and emotional data. Analysis functions within the server customize notification messages generated based on the user's emotional state. Text generation artificial intelligence creates messages with a tone and content favorable to the user, based on feedback from the emotion engine. Furthermore, speech generation artificial intelligence converts these text messages into emotionally appropriate speech.

[0137] At regular intervals, the server sends an emotionally sensitive notification message to the mobile device. The device then communicates the notification to the user via voice and screen display, prompting them to take action. For example, if the user is feeling down, an encouraging message may be sent.

[0138] The user checks the notification and selects a response on their mobile device. The response is sent to the server via the device and recorded for analysis. The server considers not only the response data but also sentiment data to adjust the timing and content of the next notification.

[0139] As a result, this invention not only provides support for time management, but also enables more effective lifestyle support by offering communication that is sensitive to the user's emotions.

[0140] The following describes the processing flow.

[0141] Step 1:

[0142] Users set their daily schedules using their mobile devices. When entering schedules, the device collects user input and voice data, and an emotion engine analyzes their emotional state at that time.

[0143] Step 2:

[0144] The device sends the user's configured schedule data and sentiment data to the server. The server then stores this information in a database.

[0145] Step 3:

[0146] The server uses the received schedule and sentiment data to prepare for generating notification messages. The server utilizes text generation artificial intelligence to create customized text messages that respond to the user's emotions. For example, if the user is relaxed, it will generate an encouraging message.

[0147] Step 4:

[0148] The server uses speech-generating artificial intelligence to convert text messages into audio data. The voice tone is also appropriately adjusted based on emotional data to generate a more natural and friendly voice.

[0149] Step 5:

[0150] As the designated time approaches, the server generates notification data and sends it to the device. The device receives the notification, displays it on the screen, and simultaneously outputs an audio signal.

[0151] Step 6:

[0152] The user checks the notification and chooses a response from the options on their device. Options such as "Yes" and "No" are provided, and the user taps accordingly.

[0153] Step 7:

[0154] The device sends the user's selected response data back to the server. This response data is recorded and analyzed on the server along with sentiment data.

[0155] Step 8:

[0156] The server analyzes recorded response and sentiment data, using it as a feedback loop to improve the content and timing of future notification messages. It provides more effective notifications based on the user's past responses.

[0157] (Example 2)

[0158] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0159] In users' daily lives, it is necessary to provide appropriate support and advice that takes into account not only time management but also their emotional state. Traditional systems failed to consider emotions, making it difficult to generate notifications and messages at the appropriate time in response to user behavior and emotions. There is a need to provide a system that can provide notifications that are sensitive to the user's emotions.

[0160] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0161] In this invention, the server includes means for receiving and analyzing user emotion data from a mobile information terminal, means for generating notification data corresponding to the user's emotions using a generative AI model, and means for analyzing the user's response data and adjusting the timing and content of the next notification. This makes it possible to provide information with appropriate notification content and timing while considering the user's emotional state in real time.

[0162] A "personal information terminal" refers to a multi-functional computer device that is portable and used daily by users, and includes smartphones and tablets.

[0163] "User input" refers to information provided by users through their mobile devices, including audio and text indicating action plans and emotional states.

[0164] "Emotional state" refers to the user's psychological and emotional state, which is estimated through voice and text analysis.

[0165] A "server device" is a central computer system responsible for storing, analyzing, and generating data, and it also plays a role in communicating with client mobile information terminals.

[0166] A "generative AI model" is an algorithm based on artificial intelligence that generates notification data based on the user's emotions.

[0167] "Notification data" refers to messages and information generated by the server device and provided to the user, which are customized according to the user's emotional state.

[0168] "Text-generating artificial intelligence" is artificial intelligence designed to generate text data and output language expressions that match the user's emotions.

[0169] "Speech-generating artificial intelligence" is artificial intelligence that converts text data into speech format, and is used to generate voice notifications that reflect emotions.

[0170] "Response data" refers to the digital data representing the user's chosen response to a notification, and is used to improve future notifications.

[0171] This invention is a system that combines a mobile information terminal and a server device to support the user's daily routine and consider the user's emotional state using an emotion engine. In addition to managing the user's daily routine, this system enhances support for the user by providing communication that responds to their emotions.

[0172] Users input their daily activity plans via their mobile devices. This includes entering appointments and emotional states in voice and text format. The mobile device uses speech recognition software (e.g., a speech recognition API) to convert speech to text, which is then analyzed by an emotion engine.

[0173] The sentiment-analyzed data is transmitted from the mobile device to the server. Data security is guaranteed by using an encrypted communication protocol. The server stores the received data and generates a notification message corresponding to the emotion through its analysis function. A generative AI model is used for generation, employing both text-generating and speech-generating artificial intelligence. Specifically, the generative AI model receives a prompt and creates a message with an appropriate tone for the user. An example of a prompt might be, "The user appears to be stressed. Please generate a message to help them relax."

[0174] At regular intervals, the server sends the generated message to the mobile device. The device presents this notification to the user in both audio and text, prompting action. For example, when the user is heading to a meeting, it can generate a message such as, "Enjoy your next meeting! Relax and be prepared."

[0175] Users provide feedback to the system by reviewing and responding to notifications. Response data is sent to the server and used to improve future notifications. Through this process, the present invention provides effective lifestyle support that is sensitive to the user's emotions.

[0176] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0177] Step 1:

[0178] The user enters their action plan into a mobile device. This input includes both audio and text regarding their schedule and emotional state. Speech recognition software converts the audio to text, and an emotion analysis engine estimates the emotional state from the text, outputting the results. This output consists of schedule data and emotion data.

[0179] Step 2:

[0180] The terminal sends sentiment-analyzed data to the server device. An encryption protocol is used for data transmission, ensuring secure transfer to the server. Inputs are schedule data and sentiment data from the terminal, while outputs are these data stored on the server.

[0181] Step 3:

[0182] The server stores the received data and generates notification messages using a generative AI model based on sentiment analysis. The server analyzes the input sentiment data and provides prompts to the generative AI model, which then generates a text message that aligns with the user's emotional state. An example of such a prompt is, "The user appears to be stressed. Please generate a message to help them relax."

[0183] Step 4:

[0184] The server performs the process of converting the generated text message into speech using speech-generating artificial intelligence. The input is the generated text message, and the output is speech data that corresponds to the user's emotions.

[0185] Step 5:

[0186] The server sends the generated audio and text messages to the mobile device. The input is audio and text data, and the output is a user notification on the device. This notification is presented to the user through screen display and audio playback. A specific example might be, "Enjoy your next meeting! Try to relax."

[0187] Step 6:

[0188] The user checks the notification and enters a response into their mobile device. The input is the content of the response to the notification, and the output is the data that is sent to the server.

[0189] Step 7:

[0190] The server analyzes user response data and uses it to adjust the timing and content of future notifications. Inputs are response data and sentiment data, while output is the adjusted notification schedule and content. This enables the continuous delivery of information that is sensitive to the user's emotions.

[0191] (Application Example 2)

[0192] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0193] In modern society, users lead busy lives and require time management and appropriate information tailored to their emotions. However, conventional systems have struggled to customize information based on users' emotional states, making it difficult to provide timely payment information and campaigns that meet individual user needs. Therefore, new solutions are needed to improve user satisfaction.

[0194] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0195] In this invention, the server includes means for performing emotion analysis and customizing notification information generated based on the user's emotional state; means for generating emotion-appropriate notification information using language-generating artificial intelligence and speech-generating artificial intelligence and providing information related to payment transactions; and means for analyzing the user's response information and adjusting the timing of the next notification and the content of the information according to the emotion based on the results. This enables more personalized information provision and time management for the user.

[0196] A "portable information device" is an electronic device that a user can carry around and use for inputting and outputting information.

[0197] A "time management sheet" is data containing information for managing a user's schedule.

[0198] A "server device" is a computing device used to store, analyze, and distribute information over a network.

[0199] "Notification information" refers to messages and alerts sent to users.

[0200] "Emotional analysis" is the process of analyzing a user's emotional state.

[0201] "Language-generating artificial intelligence" refers to artificial intelligence technology used to generate natural language.

[0202] "Speech-generating artificial intelligence" is an artificial intelligence technology that converts text into speech.

[0203] "User response information" refers to data that shows how users respond to notifications and messages.

[0204] A "settlement transaction" is a procedure necessary for making a purchase or payment.

[0205] "Means of adjusting information content" refers to methods for changing the content of information according to the user's circumstances and needs.

[0206] "Customizing" means optimizing the system to suit the individual needs and circumstances of each user.

[0207] To implement this invention, it is first necessary to input the user's time management schedule and record their emotional state using a portable information device. The portable information device has a schedule management application that allows the user to easily input and manage their daily schedule. This application receives the user's voice and text input and performs emotional analysis. At this time, an emotional engine and related analysis tools are used to understand the user's current emotional state.

[0208] Next, time management charts and emotional data are transmitted from the mobile device to the server. The server stores and analyzes this data. The server uses language-generating artificial intelligence (e.g., OpenAI® GPT-3®) and speech-generating artificial intelligence (e.g., AWS® Polly) to generate notification information tailored to the user's emotional state. This notification information may include, for example, special payment offers or campaign announcements that are tailored to the user's preferences and emotions.

[0209] The generated notification information is sent to the user's mobile device and displayed to them. The notifications are provided in text or audio format and are tailored to the user's emotional state. For example, if a user is relaxing after a busy day, the emotion engine detects the emotion of "peace" and notifies them of a discount on an online salon.

[0210] This sequence of events allows the system to provide information tailored to each individual user, making their daily life more comfortable. An example of a prompt message is: "You are relaxing at the end of the day. How can you enjoy this time by receiving a special offer tailored to your current mood?"

[0211] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0212] Step 1:

[0213] The device accepts input from the user, including a time management sheet and emotional state. The user uses an application to input their daily schedule and provides this information via voice or text. This input data is processed by an emotion analysis engine to extract the user's emotional state.

[0214] Step 2:

[0215] The terminal sends the entered time management sheet and analyzed emotional state to the server. The server receives this data and stores it in a database. The stored data is used in subsequent processing and forms the basis for understanding the user's schedule and emotional tendencies.

[0216] Step 3:

[0217] The server uses language-generating artificial intelligence to generate optimal notification information for the user based on their stored emotional state. Here, the emotional state is used as input to generate a prompt, and based on that, a message with an appropriate tone and content is generated. This process ensures that messages are tailored to the user's emotions.

[0218] Step 4:

[0219] The generated text messages are converted into audio data using speech-generating artificial intelligence. The text data is input into the speech engine, which then produces emotionally responsive audio output. This prepares the system to deliver notifications to users that are both auditory and emotionally sensitive.

[0220] Step 5:

[0221] The server sends generated audio and text notification information to the device. The device notifies the user of this information visually and audibly. The user receives the notification from the device and chooses an action based on it.

[0222] Step 6:

[0223] The user's response to a notification is sent back to the server via the terminal. The server analyzes this response information as input and readjusts the content and timing of the next notification. This ensures that the user receives the most optimal information in stages.

[0224] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0225] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0226] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0227] [Second Embodiment]

[0228] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0229] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0230] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0231] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0232] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0233] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0234] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0235] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0236] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0237] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0238] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0239] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0240] The system according to the present invention supports a child's daily routine via a mobile information terminal and is realized through the cooperation of three parties: a server device, a mobile information terminal, and a user. A specific embodiment of this system is described below.

[0241] First, the user sets their daily schedule using an application installed on their mobile device. The user can set routines for activities such as homework, bathing, and bedtime, and customize them as needed. The set schedule is then sent from the mobile device to a server device and stored in a database.

[0242] The server device manages notification timing for each user based on the received schedule information. Text-generating artificial intelligence creates user-friendly notification messages based on the user's schedule. Furthermore, speech-generating artificial intelligence converts the text into speech format, making it easier to recognize by ear.

[0243] As the set time approaches, the server device sends the generated notification data to the mobile device. The mobile device displays the received notification on its screen and also prompts the user to confirm their actions via voice. For example, a message such as "It's 8 PM. Have you finished your homework?" might be sent.

[0244] After the user confirms the notification, they respond by tapping an option on their mobile device. This selection data is then sent back to the server device, where it is recorded and analyzed. For example, if the user answers "No," the server device will take appropriate action, such as reviewing the timing of the next notification, to help the user manage their time.

[0245] Therefore, the system based on the present invention enables users to habitualize their behavior, improve their time management skills, and consequently reduce parental intervention.

[0246] The following describes the processing flow.

[0247] Step 1:

[0248] The user operates their mobile device to launch the application and open the daily schedule settings screen. The user enters their time schedule, including homework, bath time, and bedtime, and saves the settings.

[0249] Step 2:

[0250] The terminal sends the schedule data entered by the user to the server. The transmitted data is organized by user and stored in a database.

[0251] Step 3:

[0252] The server calculates the time to send a notification based on the received schedule and determines the content of the notification to be sent at that time.

[0253] Step 4:

[0254] The server generates notification messages using text generation artificial intelligence. For example, a message like "It's 8 PM. Have you finished your homework?" might be created.

[0255] Step 5:

[0256] The server uses speech-generating artificial intelligence to convert the generated text messages into audio data. This is to make the audio data more naturally recognizable to the user.

[0257] Step 6:

[0258] As the specified time approaches, the server sends the generated notification data to the device. The device then processes the received data immediately.

[0259] Step 7:

[0260] The device displays a notification on the screen and plays it aloud for the user. This allows the user to confirm the notification content both visually and aurally.

[0261] Step 8:

[0262] The user selects a response to the notification. Options such as "Yes" or "No" are presented, and the user taps the appropriate option.

[0263] Step 9:

[0264] The terminal sends the user's response back to the server. This response data is used for subsequent processing.

[0265] Step 10:

[0266] The server analyzes the response data it receives to understand user behavior patterns. Based on the analysis results, it plans to adjust the timing and content of future notifications.

[0267] (Example 1)

[0268] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0269] In modern society, carefully managing children's daily routines and promoting habit formation is crucial for busy parents. However, a lack of timely notifications and feedback hinders the development of children's time management skills. To address this issue, it is necessary to support children's behavior through appropriately tuned notification systems and response analysis.

[0270] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0271] In this invention, the server includes means for setting an action plan based on information input in a mobile information terminal, means for storing and analyzing the action plan in an information processing device, and means for transmitting notification information generated by the information processing device to the mobile information terminal and notifying the user. This makes it possible to manage the user's daily rhythm and promote actions at appropriate times.

[0272] A "portable information terminal" refers to a device that a user can carry around and use for inputting and displaying information.

[0273] "Information input" refers to the act of a user entering data via a mobile device.

[0274] An "action plan" refers to a schedule or plan for managing and implementing the user's daily routine.

[0275] An "information processing device" refers to a device that receives, stores, analyzes, and generates notification information for data.

[0276] "Notification information" refers to messages and audio data generated for users based on action plans.

[0277] "Text generation artificial intelligence" refers to an algorithm that automatically generates text data.

[0278] "Speech-generating artificial intelligence" refers to an algorithm that converts text data into speech.

[0279] "User" refers to an individual who implements an action plan and receives notifications via a mobile device.

[0280] "Response information" refers to data obtained when a user responds to a notification using their mobile device.

[0281] "Analysis results" refer to the results of the analysis derived by the information processing device based on the user's response information.

[0282] The system of the present invention consists of the cooperation of three parties: an information processing device, a mobile information terminal, and a user. Specifically, the user sets their own action plan using an application installed on the mobile information terminal. This action plan includes specific activities such as homework and bedtime. The set action plan is transmitted from the mobile information terminal to the information processing device.

[0283] The information processing device stores the received action plan in a database and analyzes it. The analysis includes a process of optimizing using a generative AI model based on past action patterns and the current action plan. Also, the information processing device utilizes a character generation artificial intelligence to generate notification information that is easy for the user to understand. This notification information is vocalized by a voice generation artificial intelligence to provide the user with instructions visually and aurally.

[0284] When the set time has elapsed, the information processing device transmits the notification information to the mobile information terminal. The mobile information terminal displays the notification on the screen and prompts the user to act by voice. For example, a specific notification message such as "It's 8 PM. Have you finished your homework?" is sent.

[0285] The user checks the notification and responds using the mobile information terminal. The user's response is transmitted to the information processing device and recorded in the database. The information processing device analyzes the response information and adjusts the next notification time based on the result.

[0286] As a specific example, the following prompt sentence can be used to input into the generative AI model. "Please generate a notification message to check if the user has finished their homework at 8 PM." In this way, the system manages the user's life rhythm and promotes action at an appropriate timing.

[0287] The flow of the specific process in Example 1 will be described using FIG. 11.

[0288] Step 1:

[0289] The user sets an action plan using the application on the mobile information terminal. Specifically, the user inputs schedules such as homework and bedtime on the screen. The input data is converted into an internal data format and temporarily stored in the terminal. Thereby, the user's specific schedule is prepared as digital data.

[0290] Step 2:

[0291] The terminal transmits the user-configured action plan data to the information processing device. For transmission, the terminal uses a network connection and securely encrypts the data, packaging it into packets. During this process, the action plan data is properly delivered to the server and prepared for reception.

[0292] Step 3:

[0293] The server analyzes the received action plan data and stores it in the database. To maintain data integrity, the server performs transaction processing and optimizes the storage process. Based on the input data, it accumulates user history and uses it for later analysis.

[0294] Step 4:

[0295] The server uses a generated AI model to determine the timing of the next notification, based on the action plan recorded in the database. The server inputs prompts into the AI ​​model, referencing past data, to calculate the optimal notification time. This process creates a plan for the next notification.

[0296] Step 5:

[0297] The server uses text generation artificial intelligence to generate notification information for the user. Specifically, it generates a message prompting the user, such as, "Have you finished your homework yet?" This information is formatted as text data and prepared for subsequent processing.

[0298] Step 6:

[0299] The server then uses speech-generating artificial intelligence to convert the generated text messages into audio data. The server selects the optimal audio format and outputs an audio file suitable for playback on the device.

[0300] Step 7:

[0301] Based on the determined notification time, the server transmits the generated notification information (text and voice) to the terminal. The server manages the transmission timing and executes the process according to the user's schedule.

[0302] Step 8:

[0303] The terminal displays the notification information received from the server on the screen and outputs it as voice. The terminal plays the voice through the speaker and presents response options that the user can select on the screen.

[0304] Step 9:

[0305] The user taps on the options displayed on the terminal to respond. Which option is selected is recorded on the terminal as a digital signal. In this way, the user's feedback is collected.

[0306] Step 10:

[0307] The terminal transmits the user's response to the server. The terminal forms an encrypted data packet and promptly conveys it to the server. This input data is passed to the server for the next feedback cycle.

[0308] Step 11:

[0309] The server analyzes the received response data and adjusts the next notification timing. Analysis is performed using the generated AI model based on the response data, and adjustment results in line with the operation pattern are calculated. In this way, continuous notification optimization is implemented.

[0310] (Application Example 1)

[0311] Next, Application Example 1 will be described. In the following description, the data processing device 12 is referred to as the "server", and the smart glasses 214 are referred to as the "terminal".

[0312] In modern consumer behavior, controlling purchasing behavior and managing budgets remains a challenge, especially for younger generations. There is a need for effective methods to help young people develop healthy spending habits while minimizing parental intervention. There is a growing need for systems that can provide timely and appropriate advice, rather than simply issuing notifications or suggesting reductions.

[0313] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0314] In this invention, the server includes means for setting information about consumption activities based on user input in a mobile information terminal, means for storing and analyzing the information about consumption activities in the server device, and means for predicting consumption behavior using an AI model generated by the server device and generating presentation information based on that prediction. This makes it possible to provide optimal advice and warnings in real time according to the user's consumption behavior.

[0315] A "portable information terminal" refers to an electronic communication device owned and carried by an individual, such as a smartphone or tablet.

[0316] "User input" refers to the act of a user providing information to a system via a mobile device, including touch operations and voice input.

[0317] "Information related to consumer activity" refers to data related to users' daily purchasing behavior and budgets.

[0318] A "server device" is a central management device that processes and stores data on a network and coordinates with other devices.

[0319] A "generative AI model" refers to a set of algorithms built to perform specific tasks using machine learning and artificial intelligence technologies.

[0320] "Notification data" refers to messages or alerts sent to a mobile device to inform the user of specific information.

[0321] "Presented information" refers to advice and warnings provided to users in relation to their consumer activities.

[0322] "Real-time optimal advice" refers to the most appropriate guidance and instructions that are generated and provided instantly based on the user's actions and circumstances.

[0323] This invention utilizes a mobile information terminal, a server device, and a generative AI model to construct a management system related to consumer activity. The mobile information terminal provides a means for the user to input information about their consumer activity. This input information is transmitted to the server device in real time.

[0324] The server centrally stores received consumer activity information and uses a generated AI model for analysis. The AI ​​model analyzes the user's spending trends and predicted purchasing behavior patterns, and generates the most appropriate information regarding future consumer behavior. This allows the server to generate user-friendly notification data regarding budget management and spending pace. For example, it can send messages that include specific suggestions such as, "You're likely to exceed your budget this month. If your next purchase isn't urgent, let's wait until next month."

[0325] In this way, users can more effectively manage their own consumption habits, enabling the establishment of reliable consumption habits without excessive parental intervention. An example of a specific prompt message would be, "Predict the timing of the next purchase based on the child's purchase history and generate friendly notifications. Provide messages that support purchasing behavior."

[0326] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0327] Step 1:

[0328] The user operates a mobile device and inputs information about their purchasing activities. Here, the user enters the products they plan to purchase and their budget into the device, and the entered data is sent directly to the server.

[0329] Step 2:

[0330] The server temporarily records consumer activity information received from mobile devices and supplies it as input data to a generating AI model. This AI model analyzes the user's purchasing behavior based on past consumer data and performs calculations to predict future consumer behavior.

[0331] Step 3:

[0332] The generative AI model generates notification data, including an optimal spending plan, based on predicted purchasing behavior. Here, the AI ​​uses the analysis results to create prompt sentences, which in turn generate user-friendly notification messages. For example, a message like, "You're likely to exceed your budget. Let's reconsider your next purchase," might be generated.

[0333] Step 4:

[0334] The server sends the generated notification data to the mobile device. At this time, the server converts the text obtained from the generating AI model into a format that is easy for the user to receive, such as converting it into an audio format, before sending it.

[0335] Step 5:

[0336] The mobile device displays notifications received from the server to the user and informs the user of their spending status via voice. The user can also respond to notifications using options on the device.

[0337] Step 6:

[0338] The user's response data is sent back to the server, which stores this data and uses it as information for subsequent analysis and notification data generation. Based on user feedback, the timing of the next notification may also be adjusted.

[0339] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0340] This invention provides a system that combines a mobile information terminal and a server device to support the user's daily routine while responding to the user's emotional state using an emotion engine. In this embodiment, the mobile information terminal is equipped with an interface for receiving schedule information and emotional state from the user as input.

[0341] Users set their daily schedules on their mobile devices, which helps manage their planned daily rhythms. During schedule setting, the device analyzes the user's emotional state using an emotion engine based on their voice and text. This emotional information is transmitted from the mobile device to a server device, influencing subsequent processes.

[0342] The server device stores the user's daily schedule and emotional data. Analysis functions within the server customize notification messages generated based on the user's emotional state. Text generation artificial intelligence creates messages with a tone and content favorable to the user, based on feedback from the emotion engine. Furthermore, speech generation artificial intelligence converts these text messages into emotionally appropriate speech.

[0343] At regular intervals, the server sends an emotionally sensitive notification message to the mobile device. The device then communicates the notification to the user via voice and screen display, prompting them to take action. For example, if the user is feeling down, an encouraging message may be sent.

[0344] The user checks the notification and selects a response on their mobile device. The response is sent to the server via the device and recorded for analysis. The server considers not only the response data but also sentiment data to adjust the timing and content of the next notification.

[0345] As a result, this invention not only provides support for time management, but also enables more effective lifestyle support by offering communication that is sensitive to the user's emotions.

[0346] The following describes the processing flow.

[0347] Step 1:

[0348] Users set their daily schedules using their mobile devices. When entering schedules, the device collects user input and voice data, and an emotion engine analyzes their emotional state at that time.

[0349] Step 2:

[0350] The device sends the user's configured schedule data and sentiment data to the server. The server then stores this information in a database.

[0351] Step 3:

[0352] The server uses the received schedule and sentiment data to prepare for generating notification messages. The server utilizes text generation artificial intelligence to create customized text messages that respond to the user's emotions. For example, if the user is relaxed, it will generate an encouraging message.

[0353] Step 4:

[0354] The server uses speech-generating artificial intelligence to convert text messages into audio data. The voice tone is also appropriately adjusted based on emotional data to generate a more natural and friendly voice.

[0355] Step 5:

[0356] As the designated time approaches, the server generates notification data and sends it to the device. The device receives the notification, displays it on the screen, and simultaneously outputs an audio signal.

[0357] Step 6:

[0358] The user checks the notification and chooses a response from the options on their device. Options such as "Yes" and "No" are provided, and the user taps accordingly.

[0359] Step 7:

[0360] The device sends the user's selected response data back to the server. This response data is recorded and analyzed on the server along with sentiment data.

[0361] Step 8:

[0362] The server analyzes recorded response and sentiment data, using it as a feedback loop to improve the content and timing of future notification messages. It provides more effective notifications based on the user's past responses.

[0363] (Example 2)

[0364] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0365] In users' daily lives, it is necessary to provide appropriate support and advice that takes into account not only time management but also their emotional state. Traditional systems failed to consider emotions, making it difficult to generate notifications and messages at the appropriate time in response to user behavior and emotions. There is a need to provide a system that can provide notifications that are sensitive to the user's emotions.

[0366] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0367] In this invention, the server includes means for receiving and analyzing user emotion data from a mobile information terminal, means for generating notification data corresponding to the user's emotions using a generative AI model, and means for analyzing the user's response data and adjusting the timing and content of the next notification. This makes it possible to provide information with appropriate notification content and timing while considering the user's emotional state in real time.

[0368] A "personal information terminal" refers to a multi-functional computer device that is portable and used daily by users, and includes smartphones and tablets.

[0369] "User input" refers to information provided by users through their mobile devices, including audio and text indicating action plans and emotional states.

[0370] "Emotional state" refers to the user's psychological and emotional state, which is estimated through voice and text analysis.

[0371] A "server device" is a central computer system responsible for storing, analyzing, and generating data, and it also plays a role in communicating with client mobile information terminals.

[0372] A "generative AI model" is an algorithm based on artificial intelligence that generates notification data based on the user's emotions.

[0373] "Notification data" refers to messages and information generated by the server device and provided to the user, which are customized according to the user's emotional state.

[0374] "Text-generating artificial intelligence" is artificial intelligence designed to generate text data and output language expressions that match the user's emotions.

[0375] "Speech-generating artificial intelligence" is artificial intelligence that converts text data into speech format, and is used to generate voice notifications that reflect emotions.

[0376] "Response data" refers to the digital data representing the user's chosen response to a notification, and is used to improve future notifications.

[0377] This invention is a system that combines a mobile information terminal and a server device to support the user's daily routine and consider the user's emotional state using an emotion engine. In addition to managing the user's daily routine, this system enhances support for the user by providing communication that responds to their emotions.

[0378] Users input their daily activity plans via their mobile devices. This includes entering appointments and emotional states in voice and text format. The mobile device uses speech recognition software (e.g., a speech recognition API) to convert speech to text, which is then analyzed by an emotion engine.

[0379] The sentiment-analyzed data is transmitted from the mobile device to the server. Data security is guaranteed by using an encrypted communication protocol. The server stores the received data and generates a notification message corresponding to the emotion through its analysis function. A generative AI model is used for generation, employing both text-generating and speech-generating artificial intelligence. Specifically, the generative AI model receives a prompt and creates a message with an appropriate tone for the user. An example of a prompt might be, "The user appears to be stressed. Please generate a message to help them relax."

[0380] At regular intervals, the server sends the generated message to the mobile device. The device presents this notification to the user in both audio and text, prompting action. For example, when the user is heading to a meeting, it can generate a message such as, "Enjoy your next meeting! Relax and be prepared."

[0381] Users provide feedback to the system by reviewing and responding to notifications. Response data is sent to the server and used to improve future notifications. Through this process, the present invention provides effective lifestyle support that is sensitive to the user's emotions.

[0382] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0383] Step 1:

[0384] The user enters their action plan into a mobile device. This input includes both audio and text regarding their schedule and emotional state. Speech recognition software converts the audio to text, and an emotion analysis engine estimates the emotional state from the text, outputting the results. This output consists of schedule data and emotion data.

[0385] Step 2:

[0386] The terminal sends sentiment-analyzed data to the server device. An encryption protocol is used for data transmission, ensuring secure transfer to the server. Inputs are schedule data and sentiment data from the terminal, while outputs are these data stored on the server.

[0387] Step 3:

[0388] The server stores the received data and generates notification messages using a generative AI model based on sentiment analysis. The server analyzes the input sentiment data and provides prompts to the generative AI model, which then generates a text message that aligns with the user's emotional state. An example of such a prompt is, "The user appears to be stressed. Please generate a message to help them relax."

[0389] Step 4:

[0390] The server performs the process of converting the generated text message into speech using speech-generating artificial intelligence. The input is the generated text message, and the output is speech data that corresponds to the user's emotions.

[0391] Step 5:

[0392] The server sends the generated audio and text messages to the mobile device. The input is audio and text data, and the output is a user notification on the device. This notification is presented to the user through screen display and audio playback. A specific example might be, "Enjoy your next meeting! Try to relax."

[0393] Step 6:

[0394] The user checks the notification and enters a response into their mobile device. The input is the content of the response to the notification, and the output is the data that is sent to the server.

[0395] Step 7:

[0396] The server analyzes user response data and uses it to adjust the timing and content of future notifications. Inputs are response data and sentiment data, while output is the adjusted notification schedule and content. This enables the continuous delivery of information that is sensitive to the user's emotions.

[0397] (Application Example 2)

[0398] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart glasses 214 as the "terminal".

[0399] In modern society, users lead busy lives and require time management and appropriate information tailored to their emotions. However, conventional systems have struggled to customize information based on users' emotional states, making it difficult to provide timely payment information and campaigns that meet individual user needs. Therefore, new solutions are needed to improve user satisfaction.

[0400] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0401] In this invention, the server includes means for performing emotion analysis and customizing notification information generated based on the user's emotional state; means for generating emotion-appropriate notification information using language-generating artificial intelligence and speech-generating artificial intelligence and providing information related to payment transactions; and means for analyzing the user's response information and adjusting the timing of the next notification and the content of the information according to the emotion based on the results. This enables more personalized information provision and time management for the user.

[0402] A "portable information device" is an electronic device that a user can carry around and use for inputting and outputting information.

[0403] A "time management sheet" is data containing information for managing a user's schedule.

[0404] A "server device" is a computing device used to store, analyze, and distribute information over a network.

[0405] "Notification information" refers to messages and alerts sent to users.

[0406] "Emotional analysis" is the process of analyzing a user's emotional state.

[0407] "Language-generating artificial intelligence" refers to artificial intelligence technology used to generate natural language.

[0408] "Speech-generating artificial intelligence" is an artificial intelligence technology that converts text into speech.

[0409] "User response information" refers to data that shows how users respond to notifications and messages.

[0410] A "settlement transaction" is a procedure necessary for making a purchase or payment.

[0411] "Means of adjusting information content" refers to methods for changing the content of information according to the user's circumstances and needs.

[0412] "Customizing" means optimizing the system to suit the individual needs and circumstances of each user.

[0413] To implement this invention, it is first necessary to input the user's time management schedule and record their emotional state using a portable information device. The portable information device has a schedule management application that allows the user to easily input and manage their daily schedule. This application receives the user's voice and text input and performs emotional analysis. At this time, an emotional engine and related analysis tools are used to understand the user's current emotional state.

[0414] Next, time management charts and emotional data are transmitted from the mobile device to the server. The server stores and analyzes this data. The server uses language-generating artificial intelligence (e.g., OpenAI GPT-3) or speech-generating artificial intelligence (e.g., AWS Polly) to generate notification information tailored to the user's emotional state. This notification information may include, for example, special payment offers or campaign announcements that match the user's preferences and emotions.

[0415] The generated notification information is sent to the user's mobile device and displayed to them. The notifications are provided in text or audio format and are tailored to the user's emotional state. For example, if a user is relaxing after a busy day, the emotion engine detects the emotion of "peace" and notifies them of a discount on an online salon.

[0416] This sequence of events allows the system to provide information tailored to each individual user, making their daily life more comfortable. An example of a prompt message is: "You are relaxing at the end of the day. How can you enjoy this time by receiving a special offer tailored to your current mood?"

[0417] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0418] Step 1:

[0419] The device accepts input from the user, including a time management sheet and emotional state. The user uses an application to input their daily schedule and provides this information via voice or text. This input data is processed by an emotion analysis engine to extract the user's emotional state.

[0420] Step 2:

[0421] The terminal sends the entered time management sheet and analyzed emotional state to the server. The server receives this data and stores it in a database. The stored data is used in subsequent processing and forms the basis for understanding the user's schedule and emotional tendencies.

[0422] Step 3:

[0423] The server uses language-generating artificial intelligence to generate optimal notification information for the user based on their stored emotional state. Here, the emotional state is used as input to generate a prompt, and based on that, a message with an appropriate tone and content is generated. This process ensures that messages are tailored to the user's emotions.

[0424] Step 4:

[0425] The generated text messages are converted into audio data using speech-generating artificial intelligence. The text data is input into the speech engine, which then produces emotionally responsive audio output. This prepares the system to deliver notifications to users that are both auditory and emotionally sensitive.

[0426] Step 5:

[0427] The server sends generated audio and text notification information to the device. The device notifies the user of this information visually and audibly. The user receives the notification from the device and chooses an action based on it.

[0428] Step 6:

[0429] The user's response to a notification is sent back to the server via the terminal. The server analyzes this response information as input and readjusts the content and timing of the next notification. This ensures that the user receives the most optimal information in stages.

[0430] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0431] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0432] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0433] [Third Embodiment]

[0434] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0435] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0436] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0437] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0438] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0439] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0440] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0441] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0442] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0443] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0444] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0445] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0446] The system according to the present invention supports a child's daily routine via a mobile information terminal and is realized through the cooperation of three parties: a server device, a mobile information terminal, and a user. A specific embodiment of this system is described below.

[0447] First, the user sets their daily schedule using an application installed on their mobile device. The user can set routines for activities such as homework, bathing, and bedtime, and customize them as needed. The set schedule is then sent from the mobile device to a server device and stored in a database.

[0448] The server device manages notification timing for each user based on the received schedule information. Text-generating artificial intelligence creates user-friendly notification messages based on the user's schedule. Furthermore, speech-generating artificial intelligence converts the text into speech format, making it easier to recognize by ear.

[0449] As the set time approaches, the server device sends the generated notification data to the mobile device. The mobile device displays the received notification on its screen and also prompts the user to confirm their actions via voice. For example, a message such as "It's 8 PM. Have you finished your homework?" might be sent.

[0450] After the user confirms the notification, they respond by tapping an option on their mobile device. This selection data is then sent back to the server device, where it is recorded and analyzed. For example, if the user answers "No," the server device will take appropriate action, such as reviewing the timing of the next notification, to help the user manage their time.

[0451] Therefore, the system based on the present invention enables users to habitualize their behavior, improve their time management skills, and consequently reduce parental intervention.

[0452] The following describes the processing flow.

[0453] Step 1:

[0454] The user operates their mobile device to launch the application and open the daily schedule settings screen. The user enters their time schedule, including homework, bath time, and bedtime, and saves the settings.

[0455] Step 2:

[0456] The terminal sends the schedule data entered by the user to the server. The transmitted data is organized by user and stored in a database.

[0457] Step 3:

[0458] The server calculates the time to send a notification based on the received schedule and determines the content of the notification to be sent at that time.

[0459] Step 4:

[0460] The server generates notification messages using text generation artificial intelligence. For example, a message like "It's 8 PM. Have you finished your homework?" might be created.

[0461] Step 5:

[0462] The server uses speech-generating artificial intelligence to convert the generated text messages into audio data. This is to make the audio data more naturally recognizable to the user.

[0463] Step 6:

[0464] As the specified time approaches, the server sends the generated notification data to the device. The device then processes the received data immediately.

[0465] Step 7:

[0466] The device displays a notification on the screen and plays it aloud for the user. This allows the user to confirm the notification content both visually and aurally.

[0467] Step 8:

[0468] The user selects a response to the notification. Options such as "Yes" or "No" are presented, and the user taps the appropriate option.

[0469] Step 9:

[0470] The terminal sends the user's response back to the server. This response data is used for subsequent processing.

[0471] Step 10:

[0472] The server analyzes the response data it receives to understand user behavior patterns. Based on the analysis results, it plans to adjust the timing and content of future notifications.

[0473] (Example 1)

[0474] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0475] In modern society, carefully managing children's daily routines and promoting habit formation is crucial for busy parents. However, a lack of timely notifications and feedback hinders the development of children's time management skills. To address this issue, it is necessary to support children's behavior through appropriately tuned notification systems and response analysis.

[0476] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0477] In this invention, the server includes means for setting an action plan based on information input in a mobile information terminal, means for storing and analyzing the action plan in an information processing device, and means for transmitting notification information generated by the information processing device to the mobile information terminal and notifying the user. This makes it possible to manage the user's daily rhythm and promote actions at appropriate times.

[0478] A "portable information terminal" refers to a device that a user can carry around and use for inputting and displaying information.

[0479] "Information input" refers to the act of a user entering data via a mobile device.

[0480] An "action plan" refers to a schedule or plan for managing and implementing the user's daily routine.

[0481] An "information processing device" refers to a device that receives, stores, analyzes, and generates notification information for data.

[0482] "Notification information" refers to messages and audio data generated for users based on action plans.

[0483] "Text generation artificial intelligence" refers to an algorithm that automatically generates text data.

[0484] "Speech-generating artificial intelligence" refers to an algorithm that converts text data into speech.

[0485] "User" refers to an individual who implements an action plan and receives notifications via a mobile device.

[0486] "Response information" refers to data obtained when a user responds to a notification using their mobile device.

[0487] "Analysis results" refer to the results of the analysis derived by the information processing device based on the user's response information.

[0488] The system of the present invention consists of the cooperation of three parties: an information processing device, a mobile information terminal, and a user. Specifically, the user sets their own action plan using an application installed on the mobile information terminal. This action plan includes specific activities such as homework and bedtime. The set action plan is transmitted from the mobile information terminal to the information processing device.

[0489] The information processing device stores and analyzes received action plans in a database. This analysis includes a process of optimization using a generative AI model based on past behavioral patterns and current action plans. Furthermore, the information processing device utilizes text-generating artificial intelligence to generate user-friendly notification information. This notification information is then converted into speech by speech-generating artificial intelligence, providing users with visual and auditory instructions.

[0490] When a set time has elapsed, the information processing device sends a notification to the mobile device. The mobile device displays the notification on its screen and prompts the user to take action via voice. For example, a specific notification message such as "It's 8 PM. Have you finished your homework?" is sent.

[0491] The user checks the notification and responds using their mobile device. The user's response is sent to the information processing device and recorded in the database. The information processing device analyzes the response information and adjusts the next notification time based on the results.

[0492] As a concrete example, the following prompt can be used to input into the generation AI model: "Generate a notification message to check if the user has finished their homework by 8 PM." In this way, the system manages the user's daily routine and prompts them to take action at the appropriate time.

[0493] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0494] Step 1:

[0495] Users set their activity plans using an application on their mobile device. Specifically, they input their schedules, such as homework and bedtime, on the screen. The entered data is converted into an internal data format and temporarily stored on the device. This prepares the user's specific schedule as digital data.

[0496] Step 2:

[0497] The terminal transmits the user-configured action plan data to the information processing device. For transmission, the terminal uses a network connection and securely encrypts the data, packaging it into packets. During this process, the action plan data is properly delivered to the server and prepared for reception.

[0498] Step 3:

[0499] The server analyzes the received action plan data and stores it in the database. To maintain data integrity, the server performs transaction processing and optimizes the storage process. Based on the input data, it accumulates user history and uses it for later analysis.

[0500] Step 4:

[0501] The server uses a generated AI model to determine the timing of the next notification, based on the action plan recorded in the database. The server inputs prompts into the AI ​​model, referencing past data, to calculate the optimal notification time. This process creates a plan for the next notification.

[0502] Step 5:

[0503] The server uses text generation artificial intelligence to generate notification information for the user. Specifically, it generates a message prompting the user, such as, "Have you finished your homework yet?" This information is formatted as text data and prepared for subsequent processing.

[0504] Step 6:

[0505] The server then uses speech-generating artificial intelligence to convert the generated text messages into audio data. The server selects the optimal audio format and outputs an audio file suitable for playback on the device.

[0506] Step 7:

[0507] The server sends generated notification information (text and audio) to the terminal based on the determined notification time. The server manages the sending timing and executes the process according to the user's schedule.

[0508] Step 8:

[0509] The terminal displays notification information received from the server on its screen and outputs it as audio. The terminal plays the audio through its speaker, while the screen displays response options that the user can select.

[0510] Step 9:

[0511] The user responds by tapping the options displayed on the device. The selected option is recorded on the device as a digital signal. This allows for the collection of user feedback.

[0512] Step 10:

[0513] The terminal sends the user's response to the server. The terminal forms an encrypted data packet and promptly transmits it to the server. This input data is then passed to the server for the next feedback cycle.

[0514] Step 11:

[0515] The server analyzes the received response data and adjusts the timing of the next notification. Based on the response data, it performs analysis using a generated AI model and calculates adjustment results that match the operating pattern. This enables continuous notification optimization.

[0516] (Application Example 1)

[0517] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0518] In modern consumer behavior, controlling purchasing behavior and managing budgets remains a challenge, especially for younger generations. There is a need for effective methods to help young people develop healthy spending habits while minimizing parental intervention. There is a growing need for systems that can provide timely and appropriate advice, rather than simply issuing notifications or suggesting reductions.

[0519] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0520] In this invention, the server includes means for setting information about consumption activities based on user input in a mobile information terminal, means for storing and analyzing the information about consumption activities in the server device, and means for predicting consumption behavior using an AI model generated by the server device and generating presentation information based on that prediction. This makes it possible to provide optimal advice and warnings in real time according to the user's consumption behavior.

[0521] A "portable information terminal" refers to an electronic communication device owned and carried by an individual, such as a smartphone or tablet.

[0522] "User input" refers to the act of a user providing information to a system via a mobile device, including touch operations and voice input.

[0523] "Information related to consumer activity" refers to data related to users' daily purchasing behavior and budgets.

[0524] A "server device" is a central management device that processes and stores data on a network and coordinates with other devices.

[0525] A "generative AI model" refers to a set of algorithms built to perform specific tasks using machine learning and artificial intelligence technologies.

[0526] "Notification data" refers to messages or alerts sent to a mobile device to inform the user of specific information.

[0527] "Presented information" refers to advice and warnings provided to users in relation to their consumer activities.

[0528] "Real-time optimal advice" refers to the most appropriate guidance and instructions that are generated and provided instantly based on the user's actions and circumstances.

[0529] This invention utilizes a mobile information terminal, a server device, and a generative AI model to construct a management system related to consumer activity. The mobile information terminal provides a means for the user to input information about their consumer activity. This input information is transmitted to the server device in real time.

[0530] The server centrally stores received consumer activity information and uses a generated AI model for analysis. The AI ​​model analyzes the user's spending trends and predicted purchasing behavior patterns, and generates the most appropriate information regarding future consumer behavior. This allows the server to generate user-friendly notification data regarding budget management and spending pace. For example, it can send messages that include specific suggestions such as, "You're likely to exceed your budget this month. If your next purchase isn't urgent, let's wait until next month."

[0531] In this way, users can more effectively manage their own consumption habits, enabling the establishment of reliable consumption habits without excessive parental intervention. An example of a specific prompt message would be, "Predict the timing of the next purchase based on the child's purchase history and generate friendly notifications. Provide messages that support purchasing behavior."

[0532] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0533] Step 1:

[0534] The user operates a mobile device and inputs information about their purchasing activities. Here, the user enters the products they plan to purchase and their budget into the device, and the entered data is sent directly to the server.

[0535] Step 2:

[0536] The server temporarily records consumer activity information received from mobile devices and supplies it as input data to a generating AI model. This AI model analyzes the user's purchasing behavior based on past consumer data and performs calculations to predict future consumer behavior.

[0537] Step 3:

[0538] The generative AI model generates notification data, including an optimal spending plan, based on predicted purchasing behavior. Here, the AI ​​uses the analysis results to create prompt sentences, which in turn generate user-friendly notification messages. For example, a message like, "You're likely to exceed your budget. Let's reconsider your next purchase," might be generated.

[0539] Step 4:

[0540] The server sends the generated notification data to the mobile device. At this time, the server converts the text obtained from the generating AI model into a format that is easy for the user to receive, such as converting it into an audio format, before sending it.

[0541] Step 5:

[0542] The mobile device displays notifications received from the server to the user and informs the user of their spending status via voice. The user can also respond to notifications using options on the device.

[0543] Step 6:

[0544] The user's response data is sent back to the server, which stores this data and uses it as information for subsequent analysis and notification data generation. Based on user feedback, the timing of the next notification may also be adjusted.

[0545] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0546] This invention provides a system that combines a mobile information terminal and a server device to support the user's daily routine while responding to the user's emotional state using an emotion engine. In this embodiment, the mobile information terminal is equipped with an interface for receiving schedule information and emotional state from the user as input.

[0547] Users set their daily schedules on their mobile devices, which helps manage their planned daily rhythms. During schedule setting, the device analyzes the user's emotional state using an emotion engine based on their voice and text. This emotional information is transmitted from the mobile device to a server device, influencing subsequent processes.

[0548] The server device stores the user's daily schedule and emotional data. Analysis functions within the server customize notification messages generated based on the user's emotional state. Text generation artificial intelligence creates messages with a tone and content favorable to the user, based on feedback from the emotion engine. Furthermore, speech generation artificial intelligence converts these text messages into emotionally appropriate speech.

[0549] At regular intervals, the server sends an emotionally sensitive notification message to the mobile device. The device then communicates the notification to the user via voice and screen display, prompting them to take action. For example, if the user is feeling down, an encouraging message may be sent.

[0550] The user checks the notification and selects a response on their mobile device. The response is sent to the server via the device and recorded for analysis. The server considers not only the response data but also sentiment data to adjust the timing and content of the next notification.

[0551] As a result, this invention not only provides support for time management, but also enables more effective lifestyle support by offering communication that is sensitive to the user's emotions.

[0552] The following describes the processing flow.

[0553] Step 1:

[0554] Users set their daily schedules using their mobile devices. When entering schedules, the device collects user input and voice data, and an emotion engine analyzes their emotional state at that time.

[0555] Step 2:

[0556] The device sends the user's configured schedule data and sentiment data to the server. The server then stores this information in a database.

[0557] Step 3:

[0558] The server uses the received schedule and sentiment data to prepare for generating notification messages. The server utilizes text generation artificial intelligence to create customized text messages that respond to the user's emotions. For example, if the user is relaxed, it will generate an encouraging message.

[0559] Step 4:

[0560] The server uses speech-generating artificial intelligence to convert text messages into audio data. The voice tone is also appropriately adjusted based on emotional data to generate a more natural and friendly voice.

[0561] Step 5:

[0562] As the designated time approaches, the server generates notification data and sends it to the device. The device receives the notification, displays it on the screen, and simultaneously outputs an audio signal.

[0563] Step 6:

[0564] The user checks the notification and chooses a response from the options on their device. Options such as "Yes" and "No" are provided, and the user taps accordingly.

[0565] Step 7:

[0566] The device sends the user's selected response data back to the server. This response data is recorded and analyzed on the server along with sentiment data.

[0567] Step 8:

[0568] The server analyzes recorded response and sentiment data, using it as a feedback loop to improve the content and timing of future notification messages. It provides more effective notifications based on the user's past responses.

[0569] (Example 2)

[0570] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0571] In users' daily lives, it is necessary to provide appropriate support and advice that takes into account not only time management but also their emotional state. Traditional systems failed to consider emotions, making it difficult to generate notifications and messages at the appropriate time in response to user behavior and emotions. There is a need to provide a system that can provide notifications that are sensitive to the user's emotions.

[0572] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0573] In this invention, the server includes means for receiving and analyzing user emotion data from a mobile information terminal, means for generating notification data corresponding to the user's emotions using a generative AI model, and means for analyzing the user's response data and adjusting the timing and content of the next notification. This makes it possible to provide information with appropriate notification content and timing while considering the user's emotional state in real time.

[0574] A "personal information terminal" refers to a multi-functional computer device that is portable and used daily by users, and includes smartphones and tablets.

[0575] "User input" refers to information provided by users through their mobile devices, including audio and text indicating action plans and emotional states.

[0576] "Emotional state" refers to the user's psychological and emotional state, which is estimated through voice and text analysis.

[0577] A "server device" is a central computer system responsible for storing, analyzing, and generating data, and it also plays a role in communicating with client mobile information terminals.

[0578] A "generative AI model" is an algorithm based on artificial intelligence that generates notification data based on the user's emotions.

[0579] "Notification data" refers to messages and information generated by the server device and provided to the user, which are customized according to the user's emotional state.

[0580] "Text-generating artificial intelligence" is artificial intelligence designed to generate text data and output language expressions that match the user's emotions.

[0581] "Speech-generating artificial intelligence" is artificial intelligence that converts text data into speech format, and is used to generate voice notifications that reflect emotions.

[0582] "Response data" refers to the digital data representing the user's chosen response to a notification, and is used to improve future notifications.

[0583] This invention is a system that combines a mobile information terminal and a server device to support the user's daily routine and consider the user's emotional state using an emotion engine. In addition to managing the user's daily routine, this system enhances support for the user by providing communication that responds to their emotions.

[0584] Users input their daily activity plans via their mobile devices. This includes entering appointments and emotional states in voice and text format. The mobile device uses speech recognition software (e.g., a speech recognition API) to convert speech to text, which is then analyzed by an emotion engine.

[0585] The sentiment-analyzed data is transmitted from the mobile device to the server. Data security is guaranteed by using an encrypted communication protocol. The server stores the received data and generates a notification message corresponding to the emotion through its analysis function. A generative AI model is used for generation, employing both text-generating and speech-generating artificial intelligence. Specifically, the generative AI model receives a prompt and creates a message with an appropriate tone for the user. An example of a prompt might be, "The user appears to be stressed. Please generate a message to help them relax."

[0586] At regular intervals, the server sends the generated message to the mobile device. The device presents this notification to the user in both audio and text, prompting action. For example, when the user is heading to a meeting, it can generate a message such as, "Enjoy your next meeting! Relax and be prepared."

[0587] Users provide feedback to the system by reviewing and responding to notifications. Response data is sent to the server and used to improve future notifications. Through this process, the present invention provides effective lifestyle support that is sensitive to the user's emotions.

[0588] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0589] Step 1:

[0590] The user enters their action plan into a mobile device. This input includes both audio and text regarding their schedule and emotional state. Speech recognition software converts the audio to text, and an emotion analysis engine estimates the emotional state from the text, outputting the results. This output consists of schedule data and emotion data.

[0591] Step 2:

[0592] The terminal sends sentiment-analyzed data to the server device. An encryption protocol is used for data transmission, ensuring secure transfer to the server. Inputs are schedule data and sentiment data from the terminal, while outputs are these data stored on the server.

[0593] Step 3:

[0594] The server stores the received data and generates notification messages using a generative AI model based on sentiment analysis. The server analyzes the input sentiment data and provides prompts to the generative AI model, which then generates a text message that aligns with the user's emotional state. An example of such a prompt is, "The user appears to be stressed. Please generate a message to help them relax."

[0595] Step 4:

[0596] The server performs the process of converting the generated text message into speech using speech-generating artificial intelligence. The input is the generated text message, and the output is speech data that corresponds to the user's emotions.

[0597] Step 5:

[0598] The server sends the generated audio and text messages to the mobile device. The input is audio and text data, and the output is a user notification on the device. This notification is presented to the user through screen display and audio playback. A specific example might be, "Enjoy your next meeting! Try to relax."

[0599] Step 6:

[0600] The user checks the notification and enters a response into their mobile device. The input is the content of the response to the notification, and the output is the data that is sent to the server.

[0601] Step 7:

[0602] The server analyzes user response data and uses it to adjust the timing and content of future notifications. Inputs are response data and sentiment data, while output is the adjusted notification schedule and content. This enables the continuous delivery of information that is sensitive to the user's emotions.

[0603] (Application Example 2)

[0604] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0605] In modern society, users lead busy lives and require time management and appropriate information tailored to their emotions. However, conventional systems have struggled to customize information based on users' emotional states, making it difficult to provide timely payment information and campaigns that meet individual user needs. Therefore, new solutions are needed to improve user satisfaction.

[0606] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0607] In this invention, the server includes means for performing emotion analysis and customizing notification information generated based on the user's emotional state; means for generating emotion-appropriate notification information using language-generating artificial intelligence and speech-generating artificial intelligence and providing information related to payment transactions; and means for analyzing the user's response information and adjusting the timing of the next notification and the content of the information according to the emotion based on the results. This enables more personalized information provision and time management for the user.

[0608] A "portable information device" is an electronic device that a user can carry around and use for inputting and outputting information.

[0609] A "time management sheet" is data containing information for managing a user's schedule.

[0610] A "server device" is a computing device used to store, analyze, and distribute information over a network.

[0611] "Notification information" refers to messages and alerts sent to users.

[0612] "Emotional analysis" is the process of analyzing a user's emotional state.

[0613] "Language-generating artificial intelligence" refers to artificial intelligence technology used to generate natural language.

[0614] "Speech-generating artificial intelligence" is an artificial intelligence technology that converts text into speech.

[0615] "User response information" refers to data that shows how users respond to notifications and messages.

[0616] A "settlement transaction" is a procedure necessary for making a purchase or payment.

[0617] "Means of adjusting information content" refers to methods for changing the content of information according to the user's circumstances and needs.

[0618] "Customizing" means optimizing the system to suit the individual needs and circumstances of each user.

[0619] To implement this invention, it is first necessary to input the user's time management schedule and record their emotional state using a portable information device. The portable information device has a schedule management application that allows the user to easily input and manage their daily schedule. This application receives the user's voice and text input and performs emotional analysis. At this time, an emotional engine and related analysis tools are used to understand the user's current emotional state.

[0620] Next, time management charts and emotional data are transmitted from the mobile device to the server. The server stores and analyzes this data. The server uses language-generating artificial intelligence (e.g., OpenAI GPT-3) or speech-generating artificial intelligence (e.g., AWS Polly) to generate notification information tailored to the user's emotional state. This notification information may include, for example, special payment offers or campaign announcements that match the user's preferences and emotions.

[0621] The generated notification information is sent to the user's mobile device and displayed to them. The notifications are provided in text or audio format and are tailored to the user's emotional state. For example, if a user is relaxing after a busy day, the emotion engine detects the emotion of "peace" and notifies them of a discount on an online salon.

[0622] This sequence of events allows the system to provide information tailored to each individual user, making their daily life more comfortable. An example of a prompt message is: "You are relaxing at the end of the day. How can you enjoy this time by receiving a special offer tailored to your current mood?"

[0623] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0624] Step 1:

[0625] The device accepts input from the user, including a time management sheet and emotional state. The user uses an application to input their daily schedule and provides this information via voice or text. This input data is processed by an emotion analysis engine to extract the user's emotional state.

[0626] Step 2:

[0627] The terminal sends the entered time management sheet and analyzed emotional state to the server. The server receives this data and stores it in a database. The stored data is used in subsequent processing and forms the basis for understanding the user's schedule and emotional tendencies.

[0628] Step 3:

[0629] The server uses language-generating artificial intelligence to generate optimal notification information for the user based on their stored emotional state. Here, the emotional state is used as input to generate a prompt, and based on that, a message with an appropriate tone and content is generated. This process ensures that messages are tailored to the user's emotions.

[0630] Step 4:

[0631] The generated text messages are converted into audio data using speech-generating artificial intelligence. The text data is input into the speech engine, which then produces emotionally responsive audio output. This prepares the system to deliver notifications to users that are both auditory and emotionally sensitive.

[0632] Step 5:

[0633] The server sends generated audio and text notification information to the device. The device notifies the user of this information visually and audibly. The user receives the notification from the device and chooses an action based on it.

[0634] Step 6:

[0635] The user's response to a notification is sent back to the server via the terminal. The server analyzes this response information as input and readjusts the content and timing of the next notification. This ensures that the user receives the most optimal information in stages.

[0636] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0637] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0638] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0639] [Fourth Embodiment]

[0640] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0641] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0642] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0643] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0644] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0645] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0646] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0647] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0648] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0649] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0650] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0651] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0652] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0653] The system according to the present invention supports a child's daily routine via a mobile information terminal and is realized through the cooperation of three parties: a server device, a mobile information terminal, and a user. A specific embodiment of this system is described below.

[0654] First, the user sets their daily schedule using an application installed on their mobile device. The user can set routines for activities such as homework, bathing, and bedtime, and customize them as needed. The set schedule is then sent from the mobile device to a server device and stored in a database.

[0655] The server device manages notification timing for each user based on the received schedule information. Text-generating artificial intelligence creates user-friendly notification messages based on the user's schedule. Furthermore, speech-generating artificial intelligence converts the text into speech format, making it easier to recognize by ear.

[0656] As the set time approaches, the server device sends the generated notification data to the mobile device. The mobile device displays the received notification on its screen and also prompts the user to confirm their actions via voice. For example, a message such as "It's 8 PM. Have you finished your homework?" might be sent.

[0657] After the user confirms the notification, they respond by tapping an option on their mobile device. This selection data is then sent back to the server device, where it is recorded and analyzed. For example, if the user answers "No," the server device will take appropriate action, such as reviewing the timing of the next notification, to help the user manage their time.

[0658] Therefore, the system based on the present invention enables users to habitualize their behavior, improve their time management skills, and consequently reduce parental intervention.

[0659] The following describes the processing flow.

[0660] Step 1:

[0661] The user operates their mobile device to launch the application and open the daily schedule settings screen. The user enters their time schedule, including homework, bath time, and bedtime, and saves the settings.

[0662] Step 2:

[0663] The terminal sends the schedule data entered by the user to the server. The transmitted data is organized by user and stored in a database.

[0664] Step 3:

[0665] The server calculates the time to send a notification based on the received schedule and determines the content of the notification to be sent at that time.

[0666] Step 4:

[0667] The server generates notification messages using text generation artificial intelligence. For example, a message like "It's 8 PM. Have you finished your homework?" might be created.

[0668] Step 5:

[0669] The server uses speech-generating artificial intelligence to convert the generated text messages into audio data. This is to make the audio data more naturally recognizable to the user.

[0670] Step 6:

[0671] As the specified time approaches, the server sends the generated notification data to the device. The device then processes the received data immediately.

[0672] Step 7:

[0673] The device displays a notification on the screen and plays it aloud for the user. This allows the user to confirm the notification content both visually and aurally.

[0674] Step 8:

[0675] The user selects a response to the notification. Options such as "Yes" or "No" are presented, and the user taps the appropriate option.

[0676] Step 9:

[0677] The terminal sends the user's response back to the server. This response data is used for subsequent processing.

[0678] Step 10:

[0679] The server analyzes the response data it receives to understand user behavior patterns. Based on the analysis results, it plans to adjust the timing and content of future notifications.

[0680] (Example 1)

[0681] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0682] In modern society, carefully managing children's daily routines and promoting habit formation is crucial for busy parents. However, a lack of timely notifications and feedback hinders the development of children's time management skills. To address this issue, it is necessary to support children's behavior through appropriately tuned notification systems and response analysis.

[0683] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0684] In this invention, the server includes means for setting an action plan based on information input in a mobile information terminal, means for storing and analyzing the action plan in an information processing device, and means for transmitting notification information generated by the information processing device to the mobile information terminal and notifying the user. This makes it possible to manage the user's daily rhythm and promote actions at appropriate times.

[0685] A "portable information terminal" refers to a device that a user can carry around and use for inputting and displaying information.

[0686] "Information input" refers to the act of a user entering data via a mobile device.

[0687] An "action plan" refers to a schedule or plan for managing and implementing the user's daily routine.

[0688] An "information processing device" refers to a device that receives, stores, analyzes, and generates notification information for data.

[0689] "Notification information" refers to messages and audio data generated for users based on action plans.

[0690] "Text generation artificial intelligence" refers to an algorithm that automatically generates text data.

[0691] "Speech-generating artificial intelligence" refers to an algorithm that converts text data into speech.

[0692] "User" refers to an individual who implements an action plan and receives notifications via a mobile device.

[0693] "Response information" refers to data obtained when a user responds to a notification using their mobile device.

[0694] "Analysis results" refer to the results of the analysis derived by the information processing device based on the user's response information.

[0695] The system of the present invention consists of the cooperation of three parties: an information processing device, a mobile information terminal, and a user. Specifically, the user sets their own action plan using an application installed on the mobile information terminal. This action plan includes specific activities such as homework and bedtime. The set action plan is transmitted from the mobile information terminal to the information processing device.

[0696] The information processing device stores and analyzes received action plans in a database. This analysis includes a process of optimization using a generative AI model based on past behavioral patterns and current action plans. Furthermore, the information processing device utilizes text-generating artificial intelligence to generate user-friendly notification information. This notification information is then converted into speech by speech-generating artificial intelligence, providing users with visual and auditory instructions.

[0697] When a set time has elapsed, the information processing device sends a notification to the mobile device. The mobile device displays the notification on its screen and prompts the user to take action via voice. For example, a specific notification message such as "It's 8 PM. Have you finished your homework?" is sent.

[0698] The user checks the notification and responds using their mobile device. The user's response is sent to the information processing device and recorded in the database. The information processing device analyzes the response information and adjusts the next notification time based on the results.

[0699] As a concrete example, the following prompt can be used to input into the generation AI model: "Generate a notification message to check if the user has finished their homework by 8 PM." In this way, the system manages the user's daily routine and prompts them to take action at the appropriate time.

[0700] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0701] Step 1:

[0702] Users set their activity plans using an application on their mobile device. Specifically, they input their schedules, such as homework and bedtime, on the screen. The entered data is converted into an internal data format and temporarily stored on the device. This prepares the user's specific schedule as digital data.

[0703] Step 2:

[0704] The terminal transmits the user-configured action plan data to the information processing device. For transmission, the terminal uses a network connection and securely encrypts the data, packaging it into packets. During this process, the action plan data is properly delivered to the server and prepared for reception.

[0705] Step 3:

[0706] The server analyzes the received action plan data and stores it in the database. To maintain data integrity, the server performs transaction processing and optimizes the storage process. Based on the input data, it accumulates user history and uses it for later analysis.

[0707] Step 4:

[0708] The server uses a generated AI model to determine the timing of the next notification, based on the action plan recorded in the database. The server inputs prompts into the AI ​​model, referencing past data, to calculate the optimal notification time. This process creates a plan for the next notification.

[0709] Step 5:

[0710] The server uses text generation artificial intelligence to generate notification information for the user. Specifically, it generates a message prompting the user, such as, "Have you finished your homework yet?" This information is formatted as text data and prepared for subsequent processing.

[0711] Step 6:

[0712] The server then uses speech-generating artificial intelligence to convert the generated text messages into audio data. The server selects the optimal audio format and outputs an audio file suitable for playback on the device.

[0713] Step 7:

[0714] The server sends generated notification information (text and audio) to the terminal based on the determined notification time. The server manages the sending timing and executes the process according to the user's schedule.

[0715] Step 8:

[0716] The terminal displays notification information received from the server on its screen and outputs it as audio. The terminal plays the audio through its speaker, while the screen displays response options that the user can select.

[0717] Step 9:

[0718] The user responds by tapping the options displayed on the device. The selected option is recorded on the device as a digital signal. This allows for the collection of user feedback.

[0719] Step 10:

[0720] The terminal sends the user's response to the server. The terminal forms an encrypted data packet and promptly transmits it to the server. This input data is then passed to the server for the next feedback cycle.

[0721] Step 11:

[0722] The server analyzes the received response data and adjusts the timing of the next notification. Based on the response data, it performs analysis using a generated AI model and calculates adjustment results that match the operating pattern. This enables continuous notification optimization.

[0723] (Application Example 1)

[0724] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0725] In modern consumer behavior, controlling purchasing behavior and managing budgets remains a challenge, especially for younger generations. There is a need for effective methods to help young people develop healthy spending habits while minimizing parental intervention. There is a growing need for systems that can provide timely and appropriate advice, rather than simply issuing notifications or suggesting reductions.

[0726] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0727] In this invention, the server includes means for setting information about consumption activities based on user input in a mobile information terminal, means for storing and analyzing the information about consumption activities in the server device, and means for predicting consumption behavior using an AI model generated by the server device and generating presentation information based on that prediction. This makes it possible to provide optimal advice and warnings in real time according to the user's consumption behavior.

[0728] A "portable information terminal" refers to an electronic communication device owned and carried by an individual, such as a smartphone or tablet.

[0729] "User input" refers to the act of a user providing information to a system via a mobile device, including touch operations and voice input.

[0730] "Information related to consumer activity" refers to data related to users' daily purchasing behavior and budgets.

[0731] A "server device" is a central management device that processes and stores data on a network and coordinates with other devices.

[0732] A "generative AI model" refers to a set of algorithms built to perform specific tasks using machine learning and artificial intelligence technologies.

[0733] "Notification data" refers to messages or alerts sent to a mobile device to inform the user of specific information.

[0734] "Presented information" refers to advice and warnings provided to users in relation to their consumer activities.

[0735] "Real-time optimal advice" refers to the most appropriate guidance and instructions that are generated and provided instantly based on the user's actions and circumstances.

[0736] This invention utilizes a mobile information terminal, a server device, and a generative AI model to construct a management system related to consumer activity. The mobile information terminal provides a means for the user to input information about their consumer activity. This input information is transmitted to the server device in real time.

[0737] The server centrally stores received consumer activity information and uses a generated AI model for analysis. The AI ​​model analyzes the user's spending trends and predicted purchasing behavior patterns, and generates the most appropriate information regarding future consumer behavior. This allows the server to generate user-friendly notification data regarding budget management and spending pace. For example, it can send messages that include specific suggestions such as, "You're likely to exceed your budget this month. If your next purchase isn't urgent, let's wait until next month."

[0738] In this way, users can more effectively manage their own consumption habits, enabling the establishment of reliable consumption habits without excessive parental intervention. An example of a specific prompt message would be, "Predict the timing of the next purchase based on the child's purchase history and generate friendly notifications. Provide messages that support purchasing behavior."

[0739] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0740] Step 1:

[0741] The user operates a mobile device and inputs information about their purchasing activities. Here, the user enters the products they plan to purchase and their budget into the device, and the entered data is sent directly to the server.

[0742] Step 2:

[0743] The server temporarily records consumer activity information received from mobile devices and supplies it as input data to a generating AI model. This AI model analyzes the user's purchasing behavior based on past consumer data and performs calculations to predict future consumer behavior.

[0744] Step 3:

[0745] The generative AI model generates notification data, including an optimal spending plan, based on predicted purchasing behavior. Here, the AI ​​uses the analysis results to create prompt sentences, which in turn generate user-friendly notification messages. For example, a message like, "You're likely to exceed your budget. Let's reconsider your next purchase," might be generated.

[0746] Step 4:

[0747] The server sends the generated notification data to the mobile device. At this time, the server converts the text obtained from the generating AI model into a format that is easy for the user to receive, such as converting it into an audio format, before sending it.

[0748] Step 5:

[0749] The mobile device displays notifications received from the server to the user and informs the user of their spending status via voice. The user can also respond to notifications using options on the device.

[0750] Step 6:

[0751] The user's response data is sent back to the server, which stores this data and uses it as information for subsequent analysis and notification data generation. Based on user feedback, the timing of the next notification may also be adjusted.

[0752] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0753] This invention provides a system that combines a mobile information terminal and a server device to support the user's daily routine while responding to the user's emotional state using an emotion engine. In this embodiment, the mobile information terminal is equipped with an interface for receiving schedule information and emotional state from the user as input.

[0754] Users set their daily schedules on their mobile devices, which helps manage their planned daily rhythms. During schedule setting, the device analyzes the user's emotional state using an emotion engine based on their voice and text. This emotional information is transmitted from the mobile device to a server device, influencing subsequent processes.

[0755] The server device stores the user's daily schedule and emotional data. Analysis functions within the server customize notification messages generated based on the user's emotional state. Text generation artificial intelligence creates messages with a tone and content favorable to the user, based on feedback from the emotion engine. Furthermore, speech generation artificial intelligence converts these text messages into emotionally appropriate speech.

[0756] At regular intervals, the server sends an emotionally sensitive notification message to the mobile device. The device then communicates the notification to the user via voice and screen display, prompting them to take action. For example, if the user is feeling down, an encouraging message may be sent.

[0757] The user checks the notification and selects a response on their mobile device. The response is sent to the server via the device and recorded for analysis. The server considers not only the response data but also sentiment data to adjust the timing and content of the next notification.

[0758] As a result, this invention not only provides support for time management, but also enables more effective lifestyle support by offering communication that is sensitive to the user's emotions.

[0759] The following describes the processing flow.

[0760] Step 1:

[0761] Users set their daily schedules using their mobile devices. When entering schedules, the device collects user input and voice data, and an emotion engine analyzes their emotional state at that time.

[0762] Step 2:

[0763] The device sends the user's configured schedule data and sentiment data to the server. The server then stores this information in a database.

[0764] Step 3:

[0765] The server uses the received schedule and sentiment data to prepare for generating notification messages. The server utilizes text generation artificial intelligence to create customized text messages that respond to the user's emotions. For example, if the user is relaxed, it will generate an encouraging message.

[0766] Step 4:

[0767] The server uses speech-generating artificial intelligence to convert text messages into audio data. The voice tone is also appropriately adjusted based on emotional data to generate a more natural and friendly voice.

[0768] Step 5:

[0769] As the designated time approaches, the server generates notification data and sends it to the device. The device receives the notification, displays it on the screen, and simultaneously outputs an audio signal.

[0770] Step 6:

[0771] The user checks the notification and chooses a response from the options on their device. Options such as "Yes" and "No" are provided, and the user taps accordingly.

[0772] Step 7:

[0773] The device sends the user's selected response data back to the server. This response data is recorded and analyzed on the server along with sentiment data.

[0774] Step 8:

[0775] The server analyzes recorded response and sentiment data, using it as a feedback loop to improve the content and timing of future notification messages. It provides more effective notifications based on the user's past responses.

[0776] (Example 2)

[0777] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0778] In users' daily lives, it is necessary to provide appropriate support and advice that takes into account not only time management but also their emotional state. Traditional systems failed to consider emotions, making it difficult to generate notifications and messages at the appropriate time in response to user behavior and emotions. There is a need to provide a system that can provide notifications that are sensitive to the user's emotions.

[0779] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0780] In this invention, the server includes means for receiving and analyzing user emotion data from a mobile information terminal, means for generating notification data corresponding to the user's emotions using a generative AI model, and means for analyzing the user's response data and adjusting the timing and content of the next notification. This makes it possible to provide information with appropriate notification content and timing while considering the user's emotional state in real time.

[0781] A "personal information terminal" refers to a multi-functional computer device that is portable and used daily by users, and includes smartphones and tablets.

[0782] "User input" refers to information provided by users through their mobile devices, including audio and text indicating action plans and emotional states.

[0783] "Emotional state" refers to the user's psychological and emotional state, which is estimated through voice and text analysis.

[0784] A "server device" is a central computer system responsible for storing, analyzing, and generating data, and it also plays a role in communicating with client mobile information terminals.

[0785] A "generative AI model" is an algorithm based on artificial intelligence that generates notification data based on the user's emotions.

[0786] "Notification data" refers to messages and information generated by the server device and provided to the user, which are customized according to the user's emotional state.

[0787] "Text-generating artificial intelligence" is artificial intelligence designed to generate text data and output language expressions that match the user's emotions.

[0788] "Speech-generating artificial intelligence" is artificial intelligence that converts text data into speech format, and is used to generate voice notifications that reflect emotions.

[0789] "Response data" refers to the digital data representing the user's chosen response to a notification, and is used to improve future notifications.

[0790] This invention is a system that combines a mobile information terminal and a server device to support the user's daily routine and consider the user's emotional state using an emotion engine. In addition to managing the user's daily routine, this system enhances support for the user by providing communication that responds to their emotions.

[0791] Users input their daily activity plans via their mobile devices. This includes entering appointments and emotional states in voice and text format. The mobile device uses speech recognition software (e.g., a speech recognition API) to convert speech to text, which is then analyzed by an emotion engine.

[0792] The sentiment-analyzed data is transmitted from the mobile device to the server. Data security is guaranteed by using an encrypted communication protocol. The server stores the received data and generates a notification message corresponding to the emotion through its analysis function. A generative AI model is used for generation, employing both text-generating and speech-generating artificial intelligence. Specifically, the generative AI model receives a prompt and creates a message with an appropriate tone for the user. An example of a prompt might be, "The user appears to be stressed. Please generate a message to help them relax."

[0793] At regular intervals, the server sends the generated message to the mobile device. The device presents this notification to the user in both audio and text, prompting action. For example, when the user is heading to a meeting, it can generate a message such as, "Enjoy your next meeting! Relax and be prepared."

[0794] Users provide feedback to the system by reviewing and responding to notifications. Response data is sent to the server and used to improve future notifications. Through this process, the present invention provides effective lifestyle support that is sensitive to the user's emotions.

[0795] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0796] Step 1:

[0797] The user enters their action plan into a mobile device. This input includes both audio and text regarding their schedule and emotional state. Speech recognition software converts the audio to text, and an emotion analysis engine estimates the emotional state from the text, outputting the results. This output consists of schedule data and emotion data.

[0798] Step 2:

[0799] The terminal sends sentiment-analyzed data to the server device. An encryption protocol is used for data transmission, ensuring secure transfer to the server. Inputs are schedule data and sentiment data from the terminal, while outputs are these data stored on the server.

[0800] Step 3:

[0801] The server stores the received data and generates notification messages using a generative AI model based on sentiment analysis. The server analyzes the input sentiment data and provides prompts to the generative AI model, which then generates a text message that aligns with the user's emotional state. An example of such a prompt is, "The user appears to be stressed. Please generate a message to help them relax."

[0802] Step 4:

[0803] The server performs the process of converting the generated text message into speech using speech-generating artificial intelligence. The input is the generated text message, and the output is speech data that corresponds to the user's emotions.

[0804] Step 5:

[0805] The server sends the generated audio and text messages to the mobile device. The input is audio and text data, and the output is a user notification on the device. This notification is presented to the user through screen display and audio playback. A specific example might be, "Enjoy your next meeting! Try to relax."

[0806] Step 6:

[0807] The user checks the notification and enters a response into their mobile device. The input is the content of the response to the notification, and the output is the data that is sent to the server.

[0808] Step 7:

[0809] The server analyzes user response data and uses it to adjust the timing and content of future notifications. Inputs are response data and sentiment data, while output is the adjusted notification schedule and content. This enables the continuous delivery of information that is sensitive to the user's emotions.

[0810] (Application Example 2)

[0811] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0812] In modern society, users lead busy lives and require time management and appropriate information tailored to their emotions. However, conventional systems have struggled to customize information based on users' emotional states, making it difficult to provide timely payment information and campaigns that meet individual user needs. Therefore, new solutions are needed to improve user satisfaction.

[0813] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0814] In this invention, the server includes means for performing emotion analysis and customizing notification information generated based on the user's emotional state; means for generating emotion-appropriate notification information using language-generating artificial intelligence and speech-generating artificial intelligence and providing information related to payment transactions; and means for analyzing the user's response information and adjusting the timing of the next notification and the content of the information according to the emotion based on the results. This enables more personalized information provision and time management for the user.

[0815] A "portable information device" is an electronic device that a user can carry around and use for inputting and outputting information.

[0816] A "time management sheet" is data containing information for managing a user's schedule.

[0817] A "server device" is a computing device used to store, analyze, and distribute information over a network.

[0818] "Notification information" refers to messages and alerts sent to users.

[0819] "Emotional analysis" is the process of analyzing a user's emotional state.

[0820] "Language-generating artificial intelligence" refers to artificial intelligence technology used to generate natural language.

[0821] "Speech-generating artificial intelligence" is an artificial intelligence technology that converts text into speech.

[0822] "User response information" refers to data that shows how users respond to notifications and messages.

[0823] A "settlement transaction" is a procedure necessary for making a purchase or payment.

[0824] "Means of adjusting information content" refers to methods for changing the content of information according to the user's circumstances and needs.

[0825] "Customizing" means optimizing the system to suit the individual needs and circumstances of each user.

[0826] To implement this invention, it is first necessary to input the user's time management schedule and record their emotional state using a portable information device. The portable information device has a schedule management application that allows the user to easily input and manage their daily schedule. This application receives the user's voice and text input and performs emotional analysis. At this time, an emotional engine and related analysis tools are used to understand the user's current emotional state.

[0827] Next, time management charts and emotional data are transmitted from the mobile device to the server. The server stores and analyzes this data. The server uses language-generating artificial intelligence (e.g., OpenAI GPT-3) or speech-generating artificial intelligence (e.g., AWS Polly) to generate notification information tailored to the user's emotional state. This notification information may include, for example, special payment offers or campaign announcements that match the user's preferences and emotions.

[0828] The generated notification information is sent to the user's mobile device and displayed to them. The notifications are provided in text or audio format and are tailored to the user's emotional state. For example, if a user is relaxing after a busy day, the emotion engine detects the emotion of "peace" and notifies them of a discount on an online salon.

[0829] This sequence of events allows the system to provide information tailored to each individual user, making their daily life more comfortable. An example of a prompt message is: "You are relaxing at the end of the day. How can you enjoy this time by receiving a special offer tailored to your current mood?"

[0830] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0831] Step 1:

[0832] The device accepts input from the user, including a time management sheet and emotional state. The user uses an application to input their daily schedule and provides this information via voice or text. This input data is processed by an emotion analysis engine to extract the user's emotional state.

[0833] Step 2:

[0834] The terminal sends the entered time management sheet and analyzed emotional state to the server. The server receives this data and stores it in a database. The stored data is used in subsequent processing and forms the basis for understanding the user's schedule and emotional tendencies.

[0835] Step 3:

[0836] The server uses language-generating artificial intelligence to generate optimal notification information for the user based on their stored emotional state. Here, the emotional state is used as input to generate a prompt, and based on that, a message with an appropriate tone and content is generated. This process ensures that messages are tailored to the user's emotions.

[0837] Step 4:

[0838] The generated text messages are converted into audio data using speech-generating artificial intelligence. The text data is input into the speech engine, which then produces emotionally responsive audio output. This prepares the system to deliver notifications to users that are both auditory and emotionally sensitive.

[0839] Step 5:

[0840] The server sends generated audio and text notification information to the device. The device notifies the user of this information visually and audibly. The user receives the notification from the device and chooses an action based on it.

[0841] Step 6:

[0842] The user's response to a notification is sent back to the server via the terminal. The server analyzes this response information as input and readjusts the content and timing of the next notification. This ensures that the user receives the most optimal information in stages.

[0843] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0844] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0845] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0846] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0847] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0848] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0849] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0850] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0851] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0852] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0853] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0854] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0855] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0856] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0857] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0858] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0859] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0860] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0861] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0862] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0863] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted as being incorporated by reference.

[0864] The following is further disclosed regarding the embodiments described above.

[0865] (Claim 1)

[0866] A means of setting a daily schedule based on user input on a mobile information terminal,

[0867] A server device provides means for saving and analyzing the aforementioned lifestyle schedule,

[0868] A means for transmitting notification data generated by the server device to the mobile information terminal and notifying the user,

[0869] Means for transmitting the user's response to the server device via the mobile information terminal,

[0870] A system that includes this.

[0871] (Claim 2)

[0872] The system according to claim 1, wherein a server device generates notification data using text-generating artificial intelligence and speech-generating artificial intelligence.

[0873] (Claim 3)

[0874] The system according to claim 1, wherein the server device analyzes the user's response data and adjusts the timing of the next notification based on the results.

[0875] "Example 1"

[0876] (Claim 1)

[0877] A means for setting an action plan based on information input on a mobile information terminal,

[0878] An information processing device includes means for storing and analyzing the aforementioned action plan,

[0879] A means for transmitting notification information generated by the information processing device to the mobile information terminal and notifying the user,

[0880] Means for transmitting the user's response via the mobile information terminal to the information processing device,

[0881] A system that includes this.

[0882] (Claim 2)

[0883] The system according to claim 1, wherein the information processing device generates notification information using artificial intelligence for character generation and artificial intelligence for speech generation.

[0884] (Claim 3)

[0885] The system according to claim 1, wherein the information processing device analyzes the user's response information and adjusts the next notification time based on the analysis results.

[0886] "Application Example 1"

[0887] (Claim 1)

[0888] A means for setting information about consumer activities based on user input in a mobile information terminal,

[0889] The server device includes means for storing and analyzing information related to the aforementioned consumption activity,

[0890] A means for transmitting notification data generated by the server device to the mobile information terminal and notifying the user,

[0891] Means for transmitting the user's response to the server device via the mobile information terminal,

[0892] The server device includes means for predicting consumer behavior using a generated AI model and generating presentation information based on that prediction,

[0893] A system that includes this.

[0894] (Claim 2)

[0895] The system according to claim 1, wherein a server device uses text-generating artificial intelligence and speech-generating artificial intelligence to generate notification data related to consumer activities.

[0896] (Claim 3)

[0897] The system according to claim 1, wherein the server device analyzes the user's response data and consumption activity history, and adjusts the timing of the next notification and the information to be presented based on the results.

[0898] "Example 2 of combining an emotion engine"

[0899] (Claim 1)

[0900] A means for setting an action plan based on user input on a mobile information terminal,

[0901] A means for analyzing a user's voice and text input to estimate their emotional state,

[0902] The aforementioned mobile information terminal includes means for transmitting the emotion data to the server device,

[0903] The server device includes means for storing and analyzing the aforementioned action plan and emotion data,

[0904] A server device generates notification data corresponding to the user's emotions using an AI model,

[0905] A means for transmitting notification data generated by a server device to the mobile information terminal and providing information to the user,

[0906] Means for transmitting the user's response to the server device via the mobile information terminal,

[0907] A system that includes this.

[0908] (Claim 2)

[0909] The system according to claim 1, wherein a server device uses text-generating artificial intelligence and speech-generating artificial intelligence to generate emotion-based notification data.

[0910] (Claim 3)

[0911] The system according to claim 1, wherein the server device analyzes the user's response data and adjusts the timing and content of the next notification based on the results.

[0912] "Application example 2 of combining emotional engines"

[0913] (Claim 1)

[0914] A means for setting up a time management schedule based on user input in a mobile information device,

[0915] A server device provides means for saving and analyzing the aforementioned time management table,

[0916] A means for transmitting notification information generated by the server device to the mobile information device and presenting the information to the user,

[0917] Means for transmitting user responses to the server device via the mobile information device,

[0918] A means for performing emotion analysis and customizing notification information generated based on the user's emotional state,

[0919] A system that includes this.

[0920] (Claim 2)

[0921] The system according to claim 1, wherein a server device uses language-generating artificial intelligence and speech-generating artificial intelligence to generate emotion-responsive notification information and provide information related to payment transactions.

[0922] (Claim 3)

[0923] The system according to claim 1, wherein the server equipment analyzes user response information and adjusts the timing of the next notification and the content of the information according to the user's emotions based on the results. [Explanation of Symbols]

[0924] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means of setting a daily schedule based on user input on a mobile information terminal, A server device provides means for saving and analyzing the aforementioned lifestyle schedule, A means for transmitting notification data generated by the server device to the mobile information terminal and notifying the user, Means for transmitting the user's response to the server device via the mobile information terminal, A system that includes this.

2. The system according to claim 1, wherein the server device generates notification data using text generation artificial intelligence and speech generation artificial intelligence.

3. The system according to claim 1, wherein the server device analyzes the user's response data and adjusts the timing of the next notification based on the results.

Images