system

Abstract

We provide the system. [Solution] A means for collecting user activity data, A means for analyzing the aforementioned activity data to identify the user's lifestyle patterns, Means for generating individual proposals based on the identified lifestyle patterns, A means for sending the generated proposal to the user's terminal, A means for collecting user feedback on the aforementioned proposal, A means of updating the proposed content based on the aforementioned feedback, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, the method including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a character of the chatbot, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In modern society, individual lifestyles are diverse, and in many cases, general support cannot fully meet the needs. In a system that provides short-circuited proposals or supports only some actions, the user satisfaction and effectiveness are limited. Furthermore, there is a lack of a mechanism to reflect user feedback in existing systems, resulting in a problem of reduced accuracy and usefulness of proposals. Against this background, there is a need to provide a system that can provide individual support according to the user's lifestyle and can be continuously improved by feedback.

Means for Solving the Problems

[0005] This invention provides a means for collecting user activity data and analyzing that data to identify the user's lifestyle patterns. Furthermore, it includes means for generating individualized suggestions based on the identified lifestyle patterns and for transmitting the generated suggestions to the user's terminal. It also includes means for collecting user feedback on the suggestions and updating the suggestion content based on that feedback. This invention enables the provision of support optimized for each user, and the system is continuously improved through feedback, thereby realizing personalized and enjoyable lifestyle support.

[0006] "Activity data" refers to information about a user's behavior and status, and specifically includes data such as steps taken, heart rate, and schedule.

[0007] "Lifestyle patterns" refer to information that shows the repetitive tendencies of a user's actions and habits in their daily life.

[0008] A "suggestion" is advice or recommendations regarding specific actions or options provided to the user.

[0009] "Device" refers to a device used by a user, and includes smartphones, tablets, wearable devices, etc.

[0010] "Feedback" refers to information about users' reactions and evaluations of suggestions.

[0011] "Analysis" is the process of processing collected data to derive patterns and trends.

[0012] A "machine learning algorithm" is a mathematical model used to automatically learn from data and perform analysis and predictions.

[0013] "Means for updating proposals" refers to the process of modifying existing proposals or generating new ones based on user feedback.

[0014] A "database" is an organized collection of data used to efficiently store, manage, and retrieve collected data. [Brief explanation of the drawing]

[0015] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] This is a sequence diagram showing the processing flow of the data processing system in Example 2, which incorporates an emotion engine. [Figure 14]It is a sequence diagram showing the processing flow of a data processing system in Application Example 2 when a sentiment engine is combined.

Embodiments for Carrying Out the Invention

[0016] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0017] First, the terms used in the following description will be explained.

[0018] In the following embodiments, a labeled processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0019] In the following embodiments, a labeled RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0020] In the following embodiments, a labeled storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, etc.

[0021] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0022] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0023] [First Embodiment]

[0024] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0025] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0026] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0027] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0028] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0029] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0030] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0031] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0032] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0033] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0034] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0035] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0036] The embodiments for carrying out the present invention will be described in detail. The system of the present invention collects user activity data, analyzes that data to identify the user's lifestyle pattern, generates individual suggestions based on the identified lifestyle pattern, and transmits those suggestions to the user's terminal. By collecting feedback from the user and updating the suggestions based on that feedback, the system provides appropriate and personalized suggestions.

[0037] System Configuration

[0038] Server side

[0039] Data collection

[0040] The server receives activity data transmitted from the user's device. This activity data includes behavioral data, schedule data, and biometric data.

[0041] Data Analysis

[0042] The server analyzes the received data to identify the user's lifestyle patterns. Statistical models and machine learning algorithms are used for this analysis.

[0043] Model generation

[0044] The server generates a predictive model based on the identified lifestyle patterns. This model is used to provide optimal support suggestions for each user.

[0045] Proposal generation

[0046] The server creates specific suggestions from the generated predictive model. For example, it might generate suggestions such as "do some stretching" or "take a 30-minute walk" based on the user's exercise habits.

[0047] Submit Proposal

[0048] The server sends the generated suggestions to the user's terminal.

[0049] Terminal side

[0050] Data transmission

[0051] The device transmits user behavior data and schedules to the server in real time.

[0052] Proposal received

[0053] The terminal receives the proposal content sent from the server.

[0054] Execution notification

[0055] The device notifies the user of the proposed content. For example, notifications are sent using smartphone push notifications or the alert function of wearable devices.

[0056] Send feedback

[0057] The device sends user feedback to the server. This feedback includes evaluations of suggestions and the results of their implementation.

[0058] Specific example

[0059] Consider a user who wakes up at 6 AM to their smartphone alarm and has breakfast at 7 AM. This user spends most of their day doing desk work and tends to be sedentary.

[0060] Data collection

[0061] The device sends the user's wake-up time and meal times to the server.

[0062] Data Analysis

[0063] The server analyzes the user's behavioral data and identifies that they are not getting enough exercise.

[0064] Model generation

[0065] The server generates a model for introducing exercise habits that are suitable for the user.

[0066] Proposal generation

[0067] The server generates specific suggestions, such as "We suggest a 10-minute stretching session after breakfast."

[0068] Submit Proposal

[0069] The server sends the proposal to the user's terminal.

[0070] Proposal received

[0071] The terminal receives a proposal sent from the server and notifies the user.

[0072] Execution notification

[0073] The device sends a push notification to the user suggesting a "10-minute stretch after breakfast."

[0074] Send feedback

[0075] The user implements the suggested actions and enters feedback on their satisfaction level and effectiveness into the device.

[0076] The device sends user feedback to the server.

[0077] In this way, the system of the present invention can provide optimal support tailored to each user's lifestyle and continuously improve its suggestions.

[0078] The following describes the processing flow.

[0079] Step 1: Data Collection

[0080] Users record their daily activities and schedules using smartphones or wearable devices. For example, they accumulate data by entering steps, heart rate, and appointments into their calendars.

[0081] The device periodically transmits recorded data to a server. This data includes behavioral data, schedule data, and biometric data.

[0082] Step 2: Data Reception

[0083] The server receives activity data sent from the terminal. The received data is temporarily held in a buffer.

[0084] Step 3: Data Storage

[0085] The server stores the received data in a database. During this process, the data is organized by user to facilitate efficient analysis later.

[0086] Step 4: Data Cleansing

[0087] The server cleanses the stored data. This process involves deleting unnecessary data and standardizing the format. For example, it removes outliers and standardizes data formats.

[0088] Step 5: Data Analysis

[0089] The server analyzes the cleansed data. Here, statistical models and machine learning algorithms are used to identify user lifestyle patterns. For example, it extracts patterns such as whether or not a user exercises at a specific time.

[0090] Step 6: Model Generation

[0091] The server generates a predictive model based on the analysis results. This model is used to reflect each user's behavior and preferences and provide optimal suggestions.

[0092] Step 7: Proposal Generation

[0093] The server creates specific suggestions for the user based on the predictive model it has generated. For example, based on the user's lifestyle patterns, it might create suggestions such as "do 15 minutes of stretching" or "we recommend a 30-minute walk."

[0094] Step 8: Submit Proposal

[0095] The server sends the generated suggestions to the user's terminal.

[0096] Step 9: Receive Proposal

[0097] The device receives suggestions sent from the server. The received suggestions are displayed in the device's notification center or within the app.

[0098] Step 10: Execution Notification

[0099] The device will notify the user based on the received suggestions. For example, it may use push notifications on a smartphone or the alert function of a wearable device to send notifications.

[0100] Step 11: User Actions

[0101] The user checks notifications from their device and takes the suggested action. For example, they might start stretching or eat a suggested healthy breakfast.

[0102] Step 12: Feedback Record

[0103] When a user takes action based on a suggestion, the results and evaluation are entered into the device. For example, feedback such as "I liked the breakfast menu" or "I went for a 30-minute walk" can be entered.

[0104] Step 13: Submit Feedback

[0105] The device sends user feedback to the server. This feedback is used as training data to inform future suggestions.

[0106] Step 14: Update the proposal

[0107] The server updates its suggestions based on user feedback. It analyzes the feedback data and makes new action suggestions or adjusts existing ones.

[0108] (Example 1)

[0109] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0110] In modern society, many people face health risks due to unhealthy lifestyle habits. Lack of exercise, stress, and sleep deprivation are particularly serious problems. Conventional systems struggle to provide optimal lifestyle improvement suggestions to individual users, and personalized support that takes into account each user's characteristics is not adequately offered. Therefore, there is a need for a system that effectively supports health maintenance and lifestyle improvement by collecting and analyzing user activity data to generate individualized suggestions.

[0111] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0112] In this invention, the server includes means for collecting user activity data, means for analyzing the activity data to identify the user's lifestyle patterns, means for generating a predictive model based on the identified lifestyle patterns, means for generating individual suggestions from the generated predictive model, means for transmitting the generated suggestions to the user's computer, means for collecting user feedback on the suggestions, and means for updating the suggestion content based on the feedback. This enables optimal suggestions based on each user's detailed lifestyle patterns, effectively supporting the user's health maintenance and improvement of lifestyle habits.

[0113] "User activity data" refers to data that shows the user's behavior and physiological state in their daily life, and includes behavioral data, schedule data, and biometric data.

[0114] "Analysis" refers to the process of analyzing collected activity data using statistical models and machine learning algorithms to identify specific patterns and trends.

[0115] "Lifestyle patterns" refer to data and characteristics that show the regularity of a user's habits and behaviors in their daily life.

[0116] A "predictive model" refers to a mathematical or algorithmic model used to predict a user's future behavior or state based on their lifestyle patterns.

[0117] "Proposal" refers to specific action plans and advice provided to users based on predictive models.

[0118] "Feedback" refers to the information that users send back to the server regarding the results and evaluations of their suggestions.

[0119] "Update" refers to the process of improving and refining the suggested content and predictive models based on feedback received from users.

[0120] The system of the present invention collects user activity data and analyzes that data to identify the user's lifestyle patterns. Based on the identified lifestyle patterns, it generates individual suggestions and sends these suggestions to the user's terminal. Furthermore, it collects feedback from the user and updates the suggestions based on that feedback to provide appropriate and personalized suggestions.

[0121] System Configuration

[0122] Server side

[0123] Data collection

[0124] The server receives activity data transmitted from the user's device. This activity data includes behavioral data, schedule data, and biometric data. Specific hardware used includes smartphones and wearable devices.

[0125] Data Analysis

[0126] The server analyzes the received data to identify the user's lifestyle patterns. Machine learning algorithms such as Python's scikit-learn and TENSORFLOW® are used for the analysis. This allows the server to determine, for example, whether the user is sedentary or needs to adopt a new exercise habit.

[0127] Model generation

[0128] The server generates a predictive model based on identified lifestyle patterns. This generated model is used to provide optimal suggestions for each user. For example, it might use TensorFlow to generate a neural network model.

[0129] Proposal generation

[0130] The server creates specific suggestions from the generated predictive model. For example, based on the user's exercise habits, it might generate suggestions such as "Do 10 minutes of stretching after breakfast" or "We recommend a 30-minute walk."

[0131] Submit Proposal

[0132] The server sends the generated suggestions to the user's terminal. This communication utilizes cloud services via the internet.

[0133] Terminal side

[0134] Data transmission

[0135] The device transmits user behavior data and schedules to the server in real time. Specifically, it uses data collected from sensors in smartphones and wearable devices.

[0136] Proposal received

[0137] The terminal receives the proposal content sent from the server. The received proposal is stored in the terminal's memory.

[0138] Execution notification

[0139] The device notifies the user of the suggested actions. For example, notifications are sent using smartphone push notifications or the alert function of wearable devices. The timing and display method of notifications are adjusted to make it easy for the user to take action.

[0140] Send feedback

[0141] Users implement the suggestions and provide feedback on their satisfaction level and effectiveness via their device. For example, they might use an input form provided as a smartphone app.

[0142] The device sends user feedback to the server. The feedback data is analyzed on the server and used to update suggestions and predictive models.

[0143] Specific example

[0144] Consider a user who wakes up at 6 AM to their smartphone alarm and has breakfast at 7 AM. This user spends most of their day doing desk work and tends to be sedentary.

[0145] Data collection

[0146] The device sends the user's wake-up time and meal times to the server.

[0147] Data Analysis

[0148] The server analyzes the user's behavioral data and identifies that they are not getting enough exercise.

[0149] Model generation

[0150] The server generates a model for introducing exercise habits that are suitable for the user.

[0151] Proposal generation

[0152] The server generates specific suggestions, such as "We suggest a 10-minute stretching session after breakfast."

[0153] Submit Proposal

[0154] The server sends the proposal to the user's terminal.

[0155] Proposal received

[0156] The terminal receives a proposal sent from the server and notifies the user.

[0157] Execution notification

[0158] The device sends a push notification to the user suggesting a "10-minute stretch after breakfast."

[0159] Send feedback

[0160] The user implements the suggested actions and enters feedback on their satisfaction level and effectiveness into the device.

[0161] The device sends user feedback to the server.

[0162] In this way, the system of the present invention can provide optimal support tailored to each user's lifestyle and continuously improve its suggestions.

[0163] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0164] System program processing flow

[0165] Step 1: Data Collection

[0166] Input: User behavior data, schedule data, biometric data

[0167] Specific operation:

[0168] The device collects user activity data in real time through smartphones and wearable devices.

[0169] The device transmits the collected data to the server via the internet.

[0170] Output: User activity data sent to the server

[0171] Step 2: Data Analysis

[0172] Input: User activity data sent to the server

[0173] Specific operation:

[0174] The server analyzes the received data using statistical models and machine learning algorithms (such as Python's scikit-learn and TensorFlow).

[0175] The server analyzes each data point to identify the user's lifestyle patterns.

[0176] Output: Identification results regarding the user's lifestyle patterns

[0177] Step 3: Model Generation

[0178] Input: Identification results regarding the user's lifestyle patterns

[0179] Specific operation:

[0180] The server generates a predictive model based on the identified lifestyle patterns.

[0181] The server uses TensorFlow to train a neural network model that provides the best possible suggestions for each user.

[0182] Output: Predictive model

[0183] Step 4: Proposal Generation

[0184] Input: Predictive model

[0185] Specific operation:

[0186] The server generates specific suggestions based on the predictive model.

[0187] The server develops an action plan that is suitable for the user's lifestyle (e.g., doing 10 minutes of stretching after breakfast).

[0188] Output: Specific proposals

[0189] Step 5: Submit Proposal

[0190] Input: Specific proposal

[0191] Specific operation:

[0192] The server sends the generated suggestions to the user's terminal.

[0193] The server records communication logs to verify that the proposal was sent correctly.

[0194] Output: Suggestions sent to the user's terminal

[0195] Step 6: Receive Proposal

[0196] Input: Suggestion sent from the server

[0197] Specific operation:

[0198] The terminal saves the suggestions received from the server to its memory.

[0199] The device prepares to notify the user of the received information.

[0200] Output: Suggestions saved on the device

[0201] Step 7: Execution Notification

[0202] Input: Suggestions saved on the device

[0203] Specific operation:

[0204] The device will notify the user of specific suggestions. This will be done using smartphone push notifications or the vibration function of wearable devices.

[0205] The device adjusts the timing and display method of notifications to make them easier for the user to act upon.

[0206] Output: Suggestions notified to the user

[0207] Step 8: Submit Feedback

[0208] Input: User feedback

[0209] Specific operation:

[0210] Users execute the suggestions and input the results and feedback into their devices. For example, they might use an input form within a smartphone app.

[0211] The device sends user feedback information to the server.

[0212] Output: Feedback information and execution results sent to the server

[0213] In this way, by processing each step, a system is built that effectively provides support tailored to the user's lifestyle.

[0214] (Application Example 1)

[0215] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0216] While conventional systems could provide personalized suggestions based on users' lifestyle patterns, they lacked concrete support such as optimizing work efficiency in factory labor and reducing the burden on workers. Furthermore, dynamic suggestions based on real-time worker data and improvements to those suggestions using feedback were not adequately implemented.

[0217] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0218] In this invention, the server includes means for collecting user activity data, means for analyzing the activity data to identify the user's lifestyle pattern, means for generating individual suggestions based on the identified lifestyle pattern, means for transmitting the generated suggestions to the user's terminal, means for collecting user feedback on the suggestions, means for updating the suggestion content based on the feedback, means for the robot terminal to collect worker motion data and work speed data in real time, means for analyzing the worker's work efficiency based on the motion data and work speed data and generating appropriate work suggestions, means for transmitting the generated work suggestions to the robot terminal and notifying the worker, and means for collecting worker feedback after notification by the robot terminal. This makes it possible to optimize the work efficiency and reduce the burden on workers in factory work.

[0219] "Means for collecting user activity data" refers to devices or systems that collect data such as user behavior and biometric information.

[0220] "Means of analyzing data to identify users' lifestyle patterns" refers to the process of analyzing collected data to identify users' behavioral patterns and lifestyle habits.

[0221] "Means for generating individual suggestions" refers to devices or systems that generate specific suggestions for actions or behaviors for users based on identified lifestyle patterns.

[0222] "Means for sending generated proposals to the user's terminal" refers to a system or method for delivering proposal content to the user's terminal.

[0223] "Means for collecting user feedback" refers to devices or systems that collect user responses and results to suggestions.

[0224] "Methods for updating proposals based on feedback" refers to the process of improving and updating proposals for future versions based on the feedback collected.

[0225] "Means for robot terminals to collect worker motion data and work speed data in real time" refers to robots or devices that monitor and record the movements and work efficiency of factory workers in real time.

[0226] "Means for analyzing worker efficiency based on work data" refers to a process of analyzing collected motion data and work speed data to evaluate worker efficiency and performance.

[0227] "Means for generating appropriate work suggestions" refers to devices or systems that generate optimal actions and break suggestions for workers based on analysis results.

[0228] "Means for sending generated work proposals to a robot terminal and notifying the worker" refers to the process of sending generated work proposals to a robot terminal and notifying the worker via the robot.

[0229] "Means for collecting worker feedback after notification by a robot terminal" refers to a system or method in which a robot terminal collects worker reactions and execution results.

[0230] The embodiments for carrying out the present invention will be described in detail. The system of the present invention is designed to optimize the work efficiency of workers in factory labor and reduce their burden. This system collects user activity data and worker motion data, analyzes them to generate optimal suggestions, and provides effective support by notifying users.

[0231] System Configuration

[0232] Server side

[0233] Data collection

[0234] The server receives worker motion data and work speed data transmitted from the robot terminal. This uses built-in sensors, cameras, and RFID tag recognition technology.

[0235] Data Analysis

[0236] The server analyzes the received data to identify the worker's work patterns and efficiency. Statistical models and machine learning algorithms (e.g., Python's Scikit-Learn and TensorFlow) are used for the analysis.

[0237] Model generation

[0238] The server generates a predictive model based on the identified work patterns. This model is used to provide optimal support suggestions for each worker.

[0239] Proposal generation

[0240] The server creates specific suggestions from the generated predictive model. For example, based on worker action data, it generates suggestions such as "recommend a 10-minute break" or "increase work speed."

[0241] Submit Proposal

[0242] The server sends the generated proposal to the robot terminal.

[0243] Terminal side

[0244] Data transmission

[0245] The robot terminal transmits worker motion data and work speed data to the server in real time.

[0246] Proposal received

[0247] The robot terminal receives the proposal content sent from the server.

[0248] Execution notification

[0249] The robot terminal notifies the worker of the proposed content. Notification methods include voice guidance and display screens.

[0250] Send feedback

[0251] The worker implements the proposed solution and inputs the results and feedback into the robot terminal. The robot terminal then sends this feedback to the server.

[0252] Hardware and software to be used

[0253] Hardware: Built-in sensors, camera, RFID tag reader, robot terminal

[0254] Software: Python, Scikit-Learn, TensorFlow

[0255] Specific example

[0256] The robot terminal collects motion and work speed data from a worker in real time and sends it to a server. The server analyzes the received data and identifies if the worker is fatigued. As a result, the server uses a predictive model to generate a suggestion to "take a 10-minute break" and sends this to the robot terminal. The robot terminal notifies the worker of the suggestion by voice, and the worker takes a break as suggested. After that, the worker enters feedback into the robot terminal, and this is sent to the server.

[0257] Example of a prompt

[0258] "Based on the worker's current work speed and motion data, please suggest the optimal break times."

[0259] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0260] Step 1: Data Collection

[0261] The robot terminal collects worker motion data and work speed data in real time. It uses built-in sensors, cameras, and RFID tag readers to acquire information about each worker's actions. For example, the worker's hand movements and walking speed are measured by sensors and recorded as data on the terminal. The input is worker motion data and work speed data, and the output is the raw data recorded on the robot terminal.

[0262] Step 2: Data transmission

[0263] The robot terminal transmits collected motion data and work speed data to the server in real time. Data recorded on the terminal is transferred to the server using an internet connection or a dedicated communication protocol. The input is the raw data recorded on the terminal, and the output is the data sent to the server for analysis.

[0264] Step 3: Data Analysis

[0265] The server analyzes the received data to identify the worker's work patterns and efficiency. The analysis uses methods for normalizing the data (e.g., StandardScaler) and the KMeans algorithm for clustering. Specifically, the operational data is analyzed using Python's Scikit-Learn and TensorFlow. The input is the data to be analyzed sent to the server, and the output is information about the worker's work patterns and efficiency.

[0266] Step 4: Model Generation

[0267] The server generates a predictive model based on the identified work patterns. Here, a machine learning model is trained to create a predictive model that provides optimal support suggestions for each worker. This process involves training the model using a large amount of work data to improve its accuracy. The input is information about work patterns and efficiency, and the output is the trained predictive model.

[0268] Step 5: Proposal Generation

[0269] The server generates specific suggestions from the predictive model it has created. For example, individual suggestions such as "recommend taking a 10-minute break" or "increase work speed" are automatically generated. The input is the trained predictive model, and the output is specific support suggestions.

[0270] Step 6: Submit Proposal

[0271] The server sends the generated suggestions to the robot terminal. The suggestions are transferred to the robot terminal using an internet connection or a dedicated communication protocol. The input is the specific support suggestion, and the output is the suggestion information sent to the robot terminal.

[0272] Step 7: Receive proposal

[0273] The robot terminal receives the suggested content sent from the server. The suggested content is stored on the robot terminal and ready to be notified to the worker. The input is the suggested information sent from the server, and the output is the suggested content stored on the robot terminal.

[0274] Step 8: Execution Notification

[0275] The robot terminal notifies the worker of the suggested actions. Notification methods include voice guidance and display screens. Specifically, the robot terminal prompts the worker to take breaks or change speed through voice messages and screen displays. The input is the suggested actions stored in the robot terminal, and the output is notifications to the worker via voice and display.

[0276] Step 9: Submit Feedback

[0277] The worker implements the proposed solution and inputs the results and feedback into the robot terminal. The robot terminal sends this feedback to the server. The input is the worker's feedback, and the output is the feedback information sent to the server.

[0278] Step 10: Feedback Analysis and Update of Proposal Content

[0279] The server updates the proposal content based on the received feedback. By adding new data to the training dataset and retraining the prediction model, the accuracy of the proposals presented in subsequent times is improved. The input is the feedback information sent to the server, and the output is the updated prediction model and improved proposal content.

[0280] Furthermore, an emotion engine for estimating the user's emotion may be combined. That is, the specific processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform specific processing using the user's emotion.

[0281] The embodiments for implementing the present invention will be described in detail. A system that collects emotion data in addition to the user's activity data and integrally analyzes these data to provide optimal support for the user's lifestyle. The purpose of this system is to generate more individualized proposals and improve the usefulness of the proposals by also considering the user's emotional state.

[0282] System Configuration

[0283] Server Side

[0284] Data Collection

[0285] The server receives the activity data and emotion data transmitted from the user's terminal. The activity data includes behavior data, schedule data, and biological data.

[0286] Collection of Emotion Data

[0287] The server receives emotion data from the emotion engine installed on the terminal. The emotion engine uses at least one of voice analysis, facial expression recognition, and text analysis to identify the user's emotion.

[0288] Data Analysis

[0289] The server cleanses and analyzes received activity and emotional data to identify the user's lifestyle patterns and emotional state. Machine learning algorithms are used to extract statistically significant patterns.

[0290] Model generation

[0291] The server generates a predictive model based on the analysis results. This model reflects the user's lifestyle patterns and emotional state and is used to provide personalized suggestions.

[0292] Proposal generation

[0293] The server creates specific suggestions from the generated predictive model. For example, for a user whose emotional state is stressed, this might include a suggestion to "take deep breaths to relax."

[0294] Submit Proposal

[0295] The server sends the generated suggestions to the user's terminal.

[0296] Terminal side

[0297] Data transmission

[0298] The device transmits user behavior data, schedule data, biometric data, and emotional data to the server in real time.

[0299] Emotion data generation

[0300] The emotion engine built into the device identifies emotions from the user's voice, facial expressions, and text, and generates data based on those emotions.

[0301] Proposal received

[0302] The terminal receives the proposal sent from the server and notifies the user.

[0303] Execution notice

[0304] The terminal notifies the user of the received proposal. For example, it notifies using the push notification of a smartphone or the alert function of a wearable device.

[0305] Feedback transmission

[0306] The terminal sends the user's feedback to the server. The feedback includes the execution result and evaluation of the proposal.

[0307] Specific example

[0308] Consider an example for dealing with the stress that users feel daily. The user records activities and emotions using a smartphone or a wearable device.

[0309] C Data collection

[0310] The terminal sends the user's activity data (e.g., the number of steps and schedule) and emotion data (such as stress and happiness based on voice and facial expression analysis) to the server.

[0311] Data analysis

[0312] The server analyzes the activity data and emotion data and identifies that the user is likely to feel stress at a specific time period.

[0313] Model generation

[0314] The server generates a prediction model based on the user's life pattern and emotional state. This model provides appropriate proposals for stress reduction.

[0315] Proposal generation

[0316] The server generates specific proposals such as "proposal for deep breathing to relax" and "take a temporary break".

[0317] Submit Proposal

[0318] The server sends the generated suggestions to the user's terminal.

[0319] Proposal received

[0320] The terminal receives a proposal sent from the server and notifies the user.

[0321] Execution notification

[0322] The device notifies the user to "take a deep breath."

[0323] Send feedback

[0324] The user takes a deep breath and enters the result and evaluation into the device.

[0325] The device sends user feedback to the server.

[0326] In this way, the system of the present invention can take into account the user's emotional state, provide optimal support tailored to their lifestyle, and continuously improve its suggestions.

[0327] The following describes the processing flow.

[0328] Step 1: Collect activity and emotional data

[0329] Users record their daily activities through smartphones and wearable devices, such as steps taken, heart rate, and scheduled events.

[0330] The emotion engine built into the device analyzes the user's voice, facial expressions, and text to generate emotion data. For example, it can determine the user's stress level from conversation content and changes in facial expressions.

[0331] Step 2: Data transmission

[0332] The device periodically sends user activity data and sentiment data to the server. Communication protocols such as HTTPS and MQTT are used.

[0333] Step 3: Data reception and storage

[0334] The server receives activity and sentiment data transmitted from the terminal. The received data is stored in a database.

[0335] Step 4: Data Cleansing

[0336] The server cleanses the stored data. For example, it improves the accuracy of the analysis by removing outliers and standardizing the data format.

[0337] Step 5: Data Analysis

[0338] The server uses the cleansed data to analyze the user's lifestyle patterns and emotional state. This analysis employs machine learning algorithms to extract the user's daily behavioral and emotional tendencies.

[0339] Step 6: Model Generation

[0340] The server generates a predictive model based on the analysis results. This model is used to provide optimal suggestions adapted to the user's behavioral patterns and emotional state.

[0341] Step 7: Proposal Generation

[0342] The server uses the generated predictive model to create specific suggestions for the user. For example, if stress levels are high due to prolonged desk work, it will generate recommendations such as "Take a 5-minute break every hour."

[0343] Step 8: Submit Proposal

[0344] The server sends the generated suggestions to the user's terminal.

[0345] Step 9: Receive Proposal

[0346] The terminal receives the proposal content sent from the server. The received proposal is immediately prepared to be notified to the user.

[0347] Step 10: Execution Notification

[0348] The device notifies the user based on the received suggestions. For example, it might use smartphone push notifications or wearable device alerts to notify the user of suggestions such as "take a deep breath" or "take a short walk."

[0349] Step 11: User Actions

[0350] The user checks notifications from their device and takes the suggested action. For example, they might take a 5-minute break and go for a walk or meditate.

[0351] Step 12: Feedback Record

[0352] When a user completes an action based on a suggestion, the results and evaluation are entered into the device. For example, it might record how much deep breathing or walking helped reduce stress.

[0353] Step 13: Submit Feedback

[0354] The device sends user feedback to the server.

[0355] Step 14: Update the proposal

[0356] The server updates its suggestions based on the feedback received. Newly collected data is also analyzed and incorporated into future suggestions. This allows for the provision of more optimized suggestions to users.

[0357] In this way, through a series of processing steps, we realize a system that provides optimal support considering the user's emotional state and daily behavior, and that allows for continuous improvement.

[0358] (Example 2)

[0359] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0360] Current lifestyle support systems make suggestions based solely on user activity data, making it difficult to provide personalized suggestions that take emotional states into account. This can result in insufficient assurance of the usefulness and suitability of the suggestions. Furthermore, the current system struggles to quickly collect feedback on changes in users' lifestyle patterns and emotional states, and to update the suggestions accordingly.

[0361] The identification processing performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for collecting user activity data and emotional data; means for cleansing and analyzing the activity data and emotional data to identify the user's lifestyle patterns and emotional states; means for generating a predictive model based on the identified lifestyle patterns and emotional states using a machine learning algorithm; means for generating individual suggestions based on the generated predictive model; means for transmitting the generated suggestions to the user's terminal; means for collecting user feedback on the suggestions; and means for updating the suggestion content based on the feedback. This enables integrated analysis of the user's activity data and emotional data, and provides optimal support tailored to their lifestyle and emotional state.

[0362] "Activity data" refers to information about a user's daily behavior, and includes behavioral data, schedule data, biometric data, etc.

[0363] "Emotional data" refers to information about a user's emotional state, and includes voice data, facial expression data, text data, and other similar data.

[0364] "Data cleansing" refers to processes aimed at improving data quality, such as handling missing data values ​​and detecting outliers.

[0365] "Analysis" refers to the process of identifying patterns and trends based on collected data using statistical methods and machine learning algorithms.

[0366] "Lifestyle patterns" refer to identifying tendencies and regularities in a user's daily behavior.

[0367] "Emotional state" refers to the emotions a user feels at a particular time or in a particular situation.

[0368] A "machine learning algorithm" refers to a computational method used to analyze large amounts of data, learn patterns, and perform predictions and classifications.

[0369] A "predictive model" refers to a mathematical model used to predict future behavior or states based on analyzed data.

[0370] "Individualized suggestions" refer to advice or guidelines tailored to address a user's specific situation or emotional state.

[0371] "Feedback" refers to responding to a user's suggestions by providing results and evaluations of their actions.

[0372] "Updating the proposal" refers to the process of improving the proposal based on the feedback collected.

[0373] This invention is a system that comprehensively analyzes user activity data and emotional data to provide personalized support tailored to the user's lifestyle and emotional state. The system aims to make more appropriate suggestions by taking the user's emotional state into consideration.

[0374] System Configuration

[0375] Server side

[0376] Data collection

[0377] The server has the functionality to receive activity data and emotional data transmitted from the user's terminal. Activity data includes behavioral data, schedule data, and biometric data. Emotional data includes voice data, facial expression data, and text data.

[0378] Specific hardware: Data center servers

[0379] Specific software: Apache® Kafka (messaging system), InfluxDB (database)

[0380] Collection of emotional data

[0381] The server receives emotion data from the emotion engine installed in the terminal. The emotion engine uses voice analysis, facial recognition, and text analysis to identify the user's emotions.

[0382] Specific hardware: High-performance CPU / GPU server

[0383] Specific software: Python libraries (OpenCV, TensorFlow)

[0384] Data Analysis

[0385] The server cleanses and analyzes the received activity and emotional data to identify the user's lifestyle patterns and emotional states. This allows it to extract statistically significant patterns.

[0386] Specific software used: Pandas (data cleansing), Scikit-learn (machine learning)

[0387] Model generation

[0388] The server generates a predictive model based on the analysis results. This model reflects the user's lifestyle patterns and emotional state and is used to provide personalized suggestions.

[0389] Specific software: Scikit-learn, TensorFlow

[0390] Proposal generation

[0391] The server creates specific suggestions from the generated predictive model. For example, for a user whose emotional state is stressed, this might include a suggestion to "take deep breaths to relax."

[0392] Specific software: Django (Web framework)

[0393] Submit Proposal

[0394] The server sends the generated suggestions to the user's terminal.

[0395] Specific hardware: Network infrastructure

[0396] Specific software: REST API

[0397] Terminal side

[0398] Data transmission

[0399] The device transmits user behavior data, schedule data, biometric data, and emotional data to the server in real time.

[0400] Specific hardware: smartphones, wearable devices

[0401] Specific software: ANDROID (registered trademark) / iOS app

[0402] Emotion data generation

[0403] The emotion engine built into the device identifies emotions from the user's voice, facial expressions, and text, and generates data based on those emotions.

[0404] Specific software: Python libraries (OpenCV, TensorFlow)

[0405] Proposal received

[0406] The terminal receives the proposal sent from the server and notifies the user.

[0407] Specific software: Firebase Cloud Messaging

[0408] Execution notification

[0409] The device notifies the user of the received proposal. For example, it may use push notifications on a smartphone or the alert function of a wearable device to notify the user.

[0410] Specific hardware: smartphones, wearable devices

[0411] Specific software: Android / iOS app

[0412] Send feedback

[0413] The device sends user feedback to the server. This feedback includes the results and evaluation of the proposed changes.

[0414] Specific hardware: smartphones, wearable devices

[0415] Specific software: Android / iOS app

[0416] Specific example

[0417] Consider examples of how users can cope with everyday stress. Users record their activities and emotions using smartphones or wearable devices.

[0418] Data collection

[0419] The device sends user activity data (e.g., steps taken and schedule) and emotional data (stress and happiness levels based on voice and facial expression analysis) to the server.

[0420] Data Analysis

[0421] The server analyzes activity and emotional data to identify when users are more likely to experience stress during specific time periods.

[0422] Model generation

[0423] The server generates a predictive model based on the user's lifestyle patterns and emotional state. This model provides appropriate suggestions for stress reduction.

[0424] Proposal generation

[0425] The server generates specific suggestions such as "take deep breaths to relax" or "take a short break."

[0426] Submit Proposal

[0427] The server sends the generated suggestions to the user's terminal.

[0428] Proposal received

[0429] The terminal receives a proposal sent from the server and notifies the user.

[0430] Execution notification

[0431] The device notifies the user to "take a deep breath."

[0432] Send feedback

[0433] The user takes a deep breath and enters the result and evaluation into the device.

[0434] The device sends user feedback to the server.

[0435] In this way, the system of the present invention can take into account the user's emotional state, provide optimal support tailored to their lifestyle, and continuously improve its suggestions.

[0436] Example of a prompt

[0437] "Based on emotional and activity data collected by users over a certain period, predict when they will experience stress and generate suggestions for appropriate relaxation methods."

[0438] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0439] Step 1:

[0440] Data collection

[0441] The device transmits user activity and emotional data to the server in real time. Specifically, this includes behavioral data (steps and schedule), biometric data (heart rate and sleep information), voice data, facial expression data, and text data collected from smartphones and wearable devices.

[0442] Input: Behavioral data, schedule data, biometric data, voice data, facial expression data, text data

[0443] Output: Sending data to the server

[0444] Step 2:

[0445] Collection of emotional data

[0446] The server receives emotion data from the emotion engine installed in the terminal. The emotion engine identifies the user's emotions using speech analysis, facial recognition, and text analysis. For example, it analyzes the user's facial expressions using OpenCV and TensorFlow to determine their emotional state. This information is also sent to the server.

[0447] Input: Audio data, facial expression data, text data

[0448] Output: Sentiment data

[0449] Step 3:

[0450] Data cleansing

[0451] The server cleanses the received activity and sentiment data. Specifically, it performs data washing such as imputing missing values ​​and removing outliers to prepare a dataset suitable for analysis. The Pandas library is used in this process.

[0452] Input: Raw activity data, emotion data

[0453] Output: Cleansed data

[0454] Step 4:

[0455] Data Analysis

[0456] The server analyzes the cleansed data to identify the user's lifestyle patterns and emotional states. Specifically, it uses the Scikit-learn library to perform clustering and anomaly detection to analyze when and what emotional states the user is experiencing.

[0457] Input: Cleansed data

[0458] Output: Analysis results (lifestyle patterns, emotional state)

[0459] Step 5:

[0460] Model generation

[0461] The server generates a predictive model using machine learning algorithms based on the analysis results. Using Scikit-learn and TensorFlow, it creates a predictive model that reflects the user's lifestyle patterns and emotional state.

[0462] Input: Analysis results

[0463] Output: Predictive model

[0464] Step 6:

[0465] Proposal generation

[0466] The server generates specific suggestions based on a predictive model. For example, it might generate a "suggestion for deep breathing to relax" during times when the user is likely to feel stressed. The suggestions are built using the Django framework.

[0467] Input: Predictive model

[0468] Output: Specific proposals

[0469] Step 7:

[0470] Submit Proposal

[0471] The server sends the generated suggestions to the user's device. Suggestions are sent in real time using a REST API.

[0472] Input: Specific proposal

[0473] Output: Sending suggestions to the terminal

[0474] Step 8:

[0475] Proposal received

[0476] The device receives suggestions sent from the server and notifies the user. Firebase Cloud Messaging is used to provide push notifications and alerts.

[0477] Input: Suggestion from the server

[0478] Output: Notification to the user

[0479] Step 9:

[0480] Execution notification

[0481] The device sends notifications to the user prompting them to take action on the suggested actions. For example, a smartphone might receive a notification instructing it to "take a deep breath before the next meeting."

[0482] Input: Received proposals

[0483] Output: Execution notification to the user

[0484] Step 10:

[0485] Send feedback

[0486] The terminal sends user feedback to the server. The user inputs the results and evaluation of the suggested actions into the terminal, and this information is sent to the server.

[0487] Input: User feedback

[0488] Output: Sending feedback to the server

[0489] These processing steps enable the integrated analysis of user activity and emotional data, allowing for the provision of optimal support tailored to their lifestyle and emotional state.

[0490] (Application Example 2)

[0491] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0492] Traditional user support systems typically made suggestions based on user activity data. However, these systems failed to consider the user's emotional state and were unable to adequately respond to temporary emotions or psychological conditions, limiting the usefulness of their suggestions. In particular, existing systems failed to provide sufficient support in situations where a rapid response was required when the user was experiencing negative emotions such as anxiety or fear.

[0493] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0494] In this invention, the server includes means for collecting user activity data and emotional data, means for analyzing the activity data and emotional data to identify the user's lifestyle patterns and emotional state, and means for generating personalized suggestions based on the identified lifestyle patterns and emotional state. This enables the prompt and appropriate safety measures to be suggested when the user feels anxiety or fear.

[0495] "Activity data" refers to information about a user's daily activities, such as location information, activity history, and schedule data.

[0496] "Emotional data" refers to information about a user's emotional state, analyzed from their voice data, text data, and facial expression data.

[0497] A "device" is an electronic device that a user possesses and uses to send, receive, and display information, and includes smartphones and wearable devices.

[0498] "Feedback" refers to the reactions and evaluations that users give to suggestions they receive.

[0499] A "machine learning algorithm" is a computational method used by computers to learn patterns from data and perform predictions and classifications.

[0500] A "suggestion" refers to specific instructions or advice regarding actions or measures for the user, generated based on the analyzed data.

[0501] This invention provides a system that collects and analyzes user activity data and emotional data, generates personalized suggestions, and transmits them to the user's device. Specific embodiments for carrying out this invention are described below.

[0502] System Configuration

[0503] Server side

[0504] Data collection:

[0505] The server receives activity data and emotion data transmitted from the user's device. This activity data includes location information, activity history, and schedule data. Emotion data consists of voice data, text data, and facial expression data.

[0506] Collection of emotional data:

[0507] The server receives emotion data from the emotion engine installed in the terminal. This emotion engine identifies the user's emotions using at least one of the following: voice analysis, facial expression recognition, and text analysis.

[0508] Data analysis:

[0509] The server cleanses and analyzes received activity and emotional data to identify the user's lifestyle patterns and emotional state. Machine learning algorithms are used to extract statistically significant patterns.

[0510] Model generation:

[0511] The server generates a predictive model based on the analysis results. This model reflects the user's lifestyle patterns and emotional state and is used to provide personalized suggestions.

[0512] Suggestion generation:

[0513] The server creates specific suggestions from the generated predictive model. For example, for a user whose emotional state is one of anxiety or fear, these suggestions might include "location of the nearest police station" or "instructions to choose a safe route."

[0514] Submit proposal:

[0515] The server sends the generated suggestions to the user's device.

[0516] Terminal side

[0517] Data transmission:

[0518] The device transmits user behavior data, schedule data, biometric data, and emotional data to the server in real time.

[0519] Emotion data generation:

[0520] The emotion engine built into the device identifies emotions from the user's voice, facial expressions, and text, and generates data based on those emotions.

[0521] Proposal received:

[0522] The terminal receives the proposal sent from the server and notifies the user.

[0523] Execution notification:

[0524] The device notifies the user of the received suggestions. For example, it may use push notifications on a smartphone or the alert function of a wearable device to notify the user.

[0525] Send feedback:

[0526] The device sends user feedback to the server. This feedback includes the results and evaluation of the proposed changes.

[0527] Specific example

[0528] Consider specific examples of how users can cope with the anxiety and fear they experience on a daily basis. Users record their activities and emotions using smartphones or wearable devices. The device sends the user's activity data (e.g., location information and activity history) and emotional data (e.g., anxiety and fear based on voice and facial expression analysis) to a server. The server analyzes the activity and emotional data to identify when the user is likely to feel anxiety or fear. The server generates a predictive model based on the user's lifestyle patterns and emotional state. This model provides appropriate suggestions for reducing anxiety and fear. The server generates specific suggestions, such as "location of the nearest police station" or "instructions for choosing a safe route." The server sends the generated suggestions to the user's device. The device receives the suggestions from the server and notifies the user. The user receives the notification, acts according to the suggestions, and inputs the results and evaluation into the device. The device sends the user's feedback to the server.

[0529] Example of a prompt

[0530] "Analyze the user's voice and text data to extract their current emotional state. If they are feeling anxious or frightened, generate a notification suggesting the location of the nearest police station based on their current location."

[0531] In this way, the system of the present invention can take into account the user's emotional state, provide optimal support tailored to their lifestyle, and continuously improve its suggestions.

[0532] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0533] Step 1:

[0534] The user's device collects user activity data (location information, activity history, schedule data) and emotional data (voice data, text data, facial expression data) in real time. This data is processed using an emotion engine installed in the device and sent to the server. The input is the user's activity data and emotional data, and the output is the transmission of this data to the server.

[0535] Step 2:

[0536] The server receives activity and sentiment data sent from the terminal. The received data is cleansed, and any missing or inaccurate data is corrected or supplemented. Next, the data is ready for analysis. The input is the activity and sentiment data sent from the terminal, and the output is the cleansed data.

[0537] Step 3:

[0538] The server analyzes cleansed activity and emotional data. This analysis uses machine learning algorithms to extract statistically significant patterns. The user's lifestyle patterns and emotional states are identified. The input is the cleansed data, and the output is the identified lifestyle patterns and emotional states.

[0539] Step 4:

[0540] The server uses an AI model based on the analysis results to generate a predictive model. This model reflects the user's lifestyle patterns and emotional state and is used to provide personalized suggestions. The input is the analysis results, and the output is the predictive model.

[0541] Step 5:

[0542] The server generates specific suggestions from the predictive model it has created. These suggestions are customized according to the user's current emotional state and lifestyle. For example, if the user is feeling anxious or fearful, the suggestions might include the location of the nearest police station and directions to a safe route. The input is the predictive model, and the output is the specific suggestions.

[0543] Step 6:

[0544] The server sends the generated suggestions to the user's terminal. The input is the specific suggestions, and the output is the suggestions sent to the user's terminal.

[0545] Step 7:

[0546] The device receives suggestions sent from the server and notifies the user. This notification is delivered using smartphone push notifications or the alert function of a wearable device. The input is the suggestion content sent from the server, and the output is the information notified to the user.

[0547] Step 8:

[0548] The user acts according to the received suggestions and inputs the results and evaluations into the device. This includes feedback on how helpful the suggestions were and the actions taken. The input is the user's feedback, and the output is the state in which the feedback has been entered into the device.

[0549] Step 9:

[0550] The terminal sends user feedback to the server. The server receives this feedback and uses it to update the suggestions. The input is the user feedback, and the output is the feedback sent to the server.

[0551] Step 10:

[0552] The server updates the suggestions based on the feedback received. This makes future suggestions more accurate and useful to the user. The input is the user's feedback, and the output is the updated suggestions.

[0553] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0554] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0555] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0556] [Second Embodiment]

[0557] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0558] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0559] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0560] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0561] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0562] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0563] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0564] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0565] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0566] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0567] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0568] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0569] The embodiments for carrying out the present invention will be described in detail. The system of the present invention collects user activity data, analyzes that data to identify the user's lifestyle pattern, generates individual suggestions based on the identified lifestyle pattern, and transmits those suggestions to the user's terminal. By collecting feedback from the user and updating the suggestions based on that feedback, the system provides appropriate and personalized suggestions.

[0570] System Configuration

[0571] Server side

[0572] Data collection

[0573] The server receives activity data transmitted from the user's device. This activity data includes behavioral data, schedule data, and biometric data.

[0574] Data Analysis

[0575] The server analyzes the received data to identify the user's lifestyle patterns. Statistical models and machine learning algorithms are used for this analysis.

[0576] Model generation

[0577] The server generates a predictive model based on the identified lifestyle patterns. This model is used to provide optimal support suggestions for each user.

[0578] Proposal generation

[0579] The server creates specific suggestions from the generated predictive model. For example, it might generate suggestions such as "do some stretching" or "take a 30-minute walk" based on the user's exercise habits.

[0580] Submit Proposal

[0581] The server sends the generated suggestions to the user's terminal.

[0582] Terminal side

[0583] Data transmission

[0584] The device transmits user behavior data and schedules to the server in real time.

[0585] Proposal received

[0586] The terminal receives the proposal content sent from the server.

[0587] Execution notification

[0588] The device notifies the user of the proposed content. For example, notifications are sent using smartphone push notifications or the alert function of wearable devices.

[0589] Send feedback

[0590] The device sends user feedback to the server. This feedback includes evaluations of suggestions and the results of their implementation.

[0591] Specific example

[0592] Consider a user who wakes up at 6 AM to their smartphone alarm and has breakfast at 7 AM. This user spends most of their day doing desk work and tends to be sedentary.

[0593] Data collection

[0594] The device sends the user's wake-up time and meal times to the server.

[0595] Data Analysis

[0596] The server analyzes the user's behavioral data and identifies that they are not getting enough exercise.

[0597] Model generation

[0598] The server generates a model for introducing exercise habits that are suitable for the user.

[0599] Proposal generation

[0600] The server generates specific suggestions, such as "We suggest a 10-minute stretching session after breakfast."

[0601] Submit Proposal

[0602] The server sends the proposal to the user's terminal.

[0603] Proposal received

[0604] The terminal receives a proposal sent from the server and notifies the user.

[0605] Execution notification

[0606] The device sends a push notification to the user suggesting a "10-minute stretch after breakfast."

[0607] Send feedback

[0608] The user implements the suggested actions and enters feedback on their satisfaction level and effectiveness into the device.

[0609] The device sends user feedback to the server.

[0610] In this way, the system of the present invention can provide optimal support tailored to each user's lifestyle and continuously improve its suggestions.

[0611] The following describes the processing flow.

[0612] Step 1: Data Collection

[0613] Users record their daily activities and schedules using smartphones or wearable devices. For example, they accumulate data by entering steps, heart rate, and appointments into their calendars.

[0614] The device periodically transmits recorded data to a server. This data includes behavioral data, schedule data, and biometric data.

[0615] Step 2: Data Reception

[0616] The server receives activity data sent from the terminal. The received data is temporarily held in a buffer.

[0617] Step 3: Data Storage

[0618] The server stores the received data in a database. During this process, the data is organized by user to facilitate efficient analysis later.

[0619] Step 4: Data Cleansing

[0620] The server cleanses the stored data. This process involves deleting unnecessary data and standardizing the format. For example, it removes outliers and standardizes data formats.

[0621] Step 5: Data Analysis

[0622] The server analyzes the cleansed data. Here, statistical models and machine learning algorithms are used to identify user lifestyle patterns. For example, it extracts patterns such as whether or not a user exercises at a specific time.

[0623] Step 6: Model Generation

[0624] The server generates a predictive model based on the analysis results. This model is used to reflect each user's behavior and preferences and provide optimal suggestions.

[0625] Step 7: Proposal Generation

[0626] The server creates specific suggestions for the user based on the predictive model it has generated. For example, based on the user's lifestyle patterns, it might create suggestions such as "do 15 minutes of stretching" or "we recommend a 30-minute walk."

[0627] Step 8: Submit Proposal

[0628] The server sends the generated suggestions to the user's terminal.

[0629] Step 9: Receive Proposal

[0630] The device receives suggestions sent from the server. The received suggestions are displayed in the device's notification center or within the app.

[0631] Step 10: Execution Notification

[0632] The device will notify the user based on the received suggestions. For example, it may use push notifications on a smartphone or the alert function of a wearable device to send notifications.

[0633] Step 11: User Actions

[0634] The user checks notifications from their device and takes the suggested action. For example, they might start stretching or eat a suggested healthy breakfast.

[0635] Step 12: Feedback Record

[0636] When a user takes action based on a suggestion, the results and evaluation are entered into the device. For example, feedback such as "I liked the breakfast menu" or "I went for a 30-minute walk" can be entered.

[0637] Step 13: Submit Feedback

[0638] The device sends user feedback to the server. This feedback is used as training data to inform future suggestions.

[0639] Step 14: Update the proposal

[0640] The server updates its suggestions based on user feedback. It analyzes the feedback data and makes new action suggestions or adjusts existing ones.

[0641] (Example 1)

[0642] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0643] In modern society, many people face health risks due to unhealthy lifestyle habits. Lack of exercise, stress, and sleep deprivation are particularly serious problems. Conventional systems struggle to provide optimal lifestyle improvement suggestions to individual users, and personalized support that takes into account each user's characteristics is not adequately offered. Therefore, there is a need for a system that effectively supports health maintenance and lifestyle improvement by collecting and analyzing user activity data to generate individualized suggestions.

[0644] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0645] In this invention, the server includes means for collecting user activity data, means for analyzing the activity data to identify the user's lifestyle patterns, means for generating a predictive model based on the identified lifestyle patterns, means for generating individual suggestions from the generated predictive model, means for transmitting the generated suggestions to the user's computer, means for collecting user feedback on the suggestions, and means for updating the suggestion content based on the feedback. This enables optimal suggestions based on each user's detailed lifestyle patterns, effectively supporting the user's health maintenance and improvement of lifestyle habits.

[0646] "User activity data" refers to data that shows the user's behavior and physiological state in their daily life, and includes behavioral data, schedule data, and biometric data.

[0647] "Analysis" refers to the process of analyzing collected activity data using statistical models and machine learning algorithms to identify specific patterns and trends.

[0648] "Lifestyle patterns" refer to data and characteristics that show the regularity of a user's habits and behaviors in their daily life.

[0649] A "predictive model" refers to a mathematical or algorithmic model used to predict a user's future behavior or state based on their lifestyle patterns.

[0650] "Proposal" refers to specific action plans and advice provided to users based on predictive models.

[0651] "Feedback" refers to the information that users send back to the server regarding the results and evaluations of their suggestions.

[0652] "Update" refers to the process of improving and refining the suggested content and predictive models based on feedback received from users.

[0653] The system of the present invention collects user activity data and analyzes that data to identify the user's lifestyle patterns. Based on the identified lifestyle patterns, it generates individual suggestions and sends these suggestions to the user's terminal. Furthermore, it collects feedback from the user and updates the suggestions based on that feedback to provide appropriate and personalized suggestions.

[0654] System Configuration

[0655] Server side

[0656] Data collection

[0657] The server receives activity data transmitted from the user's device. This activity data includes behavioral data, schedule data, and biometric data. Specific hardware used includes smartphones and wearable devices.

[0658] Data Analysis

[0659] The server analyzes the received data to identify the user's lifestyle patterns. Machine learning algorithms such as Python's scikit-learn and TensorFlow are used for the analysis. This allows the server to determine, for example, whether the user is sedentary or needs to adopt a new exercise habit.

[0660] Model generation

[0661] The server generates a predictive model based on identified lifestyle patterns. This generated model is used to provide optimal suggestions for each user. For example, it might use TensorFlow to generate a neural network model.

[0662] Proposal generation

[0663] The server creates specific suggestions from the generated predictive model. For example, based on the user's exercise habits, it might generate suggestions such as "Do 10 minutes of stretching after breakfast" or "We recommend a 30-minute walk."

[0664] Submit Proposal

[0665] The server sends the generated suggestions to the user's terminal. This communication utilizes cloud services via the internet.

[0666] Terminal side

[0667] Data transmission

[0668] The device transmits user behavior data and schedules to the server in real time. Specifically, it uses data collected from sensors in smartphones and wearable devices.

[0669] Proposal received

[0670] The terminal receives the proposal content sent from the server. The received proposal is stored in the terminal's memory.

[0671] Execution notification

[0672] The device notifies the user of the suggested actions. For example, notifications are sent using smartphone push notifications or the alert function of wearable devices. The timing and display method of notifications are adjusted to make it easy for the user to take action.

[0673] Send feedback

[0674] Users implement the suggestions and provide feedback on their satisfaction level and effectiveness via their device. For example, they might use an input form provided as a smartphone app.

[0675] The device sends user feedback to the server. The feedback data is analyzed on the server and used to update suggestions and predictive models.

[0676] Specific example

[0677] Consider a user who wakes up at 6 AM to their smartphone alarm and has breakfast at 7 AM. This user spends most of their day doing desk work and tends to be sedentary.

[0678] Data collection

[0679] The device sends the user's wake-up time and meal times to the server.

[0680] Data Analysis

[0681] The server analyzes the user's behavioral data and identifies that they are not getting enough exercise.

[0682] Model generation

[0683] The server generates a model for introducing exercise habits that are suitable for the user.

[0684] Proposal generation

[0685] The server generates specific suggestions, such as "We suggest a 10-minute stretching session after breakfast."

[0686] Submit Proposal

[0687] The server sends the proposal to the user's terminal.

[0688] Proposal received

[0689] The terminal receives a proposal sent from the server and notifies the user.

[0690] Execution notification

[0691] The device sends a push notification to the user suggesting a "10-minute stretch after breakfast."

[0692] Send feedback

[0693] The user implements the suggested actions and enters feedback on their satisfaction level and effectiveness into the device.

[0694] The device sends user feedback to the server.

[0695] In this way, the system of the present invention can provide optimal support tailored to each user's lifestyle and continuously improve its suggestions.

[0696] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0697] System program processing flow

[0698] Step 1: Data Collection

[0699] Input: User behavior data, schedule data, biometric data

[0700] Specific operation:

[0701] The device collects user activity data in real time through smartphones and wearable devices.

[0702] The device transmits the collected data to the server via the internet.

[0703] Output: User activity data sent to the server

[0704] Step 2: Data Analysis

[0705] Input: User activity data sent to the server

[0706] Specific operation:

[0707] The server analyzes the received data using statistical models and machine learning algorithms (such as Python's scikit-learn and TensorFlow).

[0708] The server analyzes each data point to identify the user's lifestyle patterns.

[0709] Output: Identification results regarding the user's lifestyle patterns

[0710] Step 3: Model Generation

[0711] Input: Identification results regarding the user's lifestyle patterns

[0712] Specific operation:

[0713] The server generates a predictive model based on the identified lifestyle patterns.

[0714] The server uses TensorFlow to train a neural network model that provides the best possible suggestions for each user.

[0715] Output: Predictive model

[0716] Step 4: Proposal Generation

[0717] Input: Predictive model

[0718] Specific operation:

[0719] The server generates specific suggestions based on the predictive model.

[0720] The server develops an action plan that is suitable for the user's lifestyle (e.g., doing 10 minutes of stretching after breakfast).

[0721] Output: Specific proposals

[0722] Step 5: Submit Proposal

[0723] Input: Specific proposal

[0724] Specific operation:

[0725] The server sends the generated suggestions to the user's terminal.

[0726] The server records communication logs to verify that the proposal was sent correctly.

[0727] Output: Suggestions sent to the user's terminal

[0728] Step 6: Receive Proposal

[0729] Input: Suggestion sent from the server

[0730] Specific operation:

[0731] The terminal saves the suggestions received from the server to its memory.

[0732] The device prepares to notify the user of the received information.

[0733] Output: Suggestions saved on the device

[0734] Step 7: Execution Notification

[0735] Input: Suggestions saved on the device

[0736] Specific operation:

[0737] The device will notify the user of specific suggestions. This will be done using smartphone push notifications or the vibration function of wearable devices.

[0738] The device adjusts the timing and display method of notifications to make them easier for the user to act upon.

[0739] Output: Suggestions notified to the user

[0740] Step 8: Submit Feedback

[0741] Input: User feedback

[0742] Specific operation:

[0743] Users execute the suggestions and input the results and feedback into their devices. For example, they might use an input form within a smartphone app.

[0744] The device sends user feedback information to the server.

[0745] Output: Feedback information and execution results sent to the server

[0746] In this way, by processing each step, a system is built that effectively provides support tailored to the user's lifestyle.

[0747] (Application Example 1)

[0748] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0749] While conventional systems could provide personalized suggestions based on users' lifestyle patterns, they lacked concrete support such as optimizing work efficiency in factory labor and reducing the burden on workers. Furthermore, dynamic suggestions based on real-time worker data and improvements to those suggestions using feedback were not adequately implemented.

[0750] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0751] In this invention, the server includes means for collecting user activity data, means for analyzing the activity data to identify the user's lifestyle pattern, means for generating individual suggestions based on the identified lifestyle pattern, means for transmitting the generated suggestions to the user's terminal, means for collecting user feedback on the suggestions, means for updating the suggestion content based on the feedback, means for the robot terminal to collect worker motion data and work speed data in real time, means for analyzing the worker's work efficiency based on the motion data and work speed data and generating appropriate work suggestions, means for transmitting the generated work suggestions to the robot terminal and notifying the worker, and means for collecting worker feedback after notification by the robot terminal. This makes it possible to optimize the work efficiency and reduce the burden on workers in factory work.

[0752] "Means for collecting user activity data" refers to devices or systems that collect data such as user behavior and biometric information.

[0753] "Means of analyzing data to identify users' lifestyle patterns" refers to the process of analyzing collected data to identify users' behavioral patterns and lifestyle habits.

[0754] "Means for generating individual suggestions" refers to devices or systems that generate specific suggestions for actions or behaviors for users based on identified lifestyle patterns.

[0755] "Means for sending generated proposals to the user's terminal" refers to a system or method for delivering proposal content to the user's terminal.

[0756] "Means for collecting user feedback" refers to devices or systems that collect user responses and results to suggestions.

[0757] "Methods for updating proposals based on feedback" refers to the process of improving and updating proposals for future versions based on the feedback collected.

[0758] "Means for robot terminals to collect worker motion data and work speed data in real time" refers to robots or devices that monitor and record the movements and work efficiency of factory workers in real time.

[0759] "Means for analyzing worker efficiency based on work data" refers to a process of analyzing collected motion data and work speed data to evaluate worker efficiency and performance.

[0760] "Means for generating appropriate work suggestions" refers to devices or systems that generate optimal actions and break suggestions for workers based on analysis results.

[0761] "Means for sending generated work proposals to a robot terminal and notifying the worker" refers to the process of sending generated work proposals to a robot terminal and notifying the worker via the robot.

[0762] "Means for collecting worker feedback after notification by a robot terminal" refers to a system or method in which a robot terminal collects worker reactions and execution results.

[0763] The embodiments for carrying out the present invention will be described in detail. The system of the present invention is designed to optimize the work efficiency of workers in factory labor and reduce their burden. This system collects user activity data and worker motion data, analyzes them to generate optimal suggestions, and provides effective support by notifying users.

[0764] System Configuration

[0765] Server side

[0766] Data collection

[0767] The server receives worker motion data and work speed data transmitted from the robot terminal. This uses built-in sensors, cameras, and RFID tag recognition technology.

[0768] Data Analysis

[0769] The server analyzes the received data to identify the worker's work patterns and efficiency. Statistical models and machine learning algorithms (e.g., Python's Scikit-Learn and TensorFlow) are used for the analysis.

[0770] Model generation

[0771] The server generates a predictive model based on the identified work patterns. This model is used to provide optimal support suggestions for each worker.

[0772] Proposal generation

[0773] The server creates specific suggestions from the generated predictive model. For example, based on worker action data, it generates suggestions such as "recommend a 10-minute break" or "increase work speed."

[0774] Submit Proposal

[0775] The server sends the generated proposal to the robot terminal.

[0776] Terminal side

[0777] Data transmission

[0778] The robot terminal transmits worker motion data and work speed data to the server in real time.

[0779] Proposal received

[0780] The robot terminal receives the proposal content sent from the server.

[0781] Execution notification

[0782] The robot terminal notifies the worker of the proposed content. Notification methods include voice guidance and display screens.

[0783] Send feedback

[0784] The worker implements the proposed solution and inputs the results and feedback into the robot terminal. The robot terminal then sends this feedback to the server.

[0785] Hardware and software to be used

[0786] Hardware: Built-in sensors, camera, RFID tag reader, robot terminal

[0787] Software: Python, Scikit-Learn, TensorFlow

[0788] Specific example

[0789] The robot terminal collects motion and work speed data from a worker in real time and sends it to a server. The server analyzes the received data and identifies if the worker is fatigued. As a result, the server uses a predictive model to generate a suggestion to "take a 10-minute break" and sends this to the robot terminal. The robot terminal notifies the worker of the suggestion by voice, and the worker takes a break as suggested. After that, the worker enters feedback into the robot terminal, and this is sent to the server.

[0790] Example of a prompt

[0791] "Based on the worker's current work speed and motion data, please suggest the optimal break times."

[0792] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0793] Step 1: Data Collection

[0794] The robot terminal collects worker motion data and work speed data in real time. It uses built-in sensors, cameras, and RFID tag readers to acquire information about each worker's actions. For example, the worker's hand movements and walking speed are measured by sensors and recorded as data on the terminal. The input is worker motion data and work speed data, and the output is the raw data recorded on the robot terminal.

[0795] Step 2: Data transmission

[0796] The robot terminal transmits collected motion data and work speed data to the server in real time. Data recorded on the terminal is transferred to the server using an internet connection or a dedicated communication protocol. The input is the raw data recorded on the terminal, and the output is the data sent to the server for analysis.

[0797] Step 3: Data Analysis

[0798] The server analyzes the received data to identify the worker's work patterns and efficiency. The analysis uses methods for normalizing the data (e.g., StandardScaler) and the KMeans algorithm for clustering. Specifically, the operational data is analyzed using Python's Scikit-Learn and TensorFlow. The input is the data to be analyzed sent to the server, and the output is information about the worker's work patterns and efficiency.

[0799] Step 4: Model Generation

[0800] The server generates a predictive model based on the identified work patterns. Here, a machine learning model is trained to create a predictive model that provides optimal support suggestions for each worker. This process involves training the model using a large amount of work data to improve its accuracy. The input is information about work patterns and efficiency, and the output is the trained predictive model.

[0801] Step 5: Proposal Generation

[0802] The server generates specific suggestions from the predictive model it has created. For example, individual suggestions such as "recommend taking a 10-minute break" or "increase work speed" are automatically generated. The input is the trained predictive model, and the output is specific support suggestions.

[0803] Step 6: Submit Proposal

[0804] The server sends the generated suggestions to the robot terminal. The suggestions are transferred to the robot terminal using an internet connection or a dedicated communication protocol. The input is the specific support suggestion, and the output is the suggestion information sent to the robot terminal.

[0805] Step 7: Receive proposal

[0806] The robot terminal receives the suggested content sent from the server. The suggested content is stored on the robot terminal and ready to be notified to the worker. The input is the suggested information sent from the server, and the output is the suggested content stored on the robot terminal.

[0807] Step 8: Execution Notification

[0808] The robot terminal notifies the worker of the suggested actions. Notification methods include voice guidance and display screens. Specifically, the robot terminal prompts the worker to take breaks or change speed through voice messages and screen displays. The input is the suggested actions stored in the robot terminal, and the output is notifications to the worker via voice and display.

[0809] Step 9: Submit Feedback

[0810] The worker implements the proposed solution and inputs the results and feedback into the robot terminal. The robot terminal sends this feedback to the server. The input is the worker's feedback, and the output is the feedback information sent to the server.

[0811] Step 10: Feedback analysis and updating of proposals

[0812] The server updates its suggestions based on the feedback it receives. By adding new data to the training dataset and retraining the predictive model, it improves the accuracy of subsequent suggestions. The input is the feedback information sent to the server, and the output is the updated predictive model and improved suggestions.

[0813] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0814] The embodiments for carrying out the present invention will be described in detail. This system collects emotional data in addition to user activity data and provides support that is optimal for the user's lifestyle by integrating and analyzing this data. The aim of this system is to generate more individualized suggestions and improve the usefulness of those suggestions by also considering the user's emotional state.

[0815] System Configuration

[0816] Server side

[0817] Data collection

[0818] The server receives activity data and sentiment data transmitted from the user's device. Activity data includes behavioral data, schedule data, and biometric data.

[0819] Collection of emotional data

[0820] The server receives emotion data from the emotion engine installed in the terminal. The emotion engine identifies the user's emotions using at least one of the following: voice analysis, facial recognition, and text analysis.

[0821] Data Analysis

[0822] The server cleanses and analyzes received activity and emotional data to identify the user's lifestyle patterns and emotional state. Machine learning algorithms are used to extract statistically significant patterns.

[0823] Model generation

[0824] The server generates a predictive model based on the analysis results. This model reflects the user's lifestyle patterns and emotional state and is used to provide personalized suggestions.

[0825] Proposal generation

[0826] The server creates specific suggestions from the generated predictive model. For example, for a user whose emotional state is stressed, this might include a suggestion to "take deep breaths to relax."

[0827] Submit Proposal

[0828] The server sends the generated suggestions to the user's terminal.

[0829] Terminal side

[0830] Data transmission

[0831] The device transmits user behavior data, schedule data, biometric data, and emotional data to the server in real time.

[0832] Emotion data generation

[0833] The emotion engine built into the device identifies emotions from the user's voice, facial expressions, and text, and generates data based on those emotions.

[0834] Proposal received

[0835] The terminal receives the proposal sent from the server and notifies the user.

[0836] Execution notification

[0837] The device notifies the user of the received suggestions. For example, it may use push notifications on a smartphone or the alert function of a wearable device to notify the user.

[0838] Send feedback

[0839] The device sends user feedback to the server. This feedback includes the results and evaluation of the proposed changes.

[0840] Specific example

[0841] Consider examples of how users can cope with everyday stress. Users record their activities and emotions using smartphones or wearable devices.

[0842] Data collection

[0843] The device sends user activity data (such as steps taken and schedule) and emotional data (such as stress levels and feelings of happiness, based on voice and facial expression analysis) to the server.

[0844] Data Analysis

[0845] The server analyzes activity and emotional data to identify when a user is more likely to experience stress.

[0846] Model generation

[0847] The server generates a predictive model based on the user's lifestyle patterns and emotional state. This model provides appropriate suggestions for stress reduction.

[0848] Proposal generation

[0849] The server generates specific suggestions, such as "take deep breaths to relax" or "take a short break."

[0850] Submit Proposal

[0851] The server sends the generated suggestions to the user's terminal.

[0852] Proposal received

[0853] The terminal receives a proposal sent from the server and notifies the user.

[0854] Execution notification

[0855] The device notifies the user to "take a deep breath."

[0856] Send feedback

[0857] The user takes a deep breath and enters the result and evaluation into the device.

[0858] The device sends user feedback to the server.

[0859] In this way, the system of the present invention can take into account the user's emotional state, provide optimal support tailored to their lifestyle, and continuously improve its suggestions.

[0860] The following describes the processing flow.

[0861] Step 1: Collect activity and emotional data

[0862] Users record their daily activities through smartphones and wearable devices, such as steps taken, heart rate, and scheduled events.

[0863] The emotion engine built into the device analyzes the user's voice, facial expressions, and text to generate emotion data. For example, it can determine the user's stress level from conversation content and changes in facial expressions.

[0864] Step 2: Data transmission

[0865] The device periodically sends user activity data and sentiment data to the server. Communication protocols such as HTTPS and MQTT are used.

[0866] Step 3: Data reception and storage

[0867] The server receives activity and sentiment data transmitted from the terminal. The received data is stored in a database.

[0868] Step 4: Data Cleansing

[0869] The server cleanses the stored data. For example, it improves the accuracy of the analysis by removing outliers and standardizing the data format.

[0870] Step 5: Data Analysis

[0871] The server uses the cleansed data to analyze the user's lifestyle patterns and emotional state. This analysis employs machine learning algorithms to extract the user's daily behavioral and emotional tendencies.

[0872] Step 6: Model Generation

[0873] The server generates a predictive model based on the analysis results. This model is used to provide optimal suggestions adapted to the user's behavioral patterns and emotional state.

[0874] Step 7: Proposal Generation

[0875] The server uses the generated predictive model to create specific suggestions for the user. For example, if stress levels are high due to prolonged desk work, it will generate recommendations such as "Take a 5-minute break every hour."

[0876] Step 8: Submit Proposal

[0877] The server sends the generated suggestions to the user's terminal.

[0878] Step 9: Receive Proposal

[0879] The terminal receives the proposal content sent from the server. The received proposal is immediately prepared to be notified to the user.

[0880] Step 10: Execution Notification

[0881] The device notifies the user based on the received suggestions. For example, it might use smartphone push notifications or wearable device alerts to notify the user of suggestions such as "take a deep breath" or "take a short walk."

[0882] Step 11: User Actions

[0883] The user checks notifications from their device and takes the suggested action. For example, they might take a 5-minute break and go for a walk or meditate.

[0884] Step 12: Feedback Record

[0885] When a user completes an action based on a suggestion, the results and evaluation are entered into the device. For example, it might record how much deep breathing or walking helped reduce stress.

[0886] Step 13: Submit Feedback

[0887] The device sends user feedback to the server.

[0888] Step 14: Update the proposal

[0889] The server updates its suggestions based on the feedback received. Newly collected data is also analyzed and incorporated into future suggestions. This allows for the provision of more optimized suggestions to users.

[0890] In this way, through a series of processing steps, we realize a system that provides optimal support considering the user's emotional state and daily behavior, and that allows for continuous improvement.

[0891] (Example 2)

[0892] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0893] Current lifestyle support systems make suggestions based solely on user activity data, making it difficult to provide personalized suggestions that take emotional states into account. This can result in insufficient assurance of the usefulness and suitability of the suggestions. Furthermore, the current system struggles to quickly collect feedback on changes in users' lifestyle patterns and emotional states, and to update the suggestions accordingly.

[0894] The identification processing performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for collecting user activity data and emotional data; means for cleansing and analyzing the activity data and emotional data to identify the user's lifestyle patterns and emotional states; means for generating a predictive model based on the identified lifestyle patterns and emotional states using a machine learning algorithm; means for generating individual suggestions based on the generated predictive model; means for transmitting the generated suggestions to the user's terminal; means for collecting user feedback on the suggestions; and means for updating the suggestion content based on the feedback. This enables integrated analysis of the user's activity data and emotional data, and provides optimal support tailored to their lifestyle and emotional state.

[0895] "Activity data" refers to information about a user's daily behavior, and includes behavioral data, schedule data, biometric data, etc.

[0896] "Emotional data" refers to information about a user's emotional state, and includes voice data, facial expression data, text data, and other similar data.

[0897] "Data cleansing" refers to processes aimed at improving data quality, such as handling missing data values ​​and detecting outliers.

[0898] "Analysis" refers to the process of identifying patterns and trends based on collected data using statistical methods and machine learning algorithms.

[0899] "Lifestyle patterns" refer to identifying tendencies and regularities in a user's daily behavior.

[0900] "Emotional state" refers to the emotions a user feels at a particular time or in a particular situation.

[0901] A "machine learning algorithm" refers to a computational method used to analyze large amounts of data, learn patterns, and perform predictions and classifications.

[0902] A "predictive model" refers to a mathematical model used to predict future behavior or states based on analyzed data.

[0903] "Individualized suggestions" refer to advice or guidelines tailored to address a user's specific situation or emotional state.

[0904] "Feedback" refers to responding to a user's suggestions by providing results and evaluations of their actions.

[0905] "Updating the proposal" refers to the process of improving the proposal based on the feedback collected.

[0906] This invention is a system that comprehensively analyzes user activity data and emotional data to provide personalized support tailored to the user's lifestyle and emotional state. The system aims to make more appropriate suggestions by taking the user's emotional state into consideration.

[0907] System Configuration

[0908] Server side

[0909] Data collection

[0910] The server has the functionality to receive activity data and emotional data transmitted from the user's terminal. Activity data includes behavioral data, schedule data, and biometric data. Emotional data includes voice data, facial expression data, and text data.

[0911] Specific hardware: Data center servers

[0912] Specific software: Apache Kafka (messaging system), InfluxDB (database)

[0913] Collection of emotional data

[0914] The server receives emotion data from the emotion engine installed in the terminal. The emotion engine uses voice analysis, facial recognition, and text analysis to identify the user's emotions.

[0915] Specific hardware: High-performance CPU / GPU server

[0916] Specific software: Python libraries (OpenCV, TensorFlow)

[0917] Data Analysis

[0918] The server cleanses and analyzes the received activity and emotional data to identify the user's lifestyle patterns and emotional states. This allows it to extract statistically significant patterns.

[0919] Specific software used: Pandas (data cleansing), Scikit-learn (machine learning)

[0920] Model generation

[0921] The server generates a predictive model based on the analysis results. This model reflects the user's lifestyle patterns and emotional state and is used to provide personalized suggestions.

[0922] Specific software: Scikit-learn, TensorFlow

[0923] Proposal generation

[0924] The server creates specific suggestions from the generated predictive model. For example, for a user whose emotional state is stressed, this might include a suggestion to "take deep breaths to relax."

[0925] Specific software: Django (Web framework)

[0926] Submit Proposal

[0927] The server sends the generated suggestions to the user's terminal.

[0928] Specific hardware: Network infrastructure

[0929] Specific software: REST API

[0930] Terminal side

[0931] Data transmission

[0932] The device transmits user behavior data, schedule data, biometric data, and emotional data to the server in real time.

[0933] Specific hardware: smartphones, wearable devices

[0934] Specific software: Android / iOS app

[0935] Emotion data generation

[0936] The emotion engine built into the device identifies emotions from the user's voice, facial expressions, and text, and generates data based on those emotions.

[0937] Specific software: Python libraries (OpenCV, TensorFlow)

[0938] Proposal received

[0939] The terminal receives the proposal sent from the server and notifies the user.

[0940] Specific software: Firebase Cloud Messaging

[0941] Execution notification

[0942] The device notifies the user of the received proposal. For example, it may use push notifications on a smartphone or the alert function of a wearable device to notify the user.

[0943] Specific hardware: smartphones, wearable devices

[0944] Specific software: Android / iOS app

[0945] Send feedback

[0946] The device sends user feedback to the server. This feedback includes the results and evaluation of the proposed changes.

[0947] Specific hardware: smartphones, wearable devices

[0948] Specific software: Android / iOS app

[0949] Specific example

[0950] Consider examples of how users can cope with everyday stress. Users record their activities and emotions using smartphones or wearable devices.

[0951] Data collection

[0952] The device sends user activity data (e.g., steps taken and schedule) and emotional data (stress and happiness levels based on voice and facial expression analysis) to the server.

[0953] Data Analysis

[0954] The server analyzes activity and emotional data to identify when users are more likely to experience stress during specific time periods.

[0955] Model generation

[0956] The server generates a predictive model based on the user's lifestyle patterns and emotional state. This model provides appropriate suggestions for stress reduction.

[0957] Proposal generation

[0958] The server generates specific suggestions such as "take deep breaths to relax" or "take a short break."

[0959] Submit Proposal

[0960] The server sends the generated suggestions to the user's terminal.

[0961] Proposal received

[0962] The terminal receives a proposal sent from the server and notifies the user.

[0963] Execution notification

[0964] The device notifies the user to "take a deep breath."

[0965] Send feedback

[0966] The user takes a deep breath and enters the result and evaluation into the device.

[0967] The device sends user feedback to the server.

[0968] In this way, the system of the present invention can take into account the user's emotional state, provide optimal support tailored to their lifestyle, and continuously improve its suggestions.

[0969] Example of a prompt

[0970] "Based on emotional and activity data collected by users over a certain period, predict when they will experience stress and generate suggestions for appropriate relaxation methods."

[0971] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0972] Step 1:

[0973] Data collection

[0974] The device transmits user activity and emotional data to the server in real time. Specifically, this includes behavioral data (steps and schedule), biometric data (heart rate and sleep information), voice data, facial expression data, and text data collected from smartphones and wearable devices.

[0975] Input: Behavioral data, schedule data, biometric data, voice data, facial expression data, text data

[0976] Output: Sending data to the server

[0977] Step 2:

[0978] Collection of emotional data

[0979] The server receives emotion data from the emotion engine installed in the terminal. The emotion engine identifies the user's emotions using speech analysis, facial recognition, and text analysis. For example, it analyzes the user's facial expressions using OpenCV and TensorFlow to determine their emotional state. This information is also sent to the server.

[0980] Input: Audio data, facial expression data, text data

[0981] Output: Sentiment data

[0982] Step 3:

[0983] Data cleansing

[0984] The server cleanses the received activity and sentiment data. Specifically, it performs data washing such as imputing missing values ​​and removing outliers to prepare a dataset suitable for analysis. The Pandas library is used in this process.

[0985] Input: Raw activity data, emotion data

[0986] Output: Cleansed data

[0987] Step 4:

[0988] Data Analysis

[0989] The server analyzes the cleansed data to identify the user's lifestyle patterns and emotional states. Specifically, it uses the Scikit-learn library to perform clustering and anomaly detection to analyze when and what emotional states the user is experiencing.

[0990] Input: Cleansed data

[0991] Output: Analysis results (lifestyle patterns, emotional state)

[0992] Step 5:

[0993] Model generation

[0994] The server generates a predictive model using machine learning algorithms based on the analysis results. Using Scikit-learn and TensorFlow, it creates a predictive model that reflects the user's lifestyle patterns and emotional state.

[0995] Input: Analysis results

[0996] Output: Predictive model

[0997] Step 6:

[0998] Proposal generation

[0999] The server generates specific suggestions based on a predictive model. For example, it might generate a "suggestion for deep breathing to relax" during times when the user is likely to feel stressed. The suggestions are built using the Django framework.

[1000] Input: Predictive model

[1001] Output: Specific proposals

[1002] Step 7:

[1003] Submit Proposal

[1004] The server sends the generated suggestions to the user's device. Suggestions are sent in real time using a REST API.

[1005] Input: Specific proposal

[1006] Output: Sending suggestions to the terminal

[1007] Step 8:

[1008] Proposal received

[1009] The device receives suggestions sent from the server and notifies the user. Firebase Cloud Messaging is used to provide push notifications and alerts.

[1010] Input: Suggestion from the server

[1011] Output: Notification to the user

[1012] Step 9:

[1013] Execution notification

[1014] The device sends notifications to the user prompting them to take action on the suggested actions. For example, a smartphone might receive a notification instructing it to "take a deep breath before the next meeting."

[1015] Input: Received proposals

[1016] Output: Execution notification to the user

[1017] Step 10:

[1018] Send feedback

[1019] The terminal sends user feedback to the server. The user inputs the results and evaluation of the suggested actions into the terminal, and this information is sent to the server.

[1020] Input: User feedback

[1021] Output: Sending feedback to the server

[1022] These processing steps enable the integrated analysis of user activity and emotional data, allowing for the provision of optimal support tailored to their lifestyle and emotional state.

[1023] (Application Example 2)

[1024] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[1025] Traditional user support systems typically made suggestions based on user activity data. However, these systems failed to consider the user's emotional state and were unable to adequately respond to temporary emotions or psychological conditions, limiting the usefulness of their suggestions. In particular, existing systems failed to provide sufficient support in situations where a rapid response was required when the user was experiencing negative emotions such as anxiety or fear.

[1026] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[1027] In this invention, the server includes means for collecting user activity data and emotional data, means for analyzing the activity data and emotional data to identify the user's lifestyle patterns and emotional state, and means for generating personalized suggestions based on the identified lifestyle patterns and emotional state. This enables the prompt and appropriate safety measures to be suggested when the user feels anxiety or fear.

[1028] "Activity data" refers to information about a user's daily activities, such as location information, activity history, and schedule data.

[1029] "Emotional data" refers to information about a user's emotional state, analyzed from their voice data, text data, and facial expression data.

[1030] A "device" is an electronic device that a user possesses and uses to send, receive, and display information, and includes smartphones and wearable devices.

[1031] "Feedback" refers to the reactions and evaluations that users give to suggestions they receive.

[1032] A "machine learning algorithm" is a computational method used by computers to learn patterns from data and perform predictions and classifications.

[1033] A "suggestion" refers to specific instructions or advice regarding actions or measures for the user, generated based on the analyzed data.

[1034] This invention provides a system that collects and analyzes user activity data and emotional data, generates personalized suggestions, and transmits them to the user's device. Specific embodiments for carrying out this invention are described below.

[1035] System Configuration

[1036] Server side

[1037] Data collection:

[1038] The server receives activity data and emotion data transmitted from the user's device. This activity data includes location information, activity history, and schedule data. Emotion data consists of voice data, text data, and facial expression data.

[1039] Collection of emotional data:

[1040] The server receives emotion data from the emotion engine installed in the terminal. This emotion engine identifies the user's emotions using at least one of the following: voice analysis, facial expression recognition, and text analysis.

[1041] Data analysis:

[1042] The server cleanses and analyzes received activity and emotional data to identify the user's lifestyle patterns and emotional state. Machine learning algorithms are used to extract statistically significant patterns.

[1043] Model generation:

[1044] The server generates a predictive model based on the analysis results. This model reflects the user's lifestyle patterns and emotional state and is used to provide personalized suggestions.

[1045] Suggestion generation:

[1046] The server creates specific suggestions from the generated predictive model. For example, for a user whose emotional state is one of anxiety or fear, these suggestions might include "location of the nearest police station" or "instructions to choose a safe route."

[1047] Submit proposal:

[1048] The server sends the generated suggestions to the user's device.

[1049] Terminal side

[1050] Data transmission:

[1051] The device transmits user behavior data, schedule data, biometric data, and emotional data to the server in real time.

[1052] Emotion data generation:

[1053] The emotion engine built into the device identifies emotions from the user's voice, facial expressions, and text, and generates data based on those emotions.

[1054] Proposal received:

[1055] The terminal receives the proposal sent from the server and notifies the user.

[1056] Execution notification:

[1057] The device notifies the user of the received suggestions. For example, it may use push notifications on a smartphone or the alert function of a wearable device to notify the user.

[1058] Send feedback:

[1059] The device sends user feedback to the server. This feedback includes the results and evaluation of the proposed changes.

[1060] Specific example

[1061] Consider specific examples of how users can cope with the anxiety and fear they experience on a daily basis. Users record their activities and emotions using smartphones or wearable devices. The device sends the user's activity data (e.g., location information and activity history) and emotional data (e.g., anxiety and fear based on voice and facial expression analysis) to a server. The server analyzes the activity and emotional data to identify when the user is likely to feel anxiety or fear. The server generates a predictive model based on the user's lifestyle patterns and emotional state. This model provides appropriate suggestions for reducing anxiety and fear. The server generates specific suggestions, such as "location of the nearest police station" or "instructions for choosing a safe route." The server sends the generated suggestions to the user's device. The device receives the suggestions from the server and notifies the user. The user receives the notification, acts according to the suggestions, and inputs the results and evaluation into the device. The device sends the user's feedback to the server.

[1062] Example of a prompt

[1063] "Analyze the user's voice and text data to extract their current emotional state. If they are feeling anxious or frightened, generate a notification suggesting the location of the nearest police station based on their current location."

[1064] In this way, the system of the present invention can take into account the user's emotional state, provide optimal support tailored to their lifestyle, and continuously improve its suggestions.

[1065] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1066] Step 1:

[1067] The user's device collects user activity data (location information, activity history, schedule data) and emotional data (voice data, text data, facial expression data) in real time. This data is processed using an emotion engine installed in the device and sent to the server. The input is the user's activity data and emotional data, and the output is the transmission of this data to the server.

[1068] Step 2:

[1069] The server receives activity and sentiment data sent from the terminal. The received data is cleansed, and any missing or inaccurate data is corrected or supplemented. Next, the data is ready for analysis. The input is the activity and sentiment data sent from the terminal, and the output is the cleansed data.

[1070] Step 3:

[1071] The server analyzes cleansed activity and emotional data. This analysis uses machine learning algorithms to extract statistically significant patterns. The user's lifestyle patterns and emotional states are identified. The input is the cleansed data, and the output is the identified lifestyle patterns and emotional states.

[1072] Step 4:

[1073] The server uses an AI model based on the analysis results to generate a predictive model. This model reflects the user's lifestyle patterns and emotional state and is used to provide personalized suggestions. The input is the analysis results, and the output is the predictive model.

[1074] Step 5:

[1075] The server generates specific suggestions from the predictive model it has created. These suggestions are customized according to the user's current emotional state and lifestyle. For example, if the user is feeling anxious or fearful, the suggestions might include the location of the nearest police station and directions to a safe route. The input is the predictive model, and the output is the specific suggestions.

[1076] Step 6:

[1077] The server sends the generated suggestions to the user's terminal. The input is the specific suggestions, and the output is the suggestions sent to the user's terminal.

[1078] Step 7:

[1079] The device receives suggestions sent from the server and notifies the user. This notification is delivered using smartphone push notifications or the alert function of a wearable device. The input is the suggestion content sent from the server, and the output is the information notified to the user.

[1080] Step 8:

[1081] The user acts according to the received suggestions and inputs the results and evaluations into the device. This includes feedback on how helpful the suggestions were and the actions taken. The input is the user's feedback, and the output is the state in which the feedback has been entered into the device.

[1082] Step 9:

[1083] The terminal sends user feedback to the server. The server receives this feedback and uses it to update the suggestions. The input is the user feedback, and the output is the feedback sent to the server.

[1084] Step 10:

[1085] The server updates the suggestions based on the feedback received. This makes future suggestions more accurate and useful to the user. The input is the user's feedback, and the output is the updated suggestions.

[1086] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1087] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1088] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[1089] [Third Embodiment]

[1090] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[1091] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1092] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1093] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[1094] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1095] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1096] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1097] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1098] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1099] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1100] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1101] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[1102] The embodiments for carrying out the present invention will be described in detail. The system of the present invention collects user activity data, analyzes that data to identify the user's lifestyle pattern, generates individual suggestions based on the identified lifestyle pattern, and transmits those suggestions to the user's terminal. By collecting feedback from the user and updating the suggestions based on that feedback, the system provides appropriate and personalized suggestions.

[1103] System Configuration

[1104] Server side

[1105] Data collection

[1106] The server receives activity data transmitted from the user's device. This activity data includes behavioral data, schedule data, and biometric data.

[1107] Data Analysis

[1108] The server analyzes the received data to identify the user's lifestyle patterns. Statistical models and machine learning algorithms are used for this analysis.

[1109] Model generation

[1110] The server generates a predictive model based on the identified lifestyle patterns. This model is used to provide optimal support suggestions for each user.

[1111] Proposal generation

[1112] The server creates specific suggestions from the generated predictive model. For example, it might generate suggestions such as "do some stretching" or "take a 30-minute walk" based on the user's exercise habits.

[1113] Submit Proposal

[1114] The server sends the generated suggestions to the user's terminal.

[1115] Terminal side

[1116] Data transmission

[1117] The device transmits user behavior data and schedules to the server in real time.

[1118] Proposal received

[1119] The terminal receives the proposal content sent from the server.

[1120] Execution notification

[1121] The device notifies the user of the proposed content. For example, notifications are sent using smartphone push notifications or the alert function of wearable devices.

[1122] Send feedback

[1123] The device sends user feedback to the server. This feedback includes evaluations of suggestions and the results of their implementation.

[1124] Specific example

[1125] Consider a user who wakes up at 6 AM to their smartphone alarm and has breakfast at 7 AM. This user spends most of their day doing desk work and tends to be sedentary.

[1126] Data collection

[1127] The device sends the user's wake-up time and meal times to the server.

[1128] Data Analysis

[1129] The server analyzes the user's behavioral data and identifies that they are not getting enough exercise.

[1130] Model generation

[1131] The server generates a model for introducing exercise habits that are suitable for the user.

[1132] Proposal generation

[1133] The server generates specific suggestions, such as "We suggest a 10-minute stretching session after breakfast."

[1134] Submit Proposal

[1135] The server sends the proposal to the user's terminal.

[1136] Proposal received

[1137] The terminal receives a proposal sent from the server and notifies the user.

[1138] Execution notification

[1139] The device sends a push notification to the user suggesting a "10-minute stretch after breakfast."

[1140] Send feedback

[1141] The user implements the suggested actions and enters feedback on their satisfaction level and effectiveness into the device.

[1142] The device sends user feedback to the server.

[1143] In this way, the system of the present invention can provide optimal support tailored to each user's lifestyle and continuously improve its suggestions.

[1144] The following describes the processing flow.

[1145] Step 1: Data Collection

[1146] Users record their daily activities and schedules using smartphones or wearable devices. For example, they accumulate data by entering steps, heart rate, and appointments into their calendars.

[1147] The device periodically transmits recorded data to a server. This data includes behavioral data, schedule data, and biometric data.

[1148] Step 2: Data Reception

[1149] The server receives activity data sent from the terminal. The received data is temporarily held in a buffer.

[1150] Step 3: Data Storage

[1151] The server stores the received data in a database. During this process, the data is organized by user to facilitate efficient analysis later.

[1152] Step 4: Data Cleansing

[1153] The server cleanses the stored data. This process involves deleting unnecessary data and standardizing the format. For example, it removes outliers and standardizes data formats.

[1154] Step 5: Data Analysis

[1155] The server analyzes the cleansed data. Here, statistical models and machine learning algorithms are used to identify user lifestyle patterns. For example, it extracts patterns such as whether or not a user exercises at a specific time.

[1156] Step 6: Model Generation

[1157] The server generates a predictive model based on the analysis results. This model is used to reflect each user's behavior and preferences and provide optimal suggestions.

[1158] Step 7: Proposal Generation

[1159] The server creates specific suggestions for the user based on the predictive model it has generated. For example, based on the user's lifestyle patterns, it might create suggestions such as "do 15 minutes of stretching" or "we recommend a 30-minute walk."

[1160] Step 8: Submit Proposal

[1161] The server sends the generated suggestions to the user's terminal.

[1162] Step 9: Receive Proposal

[1163] The device receives suggestions sent from the server. The received suggestions are displayed in the device's notification center or within the app.

[1164] Step 10: Execution Notification

[1165] The device will notify the user based on the received suggestions. For example, it may use push notifications on a smartphone or the alert function of a wearable device to send notifications.

[1166] Step 11: User Actions

[1167] The user checks notifications from their device and takes the suggested action. For example, they might start stretching or eat a suggested healthy breakfast.

[1168] Step 12: Feedback Record

[1169] When a user takes action based on a suggestion, the results and evaluation are entered into the device. For example, feedback such as "I liked the breakfast menu" or "I went for a 30-minute walk" can be entered.

[1170] Step 13: Submit Feedback

[1171] The device sends user feedback to the server. This feedback is used as training data to inform future suggestions.

[1172] Step 14: Update the proposal

[1173] The server updates its suggestions based on user feedback. It analyzes the feedback data and makes new action suggestions or adjusts existing ones.

[1174] (Example 1)

[1175] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1176] In modern society, many people face health risks due to unhealthy lifestyle habits. Lack of exercise, stress, and sleep deprivation are particularly serious problems. Conventional systems struggle to provide optimal lifestyle improvement suggestions to individual users, and personalized support that takes into account each user's characteristics is not adequately offered. Therefore, there is a need for a system that effectively supports health maintenance and lifestyle improvement by collecting and analyzing user activity data to generate individualized suggestions.

[1177] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1178] In this invention, the server includes means for collecting user activity data, means for analyzing the activity data to identify the user's lifestyle patterns, means for generating a predictive model based on the identified lifestyle patterns, means for generating individual suggestions from the generated predictive model, means for transmitting the generated suggestions to the user's computer, means for collecting user feedback on the suggestions, and means for updating the suggestion content based on the feedback. This enables optimal suggestions based on each user's detailed lifestyle patterns, effectively supporting the user's health maintenance and improvement of lifestyle habits.

[1179] "User activity data" refers to data that shows the user's behavior and physiological state in their daily life, and includes behavioral data, schedule data, and biometric data.

[1180] "Analysis" refers to the process of analyzing collected activity data using statistical models and machine learning algorithms to identify specific patterns and trends.

[1181] "Lifestyle patterns" refer to data and characteristics that show the regularity of a user's habits and behaviors in their daily life.

[1182] A "predictive model" refers to a mathematical or algorithmic model used to predict a user's future behavior or state based on their lifestyle patterns.

[1183] "Proposal" refers to specific action plans and advice provided to users based on predictive models.

[1184] "Feedback" refers to the information that users send back to the server regarding the results and evaluations of their suggestions.

[1185] "Update" refers to the process of improving and refining the suggested content and predictive models based on feedback received from users.

[1186] The system of the present invention collects user activity data and analyzes that data to identify the user's lifestyle patterns. Based on the identified lifestyle patterns, it generates individual suggestions and sends these suggestions to the user's terminal. Furthermore, it collects feedback from the user and updates the suggestions based on that feedback to provide appropriate and personalized suggestions.

[1187] System Configuration

[1188] Server side

[1189] Data collection

[1190] The server receives activity data transmitted from the user's device. This activity data includes behavioral data, schedule data, and biometric data. Specific hardware used includes smartphones and wearable devices.

[1191] Data Analysis

[1192] The server analyzes the received data to identify the user's lifestyle patterns. Machine learning algorithms such as Python's scikit-learn and TensorFlow are used for the analysis. This allows the server to determine, for example, whether the user is sedentary or needs to adopt a new exercise habit.

[1193] Model generation

[1194] The server generates a predictive model based on identified lifestyle patterns. This generated model is used to provide optimal suggestions for each user. For example, it might use TensorFlow to generate a neural network model.

[1195] Proposal generation

[1196] The server creates specific suggestions from the generated predictive model. For example, based on the user's exercise habits, it might generate suggestions such as "Do 10 minutes of stretching after breakfast" or "We recommend a 30-minute walk."

[1197] Submit Proposal

[1198] The server sends the generated suggestions to the user's terminal. This communication utilizes cloud services via the internet.

[1199] Terminal side

[1200] Data transmission

[1201] The device transmits user behavior data and schedules to the server in real time. Specifically, it uses data collected from sensors in smartphones and wearable devices.

[1202] Proposal received

[1203] The terminal receives the proposal content sent from the server. The received proposal is stored in the terminal's memory.

[1204] Execution notification

[1205] The device notifies the user of the suggested actions. For example, notifications are sent using smartphone push notifications or the alert function of wearable devices. The timing and display method of notifications are adjusted to make it easy for the user to take action.

[1206] Send feedback

[1207] Users implement the suggestions and provide feedback on their satisfaction level and effectiveness via their device. For example, they might use an input form provided as a smartphone app.

[1208] The device sends user feedback to the server. The feedback data is analyzed on the server and used to update suggestions and predictive models.

[1209] Specific example

[1210] Consider a user who wakes up at 6 AM to their smartphone alarm and has breakfast at 7 AM. This user spends most of their day doing desk work and tends to be sedentary.

[1211] Data collection

[1212] The device sends the user's wake-up time and meal times to the server.

[1213] Data Analysis

[1214] The server analyzes the user's behavioral data and identifies that they are not getting enough exercise.

[1215] Model generation

[1216] The server generates a model for introducing exercise habits that are suitable for the user.

[1217] Proposal generation

[1218] The server generates specific suggestions, such as "We suggest a 10-minute stretching session after breakfast."

[1219] Submit Proposal

[1220] The server sends the proposal to the user's terminal.

[1221] Proposal received

[1222] The terminal receives a proposal sent from the server and notifies the user.

[1223] Execution notification

[1224] The device sends a push notification to the user suggesting a "10-minute stretch after breakfast."

[1225] Send feedback

[1226] The user implements the suggested actions and enters feedback on their satisfaction level and effectiveness into the device.

[1227] The device sends user feedback to the server.

[1228] In this way, the system of the present invention can provide optimal support tailored to each user's lifestyle and continuously improve its suggestions.

[1229] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1230] System program processing flow

[1231] Step 1: Data Collection

[1232] Input: User behavior data, schedule data, biometric data

[1233] Specific operation:

[1234] The device collects user activity data in real time through smartphones and wearable devices.

[1235] The device transmits the collected data to the server via the internet.

[1236] Output: User activity data sent to the server

[1237] Step 2: Data Analysis

[1238] Input: User activity data sent to the server

[1239] Specific operation:

[1240] The server analyzes the received data using statistical models and machine learning algorithms (such as Python's scikit-learn and TensorFlow).

[1241] The server analyzes each data point to identify the user's lifestyle patterns.

[1242] Output: Identification results regarding the user's lifestyle patterns

[1243] Step 3: Model Generation

[1244] Input: Identification results regarding the user's lifestyle patterns

[1245] Specific operation:

[1246] The server generates a predictive model based on the identified lifestyle patterns.

[1247] The server uses TensorFlow to train a neural network model that provides the best possible suggestions for each user.

[1248] Output: Predictive model

[1249] Step 4: Proposal Generation

[1250] Input: Predictive model

[1251] Specific operation:

[1252] The server generates specific suggestions based on the predictive model.

[1253] The server develops an action plan that is suitable for the user's lifestyle (e.g., doing 10 minutes of stretching after breakfast).

[1254] Output: Specific proposals

[1255] Step 5: Submit Proposal

[1256] Input: Specific proposal

[1257] Specific operation:

[1258] The server sends the generated suggestions to the user's terminal.

[1259] The server records communication logs to verify that the proposal was sent correctly.

[1260] Output: Suggestions sent to the user's terminal

[1261] Step 6: Receive Proposal

[1262] Input: Suggestion sent from the server

[1263] Specific operation:

[1264] The terminal saves the suggestions received from the server to its memory.

[1265] The device prepares to notify the user of the received information.

[1266] Output: Suggestions saved on the device

[1267] Step 7: Execution Notification

[1268] Input: Suggestions saved on the device

[1269] Specific operation:

[1270] The device will notify the user of specific suggestions. This will be done using smartphone push notifications or the vibration function of wearable devices.

[1271] The device adjusts the timing and display method of notifications to make them easier for the user to act upon.

[1272] Output: Suggestions notified to the user

[1273] Step 8: Submit Feedback

[1274] Input: User feedback

[1275] Specific operation:

[1276] Users execute the suggestions and input the results and feedback into their devices. For example, they might use an input form within a smartphone app.

[1277] The device sends user feedback information to the server.

[1278] Output: Feedback information and execution results sent to the server

[1279] In this way, by processing each step, a system is built that effectively provides support tailored to the user's lifestyle.

[1280] (Application Example 1)

[1281] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1282] While conventional systems could provide personalized suggestions based on users' lifestyle patterns, they lacked concrete support such as optimizing work efficiency in factory labor and reducing the burden on workers. Furthermore, dynamic suggestions based on real-time worker data and improvements to those suggestions using feedback were not adequately implemented.

[1283] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1284] In this invention, the server includes means for collecting user activity data, means for analyzing the activity data to identify the user's lifestyle pattern, means for generating individual suggestions based on the identified lifestyle pattern, means for transmitting the generated suggestions to the user's terminal, means for collecting user feedback on the suggestions, means for updating the suggestion content based on the feedback, means for the robot terminal to collect worker motion data and work speed data in real time, means for analyzing the worker's work efficiency based on the motion data and work speed data and generating appropriate work suggestions, means for transmitting the generated work suggestions to the robot terminal and notifying the worker, and means for collecting worker feedback after notification by the robot terminal. This makes it possible to optimize the work efficiency and reduce the burden on workers in factory work.

[1285] "Means for collecting user activity data" refers to devices or systems that collect data such as user behavior and biometric information.

[1286] "Means of analyzing data to identify users' lifestyle patterns" refers to the process of analyzing collected data to identify users' behavioral patterns and lifestyle habits.

[1287] "Means for generating individual suggestions" refers to devices or systems that generate specific suggestions for actions or behaviors for users based on identified lifestyle patterns.

[1288] "Means for sending generated proposals to the user's terminal" refers to a system or method for delivering proposal content to the user's terminal.

[1289] "Means for collecting user feedback" refers to devices or systems that collect user responses and results to suggestions.

[1290] "Methods for updating proposals based on feedback" refers to the process of improving and updating proposals for future versions based on the feedback collected.

[1291] "Means for robot terminals to collect worker motion data and work speed data in real time" refers to robots or devices that monitor and record the movements and work efficiency of factory workers in real time.

[1292] "Means for analyzing worker efficiency based on work data" refers to a process of analyzing collected motion data and work speed data to evaluate worker efficiency and performance.

[1293] "Means for generating appropriate work suggestions" refers to devices or systems that generate optimal actions and break suggestions for workers based on analysis results.

[1294] "Means for sending generated work proposals to a robot terminal and notifying the worker" refers to the process of sending generated work proposals to a robot terminal and notifying the worker via the robot.

[1295] "Means for collecting worker feedback after notification by a robot terminal" refers to a system or method in which a robot terminal collects worker reactions and execution results.

[1296] The embodiments for carrying out the present invention will be described in detail. The system of the present invention is designed to optimize the work efficiency of workers in factory labor and reduce their burden. This system collects user activity data and worker motion data, analyzes them to generate optimal suggestions, and provides effective support by notifying users.

[1297] System Configuration

[1298] Server side

[1299] Data collection

[1300] The server receives worker motion data and work speed data transmitted from the robot terminal. This uses built-in sensors, cameras, and RFID tag recognition technology.

[1301] Data Analysis

[1302] The server analyzes the received data to identify the worker's work patterns and efficiency. Statistical models and machine learning algorithms (e.g., Python's Scikit-Learn and TensorFlow) are used for the analysis.

[1303] Model generation

[1304] The server generates a predictive model based on the identified work patterns. This model is used to provide optimal support suggestions for each worker.

[1305] Proposal generation

[1306] The server creates specific suggestions from the generated predictive model. For example, based on worker action data, it generates suggestions such as "recommend a 10-minute break" or "increase work speed."

[1307] Submit Proposal

[1308] The server sends the generated proposal to the robot terminal.

[1309] Terminal side

[1310] Data transmission

[1311] The robot terminal transmits worker motion data and work speed data to the server in real time.

[1312] Proposal received

[1313] The robot terminal receives the proposal content sent from the server.

[1314] Execution notification

[1315] The robot terminal notifies the worker of the proposed content. Notification methods include voice guidance and display screens.

[1316] Send feedback

[1317] The worker implements the proposed solution and inputs the results and feedback into the robot terminal. The robot terminal then sends this feedback to the server.

[1318] Hardware and software to be used

[1319] Hardware: Built-in sensors, camera, RFID tag reader, robot terminal

[1320] Software: Python, Scikit-Learn, TensorFlow

[1321] Specific example

[1322] The robot terminal collects motion and work speed data from a worker in real time and sends it to a server. The server analyzes the received data and identifies if the worker is fatigued. As a result, the server uses a predictive model to generate a suggestion to "take a 10-minute break" and sends this to the robot terminal. The robot terminal notifies the worker of the suggestion by voice, and the worker takes a break as suggested. After that, the worker enters feedback into the robot terminal, and this is sent to the server.

[1323] Example of a prompt

[1324] "Based on the worker's current work speed and motion data, please suggest the optimal break times."

[1325] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1326] Step 1: Data Collection

[1327] The robot terminal collects worker motion data and work speed data in real time. It uses built-in sensors, cameras, and RFID tag readers to acquire information about each worker's actions. For example, the worker's hand movements and walking speed are measured by sensors and recorded as data on the terminal. The input is worker motion data and work speed data, and the output is the raw data recorded on the robot terminal.

[1328] Step 2: Data transmission

[1329] The robot terminal transmits collected motion data and work speed data to the server in real time. Data recorded on the terminal is transferred to the server using an internet connection or a dedicated communication protocol. The input is the raw data recorded on the terminal, and the output is the data sent to the server for analysis.

[1330] Step 3: Data Analysis

[1331] The server analyzes the received data to identify the worker's work patterns and efficiency. The analysis uses methods for normalizing the data (e.g., StandardScaler) and the KMeans algorithm for clustering. Specifically, the operational data is analyzed using Python's Scikit-Learn and TensorFlow. The input is the data to be analyzed sent to the server, and the output is information about the worker's work patterns and efficiency.

[1332] Step 4: Model Generation

[1333] The server generates a predictive model based on the identified work patterns. Here, a machine learning model is trained to create a predictive model that provides optimal support suggestions for each worker. This process involves training the model using a large amount of work data to improve its accuracy. The input is information about work patterns and efficiency, and the output is the trained predictive model.

[1334] Step 5: Proposal Generation

[1335] The server generates specific suggestions from the predictive model it has created. For example, individual suggestions such as "recommend taking a 10-minute break" or "increase work speed" are automatically generated. The input is the trained predictive model, and the output is specific support suggestions.

[1336] Step 6: Submit Proposal

[1337] The server sends the generated suggestions to the robot terminal. The suggestions are transferred to the robot terminal using an internet connection or a dedicated communication protocol. The input is the specific support suggestion, and the output is the suggestion information sent to the robot terminal.

[1338] Step 7: Receive proposal

[1339] The robot terminal receives the suggested content sent from the server. The suggested content is stored on the robot terminal and ready to be notified to the worker. The input is the suggested information sent from the server, and the output is the suggested content stored on the robot terminal.

[1340] Step 8: Execution Notification

[1341] The robot terminal notifies the worker of the suggested actions. Notification methods include voice guidance and display screens. Specifically, the robot terminal prompts the worker to take breaks or change speed through voice messages and screen displays. The input is the suggested actions stored in the robot terminal, and the output is notifications to the worker via voice and display.

[1342] Step 9: Submit Feedback

[1343] The worker implements the proposed solution and inputs the results and feedback into the robot terminal. The robot terminal sends this feedback to the server. The input is the worker's feedback, and the output is the feedback information sent to the server.

[1344] Step 10: Feedback analysis and updating of proposals

[1345] The server updates its suggestions based on the feedback it receives. By adding new data to the training dataset and retraining the predictive model, it improves the accuracy of subsequent suggestions. The input is the feedback information sent to the server, and the output is the updated predictive model and improved suggestions.

[1346] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1347] The embodiments for carrying out the present invention will be described in detail. This system collects emotional data in addition to user activity data and provides support that is optimal for the user's lifestyle by integrating and analyzing this data. The aim of this system is to generate more individualized suggestions and improve the usefulness of those suggestions by also considering the user's emotional state.

[1348] System Configuration

[1349] Server side

[1350] Data collection

[1351] The server receives activity data and sentiment data transmitted from the user's device. Activity data includes behavioral data, schedule data, and biometric data.

[1352] Collection of emotional data

[1353] The server receives emotion data from the emotion engine installed in the terminal. The emotion engine identifies the user's emotions using at least one of the following: voice analysis, facial recognition, and text analysis.

[1354] Data Analysis

[1355] The server cleanses and analyzes received activity and emotional data to identify the user's lifestyle patterns and emotional state. Machine learning algorithms are used to extract statistically significant patterns.

[1356] Model generation

[1357] The server generates a predictive model based on the analysis results. This model reflects the user's lifestyle patterns and emotional state and is used to provide personalized suggestions.

[1358] Proposal generation

[1359] The server creates specific suggestions from the generated predictive model. For example, for a user whose emotional state is stressed, this might include a suggestion to "take deep breaths to relax."

[1360] Submit Proposal

[1361] The server sends the generated suggestions to the user's terminal.

[1362] Terminal side

[1363] Data transmission

[1364] The device transmits user behavior data, schedule data, biometric data, and emotional data to the server in real time.

[1365] Emotion data generation

[1366] The emotion engine built into the device identifies emotions from the user's voice, facial expressions, and text, and generates data based on those emotions.

[1367] Proposal received

[1368] The terminal receives the proposal sent from the server and notifies the user.

[1369] Execution notification

[1370] The device notifies the user of the received suggestions. For example, it may use push notifications on a smartphone or the alert function of a wearable device to notify the user.

[1371] Send feedback

[1372] The device sends user feedback to the server. This feedback includes the results and evaluation of the proposed changes.

[1373] Specific example

[1374] Consider examples of how users can cope with everyday stress. Users record their activities and emotions using smartphones or wearable devices.

[1375] Data collection

[1376] The device sends user activity data (such as steps taken and schedule) and emotional data (such as stress levels and feelings of happiness, based on voice and facial expression analysis) to the server.

[1377] Data Analysis

[1378] The server analyzes activity and emotional data to identify when a user is more likely to experience stress.

[1379] Model generation

[1380] The server generates a predictive model based on the user's lifestyle patterns and emotional state. This model provides appropriate suggestions for stress reduction.

[1381] Proposal generation

[1382] The server generates specific suggestions, such as "take deep breaths to relax" or "take a short break."

[1383] Submit Proposal

[1384] The server sends the generated suggestions to the user's terminal.

[1385] Proposal received

[1386] The terminal receives a proposal sent from the server and notifies the user.

[1387] Execution notification

[1388] The device notifies the user to "take a deep breath."

[1389] Send feedback

[1390] The user takes a deep breath and enters the result and evaluation into the device.

[1391] The device sends user feedback to the server.

[1392] In this way, the system of the present invention can take into account the user's emotional state, provide optimal support tailored to their lifestyle, and continuously improve its suggestions.

[1393] The following describes the processing flow.

[1394] Step 1: Collect activity and emotional data

[1395] Users record their daily activities through smartphones and wearable devices, such as steps taken, heart rate, and scheduled events.

[1396] The emotion engine built into the device analyzes the user's voice, facial expressions, and text to generate emotion data. For example, it can determine the user's stress level from conversation content and changes in facial expressions.

[1397] Step 2: Data transmission

[1398] The device periodically sends user activity data and sentiment data to the server. Communication protocols such as HTTPS and MQTT are used.

[1399] Step 3: Data reception and storage

[1400] The server receives activity and sentiment data transmitted from the terminal. The received data is stored in a database.

[1401] Step 4: Data Cleansing

[1402] The server cleanses the stored data. For example, it improves the accuracy of the analysis by removing outliers and standardizing the data format.

[1403] Step 5: Data Analysis

[1404] The server uses the cleansed data to analyze the user's lifestyle patterns and emotional state. This analysis employs machine learning algorithms to extract the user's daily behavioral and emotional tendencies.

[1405] Step 6: Model Generation

[1406] The server generates a predictive model based on the analysis results. This model is used to provide optimal suggestions adapted to the user's behavioral patterns and emotional state.

[1407] Step 7: Proposal Generation

[1408] The server uses the generated predictive model to create specific suggestions for the user. For example, if stress levels are high due to prolonged desk work, it will generate recommendations such as "Take a 5-minute break every hour."

[1409] Step 8: Submit Proposal

[1410] The server sends the generated suggestions to the user's terminal.

[1411] Step 9: Receive Proposal

[1412] The terminal receives the proposal content sent from the server. The received proposal is immediately prepared to be notified to the user.

[1413] Step 10: Execution Notification

[1414] The device notifies the user based on the received suggestions. For example, it might use smartphone push notifications or wearable device alerts to notify the user of suggestions such as "take a deep breath" or "take a short walk."

[1415] Step 11: User Actions

[1416] The user checks notifications from their device and takes the suggested action. For example, they might take a 5-minute break and go for a walk or meditate.

[1417] Step 12: Feedback Record

[1418] When a user completes an action based on a suggestion, the results and evaluation are entered into the device. For example, it might record how much deep breathing or walking helped reduce stress.

[1419] Step 13: Submit Feedback

[1420] The device sends user feedback to the server.

[1421] Step 14: Update the proposal

[1422] The server updates its suggestions based on the feedback received. Newly collected data is also analyzed and incorporated into future suggestions. This allows for the provision of more optimized suggestions to users.

[1423] In this way, through a series of processing steps, we realize a system that provides optimal support considering the user's emotional state and daily behavior, and that allows for continuous improvement.

[1424] (Example 2)

[1425] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1426] Current lifestyle support systems make suggestions based solely on user activity data, making it difficult to provide personalized suggestions that take emotional states into account. This can result in insufficient assurance of the usefulness and suitability of the suggestions. Furthermore, the current system struggles to quickly collect feedback on changes in users' lifestyle patterns and emotional states, and to update the suggestions accordingly.

[1427] The identification processing performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for collecting user activity data and emotional data; means for cleansing and analyzing the activity data and emotional data to identify the user's lifestyle patterns and emotional states; means for generating a predictive model based on the identified lifestyle patterns and emotional states using a machine learning algorithm; means for generating individual suggestions based on the generated predictive model; means for transmitting the generated suggestions to the user's terminal; means for collecting user feedback on the suggestions; and means for updating the suggestion content based on the feedback. This enables integrated analysis of the user's activity data and emotional data, and provides optimal support tailored to their lifestyle and emotional state.

[1428] "Activity data" refers to information about a user's daily behavior, and includes behavioral data, schedule data, biometric data, etc.

[1429] "Emotional data" refers to information about a user's emotional state, and includes voice data, facial expression data, text data, and other similar data.

[1430] "Data cleansing" refers to processes aimed at improving data quality, such as handling missing data values ​​and detecting outliers.

[1431] "Analysis" refers to the process of identifying patterns and trends based on collected data using statistical methods and machine learning algorithms.

[1432] "Lifestyle patterns" refer to identifying tendencies and regularities in a user's daily behavior.

[1433] "Emotional state" refers to the emotions a user feels at a particular time or in a particular situation.

[1434] A "machine learning algorithm" refers to a computational method used to analyze large amounts of data, learn patterns, and perform predictions and classifications.

[1435] A "predictive model" refers to a mathematical model used to predict future behavior or states based on analyzed data.

[1436] "Individualized suggestions" refer to advice or guidelines tailored to address a user's specific situation or emotional state.

[1437] "Feedback" refers to responding to a user's suggestions by providing results and evaluations of their actions.

[1438] "Updating the proposal" refers to the process of improving the proposal based on the feedback collected.

[1439] This invention is a system that comprehensively analyzes user activity data and emotional data to provide personalized support tailored to the user's lifestyle and emotional state. The system aims to make more appropriate suggestions by taking the user's emotional state into consideration.

[1440] System Configuration

[1441] Server side

[1442] Data collection

[1443] The server has the functionality to receive activity data and emotional data transmitted from the user's terminal. Activity data includes behavioral data, schedule data, and biometric data. Emotional data includes voice data, facial expression data, and text data.

[1444] Specific hardware: Data center servers

[1445] Specific software: Apache Kafka (messaging system), InfluxDB (database)

[1446] Collection of emotional data

[1447] The server receives emotion data from the emotion engine installed in the terminal. The emotion engine uses voice analysis, facial recognition, and text analysis to identify the user's emotions.

[1448] Specific hardware: High-performance CPU / GPU server

[1449] Specific software: Python libraries (OpenCV, TensorFlow)

[1450] Data Analysis

[1451] The server cleanses and analyzes the received activity and emotional data to identify the user's lifestyle patterns and emotional states. This allows it to extract statistically significant patterns.

[1452] Specific software used: Pandas (data cleansing), Scikit-learn (machine learning)

[1453] Model generation

[1454] The server generates a predictive model based on the analysis results. This model reflects the user's lifestyle patterns and emotional state and is used to provide personalized suggestions.

[1455] Specific software: Scikit-learn, TensorFlow

[1456] Proposal generation

[1457] The server creates specific suggestions from the generated predictive model. For example, for a user whose emotional state is stressed, this might include a suggestion to "take deep breaths to relax."

[1458] Specific software: Django (Web framework)

[1459] Submit Proposal

[1460] The server sends the generated suggestions to the user's terminal.

[1461] Specific hardware: Network infrastructure

[1462] Specific software: REST API

[1463] Terminal side

[1464] Data transmission

[1465] The device transmits user behavior data, schedule data, biometric data, and emotional data to the server in real time.

[1466] Specific hardware: smartphones, wearable devices

[1467] Specific software: Android / iOS app

[1468] Emotion data generation

[1469] The emotion engine built into the device identifies emotions from the user's voice, facial expressions, and text, and generates data based on those emotions.

[1470] Specific software: Python libraries (OpenCV, TensorFlow)

[1471] Proposal received

[1472] The terminal receives the proposal sent from the server and notifies the user.

[1473] Specific software: Firebase Cloud Messaging

[1474] Execution notification

[1475] The device notifies the user of the received proposal. For example, it may use push notifications on a smartphone or the alert function of a wearable device to notify the user.

[1476] Specific hardware: smartphones, wearable devices

[1477] Specific software: Android / iOS app

[1478] Send feedback

[1479] The device sends user feedback to the server. This feedback includes the results and evaluation of the proposed changes.

[1480] Specific hardware: smartphones, wearable devices

[1481] Specific software: Android / iOS app

[1482] Specific example

[1483] Consider examples of how users can cope with everyday stress. Users record their activities and emotions using smartphones or wearable devices.

[1484] Data collection

[1485] The device sends user activity data (e.g., steps taken and schedule) and emotional data (stress and happiness levels based on voice and facial expression analysis) to the server.

[1486] Data Analysis

[1487] The server analyzes activity and emotional data to identify when users are more likely to experience stress during specific time periods.

[1488] Model generation

[1489] The server generates a predictive model based on the user's lifestyle patterns and emotional state. This model provides appropriate suggestions for stress reduction.

[1490] Proposal generation

[1491] The server generates specific suggestions such as "take deep breaths to relax" or "take a short break."

[1492] Submit Proposal

[1493] The server sends the generated suggestions to the user's terminal.

[1494] Proposal received

[1495] The terminal receives a proposal sent from the server and notifies the user.

[1496] Execution notification

[1497] The device notifies the user to "take a deep breath."

[1498] Send feedback

[1499] The user takes a deep breath and enters the result and evaluation into the device.

[1500] The device sends user feedback to the server.

[1501] In this way, the system of the present invention can take into account the user's emotional state, provide optimal support tailored to their lifestyle, and continuously improve its suggestions.

[1502] Example of a prompt

[1503] "Based on emotional and activity data collected by users over a certain period, predict when they will experience stress and generate suggestions for appropriate relaxation methods."

[1504] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1505] Step 1:

[1506] Data collection

[1507] The device transmits user activity and emotional data to the server in real time. Specifically, this includes behavioral data (steps and schedule), biometric data (heart rate and sleep information), voice data, facial expression data, and text data collected from smartphones and wearable devices.

[1508] Input: Behavioral data, schedule data, biometric data, voice data, facial expression data, text data

[1509] Output: Sending data to the server

[1510] Step 2:

[1511] Collection of emotional data

[1512] The server receives emotion data from the emotion engine installed in the terminal. The emotion engine identifies the user's emotions using speech analysis, facial recognition, and text analysis. For example, it analyzes the user's facial expressions using OpenCV and TensorFlow to determine their emotional state. This information is also sent to the server.

[1513] Input: Audio data, facial expression data, text data

[1514] Output: Sentiment data

[1515] Step 3:

[1516] Data cleansing

[1517] The server cleanses the received activity and sentiment data. Specifically, it performs data washing such as imputing missing values ​​and removing outliers to prepare a dataset suitable for analysis. The Pandas library is used in this process.

[1518] Input: Raw activity data, emotion data

[1519] Output: Cleansed data

[1520] Step 4:

[1521] Data Analysis

[1522] The server analyzes the cleansed data to identify the user's lifestyle patterns and emotional states. Specifically, it uses the Scikit-learn library to perform clustering and anomaly detection to analyze when and what emotional states the user is experiencing.

[1523] Input: Cleansed data

[1524] Output: Analysis results (lifestyle patterns, emotional state)

[1525] Step 5:

[1526] Model generation

[1527] The server generates a predictive model using machine learning algorithms based on the analysis results. Using Scikit-learn and TensorFlow, it creates a predictive model that reflects the user's lifestyle patterns and emotional state.

[1528] Input: Analysis results

[1529] Output: Predictive model

[1530] Step 6:

[1531] Proposal generation

[1532] The server generates specific suggestions based on a predictive model. For example, it might generate a "suggestion for deep breathing to relax" during times when the user is likely to feel stressed. The suggestions are built using the Django framework.

[1533] Input: Predictive model

[1534] Output: Specific proposals

[1535] Step 7:

[1536] Submit Proposal

[1537] The server sends the generated suggestions to the user's device. Suggestions are sent in real time using a REST API.

[1538] Input: Specific proposal

[1539] Output: Sending suggestions to the terminal

[1540] Step 8:

[1541] Proposal received

[1542] The device receives suggestions sent from the server and notifies the user. Firebase Cloud Messaging is used to provide push notifications and alerts.

[1543] Input: Suggestion from the server

[1544] Output: Notification to the user

[1545] Step 9:

[1546] Execution notification

[1547] The device sends notifications to the user prompting them to take action on the suggested actions. For example, a smartphone might receive a notification instructing it to "take a deep breath before the next meeting."

[1548] Input: Received proposals

[1549] Output: Execution notification to the user

[1550] Step 10:

[1551] Send feedback

[1552] The terminal sends user feedback to the server. The user inputs the results and evaluation of the suggested actions into the terminal, and this information is sent to the server.

[1553] Input: User feedback

[1554] Output: Sending feedback to the server

[1555] These processing steps enable the integrated analysis of user activity and emotional data, allowing for the provision of optimal support tailored to their lifestyle and emotional state.

[1556] (Application Example 2)

[1557] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1558] Traditional user support systems typically made suggestions based on user activity data. However, these systems failed to consider the user's emotional state and were unable to adequately respond to temporary emotions or psychological conditions, limiting the usefulness of their suggestions. In particular, existing systems failed to provide sufficient support in situations where a rapid response was required when the user was experiencing negative emotions such as anxiety or fear.

[1559] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[1560] In this invention, the server includes means for collecting user activity data and emotional data, means for analyzing the activity data and emotional data to identify the user's lifestyle patterns and emotional state, and means for generating personalized suggestions based on the identified lifestyle patterns and emotional state. This enables the prompt and appropriate safety measures to be suggested when the user feels anxiety or fear.

[1561] "Activity data" refers to information about a user's daily activities, such as location information, activity history, and schedule data.

[1562] "Emotional data" refers to information about a user's emotional state, analyzed from their voice data, text data, and facial expression data.

[1563] A "device" is an electronic device that a user possesses and uses to send, receive, and display information, and includes smartphones and wearable devices.

[1564] "Feedback" refers to the reactions and evaluations that users give to suggestions they receive.

[1565] A "machine learning algorithm" is a computational method used by computers to learn patterns from data and perform predictions and classifications.

[1566] A "suggestion" refers to specific instructions or advice regarding actions or measures for the user, generated based on the analyzed data.

[1567] This invention provides a system that collects and analyzes user activity data and emotional data, generates personalized suggestions, and transmits them to the user's device. Specific embodiments for carrying out this invention are described below.

[1568] System Configuration

[1569] Server side

[1570] Data collection:

[1571] The server receives activity data and emotion data transmitted from the user's device. This activity data includes location information, activity history, and schedule data. Emotion data consists of voice data, text data, and facial expression data.

[1572] Collection of emotional data:

[1573] The server receives emotion data from the emotion engine installed in the terminal. This emotion engine identifies the user's emotions using at least one of the following: voice analysis, facial expression recognition, and text analysis.

[1574] Data analysis:

[1575] The server cleanses and analyzes received activity and emotional data to identify the user's lifestyle patterns and emotional state. Machine learning algorithms are used to extract statistically significant patterns.

[1576] Model generation:

[1577] The server generates a predictive model based on the analysis results. This model reflects the user's lifestyle patterns and emotional state and is used to provide personalized suggestions.

[1578] Suggestion generation:

[1579] The server creates specific suggestions from the generated predictive model. For example, for a user whose emotional state is one of anxiety or fear, these suggestions might include "location of the nearest police station" or "instructions to choose a safe route."

[1580] Submit proposal:

[1581] The server sends the generated suggestions to the user's device.

[1582] Terminal side

[1583] Data transmission:

[1584] The device transmits user behavior data, schedule data, biometric data, and emotional data to the server in real time.

[1585] Emotion data generation:

[1586] The emotion engine built into the device identifies emotions from the user's voice, facial expressions, and text, and generates data based on those emotions.

[1587] Proposal received:

[1588] The terminal receives the proposal sent from the server and notifies the user.

[1589] Execution notification:

[1590] The device notifies the user of the received suggestions. For example, it may use push notifications on a smartphone or the alert function of a wearable device to notify the user.

[1591] Send feedback:

[1592] The device sends user feedback to the server. This feedback includes the results and evaluation of the proposed changes.

[1593] Specific example

[1594] Consider specific examples of how users can cope with the anxiety and fear they experience on a daily basis. Users record their activities and emotions using smartphones or wearable devices. The device sends the user's activity data (e.g., location information and activity history) and emotional data (e.g., anxiety and fear based on voice and facial expression analysis) to a server. The server analyzes the activity and emotional data to identify when the user is likely to feel anxiety or fear. The server generates a predictive model based on the user's lifestyle patterns and emotional state. This model provides appropriate suggestions for reducing anxiety and fear. The server generates specific suggestions, such as "location of the nearest police station" or "instructions for choosing a safe route." The server sends the generated suggestions to the user's device. The device receives the suggestions from the server and notifies the user. The user receives the notification, acts according to the suggestions, and inputs the results and evaluation into the device. The device sends the user's feedback to the server.

[1595] Example of a prompt

[1596] "Analyze the user's voice and text data to extract their current emotional state. If they are feeling anxious or frightened, generate a notification suggesting the location of the nearest police station based on their current location."

[1597] In this way, the system of the present invention can take into account the user's emotional state, provide optimal support tailored to their lifestyle, and continuously improve its suggestions.

[1598] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1599] Step 1:

[1600] The user's device collects user activity data (location information, activity history, schedule data) and emotional data (voice data, text data, facial expression data) in real time. This data is processed using an emotion engine installed in the device and sent to the server. The input is the user's activity data and emotional data, and the output is the transmission of this data to the server.

[1601] Step 2:

[1602] The server receives activity and sentiment data sent from the terminal. The received data is cleansed, and any missing or inaccurate data is corrected or supplemented. Next, the data is ready for analysis. The input is the activity and sentiment data sent from the terminal, and the output is the cleansed data.

[1603] Step 3:

[1604] The server analyzes cleansed activity and emotional data. This analysis uses machine learning algorithms to extract statistically significant patterns. The user's lifestyle patterns and emotional states are identified. The input is the cleansed data, and the output is the identified lifestyle patterns and emotional states.

[1605] Step 4:

[1606] The server uses an AI model based on the analysis results to generate a predictive model. This model reflects the user's lifestyle patterns and emotional state and is used to provide personalized suggestions. The input is the analysis results, and the output is the predictive model.

[1607] Step 5:

[1608] The server generates specific suggestions from the predictive model it has created. These suggestions are customized according to the user's current emotional state and lifestyle. For example, if the user is feeling anxious or fearful, the suggestions might include the location of the nearest police station and directions to a safe route. The input is the predictive model, and the output is the specific suggestions.

[1609] Step 6:

[1610] The server sends the generated suggestions to the user's terminal. The input is the specific suggestions, and the output is the suggestions sent to the user's terminal.

[1611] Step 7:

[1612] The device receives suggestions sent from the server and notifies the user. This notification is delivered using smartphone push notifications or the alert function of a wearable device. The input is the suggestion content sent from the server, and the output is the information notified to the user.

[1613] Step 8:

[1614] The user acts according to the received suggestions and inputs the results and evaluations into the device. This includes feedback on how helpful the suggestions were and the actions taken. The input is the user's feedback, and the output is the state in which the feedback has been entered into the device.

[1615] Step 9:

[1616] The terminal sends user feedback to the server. The server receives this feedback and uses it to update the suggestions. The input is the user feedback, and the output is the feedback sent to the server.

[1617] Step 10:

[1618] The server updates the suggestions based on the feedback received. This makes future suggestions more accurate and useful to the user. The input is the user's feedback, and the output is the updated suggestions.

[1619] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1620] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1621] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[1622] [Fourth Embodiment]

[1623] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[1624] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1625] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1626] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[1627] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1628] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1629] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1630] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[1631] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1632] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1633] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1634] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1635] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1636] The embodiments for carrying out the present invention will be described in detail. The system of the present invention collects user activity data, analyzes that data to identify the user's lifestyle pattern, generates individual suggestions based on the identified lifestyle pattern, and transmits those suggestions to the user's terminal. By collecting feedback from the user and updating the suggestions based on that feedback, the system provides appropriate and personalized suggestions.

[1637] System Configuration

[1638] Server side

[1639] Data collection

[1640] The server receives activity data transmitted from the user's device. This activity data includes behavioral data, schedule data, and biometric data.

[1641] Data Analysis

[1642] The server analyzes the received data to identify the user's lifestyle patterns. Statistical models and machine learning algorithms are used for this analysis.

[1643] Model generation

[1644] The server generates a predictive model based on the identified lifestyle patterns. This model is used to provide optimal support suggestions for each user.

[1645] Proposal generation

[1646] The server creates specific suggestions from the generated predictive model. For example, it might generate suggestions such as "do some stretching" or "take a 30-minute walk" based on the user's exercise habits.

[1647] Submit Proposal

[1648] The server sends the generated suggestions to the user's terminal.

[1649] Terminal side

[1650] Data transmission

[1651] The device transmits user behavior data and schedules to the server in real time.

[1652] Proposal received

[1653] The terminal receives the proposal content sent from the server.

[1654] Execution notification

[1655] The device notifies the user of the proposed content. For example, notifications are sent using smartphone push notifications or the alert function of wearable devices.

[1656] Send feedback

[1657] The device sends user feedback to the server. This feedback includes evaluations of suggestions and the results of their implementation.

[1658] Specific example

[1659] Consider a user who wakes up at 6 AM to their smartphone alarm and has breakfast at 7 AM. This user spends most of their day doing desk work and tends to be sedentary.

[1660] Data collection

[1661] The device sends the user's wake-up time and meal times to the server.

[1662] Data Analysis

[1663] The server analyzes the user's behavioral data and identifies that they are not getting enough exercise.

[1664] Model generation

[1665] The server generates a model for introducing exercise habits that are suitable for the user.

[1666] Proposal generation

[1667] The server generates specific suggestions, such as "We suggest a 10-minute stretching session after breakfast."

[1668] Submit Proposal

[1669] The server sends the proposal to the user's terminal.

[1670] Proposal received

[1671] The terminal receives a proposal sent from the server and notifies the user.

[1672] Execution notification

[1673] The device sends a push notification to the user suggesting a "10-minute stretch after breakfast."

[1674] Send feedback

[1675] The user implements the suggested actions and enters feedback on their satisfaction level and effectiveness into the device.

[1676] The device sends user feedback to the server.

[1677] In this way, the system of the present invention can provide optimal support tailored to each user's lifestyle and continuously improve its suggestions.

[1678] The following describes the processing flow.

[1679] Step 1: Data Collection

[1680] Users record their daily activities and schedules using smartphones or wearable devices. For example, they accumulate data by entering steps, heart rate, and appointments into their calendars.

[1681] The device periodically transmits recorded data to a server. This data includes behavioral data, schedule data, and biometric data.

[1682] Step 2: Data Reception

[1683] The server receives activity data sent from the terminal. The received data is temporarily held in a buffer.

[1684] Step 3: Data Storage

[1685] The server stores the received data in a database. During this process, the data is organized by user to facilitate efficient analysis later.

[1686] Step 4: Data Cleansing

[1687] The server cleanses the stored data. This process involves deleting unnecessary data and standardizing the format. For example, it removes outliers and standardizes data formats.

[1688] Step 5: Data Analysis

[1689] The server analyzes the cleansed data. Here, statistical models and machine learning algorithms are used to identify user lifestyle patterns. For example, it extracts patterns such as whether or not a user exercises at a specific time.

[1690] Step 6: Model Generation

[1691] The server generates a predictive model based on the analysis results. This model is used to reflect each user's behavior and preferences and provide optimal suggestions.

[1692] Step 7: Proposal Generation

[1693] The server creates specific suggestions for the user based on the predictive model it has generated. For example, based on the user's lifestyle patterns, it might create suggestions such as "do 15 minutes of stretching" or "we recommend a 30-minute walk."

[1694] Step 8: Submit Proposal

[1695] The server sends the generated suggestions to the user's terminal.

[1696] Step 9: Receive Proposal

[1697] The device receives suggestions sent from the server. The received suggestions are displayed in the device's notification center or within the app.

[1698] Step 10: Execution Notification

[1699] The device will notify the user based on the received suggestions. For example, it may use push notifications on a smartphone or the alert function of a wearable device to send notifications.

[1700] Step 11: User Actions

[1701] The user checks notifications from their device and takes the suggested action. For example, they might start stretching or eat a suggested healthy breakfast.

[1702] Step 12: Feedback Record

[1703] When a user takes action based on a suggestion, the results and evaluation are entered into the device. For example, feedback such as "I liked the breakfast menu" or "I went for a 30-minute walk" can be entered.

[1704] Step 13: Submit Feedback

[1705] The device sends user feedback to the server. This feedback is used as training data to inform future suggestions.

[1706] Step 14: Update the proposal

[1707] The server updates its suggestions based on user feedback. It analyzes the feedback data and makes new action suggestions or adjusts existing ones.

[1708] (Example 1)

[1709] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1710] In modern society, many people face health risks due to unhealthy lifestyle habits. Lack of exercise, stress, and sleep deprivation are particularly serious problems. Conventional systems struggle to provide optimal lifestyle improvement suggestions to individual users, and personalized support that takes into account each user's characteristics is not adequately offered. Therefore, there is a need for a system that effectively supports health maintenance and lifestyle improvement by collecting and analyzing user activity data to generate individualized suggestions.

[1711] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1712] In this invention, the server includes means for collecting user activity data, means for analyzing the activity data to identify the user's lifestyle patterns, means for generating a predictive model based on the identified lifestyle patterns, means for generating individual suggestions from the generated predictive model, means for transmitting the generated suggestions to the user's computer, means for collecting user feedback on the suggestions, and means for updating the suggestion content based on the feedback. This enables optimal suggestions based on each user's detailed lifestyle patterns, effectively supporting the user's health maintenance and improvement of lifestyle habits.

[1713] "User activity data" refers to data that shows the user's behavior and physiological state in their daily life, and includes behavioral data, schedule data, and biometric data.

[1714] "Analysis" refers to the process of analyzing collected activity data using statistical models and machine learning algorithms to identify specific patterns and trends.

[1715] "Lifestyle patterns" refer to data and characteristics that show the regularity of a user's habits and behaviors in their daily life.

[1716] A "predictive model" refers to a mathematical or algorithmic model used to predict a user's future behavior or state based on their lifestyle patterns.

[1717] "Proposal" refers to specific action plans and advice provided to users based on predictive models.

[1718] "Feedback" refers to the information that users send back to the server regarding the results and evaluations of their suggestions.

[1719] "Update" refers to the process of improving and refining the suggested content and predictive models based on feedback received from users.

[1720] The system of the present invention collects user activity data and analyzes that data to identify the user's lifestyle patterns. Based on the identified lifestyle patterns, it generates individual suggestions and sends these suggestions to the user's terminal. Furthermore, it collects feedback from the user and updates the suggestions based on that feedback to provide appropriate and personalized suggestions.

[1721] System Configuration

[1722] Server side

[1723] Data collection

[1724] The server receives activity data transmitted from the user's device. This activity data includes behavioral data, schedule data, and biometric data. Specific hardware used includes smartphones and wearable devices.

[1725] Data Analysis

[1726] The server analyzes the received data to identify the user's lifestyle patterns. Machine learning algorithms such as Python's scikit-learn and TensorFlow are used for the analysis. This allows the server to determine, for example, whether the user is sedentary or needs to adopt a new exercise habit.

[1727] Model generation

[1728] The server generates a predictive model based on identified lifestyle patterns. This generated model is used to provide optimal suggestions for each user. For example, it might use TensorFlow to generate a neural network model.

[1729] Proposal generation

[1730] The server creates specific suggestions from the generated predictive model. For example, based on the user's exercise habits, it might generate suggestions such as "Do 10 minutes of stretching after breakfast" or "We recommend a 30-minute walk."

[1731] Submit Proposal

[1732] The server sends the generated suggestions to the user's terminal. This communication utilizes cloud services via the internet.

[1733] Terminal side

[1734] Data transmission

[1735] The device transmits user behavior data and schedules to the server in real time. Specifically, it uses data collected from sensors in smartphones and wearable devices.

[1736] Proposal received

[1737] The terminal receives the proposal content sent from the server. The received proposal is stored in the terminal's memory.

[1738] Execution notification

[1739] The device notifies the user of the suggested actions. For example, notifications are sent using smartphone push notifications or the alert function of wearable devices. The timing and display method of notifications are adjusted to make it easy for the user to take action.

[1740] Send feedback

[1741] Users implement the suggestions and provide feedback on their satisfaction level and effectiveness via their device. For example, they might use an input form provided as a smartphone app.

[1742] The device sends user feedback to the server. The feedback data is analyzed on the server and used to update suggestions and predictive models.

[1743] Specific example

[1744] Consider a user who wakes up at 6 AM to their smartphone alarm and has breakfast at 7 AM. This user spends most of their day doing desk work and tends to be sedentary.

[1745] Data collection

[1746] The device sends the user's wake-up time and meal times to the server.

[1747] Data Analysis

[1748] The server analyzes the user's behavioral data and identifies that they are not getting enough exercise.

[1749] Model generation

[1750] The server generates a model for introducing exercise habits that are suitable for the user.

[1751] Proposal generation

[1752] The server generates specific suggestions, such as "We suggest a 10-minute stretching session after breakfast."

[1753] Submit Proposal

[1754] The server sends the proposal to the user's terminal.

[1755] Proposal received

[1756] The terminal receives a proposal sent from the server and notifies the user.

[1757] Execution notification

[1758] The device sends a push notification to the user suggesting a "10-minute stretch after breakfast."

[1759] Send feedback

[1760] The user implements the suggested actions and enters feedback on their satisfaction level and effectiveness into the device.

[1761] The device sends user feedback to the server.

[1762] In this way, the system of the present invention can provide optimal support tailored to each user's lifestyle and continuously improve its suggestions.

[1763] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1764] System program processing flow

[1765] Step 1: Data Collection

[1766] Input: User behavior data, schedule data, biometric data

[1767] Specific operation:

[1768] The device collects user activity data in real time through smartphones and wearable devices.

[1769] The device transmits the collected data to the server via the internet.

[1770] Output: User activity data sent to the server

[1771] Step 2: Data Analysis

[1772] Input: User activity data sent to the server

[1773] Specific operation:

[1774] The server analyzes the received data using statistical models and machine learning algorithms (such as Python's scikit-learn and TensorFlow).

[1775] The server analyzes each data point to identify the user's lifestyle patterns.

[1776] Output: Identification results regarding the user's lifestyle patterns

[1777] Step 3: Model Generation

[1778] Input: Identification results regarding the user's lifestyle patterns

[1779] Specific operation:

[1780] The server generates a predictive model based on the identified lifestyle patterns.

[1781] The server uses TensorFlow to train a neural network model that provides the best possible suggestions for each user.

[1782] Output: Predictive model

[1783] Step 4: Proposal Generation

[1784] Input: Predictive model

[1785] Specific operation:

[1786] The server generates specific suggestions based on the predictive model.

[1787] The server develops an action plan that is suitable for the user's lifestyle (e.g., doing 10 minutes of stretching after breakfast).

[1788] Output: Specific proposals

[1789] Step 5: Submit Proposal

[1790] Input: Specific proposal

[1791] Specific operation:

[1792] The server sends the generated suggestions to the user's terminal.

[1793] The server records communication logs to verify that the proposal was sent correctly.

[1794] Output: Suggestions sent to the user's terminal

[1795] Step 6: Receive Proposal

[1796] Input: Suggestion sent from the server

[1797] Specific operation:

[1798] The terminal saves the suggestions received from the server to its memory.

[1799] The device prepares to notify the user of the received information.

[1800] Output: Suggestions saved on the device

[1801] Step 7: Execution Notification

[1802] Input: Suggestions saved on the device

[1803] Specific operation:

[1804] The device will notify the user of specific suggestions. This will be done using smartphone push notifications or the vibration function of wearable devices.

[1805] The device adjusts the timing and display method of notifications to make them easier for the user to act upon.

[1806] Output: Suggestions notified to the user

[1807] Step 8: Submit Feedback

[1808] Input: User feedback

[1809] Specific operation:

[1810] Users execute the suggestions and input the results and feedback into their devices. For example, they might use an input form within a smartphone app.

[1811] The device sends user feedback information to the server.

[1812] Output: Feedback information and execution results sent to the server

[1813] In this way, by processing each step, a system is built that effectively provides support tailored to the user's lifestyle.

[1814] (Application Example 1)

[1815] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1816] While conventional systems could provide personalized suggestions based on users' lifestyle patterns, they lacked concrete support such as optimizing work efficiency in factory labor and reducing the burden on workers. Furthermore, dynamic suggestions based on real-time worker data and improvements to those suggestions using feedback were not adequately implemented.

[1817] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1818] In this invention, the server includes means for collecting user activity data, means for analyzing the activity data to identify the user's lifestyle pattern, means for generating individual suggestions based on the identified lifestyle pattern, means for transmitting the generated suggestions to the user's terminal, means for collecting user feedback on the suggestions, means for updating the suggestion content based on the feedback, means for the robot terminal to collect worker motion data and work speed data in real time, means for analyzing the worker's work efficiency based on the motion data and work speed data and generating appropriate work suggestions, means for transmitting the generated work suggestions to the robot terminal and notifying the worker, and means for collecting worker feedback after notification by the robot terminal. This makes it possible to optimize the work efficiency and reduce the burden on workers in factory work.

[1819] "Means for collecting user activity data" refers to devices or systems that collect data such as user behavior and biometric information.

[1820] "Means of analyzing data to identify users' lifestyle patterns" refers to the process of analyzing collected data to identify users' behavioral patterns and lifestyle habits.

[1821] "Means for generating individual suggestions" refers to devices or systems that generate specific suggestions for actions or behaviors for users based on identified lifestyle patterns.

[1822] "Means for sending generated proposals to the user's terminal" refers to a system or method for delivering proposal content to the user's terminal.

[1823] "Means for collecting user feedback" refers to devices or systems that collect user responses and results to suggestions.

[1824] "Methods for updating proposals based on feedback" refers to the process of improving and updating proposals for future versions based on the feedback collected.

[1825] "Means for robot terminals to collect worker motion data and work speed data in real time" refers to robots or devices that monitor and record the movements and work efficiency of factory workers in real time.

[1826] "Means for analyzing worker efficiency based on work data" refers to a process of analyzing collected motion data and work speed data to evaluate worker efficiency and performance.

[1827] "Means for generating appropriate work suggestions" refers to devices or systems that generate optimal actions and break suggestions for workers based on analysis results.

[1828] "Means for sending generated work proposals to a robot terminal and notifying the worker" refers to the process of sending generated work proposals to a robot terminal and notifying the worker via the robot.

[1829] "Means for collecting worker feedback after notification by a robot terminal" refers to a system or method in which a robot terminal collects worker reactions and execution results.

[1830] The embodiments for carrying out the present invention will be described in detail. The system of the present invention is designed to optimize the work efficiency of workers in factory labor and reduce their burden. This system collects user activity data and worker motion data, analyzes them to generate optimal suggestions, and provides effective support by notifying users.

[1831] System Configuration

[1832] Server side

[1833] Data collection

[1834] The server receives worker motion data and work speed data transmitted from the robot terminal. This uses built-in sensors, cameras, and RFID tag recognition technology.

[1835] Data Analysis

[1836] The server analyzes the received data to identify the worker's work patterns and efficiency. Statistical models and machine learning algorithms (e.g., Python's Scikit-Learn and TensorFlow) are used for the analysis.

[1837] Model generation

[1838] The server generates a predictive model based on the identified work patterns. This model is used to provide optimal support suggestions for each worker.

[1839] Proposal generation

[1840] The server creates specific suggestions from the generated predictive model. For example, based on worker action data, it generates suggestions such as "recommend a 10-minute break" or "increase work speed."

[1841] Submit Proposal

[1842] The server sends the generated proposal to the robot terminal.

[1843] Terminal side

[1844] Data transmission

[1845] The robot terminal transmits worker motion data and work speed data to the server in real time.

[1846] Proposal received

[1847] The robot terminal receives the proposal content sent from the server.

[1848] Execution notification

[1849] The robot terminal notifies the worker of the proposed content. Notification methods include voice guidance and display screens.

[1850] Send feedback

[1851] The worker implements the proposed solution and inputs the results and feedback into the robot terminal. The robot terminal then sends this feedback to the server.

[1852] Hardware and software to be used

[1853] Hardware: Built-in sensors, camera, RFID tag reader, robot terminal

[1854] Software: Python, Scikit-Learn, TensorFlow

[1855] Specific example

[1856] The robot terminal collects motion and work speed data from a worker in real time and sends it to a server. The server analyzes the received data and identifies if the worker is fatigued. As a result, the server uses a predictive model to generate a suggestion to "take a 10-minute break" and sends this to the robot terminal. The robot terminal notifies the worker of the suggestion by voice, and the worker takes a break as suggested. After that, the worker enters feedback into the robot terminal, and this is sent to the server.

[1857] Example of a prompt

[1858] "Based on the worker's current work speed and motion data, please suggest the optimal break times."

[1859] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1860] Step 1: Data Collection

[1861] The robot terminal collects worker motion data and work speed data in real time. It uses built-in sensors, cameras, and RFID tag readers to acquire information about each worker's actions. For example, the worker's hand movements and walking speed are measured by sensors and recorded as data on the terminal. The input is worker motion data and work speed data, and the output is the raw data recorded on the robot terminal.

[1862] Step 2: Data transmission

[1863] The robot terminal transmits collected motion data and work speed data to the server in real time. Data recorded on the terminal is transferred to the server using an internet connection or a dedicated communication protocol. The input is the raw data recorded on the terminal, and the output is the data sent to the server for analysis.

[1864] Step 3: Data Analysis

[1865] The server analyzes the received data to identify the worker's work patterns and efficiency. The analysis uses methods for normalizing the data (e.g., StandardScaler) and the KMeans algorithm for clustering. Specifically, the operational data is analyzed using Python's Scikit-Learn and TensorFlow. The input is the data to be analyzed sent to the server, and the output is information about the worker's work patterns and efficiency.

[1866] Step 4: Model Generation

[1867] The server generates a predictive model based on the identified work patterns. Here, a machine learning model is trained to create a predictive model that provides optimal support suggestions for each worker. This process involves training the model using a large amount of work data to improve its accuracy. The input is information about work patterns and efficiency, and the output is the trained predictive model.

[1868] Step 5: Proposal Generation

[1869] The server generates specific suggestions from the predictive model it has created. For example, individual suggestions such as "recommend taking a 10-minute break" or "increase work speed" are automatically generated. The input is the trained predictive model, and the output is specific support suggestions.

[1870] Step 6: Submit Proposal

[1871] The server sends the generated suggestions to the robot terminal. The suggestions are transferred to the robot terminal using an internet connection or a dedicated communication protocol. The input is the specific support suggestion, and the output is the suggestion information sent to the robot terminal.

[1872] Step 7: Receive proposal

[1873] The robot terminal receives the suggested content sent from the server. The suggested content is stored on the robot terminal and ready to be notified to the worker. The input is the suggested information sent from the server, and the output is the suggested content stored on the robot terminal.

[1874] Step 8: Execution Notification

[1875] The robot terminal notifies the worker of the suggested actions. Notification methods include voice guidance and display screens. Specifically, the robot terminal prompts the worker to take breaks or change speed through voice messages and screen displays. The input is the suggested actions stored in the robot terminal, and the output is notifications to the worker via voice and display.

[1876] Step 9: Submit Feedback

[1877] The worker implements the proposed solution and inputs the results and feedback into the robot terminal. The robot terminal sends this feedback to the server. The input is the worker's feedback, and the output is the feedback information sent to the server.

[1878] Step 10: Feedback analysis and updating of proposals

[1879] The server updates its suggestions based on the feedback it receives. By adding new data to the training dataset and retraining the predictive model, it improves the accuracy of subsequent suggestions. The input is the feedback information sent to the server, and the output is the updated predictive model and improved suggestions.

[1880] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1881] The embodiments for carrying out the present invention will be described in detail. This system collects emotional data in addition to user activity data and provides support that is optimal for the user's lifestyle by integrating and analyzing this data. The aim of this system is to generate more individualized suggestions and improve the usefulness of those suggestions by also considering the user's emotional state.

[1882] System Configuration

[1883] Server side

[1884] Data collection

[1885] The server receives activity data and sentiment data transmitted from the user's device. Activity data includes behavioral data, schedule data, and biometric data.

[1886] Collection of emotional data

[1887] The server receives emotion data from the emotion engine installed in the terminal. The emotion engine identifies the user's emotions using at least one of the following: voice analysis, facial recognition, and text analysis.

[1888] Data Analysis

[1889] The server cleanses and analyzes received activity and emotional data to identify the user's lifestyle patterns and emotional state. Machine learning algorithms are used to extract statistically significant patterns.

[1890] Model generation

[1891] The server generates a predictive model based on the analysis results. This model reflects the user's lifestyle patterns and emotional state and is used to provide personalized suggestions.

[1892] Proposal generation

[1893] The server creates specific suggestions from the generated predictive model. For example, for a user whose emotional state is stressed, this might include a suggestion to "take deep breaths to relax."

[1894] Submit Proposal

[1895] The server sends the generated suggestions to the user's terminal.

[1896] Terminal side

[1897] Data transmission

[1898] The device transmits user behavior data, schedule data, biometric data, and emotional data to the server in real time.

[1899] Emotion data generation

[1900] The emotion engine built into the device identifies emotions from the user's voice, facial expressions, and text, and generates data based on those emotions.

[1901] Proposal received

[1902] The terminal receives the proposal sent from the server and notifies the user.

[1903] Execution notification

[1904] The device notifies the user of the received suggestions. For example, it may use push notifications on a smartphone or the alert function of a wearable device to notify the user.

[1905] Send feedback

[1906] The device sends user feedback to the server. This feedback includes the results and evaluation of the proposed changes.

[1907] Specific example

[1908] Consider examples of how users can cope with everyday stress. Users record their activities and emotions using smartphones or wearable devices.

[1909] Data collection

[1910] The device sends user activity data (such as steps taken and schedule) and emotional data (such as stress levels and feelings of happiness, based on voice and facial expression analysis) to the server.

[1911] Data Analysis

[1912] The server analyzes activity and emotional data to identify when a user is more likely to experience stress.

[1913] Model generation

[1914] The server generates a predictive model based on the user's lifestyle patterns and emotional state. This model provides appropriate suggestions for stress reduction.

[1915] Proposal generation

[1916] The server generates specific suggestions, such as "take deep breaths to relax" or "take a short break."

[1917] Submit Proposal

[1918] The server sends the generated suggestions to the user's terminal.

[1919] Proposal received

[1920] The terminal receives a proposal sent from the server and notifies the user.

[1921] Execution notification

[1922] The device notifies the user to "take a deep breath."

[1923] Send feedback

[1924] The user takes a deep breath and enters the result and evaluation into the device.

[1925] The device sends user feedback to the server.

[1926] In this way, the system of the present invention can take into account the user's emotional state, provide optimal support tailored to their lifestyle, and continuously improve its suggestions.

[1927] The following describes the processing flow.

[1928] Step 1: Collect activity and emotional data

[1929] Users record their daily activities through smartphones and wearable devices, such as steps taken, heart rate, and scheduled events.

[1930] The emotion engine built into the device analyzes the user's voice, facial expressions, and text to generate emotion data. For example, it can determine the user's stress level from conversation content and changes in facial expressions.

[1931] Step 2: Data transmission

[1932] The device periodically sends user activity data and sentiment data to the server. Communication protocols such as HTTPS and MQTT are used.

[1933] Step 3: Data reception and storage

[1934] The server receives activity and sentiment data transmitted from the terminal. The received data is stored in a database.

[1935] Step 4: Data Cleansing

[1936] The server cleanses the stored data. For example, it improves the accuracy of the analysis by removing outliers and standardizing the data format.

[1937] Step 5: Data Analysis

[1938] The server uses the cleansed data to analyze the user's lifestyle patterns and emotional state. This analysis employs machine learning algorithms to extract the user's daily behavioral and emotional tendencies.

[1939] Step 6: Model Generation

[1940] The server generates a predictive model based on the analysis results. This model is used to provide optimal suggestions adapted to the user's behavioral patterns and emotional state.

[1941] Step 7: Proposal Generation

[1942] The server uses the generated predictive model to create specific suggestions for the user. For example, if stress levels are high due to prolonged desk work, it will generate recommendations such as "Take a 5-minute break every hour."

[1943] Step 8: Submit Proposal

[1944] The server sends the generated suggestions to the user's terminal.

[1945] Step 9: Receive Proposal

[1946] The terminal receives the proposal content sent from the server. The received proposal is immediately prepared to be notified to the user.

[1947] Step 10: Execution Notification

[1948] The device notifies the user based on the received suggestions. For example, it might use smartphone push notifications or wearable device alerts to notify the user of suggestions such as "take a deep breath" or "take a short walk."

[1949] Step 11: User Actions

[1950] The user checks notifications from their device and takes the suggested action. For example, they might take a 5-minute break and go for a walk or meditate.

[1951] Step 12: Feedback Record

[1952] When a user completes an action based on a suggestion, the results and evaluation are entered into the device. For example, it might record how much deep breathing or walking helped reduce stress.

[1953] Step 13: Submit Feedback

[1954] The device sends user feedback to the server.

[1955] Step 14: Update the proposal

[1956] The server updates its suggestions based on the feedback received. Newly collected data is also analyzed and incorporated into future suggestions. This allows for the provision of more optimized suggestions to users.

[1957] In this way, through a series of processing steps, we realize a system that provides optimal support considering the user's emotional state and daily behavior, and that allows for continuous improvement.

[1958] (Example 2)

[1959] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1960] Current lifestyle support systems make suggestions based solely on user activity data, making it difficult to provide personalized suggestions that take emotional states into account. This can result in insufficient assurance of the usefulness and suitability of the suggestions. Furthermore, the current system struggles to quickly collect feedback on changes in users' lifestyle patterns and emotional states, and to update the suggestions accordingly.

[1961] The identification processing performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for collecting user activity data and emotional data; means for cleansing and analyzing the activity data and emotional data to identify the user's lifestyle patterns and emotional states; means for generating a predictive model based on the identified lifestyle patterns and emotional states using a machine learning algorithm; means for generating individual suggestions based on the generated predictive model; means for transmitting the generated suggestions to the user's terminal; means for collecting user feedback on the suggestions; and means for updating the suggestion content based on the feedback. This enables integrated analysis of the user's activity data and emotional data, and provides optimal support tailored to their lifestyle and emotional state.

[1962] "Activity data" refers to information about a user's daily behavior, and includes behavioral data, schedule data, biometric data, etc.

[1963] "Emotional data" refers to information about a user's emotional state, and includes voice data, facial expression data, text data, and other similar data.

[1964] "Data cleansing" refers to processes aimed at improving data quality, such as handling missing data values ​​and detecting outliers.

[1965] "Analysis" refers to the process of identifying patterns and trends based on collected data using statistical methods and machine learning algorithms.

[1966] "Lifestyle patterns" refer to identifying tendencies and regularities in a user's daily behavior.

[1967] "Emotional state" refers to the emotions a user feels at a particular time or in a particular situation.

[1968] A "machine learning algorithm" refers to a computational method used to analyze large amounts of data, learn patterns, and perform predictions and classifications.

[1969] A "predictive model" refers to a mathematical model used to predict future behavior or states based on analyzed data.

[1970] "Individualized suggestions" refer to advice or guidelines tailored to address a user's specific situation or emotional state.

[1971] "Feedback" refers to responding to a user's suggestions by providing results and evaluations of their actions.

[1972] "Updating the proposal" refers to the process of improving the proposal based on the feedback collected.

[1973] This invention is a system that comprehensively analyzes user activity data and emotional data to provide personalized support tailored to the user's lifestyle and emotional state. The system aims to make more appropriate suggestions by taking the user's emotional state into consideration.

[1974] System Configuration

[1975] Server side

[1976] Data collection

[1977] The server has the functionality to receive activity data and emotional data transmitted from the user's terminal. Activity data includes behavioral data, schedule data, and biometric data. Emotional data includes voice data, facial expression data, and text data.

[1978] Specific hardware: Data center servers

[1979] Specific software: Apache Kafka (messaging system), InfluxDB (database)

[1980] Collection of emotional data

[1981] The server receives emotion data from the emotion engine installed in the terminal. The emotion engine uses voice analysis, facial recognition, and text analysis to identify the user's emotions.

[1982] Specific hardware: High-performance CPU / GPU server

[1983] Specific software: Python libraries (OpenCV, TensorFlow)

[1984] Data Analysis

[1985] The server cleanses and analyzes the received activity and emotional data to identify the user's lifestyle patterns and emotional states. This allows it to extract statistically significant patterns.

[1986] Specific software used: Pandas (data cleansing), Scikit-learn (machine learning)

[1987] Model generation

[1988] The server generates a predictive model based on the analysis results. This model reflects the user's lifestyle patterns and emotional state and is used to provide personalized suggestions.

[1989] Specific software: Scikit-learn, TensorFlow

[1990] Proposal generation

[1991] The server creates specific suggestions from the generated predictive model. For example, for a user whose emotional state is stressed, this might include a suggestion to "take deep breaths to relax."

[1992] Specific software: Django (Web framework)

[1993] Submit Proposal

[1994] The server sends the generated suggestions to the user's terminal.

[1995] Specific hardware: Network infrastructure

[1996] Specific software: REST API

[1997] Terminal side

[1998] Data transmission

[1999] The device transmits user behavior data, schedule data, biometric data, and emotional data to the server in real time.

[2000] Specific hardware: smartphones, wearable devices

[2001] Specific software: Android / iOS app

[2002] Emotion data generation

[2003] The emotion engine built into the device identifies emotions from the user's voice, facial expressions, and text, and generates data based on those emotions.

[2004] Specific software: Python libraries (OpenCV, TensorFlow)

[2005] Proposal received

[2006] The terminal receives the proposal sent from the server and notifies the user.

[2007] Specific software: Firebase Cloud Messaging

[2008] Execution notification

[2009] The device notifies the user of the received proposal. For example, it may use push notifications on a smartphone or the alert function of a wearable device to notify the user.

[2010] Specific hardware: smartphones, wearable devices

[2011] Specific software: Android / iOS app

[2012] Send feedback

[2013] The device sends user feedback to the server. This feedback includes the results and evaluation of the proposed changes.

[2014] Specific hardware: smartphones, wearable devices

[2015] Specific software: Android / iOS app

[2016] Specific example

[2017] Consider examples of how users can cope with everyday stress. Users record their activities and emotions using smartphones or wearable devices.

[2018] Data collection

[2019] The device sends user activity data (e.g., steps taken and schedule) and emotional data (stress and happiness levels based on voice and facial expression analysis) to the server.

[2020] Data Analysis

[2021] The server analyzes activity and emotional data to identify when users are more likely to experience stress during specific time periods.

[2022] Model generation

[2023] The server generates a predictive model based on the user's lifestyle patterns and emotional state. This model provides appropriate suggestions for stress reduction.

[2024] Proposal generation

[2025] The server generates specific suggestions such as "take deep breaths to relax" or "take a short break."

[2026] Submit Proposal

[2027] The server sends the generated suggestions to the user's terminal.

[2028] Proposal received

[2029] The terminal receives a proposal sent from the server and notifies the user.

[2030] Execution notification

[2031] The device notifies the user to "take a deep breath."

[2032] Send feedback

[2033] The user takes a deep breath and enters the result and evaluation into the device.

[2034] The device sends user feedback to the server.

[2035] In this way, the system of the present invention can take into account the user's emotional state, provide optimal support tailored to their lifestyle, and continuously improve its suggestions.

[2036] Example of a prompt

[2037] "Based on emotional and activity data collected by users over a certain period, predict when they will experience stress and generate suggestions for appropriate relaxation methods."

[2038] The flow of the specific processing in Example 2 will be explained using Figure 13.

[2039] Step 1:

[2040] Data collection

[2041] The device transmits user activity and emotional data to the server in real time. Specifically, this includes behavioral data (steps and schedule), biometric data (heart rate and sleep information), voice data, facial expression data, and text data collected from smartphones and wearable devices.

[2042] Input: Behavioral data, schedule data, biometric data, voice data, facial expression data, text data

[2043] Output: Sending data to the server

[2044] Step 2:

[2045] Collection of emotional data

[2046] The server receives emotion data from the emotion engine installed in the terminal. The emotion engine identifies the user's emotions using speech analysis, facial recognition, and text analysis. For example, it analyzes the user's facial expressions using OpenCV and TensorFlow to determine their emotional state. This information is also sent to the server.

[2047] Input: Audio data, facial expression data, text data

[2048] Output: Sentiment data

[2049] Step 3:

[2050] Data cleansing

[2051] The server cleanses the received activity and sentiment data. Specifically, it performs data washing such as imputing missing values ​​and removing outliers to prepare a dataset suitable for analysis. The Pandas library is used in this process.

[2052] Input: Raw activity data, emotion data

[2053] Output: Cleansed data

[2054] Step 4:

[2055] Data Analysis

[2056] The server analyzes the cleansed data to identify the user's lifestyle patterns and emotional states. Specifically, it uses the Scikit-learn library to perform clustering and anomaly detection to analyze when and what emotional states the user is experiencing.

[2057] Input: Cleansed data

[2058] Output: Analysis results (lifestyle patterns, emotional state)

[2059] Step 5:

[2060] Model generation

[2061] The server generates a predictive model using machine learning algorithms based on the analysis results. Using Scikit-learn and TensorFlow, it creates a predictive model that reflects the user's lifestyle patterns and emotional state.

[2062] Input: Analysis results

[2063] Output: Predictive model

[2064] Step 6:

[2065] Proposal generation

[2066] The server generates specific suggestions based on a predictive model. For example, it might generate a "suggestion for deep breathing to relax" during times when the user is likely to feel stressed. The suggestions are built using the Django framework.

[2067] Input: Predictive model

[2068] Output: Specific proposals

[2069] Step 7:

[2070] Submit Proposal

[2071] The server sends the generated suggestions to the user's device. Suggestions are sent in real time using a REST API.

[2072] Input: Specific proposal

[2073] Output: Sending suggestions to the terminal

[2074] Step 8:

[2075] Proposal received

[2076] The device receives suggestions sent from the server and notifies the user. Firebase Cloud Messaging is used to provide push notifications and alerts.

[2077] Input: Suggestion from the server

[2078] Output: Notification to the user

[2079] Step 9:

[2080] Execution notification

[2081] The device sends notifications to the user prompting them to take action on the suggested actions. For example, a smartphone might receive a notification instructing it to "take a deep breath before the next meeting."

[2082] Input: Received proposals

[2083] Output: Execution notification to the user

[2084] Step 10:

[2085] Send feedback

[2086] The terminal sends user feedback to the server. The user inputs the results and evaluation of the suggested actions into the terminal, and this information is sent to the server.

[2087] Input: User feedback

[2088] Output: Sending feedback to the server

[2089] These processing steps enable the integrated analysis of user activity and emotional data, allowing for the provision of optimal support tailored to their lifestyle and emotional state.

[2090] (Application Example 2)

[2091] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[2092] Traditional user support systems typically made suggestions based on user activity data. However, these systems failed to consider the user's emotional state and were unable to adequately respond to temporary emotions or psychological conditions, limiting the usefulness of their suggestions. In particular, existing systems failed to provide sufficient support in situations where a rapid response was required when the user was experiencing negative emotions such as anxiety or fear.

[2093] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[2094] In this invention, the server includes means for collecting user activity data and emotional data, means for analyzing the activity data and emotional data to identify the user's lifestyle patterns and emotional state, and means for generating personalized suggestions based on the identified lifestyle patterns and emotional state. This enables the prompt and appropriate safety measures to be suggested when the user feels anxiety or fear.

[2095] "Activity data" refers to information about a user's daily activities, such as location information, activity history, and schedule data.

[2096] "Emotional data" refers to information about a user's emotional state, analyzed from their voice data, text data, and facial expression data.

[2097] A "device" is an electronic device that a user possesses and uses to send, receive, and display information, and includes smartphones and wearable devices.

[2098] "Feedback" refers to the reactions and evaluations that users give to suggestions they receive.

[2099] A "machine learning algorithm" is a computational method used by computers to learn patterns from data and perform predictions and classifications.

[2100] A "suggestion" refers to specific instructions or advice regarding actions or measures for the user, generated based on the analyzed data.

[2101] This invention provides a system that collects and analyzes user activity data and emotional data, generates personalized suggestions, and transmits them to the user's device. Specific embodiments for carrying out this invention are described below.

[2102] System Configuration

[2103] Server side

[2104] Data collection:

[2105] The server receives activity data and emotion data transmitted from the user's device. This activity data includes location information, activity history, and schedule data. Emotion data consists of voice data, text data, and facial expression data.

[2106] Collection of emotional data:

[2107] The server receives emotion data from the emotion engine installed in the terminal. This emotion engine identifies the user's emotions using at least one of the following: voice analysis, facial expression recognition, and text analysis.

[2108] Data analysis:

[2109] The server cleanses and analyzes received activity and emotional data to identify the user's lifestyle patterns and emotional state. Machine learning algorithms are used to extract statistically significant patterns.

[2110] Model generation:

[2111] The server generates a predictive model based on the analysis results. This model reflects the user's lifestyle patterns and emotional state and is used to provide personalized suggestions.

[2112] Suggestion generation:

[2113] The server creates specific suggestions from the generated predictive model. For example, for a user whose emotional state is one of anxiety or fear, these suggestions might include "location of the nearest police station" or "instructions to choose a safe route."

[2114] Submit proposal:

[2115] The server sends the generated suggestions to the user's device.

[2116] Terminal side

[2117] Data transmission:

[2118] The device transmits user behavior data, schedule data, biometric data, and emotional data to the server in real time.

[2119] Emotion data generation:

[2120] The emotion engine built into the device identifies emotions from the user's voice, facial expressions, and text, and generates data based on those emotions.

[2121] Proposal received:

[2122] The terminal receives the proposal sent from the server and notifies the user.

[2123] Execution notification:

[2124] The device notifies the user of the received suggestions. For example, it may use push notifications on a smartphone or the alert function of a wearable device to notify the user.

[2125] Send feedback:

[2126] The device sends user feedback to the server. This feedback includes the results and evaluation of the proposed changes.

[2127] Specific example

[2128] Consider specific examples of how users can cope with the anxiety and fear they experience on a daily basis. Users record their activities and emotions using smartphones or wearable devices. The device sends the user's activity data (e.g., location information and activity history) and emotional data (e.g., anxiety and fear based on voice and facial expression analysis) to a server. The server analyzes the activity and emotional data to identify when the user is likely to feel anxiety or fear. The server generates a predictive model based on the user's lifestyle patterns and emotional state. This model provides appropriate suggestions for reducing anxiety and fear. The server generates specific suggestions, such as "location of the nearest police station" or "instructions for choosing a safe route." The server sends the generated suggestions to the user's device. The device receives the suggestions from the server and notifies the user. The user receives the notification, acts according to the suggestions, and inputs the results and evaluation into the device. The device sends the user's feedback to the server.

[2129] Example of a prompt

[2130] "Analyze the user's voice and text data to extract their current emotional state. If they are feeling anxious or frightened, generate a notification suggesting the location of the nearest police station based on their current location."

[2131] In this way, the system of the present invention can take into account the user's emotional state, provide optimal support tailored to their lifestyle, and continuously improve its suggestions.

[2132] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[2133] Step 1:

[2134] The user's device collects user activity data (location information, activity history, schedule data) and emotional data (voice data, text data, facial expression data) in real time. This data is processed using an emotion engine installed in the device and sent to the server. The input is the user's activity data and emotional data, and the output is the transmission of this data to the server.

[2135] Step 2:

[2136] The server receives activity and sentiment data sent from the terminal. The received data is cleansed, and any missing or inaccurate data is corrected or supplemented. Next, the data is ready for analysis. The input is the activity and sentiment data sent from the terminal, and the output is the cleansed data.

[2137] Step 3:

[2138] The server analyzes cleansed activity and emotional data. This analysis uses machine learning algorithms to extract statistically significant patterns. The user's lifestyle patterns and emotional states are identified. The input is the cleansed data, and the output is the identified lifestyle patterns and emotional states.

[2139] Step 4:

[2140] The server uses an AI model based on the analysis results to generate a predictive model. This model reflects the user's lifestyle patterns and emotional state and is used to provide personalized suggestions. The input is the analysis results, and the output is the predictive model.

[2141] Step 5:

[2142] The server generates specific suggestions from the predictive model it has created. These suggestions are customized according to the user's current emotional state and lifestyle. For example, if the user is feeling anxious or fearful, the suggestions might include the location of the nearest police station and directions to a safe route. The input is the predictive model, and the output is the specific suggestions.

[2143] Step 6:

[2144] The server sends the generated suggestions to the user's terminal. The input is the specific suggestions, and the output is the suggestions sent to the user's terminal.

[2145] Step 7:

[2146] The device receives suggestions sent from the server and notifies the user. This notification is delivered using smartphone push notifications or the alert function of a wearable device. The input is the suggestion content sent from the server, and the output is the information notified to the user.

[2147] Step 8:

[2148] The user acts according to the received suggestions and inputs the results and evaluations into the device. This includes feedback on how helpful the suggestions were and the actions taken. The input is the user's feedback, and the output is the state in which the feedback has been entered into the device.

[2149] Step 9:

[2150] The terminal sends user feedback to the server. The server receives this feedback and uses it to update the suggestions. The input is the user feedback, and the output is the feedback sent to the server.

[2151] Step 10:

[2152] The server updates the suggestions based on the feedback received. This makes future suggestions more accurate and useful to the user. The input is the user's feedback, and the output is the updated suggestions.

[2153] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[2154] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2155] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[2156] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[2157] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[2158] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[2159] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[2160] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[2161] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[2162] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[2163] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[2164] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[2165] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[2166] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[2167] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[2168] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[2169] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[2170] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[2171] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[2172] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[2173] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[2174] The following is further disclosed regarding the embodiments described above.

[2175] (Claim 1)

[2176] Means for collecting user activity data,

[2177] A means for analyzing the aforementioned activity data to identify the user's lifestyle patterns,

[2178] Means for generating individual proposals based on the identified lifestyle patterns,

[2179] A means for sending the generated proposal to the user's terminal,

[2180] A means for collecting user feedback on the aforementioned proposal,

[2181] A means of updating the proposed content based on the aforementioned feedback,

[2182] A system that includes this.

[2183] (Claim 2)

[2184] The system according to claim 1, wherein the activity data includes at least one of behavioral data, schedule data, and biometric data.

[2185] (Claim 3)

[2186] The system according to claim 1, wherein the analysis is performed using a machine learning algorithm.

[2187] "Example 1"

[2188] (Claim 1)

[2189] Means for collecting user activity data,

[2190] A means for analyzing the aforementioned activity data to identify the user's lifestyle patterns,

[2191] Means for generating a predictive model based on the identified lifestyle patterns,

[2192] A means for generating individual proposals from the aforementioned generated predictive model,

[2193] Means for sending the generated proposal to the user's computer,

[2194] A means for collecting user feedback on the aforementioned proposal,

[2195] A means of updating the proposed content based on the aforementioned feedback,

[2196] A system that includes this.

[2197] (Claim 2)

[2198] The system according to claim 1, wherein the activity data includes at least one of behavioral data, schedule data, and biometric data.

[2199] (Claim 3)

[2200] The system according to claim 1, wherein the analysis is performed using a machine learning algorithm.

[2201] "Application Example 1"

[2202] (Claim 1)

[2203] Means for collecting user activity data,

[2204] A means for analyzing the aforementioned activity data to identify the user's lifestyle patterns,

[2205] Means for generating individual proposals based on the identified lifestyle patterns,

[2206] A means for sending the generated proposal to the user's terminal,

[2207] A means for collecting user feedback on the aforementioned proposal,

[2208] A means of updating the proposed content based on the aforementioned feedback,

[2209] A means for the robot terminal to collect worker motion data and work speed data in real time,

[2210] A means for analyzing the worker's work efficiency based on the aforementioned motion data and work speed data, and for generating appropriate work suggestions,

[2211] A means for transmitting the generated work proposal to a robot terminal and notifying the worker,

[2212] A means for collecting worker feedback after notification by the robot terminal,

[2213] A system that includes this.

[2214] (Claim 2)

[2215] The system according to claim 1, wherein the activity data includes at least one of behavioral data, schedule data, and biometric data.

[2216] (Claim 3)

[2217] The system according to claim 1, wherein the analysis is performed using a machine learning algorithm.

[2218] "Example 2 of combining an emotion engine"

[2219] (Claim 1)

[2220] Means for collecting user activity data and sentiment data,

[2221] A means for cleansing and analyzing the aforementioned activity data and emotional data to identify the user's lifestyle patterns and emotional state,

[2222] A means for generating a predictive model based on the identified lifestyle patterns and emotional states using a machine learning algorithm,

[2223] Means for generating individual proposals based on the aforementioned generated predictive model,

[2224] A means for sending the generated proposal to the user's terminal,

[2225] A means for collecting user feedback on the aforementioned proposal,

[2226] A means of updating the proposed content based on the aforementioned feedback,

[2227] A system that includes this.

[2228] (Claim 2)

[2229] The system according to claim 1, wherein the activity data and emotion data include at least one of behavioral data, schedule data, biometric data, voice data, facial expression data, and text data.

[2230] (Claim 3)

[2231] The system according to claim 1, wherein the analysis is performed using data cleansing processes and machine learning algorithms.

[2232] "Application example 2 when combining with an emotional engine"

[2233] (Claim 1)

[2234] Means for collecting user activity data and sentiment data,

[2235] A means for analyzing the aforementioned activity data and emotional data to identify the user's lifestyle patterns and emotional state,

[2236] Means for generating individual suggestions based on the identified lifestyle patterns and emotional states,

[2237] Means for sending the generated proposal to the user's device,

[2238] A means for collecting user feedback on the aforementioned proposal,

[2239] A means of updating the proposed content based on the aforementioned feedback,

[2240] A system that includes this.

[2241] (Claim 2)

[2242] The system according to claim 1, wherein the activity data and emotion data include at least one of location information, behavioral history, schedule data, voice data, and text data.

[2243] (Claim 3)

[2244] The system according to claim 1, wherein the analysis is performed using a machine learning algorithm. [Explanation of symbols]

[2245] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. Means for collecting user activity data, A means for analyzing the aforementioned activity data to identify the user's lifestyle patterns, Means for generating individual proposals based on the identified lifestyle patterns, A means for sending the generated proposal to the user's terminal, A means for collecting user feedback on the aforementioned proposal, A means of updating the proposed content based on the aforementioned feedback, A system that includes this.

2. The system according to claim 1, wherein the activity data includes at least one of behavioral data, schedule data, and biometric data.

3. The system according to claim 1, wherein the analysis is performed using a machine learning algorithm.

Images