System

Abstract

A system is provided.SOLUTION: A system including means for collecting behavior data of a user including a heart rate, the number of steps, and position information from a smart device, means for preprocessing the collected behavior data and removing abnormal values and missing values, means for constructing a generation model for distinguishing between unconscious behavior and conscious behavior using the preprocessed behavior data, means for digitizing unconscious behavior and generating statistical information based on the generation model, and means for generating advice personalized to the user based on the generated statistical information and notifying the user of the generated advice via communication means.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In modern society, lifestyle-related diseases are steadily increasing, making their prevention and management a critical issue. It is particularly difficult for elderly people and busy modern people to accurately understand their own lifestyle habits and unconscious behaviors, making it difficult to find appropriate measures to improve them. Furthermore, conventional health management systems primarily focus on conscious behaviors and do not sufficiently consider unconscious behaviors. The problem that this invention aims to solve is to provide a system that uses behavioral data collected from smart devices to accurately quantify a user's unconscious behaviors and, based on that data, supports the prevention and management of lifestyle-related diseases. [Means for solving the problem]

[0005] The present invention solves the above-mentioned problems with a system that includes the following means. First, it provides a means for collecting user behavioral data, including heart rate, step count, and location information, from a smart device. Next, it provides a means for preprocessing the collected behavioral data and removing outliers and missing values. Furthermore, it provides a means for constructing a generative model that distinguishes between unconscious and conscious behavior using the preprocessed behavioral data. Next, it provides a means for quantifying unconscious behavior based on the generative model and generating statistical information. Finally, it provides a means for generating personalized advice for the user based on the generated statistical information and notifying the user via a communication means. This system enables users to understand their own unconscious behavior and improve it, thereby preventing and managing lifestyle-related diseases.

[0006] "Smart devices" are digital devices that can measure and collect user behavioral and physiological data, and generally include smartwatches and fitness trackers.

[0007] "Behavioral data" refers to information about a user's daily activities, including heart rate, number of steps, and location information.

[0008] "Collection means" refers to the hardware and software configuration for acquiring data from smart devices and transmitting it to a server.

[0009] "Preprocessing" refers to the process of arranging collected data into a format that is easy to analyze, and includes removing outliers and filling in missing values.

[0010] A "generative model" refers to an analytical model that uses deep learning to distinguish between unconscious and conscious behavior based on user behavior data.

[0011] "Unconscious behavior" refers to behaviors that users perform without being particularly conscious of them, including everyday habits and repetitive movements.

[0012] "Conscious behavior" refers to actions that a user consciously performs with a specific purpose, including exercise and planned activities.

[0013] "Quantification" refers to statistically measuring unconscious behavior and expressing it as a number.

[0014] "Statistical information" refers to information that shows trends and patterns obtained by compiling and analyzing quantified unconscious behavior data.

[0015] "Personalized advice" refers to specific suggestions for improving lifestyle habits and managing health, generated based on each user's individual behavioral data.

[0016] "Communication means" refers to the network infrastructure and protocols for sending and receiving data between the server and the user's smart device. [Brief explanation of the drawings]

[0017] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8]FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram illustrating a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0018] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0019] First, the terms used in the following description will be explained.

[0020] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0021] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0022] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0023] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0024] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0025] [First embodiment]

[0026] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0027] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0028] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0029] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0030] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0031] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0032] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0033] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0034] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0035] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0036] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0037] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0038] This system quantifies the user's unconscious behavior and provides personalized advice for improving their health. Below, we will explain the program functions and processing of this system in natural language.

[0039] Data collection

[0040] Device:

[0041] Smart devices worn by users (smartwatches and fitness trackers) measure behavioral data such as heart rate, number of steps, and location information in real time. For example, they record the number of steps a user takes throughout the day and their heart rate fluctuations during that time.

[0042] The device sends the measurement data to the server in real time, using communication infrastructure such as LINE.

[0043] Data Preprocessing

[0044] server:

[0045] The received behavioral data is stored in a database, and outliers and missing values ​​are removed. The data stored in the database is then formatted into a format that is easier to analyze.

[0046] For example, if a user's data contains an abnormally high heart rate, it can be removed and the other data normalized.

[0047] Data analysis

[0048] server:

[0049] The preprocessed data is analyzed using a generative model (deep learning model). This model is designed to distinguish between unconscious and conscious behavior. Unconscious behavior is identified from fluctuations in walking patterns and heart rate during specific time periods.

[0050] For example, if a user walks a specific route every day between 6:00 p.m. and 8:00 p.m., this is determined to be an unconscious activity such as commuting or shopping.

[0051] Quantifying behavior

[0052] server:

[0053] The number of unconscious behaviors and time periods are quantified to generate statistical information, including the frequency, average value, and range of variation of unconscious behaviors.

[0054] For example, it may be shown that the user's daily unconscious walking count is 5,000 steps, and that this occurs mainly in the morning.

[0055] Advice Generation

[0056] server:

[0057] Based on the statistics generated, personalized advice is created for the user, which comes in the form of specific suggestions, reminders and motivational messages to improve their health.

[0058] For example, advice such as, "You unconsciously spend a lot of time sitting. Standing up and doing some light exercise every hour can have a positive effect on your health." is generated.

[0059] notification

[0060] The server generates advice and notifies the user via LINE or other communication methods.

[0061] User:

[0062] Users receive notifications and take action based on the advice, which allows them to change their unconscious behaviors and improve their health.

[0063] Ongoing support

[0064] The server continuously collects data and tracks changes in the user's behavior, generating new statistics and advice in a timely manner. This continuous process allows users to manage their health over the long term.

[0065] This system allows users to accurately understand unconscious behavior and take specific measures to improve it, contributing to the prevention and management of lifestyle-related diseases.

[0066] The processing flow will be explained below.

[0067] Step 1:

[0068] The device begins to measure the user's activity data. The smart device's sensors measure heart rate, steps, location, etc. As the user begins their daily activities, these sensors record data in real time.

[0069] Step 2:

[0070] The device sends the measurement data to the server via LINE. Using the LINE API, the device periodically uploads the data to the server. This process is done automatically, without any burden on the user.

[0071] Step 3:

[0072] The behavioral data received by the server is first stored in a database, where it is tagged and organized for each user.

[0073] Step 4:

[0074] The server performs data cleaning, detecting outliers and missing values ​​from the collected data and removing or filling them in. For example, it removes extremely abnormal heart rate data and fills in missing data points using statistical methods.

[0075] Step 5:

[0076] The server uses the preprocessed data to prepare it for feeding into the generative model. The data is converted into a time series and into a format optimized for analysis.

[0077] Step 6:

[0078] The server uses a generative model (deep learning model) to distinguish between unconscious and conscious behavior. The model is fed with behavioral data and uses it to identify unconscious behavior. For example, it can identify unconscious behavior based on fluctuations in the number of steps taken or heart rate at a specific time and place.

[0079] Step 7:

[0080] Based on the results of the generative model, the server quantifies the number of unconscious behaviors and time periods, and generates statistical information, including the frequency, time allocation, and fluctuation patterns of unconscious behaviors.

[0081] Step 8:

[0082] The server uses the generated statistics to create personalized advice for the user, including lifestyle changes, reminders, and motivational messages.

[0083] Step 9:

[0084] The server notifies the user of the generated advice via LINE. The notification is sent to the user's device using the LINE API. For example, a message such as "Getting up and doing some light exercise every hour can have health benefits" can be sent.

[0085] Step 10:

[0086] The user receives a notification and takes action based on the advice, for example, walking 30 minutes every day.

[0087] Step 11:

[0088] The server continually tracks changes in user behavior, collecting new data, then reanalyzes it based on the new behavioral data to generate new personalized advice, and the process continues.

[0089] These detailed steps clarify the overall processing flow of the system, allowing users to understand their own unconscious behaviors and take concrete actions to improve them.

[0090] Example 1

[0091] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0092] In modern life, many people unconsciously engage in unhealthy behaviors. However, there is a lack of systems that accurately identify these unconscious behaviors and implement specific measures to improve them. Furthermore, existing health management systems often lack the functionality to not only track specific behaviors but also provide users with personalized advice. Therefore, there is a need for a system that can quantify users' unconscious behaviors and provide specific, personalized advice for improving their health based on that information.

[0093] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0094] In this invention, the server includes means for collecting biometric data including heart rate, step count, and location information from the smart device, means for preprocessing the collected biometric data and removing outliers and missing values, means for using the preprocessed biometric data to build a generative model that distinguishes between unconscious and conscious behavior, means for quantifying unconscious behavior and generating statistical information based on the generative model, means for generating personalized advice for the user based on the generated statistical information and notifying the user via communication means, and means for continuously providing the generated advice to the user and tracking changes in behavior. This makes it possible to accurately understand the user's unconscious behavior and take specific improvement measures, thereby promoting improvement of the user's health condition.

[0095] A "smart device" is a device worn by a user to measure biometric data such as heart rate, number of steps, and location information.

[0096] "Biometric data" refers to data related to a user's body, such as heart rate, number of steps, and location information.

[0097] "Means of collection" refers to the method of acquiring biometric data using a smart device and transferring it to a server.

[0098] "Preprocessing means" refers to a method for removing outliers and missing values ​​from collected biometric data and standardizing the data.

[0099] A "generative model" is a mathematical model, such as a deep learning model, that distinguishes between unconscious and conscious behavior from user behavior data.

[0100] "Unconscious behavior" refers to behavior that a user performs without being aware of it, such as walking naturally during commuting or breaks.

[0101] "Conscious behavior" refers to behavior that a user intentionally performs, such as exercise or a specific activity.

[0102] "Means of quantifying" refers to methods of expressing the number of times or time periods of unconscious behavior as specific numbers.

[0103] "Statistical information" refers to statistical data such as the frequency, average value, and range of variation of quantified unconscious behavior.

[0104] "Personalized advice" refers to individually tailored health improvement suggestions and reminders generated based on a user's specific subconscious behaviors and patterns.

[0105] "Communication Method" refers to the communications infrastructure or protocol used to provide generated advice or other notifications to users, such as LINE or Bluetooth.

[0106] "Means for tracking changes in behavior" refers to methods for continuously collecting data on users' unconscious behavior and analyzing and updating statistical information.

[0107] In the embodiment of the present invention, a system for quantifying a user's unconscious behavior and providing personalized advice for improving health will be described in detail. A specific implementation method of this system will be described below.

[0108] Data collection

[0109] Device:

[0110] Smart devices worn by users (e.g., smartwatches and fitness trackers) measure biometric data such as heart rate, number of steps, and location information in real time. For example, they record the number of steps a user takes throughout the day, heart rate fluctuations during that time, and the amount of time spent in a specific location. The collected data is automatically sent to a server. Communication is achieved using technologies such as LINE and Bluetooth.

[0111] Data Preprocessing

[0112] server:

[0113] The server stores the received data in a database. Next, the data is cleansed to remove outliers (e.g., a heart rate of 1000 bpm) and missing values ​​(data that has not been measured). Standardization unifies the units of data, making subsequent analysis easier.

[0114] For example, the server inserts new data into the database, a data cleansing algorithm filters out outliers, and a standardization function is applied to convert the data into a uniform format.

[0115] Data analysis

[0116] server:

[0117] The preprocessed data is then analyzed using a generative AI model (deep learning model), which is designed to distinguish between unconscious and conscious behaviors. For example, continuous walking during a specific time period can be determined to be unconscious behavior.

[0118] As a specific example of how it works, the server inputs data into a deep learning model, and the model analyzes the data to distinguish between unconscious and conscious behavior.

[0119] Quantifying behavior

[0120] server:

[0121] The number of unconscious actions and the time periods are quantified to generate statistical information. This statistical information includes the frequency, average value, and fluctuation range of unconscious actions. For example, it shows the total number of unconscious steps a user takes in a day and the average time period during which they occur.

[0122] As a specific example of operation, the server aggregates the analysis results, and a digitization algorithm generates statistical information.

[0123] Advice Generation

[0124] server:

[0125] Based on the generated statistics, the system creates personalized advice for users. These advice come in the form of specific suggestions, reminders, and motivational messages for improving their health. For example, the system might generate advice like, "You spend a lot of time sitting without realizing it. Standing up and doing some light exercise every hour can help improve your health."

[0126] As a specific example of operation, the server executes an advice generation algorithm based on the statistical information of each user and formats the generated advice as text.

[0127] notification

[0128] server:

[0129] The generated advice will be sent to the user via LINE or other communication means.

[0130] user:

[0131] Users receive notifications and take action based on the advice, which allows them to change their unconscious behaviors and improve their health.

[0132] Specifically, the server creates a notification message and sends it via the LINE API. The user then checks the notification on their smartphone or other device and adjusts their actions accordingly.

[0133] Ongoing support

[0134] server:

[0135] It continuously collects data and tracks changes in user behavior, generating new statistics and advice in a timely manner, allowing users to manage their health over the long term.

[0136] As a specific example of operation, the server periodically receives new behavioral data, repeats preprocessing and analysis, and updates advice at regular intervals to keep notifying the user.

[0137] This system allows users to accurately understand unconscious behavior and take specific measures to improve it, contributing to the prevention and management of lifestyle-related diseases.

[0138] Prompt Sentence Examples

[0139] Sample prompt 1: "Analyze the user's walking data over a specified period of time and generate personalized health advice."

[0140] Sample prompt 2: "Remove outliers and use the preprocessed data to generate statistics on unconscious behavior."

[0141] Based on these prompts, the generative AI model analyzes the user's behavioral data and generates appropriate advice, thus realizing a system that effectively supports users in managing their health.

[0142] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0143] Step 1: Data collection

[0144] Device: A smart device worn by the user measures biometric data such as heart rate, number of steps, and location information in real time. The collected data is automatically sent to a server. Specifically, sensors detect heart rate and number of steps, and location information is measured using GPS. Communication is via LINE or Bluetooth. The input is real-time biometric data, and the output is biometric data sent to the server.

[0145] Specific behavior:

[0146] Sensors measure heart rate, steps, and location.

[0147] The measured data is temporarily stored in the smart device.

[0148] The smart device sends data to the server using LINE or Bluetooth.

[0149] Step 2: Data Preprocessing

[0150] Server: The server stores the data it receives in a database. Next, it cleanses the data to remove outliers (for example, a heart rate of 1000 bpm) and missing values ​​(data that has not been measured). It standardizes the data units and converts them into a format that is easy to analyze. The input is the biometric data sent to the server, and the output is the cleansed, standardized data.

[0151] Specific behavior:

[0152] The server inserts new data into the database.

[0153] Data cleansing algorithms filter out outliers and missing values.

[0154] A standardization function is applied to transform the data into a uniform format.

[0155] Step 3: Data analysis

[0156] Server: The preprocessed data is analyzed using a generative AI model (deep learning model). This model is designed to distinguish between unconscious and conscious behavior. As a result of the analysis, unconscious and conscious behavior are identified. The input is the preprocessed data, and the output is the classification result of unconscious and conscious behavior.

[0157] Specific behavior:

[0158] The server inputs the preprocessed data into the deep learning model.

[0159] The model analyzes the data and distinguishes between unconscious and conscious behavior.

[0160] The analysis results are saved on the server.

[0161] Step 4: Quantify behavior

[0162] Server: Quantifies the number of unconscious behaviors and time periods and generates statistical information. This statistical information includes the frequency, average value, and fluctuation range of unconscious behavior. The input is the classification result of unconscious behavior and conscious behavior, and the output is statistical information.

[0163] Specific behavior:

[0164] The server aggregates the behavioral data based on the analysis results.

[0165] A numerical algorithm calculates the frequency and time period of unconscious behavior as a number.

[0166] Statistics are generated and stored in a database.

[0167] Step 5: Advice Generation

[0168] Server: Creates personalized advice for users based on the generated statistics. These advice come in the form of specific suggestions, reminders, and motivational messages to improve their health. The input is statistics, and the output is personalized advice.

[0169] Specific behavior:

[0170] The server runs an advice generation algorithm based on each user's statistical information.

[0171] Format the generated advice into text.

[0172] Step 6: Notification

[0173] Server: Notifies the user of the generated advice via LINE or other communication methods.

[0174] User: Receives notifications and takes action based on the advice. The input is personalized advice, and the output is a change in user behavior.

[0175] Specific behavior:

[0176] The server creates a notification message and sends it via the LINE API.

[0177] The user checks the notification on their smartphone or other device and follows the advice.

[0178] Step 7: Ongoing support

[0179] Server: Continuously collects data and keeps track of changes in user behavior. New statistics and advice are generated and provided to the user in a timely manner. The input is new biometric data, and the output is updated advice.

[0180] Specific behavior:

[0181] The server periodically receives new behavioral data and repeats the preprocessing and analysis.

[0182] It updates the advice at regular intervals to keep the user informed.

[0183] In this way, users can accurately understand their unconscious behavior and take concrete measures to improve it.

[0184] (Application example 1)

[0185] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0186] Maximizing the operating efficiency of factory robots and reducing unnecessary maintenance are important challenges for improving factory productivity. However, current systems struggle to centrally manage everything from collecting and analyzing robot operation data to predicting and notifying maintenance, and many systems use individual approaches. As a result, maintenance timing is often misjudged, and resources are wasted due to excessive maintenance. There is a need to build a system that solves this problem, maximizes the uptime of factory robots, and provides optimal maintenance schedules.

[0187] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0188] In this invention, the server includes means for collecting user behavioral data, including heart rate, step count, and location information, means for preprocessing the collected behavioral data and removing outliers and missing values, means for using the preprocessed behavioral data to build a generative model that distinguishes between unconscious and conscious behavior, means for quantifying the unconscious behavior based on the generative model and generating statistical information, means for generating personalized advice for the user based on the generated statistical information and notifying the user via a communication means, means for collecting operation data using sensors mounted on the factory robot and measuring operating time and power consumption, means for removing outliers and missing values ​​from the collected operation data and formatting the data into an easily analyzable format, means for analyzing unconscious and conscious behavior using the preprocessed operation data and generating statistical information, and means for predicting factory robot maintenance based on the generated statistical information and notifying a manager. This improves the operating efficiency of the factory robot and enables the provision of an optimal maintenance schedule.

[0189] A "smart device" is a device that can collect user behavioral data (heart rate, number of steps, location information, etc.).

[0190] "Behavioral data" refers to data that includes the operating status and behavior patterns of users and factory robots.

[0191] "Preprocessing" is the process of removing outliers and missing values ​​from collected data and formatting it into a format that is easy to analyze.

[0192] A "generative model" is a deep learning model designed to distinguish between unconscious and conscious behavior.

[0193] "Unconscious behavior" refers to actions that users or factory robots automatically repeat, and refers to routine operations that occur during specific periods of time.

[0194] "Intentional behavior" refers to actions taken by users or factory robots with a specific purpose, including responding to abnormalities and maintenance activities.

[0195] "Statistical information" is information that quantifies and summarizes the frequency and time period of unconscious and conscious behavior, as well as other related data.

[0196] "Personalized advice" is specific, individualized advice tailored to the situation of the user or factory robot, based on collected data and statistical information.

[0197] A "sensor" is a device that is installed on a factory robot and measures operational data (operating time, power consumption, operating route, etc.).

[0198] "Maintenance prediction" refers to predicting when factory robots will require maintenance and the specific work required, based on the generated statistical information.

[0199] This invention provides a system that maximizes the operational efficiency of factory robots and enables the provision of optimal maintenance schedules. The system collects data from sensors installed in smart devices and factory robots, preprocesses and analyzes the data, and provides personalized advice and maintenance predictions.

[0200] Data collection

[0201] The server collects operational data (such as operating time, power consumption, and operating route) in real time from sensors installed in factory robots. Smart devices also collect behavioral data such as the user's heart rate, number of steps, and location information, and send it to the server. This data transfer uses the factory's network infrastructure (Wi-Fi, LAN).

[0202] Data Preprocessing

[0203] The server stores the collected motion and behavior data in a database (MySQL), removes outliers and missing values, and formats the preprocessed data into a format that is easy to analyze.

[0204] Data analysis

[0205] The server uses a TensorFlow-based generative model to analyze the preprocessed data. This generative model is designed to distinguish between unconscious and conscious behavior, identifying unconscious behavior from fluctuations in operating patterns and power consumption during specific time periods.

[0206] Quantification of movement

[0207] The server quantifies unconscious behavior based on the generative model and generates statistical information. This statistical information includes the frequency, average value, and fluctuation range of unconscious behavior. For example, it may show that a certain robot operates unconsciously for 8 hours a day, mainly during the daytime.

[0208] Maintenance prediction and advice generation

[0209] Based on the generated statistical information, the server predicts when factory robots will need maintenance and the specific tasks to be performed. Additionally, it creates messages for users, including advice on lifestyle improvements, daily reminders, and motivational messages. These advice and predictions are then sent to administrators and users via communication methods such as LINE using the Line Messaging API.

[0210] Specific examples

[0211] Here is an example prompt:

[0212] By collecting and analyzing the operation data of your factory robots in real time, we can predict when maintenance will be required and the specific work to be done, reducing wasted time and allowing you to operate your robots efficiently.

[0213] This system is expected to improve factory production efficiency and extend the lifespan of robots. It also allows users to understand their own unconscious behavior and take specific measures to improve it, contributing to the prevention and management of lifestyle-related diseases.

[0214] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0215] Step 1: Data collection

[0216] The terminal measures operational data (operating time, power consumption, operating route, etc.) in real time from sensors installed on factory robots and sends this data to a server. The input is raw data from the sensors, and the output is raw data stored on the server. Specifically, the sensors collect operational data at regular intervals and send it via Wi-Fi or LAN.

[0217] Step 2: Data Preprocessing

[0218] The server stores the received operational data in a database (MySQL) and removes outliers and missing values. The input is raw data, and the output is preprocessed clean data. Specific operations include inserting data into the database, filtering outliers, and completing or removing missing values.

[0219] Step 3: Data analysis

[0220] The server analyzes the preprocessed data through a generative AI model (a deep learning model using TensorFlow). The input is clean data, and the output is analyzed data that distinguishes between unconscious and conscious behavior. Specifically, the data is input into the deep learning model and pattern recognition is performed.

[0221] Step 4: Quantifying behavior

[0222] The server generates statistical information by quantifying the frequency and time period of unconscious behavior based on the data analyzed using the generative model. The input is the analyzed data, and the output is statistical information. Specific operations include frequency calculation, data aggregation by time period, and calculation of the range of variation.

[0223] Step 5: Maintenance prediction and advice generation

[0224] Based on the generated statistical information, the server predicts when factory robot maintenance will be required and the specific work to be done, and generates advice. The input is statistical information, and the output is maintenance prediction data and specific advice. Specific operations include generating a maintenance schedule based on threshold judgments and creating advice for users.

[0225] Step 6: Notification

[0226] The server uses the Line Messaging API to notify administrators and users of the generated maintenance forecast data and advice. The input is the maintenance forecast data and advice, and the output is a notification message. The specific operation involves sending a message using the API.

[0227] Step 7: Ongoing support

[0228] The server continuously collects, analyzes, and notifies the robots to improve their operational efficiency. The input is real-time operational data, and the output is updated analysis data and maintenance predictions. Specifically, the server collects and analyzes data periodically, and then provides timely maintenance predictions and notifications.

[0229] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0230] This invention combines a system that quantifies a user's unconscious behavior and provides personalized advice for improving their health with an emotion engine that identifies the user's emotions. Below, we will explain the program functions and processing of this system in natural language.

[0231] Data collection

[0232] Device:

[0233] Smart devices worn by users (smartwatches and fitness trackers) measure behavioral data in real time, such as heart rate, number of steps, location information, and facial expression data for emotion recognition. For example, they record the number of steps a user takes throughout the day, heart rate fluctuations during that time, and emotional data determined from facial expressions.

[0234] The device sends the measurement data to the server in real time, using communication infrastructure such as LINE.

[0235] Data Preprocessing

[0236] server:

[0237] The received behavioral data is stored in a database, and outliers and missing values ​​are removed. The data stored in the database is then formatted into a format that is easier to analyze.

[0238] For example, if a user's data contains an abnormally high heart rate, it can be removed and the other data normalized.

[0239] Data analysis

[0240] server:

[0241] The preprocessed data is analyzed using a generative model (deep learning model). This model is designed to distinguish between unconscious and conscious behavior. Unconscious behavior is identified from fluctuations in walking patterns and heart rate during specific time periods.

[0242] For example, if a user walks a specific route every day between 6:00 p.m. and 8:00 p.m., this is determined to be an unconscious activity such as commuting or shopping.

[0243] Quantifying behavior

[0244] server:

[0245] The number of unconscious behaviors and time periods are quantified to generate statistical information, including the frequency, average value, and range of variation of unconscious behaviors.

[0246] For example, it may be shown that the user's daily unconscious walking count is 5,000 steps, and that this occurs mainly in the morning.

[0247] Emotion Recognition and Integration

[0248] server:

[0249] The emotional engine recognizes the user's emotional state. It integrates behavioral and emotional data to generate statistical information that correlates unconscious behavior with emotional fluctuations. For example, if the emotional state while walking at a specific time indicates "stress," this can be associated with unconscious behavior.

[0250] Emotion-based advice generation

[0251] server:

[0252] Based on the results of the emotion engine, personalized advice is created for each user, including lifestyle improvements, reminders, and motivational messages, with a particular emphasis on advice tailored to the user's emotional state.

[0253] For example, advice such as "You seem to be feeling stressed lately. Try making time to refresh yourself on the weekend" may be generated.

[0254] notification

[0255] The server generates advice and notifies the user via LINE or other communication methods.

[0256] User:

[0257] Receive notifications and take action based on the advice, for example, walking 30 minutes every day.

[0258] Ongoing support

[0259] The server continuously collects data, tracking changes in user behavior and emotions, and then re-analyzes the new behavioral and emotional data to generate new personalized advice. This process is repeated continuously.

[0260] This system allows users to understand not only their unconscious behaviors but also their emotional state and take specific actions to improve it, resulting in more effective health management and prevention of lifestyle-related diseases.

[0261] The processing flow will be explained below.

[0262] Step 1:

[0263] The device begins to measure the user's behavioral data. The smart device's sensors measure heart rate, steps, location information, and facial expression data for emotion recognition. As the user begins their daily activities, these sensors record data in real time. For example, they collect information on how many steps the user took during the day, heart rate fluctuations during that time, and emotional state from facial expressions.

[0264] Step 2:

[0265] The device sends measurement data to the server via LINE. Using the LINE API, the device periodically uploads data to the server. This process is automatic, and behavioral data is sent to the server without placing any burden on the user. For example, the latest measurement data is sent to the server every hour.

[0266] Step 3:

[0267] The server stores the received behavioral data in a database, which is categorized by user.

[0268] Step 4:

[0269] The server performs data cleaning, detecting outliers and missing values ​​from the collected data and removing or filling them in. For example, extremely high heart rates or extremely high step counts are removed as outliers, and missing values ​​are filled in by guessing values ​​based on the data before and after.

[0270] Step 5:

[0271] The server uses the preprocessed data to prepare it for feeding into the generative model. The data is converted into a time series and into a format optimized for analysis.

[0272] Step 6:

[0273] The server uses a generative model (deep learning model) to distinguish between unconscious and conscious behavior. The model is fed with behavioral data, and unconscious behavior is identified based on that data. For example, unconscious behavior can be identified based on fluctuations in the number of steps taken and heart rate at a specific time and location. Walking a specific route during a specific time period (for example, from 6:00 PM to 8:00 PM) is determined to be a routine and regular unconscious behavior.

[0274] Step 7:

[0275] Based on the results of the generative model, the server quantifies the number of unconscious behaviors and time periods, and generates statistical information. This includes the frequency of unconscious behavior, time allocation, and fluctuation patterns. For example, statistics are generated on how much time a user spends walking unconsciously each day.

[0276] Step 8:

[0277] The server uses an emotion engine to recognize the user's emotional state. It integrates behavioral and emotional data to generate statistical information that correlates unconscious behavior with emotional fluctuations. For example, if the emotional state while walking at a specific time indicates "stress," it can be associated with unconscious behavior.

[0278] Step 9:

[0279] The server uses the generated statistics to create personalized advice for users. The advice includes lifestyle improvement tips, reminders, and motivational messages. The system emphasizes advice tailored to the user's emotional state. For example, the system might generate advice like, "You've been feeling stressed lately, so try taking a walk on the weekend to refresh yourself."

[0280] Step 10:

[0281] The server generates advice and notifies the user via LINE or other communication methods. Advice is sent to the user using the LINE API. For example, at 9:00 a.m., a message saying, "Try to walk for 30 minutes today" is sent.

[0282] Step 11:

[0283] The user receives a notification and takes action based on the advice. For example, they follow the notification and take a walk, recording how they felt during the walk. By following the advice, the user is expected to make effective behavioral improvements.

[0284] Step 12:

[0285] The server continuously tracks changes in the user's behavior and emotions, collecting new data, and then re-analyzing the new behavioral and emotional data to generate new personalized advice. This process is repeated continuously.

[0286] These detailed steps clarify the overall processing flow of the system, allowing users to understand their own unconscious behaviors and emotional states and take concrete measures to improve them, leading to more effective health management and the prevention of lifestyle-related diseases.

[0287] Example 2

[0288] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0289] Conventional health management systems can collect and analyze user behavioral data, but it is difficult to provide personalized advice that takes into account the user's emotional state. As a result, the effectiveness of advice is limited, making it difficult to maintain user motivation or lead to actual lifestyle improvements. In addition, there are insufficient ways to handle outliers and missing values, leading to problems with the accuracy and reliability of the data.

[0290] The identification processing by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for collecting user behavioral data from the smart device, including heart rate, number of steps, location information, and facial expression data; means for preprocessing the collected behavioral data and removing outliers and missing values; means for constructing a generative model that distinguishes between unconscious behavior and conscious behavior using the preprocessed behavioral data; means for quantifying unconscious behavior and generating statistical information based on the generative model; means for integrating the generated statistical information with emotional states and generating statistical information that associates unconscious behavior with emotional fluctuations; and means for generating personalized advice for the user based on the generated statistical information and the results of the emotion engine and notifying the user via communication means. This allows the user to receive highly accurate personalized advice based on their behavioral data and emotional data, enabling more effective health management and lifestyle improvement.

[0291] 1. A "smart device" is a wearable device that can collect behavioral data such as heart rate, number of steps, location information, and facial expression data.

[0292] 2. "Behavioral Data" means data collected by smart devices, such as a user's heart rate, number of steps, location information, and facial expression data.

[0293] 3. "Preprocessing" refers to the process of removing outliers and missing values ​​from collected behavioral data and formatting it into a format suitable for analysis.

[0294] 4. "Outliers" are abnormally high or low values ​​in the collected data and values ​​that should be removed for the system to perform accurate analysis.

[0295] 5. A "missing value" is a value that has a missing portion in the collected data.

[0296] 6. A "generative model" is a deep learning model that is built to distinguish between unconscious and conscious behaviors using preprocessed behavioral data.

[0297] 7. "Unconscious behavior" refers to behavior that a user performs unconsciously, such as everyday walking or commuting.

[0298] 8. "Conscious behavior" refers to behavior that a user consciously engages in, such as exercise or consciously performing a specific task.

[0299] 9. “Statistical information” refers to statistical data such as the frequency and time period of unconscious behavior quantified based on a generative model.

[0300] 10. "Emotion Engine" means software or algorithms that analyze a user's facial expression data to identify their emotional state.

[0301] 11. “Personalized Advice” means advice that is individually tailored to a user based on their behavioral and emotional data.

[0302] 12. "Communication Means" means an interface for transmitting information from the Server to the User, including, for example, an Internet connection, a mobile network, or a messaging service.

[0303] This invention is a system that analyzes a user's unconscious behavior and emotional state and provides personalized advice. To implement this system, the following specific hardware and software are used.

[0304] Device:

[0305] A user wears a smart device (e.g., a smartwatch or fitness tracker). The smart device measures behavioral data such as heart rate, steps, location information, and facial expression data in real time. The data is sent to a server via Bluetooth or Wi-Fi. For example, a smartwatch measures heart rate every second and sends the data to a server every hour.

[0306] server:

[0307] The server stores the behavioral data sent from the smart devices in a database. First, the data is preprocessed to remove outliers and missing values. Specifically, a Python script is used to remove outliers with heart rates above 300 and normalize the data. This preprocessing improves the reliability of the data and the accuracy of the analysis.

[0308] Next, the preprocessed data is analyzed using a deep learning model. This generative model (using a deep learning framework such as TensorFlow or PyTorch) distinguishes between unconscious and conscious behavior. For example, if a user walks a specific route between 6:00 PM and 8:00 PM every day, this is recognized as unconscious behavior. The model analyzes walking data and heart rate fluctuations to identify behavioral patterns.

[0309] The number of unconscious actions and the time periods are then quantified to generate statistical information. For example, the number of unconscious steps a user takes in a day is tallied and the time period is quantified (e.g., 5,000 steps in the morning). This statistical information forms the basis for effective advice based on the user's behavior.

[0310] Furthermore, an emotion engine is used to recognize the user's emotional state from facial expression data. Specifically, an emotion-identifying CNN model is used to analyze the facial expression data and classify the user's emotional state into categories such as "stress," "joy," and "surprise." The emotion data is then integrated with behavioral data to generate statistical information relating unconscious behavior to emotional fluctuations. For example, if the emotional state while walking at a specific time indicates "stress," this is associated with unconscious behavior.

[0311] Based on the generated statistics and the results of the emotion engine, personalized advice is generated for the user. This advice includes lifestyle improvements, reminders, and motivational messages. For example, specific advice may be generated such as, "Your stress level has been high lately, so try doing some yoga to relax." This process uses natural language generation (NLG) technology.

[0312] The server notifies the user of the generated advice via LINE or other communication methods (e.g., SMS or email). Upon receiving the notification, the user takes action based on the advice. The behavioral data is then collected again, and continuous data analysis and advice generation are repeated.

[0313] Prompt Sentence Examples

[0314] "A user uses a smartwatch to collect daily activity data, including heart rate, steps, location, and facial expression data. Analyze this data to assess the user's stress level and generate personalized health improvement advice."

[0315] This system enables users to receive highly accurate personalized advice based on their behavioral and emotional data, enabling them to effectively manage their health and improve their lifestyles. Furthermore, by processing outliers and missing values, the system can improve the reliability of data and the accuracy of analysis.

[0316] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0317] Step 1: Data collection

[0318] Device:

[0319] The user wears a smart device, which measures heart rate, steps, location information, and facial expression data in real time. These measurement data are stored in the internal memory at regular intervals (e.g., every second). The stored data is then periodically sent to a server via Bluetooth or Wi-Fi. For example, a smartwatch records heart rate every second and sends it to a server every hour.

[0320] input:

[0321] Your heart rate, steps, location, and facial expression data.

[0322] output:

[0323] The raw data sent to the server.

[0324] Step 2: Data Preprocessing

[0325] server:

[0326] The server stores the received raw data in a database. First, it performs preprocessing to remove outliers and missing values. Specifically, a Python script is used to remove data such as abnormally high heart rates and normalize the data. For example, outliers with heart rates above 300 are removed, and the entire dataset is normalized to a mean of 0 and a standard deviation of 1.

[0327] input:

[0328] Raw data sent from smart devices.

[0329] output:

[0330] Preprocessed data, normalized to remove outliers and missing values.

[0331] Step 3: Data analysis

[0332] server:

[0333] The preprocessed data is then analyzed using a deep learning model. Frameworks such as TensorFlow and PyTorch are used to distinguish between unconscious and conscious behavior using a trained generative model. This analysis detects patterns of walking data and heart rate fluctuations during specific time periods to identify user behavior. For example, walking data from 6:00 PM to 8:00 PM every day is analyzed and recognized as unconscious commuting behavior.

[0334] input:

[0335] Normalized preprocessed data.

[0336] output:

[0337] Data categorized into unconscious and conscious behaviors.

[0338] Step 4: Quantify behavior

[0339] server:

[0340] Based on the analysis results, the number of unconscious actions and time periods are quantified. Specifically, statistical information such as the frequency, average value, and fluctuation range of unconscious actions is generated. For example, the number of unconscious steps a user takes in a day is quantified as 5,000 steps, and the time period is defined as the morning.

[0341] input:

[0342] Data categorized into unconscious and conscious behaviors.

[0343] output:

[0344] Statistics on the number and time of unconscious behaviors.

[0345] Step 5: Emotion Recognition and Integration

[0346] server:

[0347] An emotion engine is used to recognize emotional states from facial expression data. Specifically, an emotion-identifying CNN model analyzes the facial expression data and classifies them into expressions such as "stress," "happiness," and "surprise." Subconscious behavior and emotion data are then integrated to generate statistical information relating unconscious behavior to emotional fluctuations. For example, if "stress" is recognized while walking at a specific time of day, it can be associated with unconscious behavior.

[0348] input:

[0349] Facial expression data, statistical information on unconscious behavior.

[0350] output:

[0351] Statistics of unconscious behavior integrated with emotion data.

[0352] Step 6: Advice Generation

[0353] server:

[0354] Based on the generated statistics and the results of the emotion engine, personalized advice is generated for the user. This advice includes lifestyle improvements, reminders, and motivational messages. For example, specific advice such as "You've been feeling stressed lately, so try doing yoga every night" is generated. Natural language generation technology (NLG) is used to present advice in a format that is easy for the user to understand.

[0355] input:

[0356] Statistics of unconscious behavior integrated with emotion data.

[0357] output:

[0358] Personalized advice.

[0359] Step 7: Notification

[0360] server:

[0361] The generated advice is notified to the user via a communication method (e.g., LINE, SMS, email). For example, the advice content is sent to the user's smartphone using the LINE Messaging API.

[0362] User:

[0363] Receive notifications and take action based on the advice, for example, walking 30 minutes every day as advised.

[0364] input:

[0365] Personalized advice.

[0366] output:

[0367] Advice given to users.

[0368] Step 8: Ongoing support

[0369] server:

[0370] It continuously collects data, tracks changes in user behavior and emotions, and re-analyzes the new data to generate new personalized advice, allowing users to continuously manage their health and improve their lifestyle.

[0371] input:

[0372] New behavioral and emotional data.

[0373] output:

[0374] Updated personalized advice.

[0375] (Application example 2)

[0376] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0377] There is a need to analyze users' unconscious behaviors and emotional states and provide personalized advice for improving their health. However, conventional systems generate advice by analyzing only user behavioral data, and therefore are unable to provide specific advice that takes into account the user's emotional state. Furthermore, even in physical store shopping experiences, it is difficult to analyze customers' unconscious behaviors and emotional states and provide personalized services in real time. To address these challenges, a new system is needed to provide more effective and personalized advice and services.

[0378] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for collecting user behavioral data, including heart rate, step count, location information, and facial expression data, from the smart device; means for preprocessing the collected behavioral data and removing outliers and missing values; means for constructing a generative model that distinguishes between unconscious and conscious behavior using the preprocessed behavioral data; means for quantifying unconscious behavior and generating statistical information based on the generative model; means for analyzing the user's emotional state and integrating it with the behavioral data; and means for generating personalized advice for the user based on the generated statistical information and emotional data and notifying the user via communication means. This makes it possible to analyze the user's unconscious behavior and emotional state and provide specific, personalized advice for improving health. Furthermore, it is possible to analyze customers' unconscious behavior and emotional state and provide personalized services in real time even during shopping experiences in physical stores.

[0379] A "smart device" is a wearable device for collecting user behavioral data and biometric information, and specifically includes devices such as smart watches and smart glasses.

[0380] "Behavioral data" refers to data related to a user's daily movements and physiological responses, such as the user's heart rate, number of steps, location information, and facial expression data.

[0381] "Preprocessing" is the process of removing outliers and missing values ​​from collected behavioral data and converting the data into a format that is easier to analyze.

[0382] A "generative model" is an algorithm or mathematical model that uses preprocessed behavioral data to distinguish between unconscious and conscious behavior.

[0383] "Unconscious behavior" refers to everyday actions and movements that users perform without being particularly conscious of them, and is the behavior that can be analyzed using a generative model.

[0384] "Statistical information" is data generated based on the quantified results of unconscious behavior, and includes statistical characteristics such as frequency, average value, and range of variation.

[0385] "Emotional state" is information that indicates the psychological state of the user, which is analyzed using data such as facial expressions of the user.

[0386] "Personalized advice" refers to specific instructions or suggestions tailored to a user's specific needs, based on their behavioral data and emotional state.

[0387] "Communication methods" are systems used to send advice and notifications to users, including messaging applications and email.

[0388] The present invention relates to a system that analyzes a user's unconscious behavior and emotional state and provides personalized advice for improving their health. This system uses smart devices to collect and analyze behavioral data and provide advice to the user. The specific hardware and software used are shown below.

[0389] Hardware Configuration

[0390] 1. Smart devices: Wearable devices such as smartwatches and smart glasses. These devices collect users' heart rate, steps, location, and facial expression data in real time.

[0391] 2. Server: Preprocesses and analyzes data, builds generative models, and generates advice.

[0392] 3. User terminal: A communication device, such as a smartphone, through which the user receives advice.

[0393] Software Configuration

[0394] 1. Data preprocessing software: Using the Python library scikit-learn, outliers and missing values ​​are removed from the collected behavioral data, and the data is standardized.

[0395] 2. Generative model: A deep learning model for emotion analysis. For example, EmotionRecognitionModel(tentative).

[0396] 3. Communication Software: We use messaging applications and email as a means of communication to send notifications.

[0397] Data processing and calculation

[0398] The server receives data collected from smart devices in real time, removes outliers and missing values, and then feeds the standardized data into a generative model to distinguish between unconscious and conscious behavior. It also analyzes the user's emotional state using an emotion engine and generates statistical information that integrates behavioral and emotional data.

[0399] Specific examples

[0400] For example, the system records the number of steps a user takes throughout the day, fluctuations in heart rate during that time, and emotional data determined from facial expressions, and identifies unconscious behavior from walking patterns and heart rate fluctuations during specific times of the day. If a user walks a specific route every day between 6:00 and 8:00 p.m. and their emotional state during that time indicates "stress," the system will integrate this as unconscious behavior and emotional data and generate and notify the user with specific advice such as, "You seem to be feeling stressed recently. Try making time to refresh yourself on the weekend."

[0401] Prompt Sentence Examples

[0402] "We've detected an elevated heart rate in your customer. They're likely interested in your product, so issue them a special discount coupon."

[0403] This invention enables detailed analysis of users' unconscious behavior and emotional state, and provides specific and personalized advice for improving their health. Furthermore, by providing personalized services in real time in brick-and-mortar stores, it is expected that customer satisfaction and sales will increase.

[0404] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0405] Step 1:

[0406] The server collects user behavioral data from smart devices (smartwatches and smart glasses), including heart rate, steps, location information, and facial expression data. The collected data is sent to the server in real time. The input is raw data from the smart devices, and the output is unprocessed behavioral data stored on the server.

[0407] Step 2:

[0408] The server preprocesses the collected behavioral data. Specifically, it removes outliers and missing values ​​and standardizes the data. For this purpose, it uses libraries such as StandardScaler from scikit-learn. The input is the raw behavioral data, and the output is the preprocessed behavioral data.

[0409] Step 3:

[0410] The server feeds the preprocessed behavioral data into a generative model to build a generative model that distinguishes between unconscious and conscious behaviors. This uses a deep learning model, where the input is the preprocessed behavioral data and the output is labeled data that distinguishes between unconscious and conscious behaviors.

[0411] Step 4:

[0412] The server quantifies unconscious behaviors based on the generative model and generates statistical information, including the number of unconscious behaviors, time periods, frequencies, average values, and fluctuation ranges. The input is labeled data, and the output is quantified unconscious behaviors and statistical information based on them.

[0413] Step 5:

[0414] The server uses an emotion engine to analyze the user's emotional state and integrate it with behavioral data. Specifically, it recognizes the user's emotions from facial expression data and combines them with statistical information on unconscious behavior to generate an integrated dataset. The input is facial expression data and statistical information on unconscious behavior, and the output is the integrated dataset.

[0415] Step 6:

[0416] The server generates personalized advice for users based on the generated integrated data. Specific advice content includes lifestyle improvement tips, reminders, and motivational messages. For example, it generates a message such as, "You seem to be feeling stressed lately. Try to make time to refresh yourself on the weekend." The input is the integrated dataset, and the output is the advice content.

[0417] Step 7:

[0418] The server notifies the user of the generated advice via a communication method (messaging application or email). The input is the advice content, and the output is the notification sent to the user's terminal.

[0419] The above processing steps enable detailed analysis of a user's unconscious behavior and emotional state, enabling the provision of specific and personalized advice for improving health. This also enables real-time personalized services to be provided in physical stores.

[0420] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0421] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0422] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0423] [Second embodiment]

[0424] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0425] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0426] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0427] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0428] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0429] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0430] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0431] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0432] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0433] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0434] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0435] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0436] This system quantifies the user's unconscious behavior and provides personalized advice for improving their health. Below, we will explain the program functions and processing of this system in natural language.

[0437] Data collection

[0438] Device:

[0439] Smart devices worn by users (smartwatches and fitness trackers) measure behavioral data such as heart rate, number of steps, and location information in real time. For example, they record the number of steps a user takes throughout the day and their heart rate fluctuations during that time.

[0440] The device sends the measurement data to the server in real time, using communication infrastructure such as LINE.

[0441] Data Preprocessing

[0442] server:

[0443] The received behavioral data is stored in a database, and outliers and missing values ​​are removed. The data stored in the database is then formatted into a format that is easier to analyze.

[0444] For example, if a user's data contains an abnormally high heart rate, it can be removed and the other data normalized.

[0445] Data analysis

[0446] server:

[0447] The preprocessed data is analyzed using a generative model (deep learning model). This model is designed to distinguish between unconscious and conscious behavior. Unconscious behavior is identified from fluctuations in walking patterns and heart rate during specific time periods.

[0448] For example, if a user walks a specific route every day between 6:00 p.m. and 8:00 p.m., this is determined to be an unconscious activity such as commuting or shopping.

[0449] Quantifying behavior

[0450] server:

[0451] The number of unconscious behaviors and time periods are quantified to generate statistical information, including the frequency, average value, and range of variation of unconscious behaviors.

[0452] For example, it may be shown that the user's daily unconscious walking count is 5,000 steps, and that this occurs mainly in the morning.

[0453] Advice Generation

[0454] server:

[0455] Based on the statistics generated, personalized advice is created for the user, which comes in the form of specific suggestions, reminders and motivational messages to improve their health.

[0456] For example, advice such as, "You unconsciously spend a lot of time sitting. Standing up and doing some light exercise every hour can have a positive effect on your health." is generated.

[0457] notification

[0458] The server generates advice and notifies the user via LINE or other communication methods.

[0459] User:

[0460] Users receive notifications and take action based on the advice, which allows them to change their unconscious behaviors and improve their health.

[0461] Ongoing support

[0462] The server continuously collects data and tracks changes in the user's behavior, generating new statistics and advice in a timely manner. This continuous process allows users to manage their health over the long term.

[0463] This system allows users to accurately understand unconscious behavior and take specific measures to improve it, contributing to the prevention and management of lifestyle-related diseases.

[0464] The processing flow will be explained below.

[0465] Step 1:

[0466] The device begins to measure the user's activity data. The smart device's sensors measure heart rate, steps, location, etc. As the user begins their daily activities, these sensors record data in real time.

[0467] Step 2:

[0468] The device sends the measurement data to the server via LINE. Using the LINE API, the device periodically uploads the data to the server. This process is done automatically, without any burden on the user.

[0469] Step 3:

[0470] The behavioral data received by the server is first stored in a database, where it is tagged and organized for each user.

[0471] Step 4:

[0472] The server performs data cleaning, detecting outliers and missing values ​​from the collected data and removing or filling them in. For example, it removes extremely abnormal heart rate data and fills in missing data points using statistical methods.

[0473] Step 5:

[0474] The server uses the preprocessed data to prepare it for feeding into the generative model. The data is converted into a time series and into a format optimized for analysis.

[0475] Step 6:

[0476] The server uses a generative model (deep learning model) to distinguish between unconscious and conscious behavior. The model is fed with behavioral data and uses it to identify unconscious behavior. For example, it can identify unconscious behavior based on fluctuations in the number of steps taken or heart rate at a specific time and place.

[0477] Step 7:

[0478] Based on the results of the generative model, the server quantifies the number of unconscious behaviors and time periods, and generates statistical information, including the frequency, time allocation, and fluctuation patterns of unconscious behaviors.

[0479] Step 8:

[0480] The server uses the generated statistics to create personalized advice for the user, including lifestyle changes, reminders, and motivational messages.

[0481] Step 9:

[0482] The server notifies the user of the generated advice via LINE. The notification is sent to the user's device using the LINE API. For example, a message such as "Getting up and doing some light exercise every hour can have health benefits" can be sent.

[0483] Step 10:

[0484] The user receives a notification and takes action based on the advice, for example, walking 30 minutes every day.

[0485] Step 11:

[0486] The server continually tracks changes in user behavior, collecting new data, then reanalyzes it based on the new behavioral data to generate new personalized advice, and the process continues.

[0487] These detailed steps clarify the overall processing flow of the system, allowing users to understand their own unconscious behaviors and take concrete actions to improve them.

[0488] Example 1

[0489] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0490] In modern life, many people unconsciously engage in unhealthy behaviors. However, there is a lack of systems that accurately identify these unconscious behaviors and implement specific measures to improve them. Furthermore, existing health management systems often lack the functionality to not only track specific behaviors but also provide users with personalized advice. Therefore, there is a need for a system that can quantify users' unconscious behaviors and provide specific, personalized advice for improving their health based on that information.

[0491] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0492] In this invention, the server includes means for collecting biometric data including heart rate, step count, and location information from the smart device, means for preprocessing the collected biometric data and removing outliers and missing values, means for using the preprocessed biometric data to build a generative model that distinguishes between unconscious and conscious behavior, means for quantifying unconscious behavior and generating statistical information based on the generative model, means for generating personalized advice for the user based on the generated statistical information and notifying the user via communication means, and means for continuously providing the generated advice to the user and tracking changes in behavior. This makes it possible to accurately understand the user's unconscious behavior and take specific improvement measures, thereby promoting improvement of the user's health condition.

[0493] A "smart device" is a device worn by a user to measure biometric data such as heart rate, number of steps, and location information.

[0494] "Biometric data" refers to data related to a user's body, such as heart rate, number of steps, and location information.

[0495] "Means of collection" refers to the method of acquiring biometric data using a smart device and transferring it to a server.

[0496] "Preprocessing means" refers to a method for removing outliers and missing values ​​from collected biometric data and standardizing the data.

[0497] A "generative model" is a mathematical model, such as a deep learning model, that distinguishes between unconscious and conscious behavior from user behavior data.

[0498] "Unconscious behavior" refers to behavior that a user performs without being aware of it, such as walking naturally during commuting or breaks.

[0499] "Conscious behavior" refers to behavior that a user intentionally performs, such as exercise or a specific activity.

[0500] "Means of quantifying" refers to methods of expressing the number of times or time periods of unconscious behavior as specific numbers.

[0501] "Statistical information" refers to statistical data such as the frequency, average value, and range of variation of quantified unconscious behavior.

[0502] "Personalized advice" refers to individually tailored health improvement suggestions and reminders generated based on a user's specific subconscious behaviors and patterns.

[0503] "Communication Method" refers to the communications infrastructure or protocol used to provide generated advice or other notifications to users, such as LINE or Bluetooth.

[0504] "Means for tracking changes in behavior" refers to methods for continuously collecting data on users' unconscious behavior and analyzing and updating statistical information.

[0505] In the embodiment of the present invention, a system for quantifying a user's unconscious behavior and providing personalized advice for improving health will be described in detail. A specific implementation method of this system will be described below.

[0506] Data collection

[0507] Device:

[0508] Smart devices worn by users (e.g., smartwatches and fitness trackers) measure biometric data such as heart rate, number of steps, and location information in real time. For example, they record the number of steps a user takes throughout the day, heart rate fluctuations during that time, and the amount of time spent in a specific location. The collected data is automatically sent to a server. Communication is achieved using technologies such as LINE and Bluetooth.

[0509] Data Preprocessing

[0510] server:

[0511] The server stores the received data in a database. Next, the data is cleansed to remove outliers (e.g., a heart rate of 1000 bpm) and missing values ​​(data that has not been measured). Standardization unifies the units of data, making subsequent analysis easier.

[0512] For example, the server inserts new data into the database, a data cleansing algorithm filters out outliers, and a standardization function is applied to convert the data into a uniform format.

[0513] Data analysis

[0514] server:

[0515] The preprocessed data is then analyzed using a generative AI model (deep learning model), which is designed to distinguish between unconscious and conscious behaviors. For example, continuous walking during a specific time period can be determined to be unconscious behavior.

[0516] As a specific example of how it works, the server inputs data into a deep learning model, and the model analyzes the data to distinguish between unconscious and conscious behavior.

[0517] Quantifying behavior

[0518] server:

[0519] The number of unconscious actions and the time periods are quantified to generate statistical information. This statistical information includes the frequency, average value, and fluctuation range of unconscious actions. For example, it shows the total number of unconscious steps a user takes in a day and the average time period during which they occur.

[0520] As a specific example of operation, the server aggregates the analysis results, and a digitization algorithm generates statistical information.

[0521] Advice Generation

[0522] server:

[0523] Based on the generated statistics, the system creates personalized advice for users. These advice come in the form of specific suggestions, reminders, and motivational messages for improving their health. For example, the system might generate advice like, "You spend a lot of time sitting without realizing it. Standing up and doing some light exercise every hour can help improve your health."

[0524] As a specific example of operation, the server executes an advice generation algorithm based on the statistical information of each user and formats the generated advice as text.

[0525] notification

[0526] server:

[0527] The generated advice will be sent to the user via LINE or other communication means.

[0528] user:

[0529] Users receive notifications and take action based on the advice, which allows them to change their unconscious behaviors and improve their health.

[0530] Specifically, the server creates a notification message and sends it via the LINE API. The user then checks the notification on their smartphone or other device and adjusts their actions accordingly.

[0531] Ongoing support

[0532] server:

[0533] It continuously collects data and tracks changes in user behavior, generating new statistics and advice in a timely manner, allowing users to manage their health over the long term.

[0534] As a specific example of operation, the server periodically receives new behavioral data, repeats preprocessing and analysis, and updates advice at regular intervals to keep notifying the user.

[0535] This system allows users to accurately understand unconscious behavior and take specific measures to improve it, contributing to the prevention and management of lifestyle-related diseases.

[0536] Prompt Sentence Examples

[0537] Sample prompt 1: "Analyze the user's walking data over a specified period of time and generate personalized health advice."

[0538] Sample prompt 2: "Remove outliers and use the preprocessed data to generate statistics on unconscious behavior."

[0539] Based on these prompts, the generative AI model analyzes the user's behavioral data and generates appropriate advice, thus realizing a system that effectively supports users in managing their health.

[0540] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0541] Step 1: Data collection

[0542] Device: A smart device worn by the user measures biometric data such as heart rate, number of steps, and location information in real time. The collected data is automatically sent to a server. Specifically, sensors detect heart rate and number of steps, and location information is measured using GPS. Communication is via LINE or Bluetooth. The input is real-time biometric data, and the output is biometric data sent to the server.

[0543] Specific behavior:

[0544] Sensors measure heart rate, steps, and location.

[0545] The measured data is temporarily stored in the smart device.

[0546] The smart device sends data to the server using LINE or Bluetooth.

[0547] Step 2: Data Preprocessing

[0548] Server: The server stores the data it receives in a database. Next, it cleanses the data to remove outliers (for example, a heart rate of 1000 bpm) and missing values ​​(data that has not been measured). It standardizes the data units and converts them into a format that is easy to analyze. The input is the biometric data sent to the server, and the output is the cleansed, standardized data.

[0549] Specific behavior:

[0550] The server inserts new data into the database.

[0551] Data cleansing algorithms filter out outliers and missing values.

[0552] A standardization function is applied to transform the data into a uniform format.

[0553] Step 3: Data analysis

[0554] Server: The preprocessed data is analyzed using a generative AI model (deep learning model). This model is designed to distinguish between unconscious and conscious behavior. As a result of the analysis, unconscious and conscious behavior are identified. The input is the preprocessed data, and the output is the classification result of unconscious and conscious behavior.

[0555] Specific behavior:

[0556] The server inputs the preprocessed data into the deep learning model.

[0557] The model analyzes the data and distinguishes between unconscious and conscious behavior.

[0558] The analysis results are saved on the server.

[0559] Step 4: Quantify behavior

[0560] Server: Quantifies the number of unconscious behaviors and time periods and generates statistical information. This statistical information includes the frequency, average value, and fluctuation range of unconscious behavior. The input is the classification result of unconscious behavior and conscious behavior, and the output is statistical information.

[0561] Specific behavior:

[0562] The server aggregates the behavioral data based on the analysis results.

[0563] A numerical algorithm calculates the frequency and time period of unconscious behavior as a number.

[0564] Statistics are generated and stored in a database.

[0565] Step 5: Advice Generation

[0566] Server: Creates personalized advice for users based on the generated statistics. These advice come in the form of specific suggestions, reminders, and motivational messages to improve their health. The input is statistics, and the output is personalized advice.

[0567] Specific behavior:

[0568] The server runs an advice generation algorithm based on each user's statistical information.

[0569] Format the generated advice into text.

[0570] Step 6: Notification

[0571] Server: Notifies the user of the generated advice via LINE or other communication methods.

[0572] User: Receives notifications and takes action based on the advice. The input is personalized advice, and the output is a change in user behavior.

[0573] Specific behavior:

[0574] The server creates a notification message and sends it via the LINE API.

[0575] The user checks the notification on their smartphone or other device and follows the advice.

[0576] Step 7: Ongoing support

[0577] Server: Continuously collects data and keeps track of changes in user behavior. New statistics and advice are generated and provided to the user in a timely manner. The input is new biometric data, and the output is updated advice.

[0578] Specific behavior:

[0579] The server periodically receives new behavioral data and repeats the preprocessing and analysis.

[0580] It updates the advice at regular intervals to keep the user informed.

[0581] In this way, users can accurately understand their unconscious behavior and take concrete measures to improve it.

[0582] (Application example 1)

[0583] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0584] Maximizing the operating efficiency of factory robots and reducing unnecessary maintenance are important challenges for improving factory productivity. However, current systems struggle to centrally manage everything from collecting and analyzing robot operation data to predicting and notifying maintenance, and many systems use individual approaches. As a result, maintenance timing is often misjudged, and resources are wasted due to excessive maintenance. There is a need to build a system that solves this problem, maximizes the uptime of factory robots, and provides optimal maintenance schedules.

[0585] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0586] In this invention, the server includes means for collecting user behavioral data, including heart rate, step count, and location information, means for preprocessing the collected behavioral data and removing outliers and missing values, means for using the preprocessed behavioral data to build a generative model that distinguishes between unconscious and conscious behavior, means for quantifying the unconscious behavior based on the generative model and generating statistical information, means for generating personalized advice for the user based on the generated statistical information and notifying the user via a communication means, means for collecting operation data using sensors mounted on the factory robot and measuring operating time and power consumption, means for removing outliers and missing values ​​from the collected operation data and formatting the data into an easily analyzable format, means for analyzing unconscious and conscious behavior using the preprocessed operation data and generating statistical information, and means for predicting factory robot maintenance based on the generated statistical information and notifying a manager. This improves the operating efficiency of the factory robot and enables the provision of an optimal maintenance schedule.

[0587] A "smart device" is a device that can collect user behavioral data (heart rate, number of steps, location information, etc.).

[0588] "Behavioral data" refers to data that includes the operating status and behavior patterns of users and factory robots.

[0589] "Preprocessing" is the process of removing outliers and missing values ​​from collected data and formatting it into a format that is easy to analyze.

[0590] A "generative model" is a deep learning model designed to distinguish between unconscious and conscious behavior.

[0591] "Unconscious behavior" refers to actions that users or factory robots automatically repeat, and refers to routine operations that occur during specific periods of time.

[0592] "Intentional behavior" refers to actions taken by users or factory robots with a specific purpose, including responding to abnormalities and maintenance activities.

[0593] "Statistical information" is information that quantifies and summarizes the frequency and time period of unconscious and conscious behavior, as well as other related data.

[0594] "Personalized advice" is specific, individualized advice tailored to the situation of the user or factory robot, based on collected data and statistical information.

[0595] A "sensor" is a device that is installed on a factory robot and measures operational data (operating time, power consumption, operating route, etc.).

[0596] "Maintenance prediction" refers to predicting when factory robots will require maintenance and the specific work required, based on the generated statistical information.

[0597] This invention provides a system that maximizes the operational efficiency of factory robots and enables the provision of optimal maintenance schedules. The system collects data from sensors installed in smart devices and factory robots, preprocesses and analyzes the data, and provides personalized advice and maintenance predictions.

[0598] Data collection

[0599] The server collects operational data (such as operating time, power consumption, and operating route) in real time from sensors installed in factory robots. Smart devices also collect behavioral data such as the user's heart rate, number of steps, and location information, and send it to the server. This data transfer uses the factory's network infrastructure (Wi-Fi, LAN).

[0600] Data Preprocessing

[0601] The server stores the collected motion and behavior data in a database (MySQL), removes outliers and missing values, and formats the preprocessed data into a format that is easy to analyze.

[0602] Data analysis

[0603] The server uses a TensorFlow-based generative model to analyze the preprocessed data. This generative model is designed to distinguish between unconscious and conscious behavior, identifying unconscious behavior from fluctuations in operating patterns and power consumption during specific time periods.

[0604] Quantification of movement

[0605] The server quantifies unconscious behavior based on the generative model and generates statistical information. This statistical information includes the frequency, average value, and fluctuation range of unconscious behavior. For example, it may show that a certain robot operates unconsciously for 8 hours a day, mainly during the daytime.

[0606] Maintenance prediction and advice generation

[0607] Based on the generated statistical information, the server predicts when factory robots will need maintenance and the specific tasks to be performed. Additionally, it creates messages for users, including advice on lifestyle improvements, daily reminders, and motivational messages. These advice and predictions are then sent to administrators and users via communication methods such as LINE using the Line Messaging API.

[0608] Specific examples

[0609] Here is an example prompt:

[0610] By collecting and analyzing the operation data of your factory robots in real time, we can predict when maintenance will be required and the specific work to be done, reducing wasted time and allowing you to operate your robots efficiently.

[0611] This system is expected to improve factory production efficiency and extend the lifespan of robots. It also allows users to understand their own unconscious behavior and take specific measures to improve it, contributing to the prevention and management of lifestyle-related diseases.

[0612] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0613] Step 1: Data collection

[0614] The terminal measures operational data (operating time, power consumption, operating route, etc.) in real time from sensors installed on factory robots and sends this data to a server. The input is raw data from the sensors, and the output is raw data stored on the server. Specifically, the sensors collect operational data at regular intervals and send it via Wi-Fi or LAN.

[0615] Step 2: Data Preprocessing

[0616] The server stores the received operational data in a database (MySQL) and removes outliers and missing values. The input is raw data, and the output is preprocessed clean data. Specific operations include inserting data into the database, filtering outliers, and completing or removing missing values.

[0617] Step 3: Data analysis

[0618] The server analyzes the preprocessed data through a generative AI model (a deep learning model using TensorFlow). The input is clean data, and the output is analyzed data that distinguishes between unconscious and conscious behavior. Specifically, the data is input into the deep learning model and pattern recognition is performed.

[0619] Step 4: Quantifying behavior

[0620] The server generates statistical information by quantifying the frequency and time period of unconscious behavior based on the data analyzed using the generative model. The input is the analyzed data, and the output is statistical information. Specific operations include frequency calculation, data aggregation by time period, and calculation of the range of variation.

[0621] Step 5: Maintenance prediction and advice generation

[0622] Based on the generated statistical information, the server predicts when factory robot maintenance will be required and the specific work to be done, and generates advice. The input is statistical information, and the output is maintenance prediction data and specific advice. Specific operations include generating a maintenance schedule based on threshold judgments and creating advice for users.

[0623] Step 6: Notification

[0624] The server uses the Line Messaging API to notify administrators and users of the generated maintenance forecast data and advice. The input is the maintenance forecast data and advice, and the output is a notification message. The specific operation involves sending a message using the API.

[0625] Step 7: Ongoing support

[0626] The server continuously collects, analyzes, and notifies the robots to improve their operational efficiency. The input is real-time operational data, and the output is updated analysis data and maintenance predictions. Specifically, the server collects and analyzes data periodically, and then provides timely maintenance predictions and notifications.

[0627] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0628] This invention combines a system that quantifies a user's unconscious behavior and provides personalized advice for improving their health with an emotion engine that identifies the user's emotions. Below, we will explain the program functions and processing of this system in natural language.

[0629] Data collection

[0630] Device:

[0631] Smart devices worn by users (smartwatches and fitness trackers) measure behavioral data in real time, such as heart rate, number of steps, location information, and facial expression data for emotion recognition. For example, they record the number of steps a user takes throughout the day, heart rate fluctuations during that time, and emotional data determined from facial expressions.

[0632] The device sends the measurement data to the server in real time, using communication infrastructure such as LINE.

[0633] Data Preprocessing

[0634] server:

[0635] The received behavioral data is stored in a database, and outliers and missing values ​​are removed. The data stored in the database is then formatted into a format that is easier to analyze.

[0636] For example, if a user's data contains an abnormally high heart rate, it can be removed and the other data normalized.

[0637] Data analysis

[0638] server:

[0639] The preprocessed data is analyzed using a generative model (deep learning model). This model is designed to distinguish between unconscious and conscious behavior. Unconscious behavior is identified from fluctuations in walking patterns and heart rate during specific time periods.

[0640] For example, if a user walks a specific route every day between 6:00 p.m. and 8:00 p.m., this is determined to be an unconscious activity such as commuting or shopping.

[0641] Quantifying behavior

[0642] server:

[0643] The number of unconscious behaviors and time periods are quantified to generate statistical information, including the frequency, average value, and range of variation of unconscious behaviors.

[0644] For example, it may be shown that the user's daily unconscious walking count is 5,000 steps, and that this occurs mainly in the morning.

[0645] Emotion Recognition and Integration

[0646] server:

[0647] The emotional engine recognizes the user's emotional state. It integrates behavioral and emotional data to generate statistical information that correlates unconscious behavior with emotional fluctuations. For example, if the emotional state while walking at a specific time indicates "stress," this can be associated with unconscious behavior.

[0648] Emotion-based advice generation

[0649] server:

[0650] Based on the results of the emotion engine, personalized advice is created for each user, including lifestyle improvements, reminders, and motivational messages, with a particular emphasis on advice tailored to the user's emotional state.

[0651] For example, advice such as "You seem to be feeling stressed lately. Try making time to refresh yourself on the weekend" may be generated.

[0652] notification

[0653] The server generates advice and notifies the user via LINE or other communication methods.

[0654] User:

[0655] Receive notifications and take action based on the advice, for example, walking 30 minutes every day.

[0656] Ongoing support

[0657] The server continuously collects data, tracking changes in user behavior and emotions, and then re-analyzes the new behavioral and emotional data to generate new personalized advice. This process is repeated continuously.

[0658] This system allows users to understand not only their unconscious behaviors but also their emotional state and take specific actions to improve it, resulting in more effective health management and prevention of lifestyle-related diseases.

[0659] The processing flow will be explained below.

[0660] Step 1:

[0661] The device begins to measure the user's behavioral data. The smart device's sensors measure heart rate, steps, location information, and facial expression data for emotion recognition. As the user begins their daily activities, these sensors record data in real time. For example, they collect information on how many steps the user took during the day, heart rate fluctuations during that time, and emotional state from facial expressions.

[0662] Step 2:

[0663] The device sends measurement data to the server via LINE. Using the LINE API, the device periodically uploads data to the server. This process is automatic, and behavioral data is sent to the server without placing any burden on the user. For example, the latest measurement data is sent to the server every hour.

[0664] Step 3:

[0665] The server stores the received behavioral data in a database, which is categorized by user.

[0666] Step 4:

[0667] The server performs data cleaning, detecting outliers and missing values ​​from the collected data and removing or filling them in. For example, extremely high heart rates or extremely high step counts are removed as outliers, and missing values ​​are filled in by guessing values ​​based on the data before and after.

[0668] Step 5:

[0669] The server uses the preprocessed data to prepare it for feeding into the generative model. The data is converted into a time series and into a format optimized for analysis.

[0670] Step 6:

[0671] The server uses a generative model (deep learning model) to distinguish between unconscious and conscious behavior. The model is fed with behavioral data, and unconscious behavior is identified based on that data. For example, unconscious behavior can be identified based on fluctuations in the number of steps taken and heart rate at a specific time and location. Walking a specific route during a specific time period (for example, from 6:00 PM to 8:00 PM) is determined to be a routine and regular unconscious behavior.

[0672] Step 7:

[0673] Based on the results of the generative model, the server quantifies the number of unconscious behaviors and time periods, and generates statistical information. This includes the frequency of unconscious behavior, time allocation, and fluctuation patterns. For example, statistics are generated on how much time a user spends walking unconsciously each day.

[0674] Step 8:

[0675] The server uses an emotion engine to recognize the user's emotional state. It integrates behavioral and emotional data to generate statistical information that correlates unconscious behavior with emotional fluctuations. For example, if the emotional state while walking at a specific time indicates "stress," it can be associated with unconscious behavior.

[0676] Step 9:

[0677] The server uses the generated statistics to create personalized advice for users. The advice includes lifestyle improvement tips, reminders, and motivational messages. The system emphasizes advice tailored to the user's emotional state. For example, the system might generate advice like, "You've been feeling stressed lately, so try taking a walk on the weekend to refresh yourself."

[0678] Step 10:

[0679] The server generates advice and notifies the user via LINE or other communication methods. Advice is sent to the user using the LINE API. For example, at 9:00 a.m., a message saying, "Try to walk for 30 minutes today" is sent.

[0680] Step 11:

[0681] The user receives a notification and takes action based on the advice. For example, they follow the notification and take a walk, recording how they felt during the walk. By following the advice, the user is expected to make effective behavioral improvements.

[0682] Step 12:

[0683] The server continuously tracks changes in the user's behavior and emotions, collecting new data, and then re-analyzing the new behavioral and emotional data to generate new personalized advice. This process is repeated continuously.

[0684] These detailed steps clarify the overall processing flow of the system, allowing users to understand their own unconscious behaviors and emotional states and take concrete measures to improve them, leading to more effective health management and the prevention of lifestyle-related diseases.

[0685] Example 2

[0686] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0687] Conventional health management systems can collect and analyze user behavioral data, but it is difficult to provide personalized advice that takes into account the user's emotional state. As a result, the effectiveness of advice is limited, making it difficult to maintain user motivation or lead to actual lifestyle improvements. In addition, there are insufficient ways to handle outliers and missing values, leading to problems with the accuracy and reliability of the data.

[0688] The identification processing by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for collecting user behavioral data from the smart device, including heart rate, number of steps, location information, and facial expression data; means for preprocessing the collected behavioral data and removing outliers and missing values; means for constructing a generative model that distinguishes between unconscious behavior and conscious behavior using the preprocessed behavioral data; means for quantifying unconscious behavior and generating statistical information based on the generative model; means for integrating the generated statistical information with emotional states and generating statistical information that associates unconscious behavior with emotional fluctuations; and means for generating personalized advice for the user based on the generated statistical information and the results of the emotion engine and notifying the user via communication means. This allows the user to receive highly accurate personalized advice based on their behavioral data and emotional data, enabling more effective health management and lifestyle improvement.

[0689] 1. A "smart device" is a wearable device that can collect behavioral data such as heart rate, number of steps, location information, and facial expression data.

[0690] 2. "Behavioral Data" means data collected by smart devices, such as a user's heart rate, number of steps, location information, and facial expression data.

[0691] 3. "Preprocessing" refers to the process of removing outliers and missing values ​​from collected behavioral data and formatting it into a format suitable for analysis.

[0692] 4. "Outliers" are abnormally high or low values ​​in the collected data and values ​​that should be removed for the system to perform accurate analysis.

[0693] 5. A "missing value" is a value that has a missing portion in the collected data.

[0694] 6. A "generative model" is a deep learning model that is built to distinguish between unconscious and conscious behaviors using preprocessed behavioral data.

[0695] 7. "Unconscious behavior" refers to behavior that a user performs unconsciously, such as everyday walking or commuting.

[0696] 8. "Conscious behavior" refers to behavior that a user consciously engages in, such as exercise or consciously performing a specific task.

[0697] 9. “Statistical information” refers to statistical data such as the frequency and time period of unconscious behavior quantified based on a generative model.

[0698] 10. "Emotion Engine" means software or algorithms that analyze a user's facial expression data to identify their emotional state.

[0699] 11. “Personalized Advice” means advice that is individually tailored to a user based on their behavioral and emotional data.

[0700] 12. "Communication Means" means an interface for transmitting information from the Server to the User, including, for example, an Internet connection, a mobile network, or a messaging service.

[0701] This invention is a system that analyzes a user's unconscious behavior and emotional state and provides personalized advice. To implement this system, the following specific hardware and software are used.

[0702] Device:

[0703] A user wears a smart device (e.g., a smartwatch or fitness tracker). The smart device measures behavioral data such as heart rate, steps, location information, and facial expression data in real time. The data is sent to a server via Bluetooth or Wi-Fi. For example, a smartwatch measures heart rate every second and sends the data to a server every hour.

[0704] server:

[0705] The server stores the behavioral data sent from the smart devices in a database. First, the data is preprocessed to remove outliers and missing values. Specifically, a Python script is used to remove outliers with heart rates above 300 and normalize the data. This preprocessing improves the reliability of the data and the accuracy of the analysis.

[0706] Next, the preprocessed data is analyzed using a deep learning model. This generative model (using a deep learning framework such as TensorFlow or PyTorch) distinguishes between unconscious and conscious behavior. For example, if a user walks a specific route between 6:00 PM and 8:00 PM every day, this is recognized as unconscious behavior. The model analyzes walking data and heart rate fluctuations to identify behavioral patterns.

[0707] The number of unconscious actions and the time periods are then quantified to generate statistical information. For example, the number of unconscious steps a user takes in a day is tallied and the time period is quantified (e.g., 5,000 steps in the morning). This statistical information forms the basis for effective advice based on the user's behavior.

[0708] Furthermore, an emotion engine is used to recognize the user's emotional state from facial expression data. Specifically, an emotion-identifying CNN model is used to analyze the facial expression data and classify the user's emotional state into categories such as "stress," "joy," and "surprise." The emotion data is then integrated with behavioral data to generate statistical information relating unconscious behavior to emotional fluctuations. For example, if the emotional state while walking at a specific time indicates "stress," this is associated with unconscious behavior.

[0709] Based on the generated statistics and the results of the emotion engine, personalized advice is generated for the user. This advice includes lifestyle improvements, reminders, and motivational messages. For example, specific advice may be generated such as, "Your stress level has been high lately, so try doing some yoga to relax." This process uses natural language generation (NLG) technology.

[0710] The server notifies the user of the generated advice via LINE or other communication methods (e.g., SMS or email). Upon receiving the notification, the user takes action based on the advice. The behavioral data is then collected again, and continuous data analysis and advice generation are repeated.

[0711] Prompt Sentence Examples

[0712] "A user uses a smartwatch to collect daily activity data, including heart rate, steps, location, and facial expression data. Analyze this data to assess the user's stress level and generate personalized health improvement advice."

[0713] This system enables users to receive highly accurate personalized advice based on their behavioral and emotional data, enabling them to effectively manage their health and improve their lifestyles. Furthermore, by processing outliers and missing values, the system can improve the reliability of data and the accuracy of analysis.

[0714] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0715] Step 1: Data collection

[0716] Device:

[0717] The user wears a smart device, which measures heart rate, steps, location information, and facial expression data in real time. These measurement data are stored in the internal memory at regular intervals (e.g., every second). The stored data is then periodically sent to a server via Bluetooth or Wi-Fi. For example, a smartwatch records heart rate every second and sends it to a server every hour.

[0718] input:

[0719] Your heart rate, steps, location, and facial expression data.

[0720] output:

[0721] The raw data sent to the server.

[0722] Step 2: Data Preprocessing

[0723] server:

[0724] The server stores the received raw data in a database. First, it performs preprocessing to remove outliers and missing values. Specifically, a Python script is used to remove data such as abnormally high heart rates and normalize the data. For example, outliers with heart rates above 300 are removed, and the entire dataset is normalized to a mean of 0 and a standard deviation of 1.

[0725] input:

[0726] Raw data sent from smart devices.

[0727] output:

[0728] Preprocessed data, normalized to remove outliers and missing values.

[0729] Step 3: Data analysis

[0730] server:

[0731] The preprocessed data is then analyzed using a deep learning model. Frameworks such as TensorFlow and PyTorch are used to distinguish between unconscious and conscious behavior using a trained generative model. This analysis detects patterns of walking data and heart rate fluctuations during specific time periods to identify user behavior. For example, walking data from 6:00 PM to 8:00 PM every day is analyzed and recognized as unconscious commuting behavior.

[0732] input:

[0733] Normalized preprocessed data.

[0734] output:

[0735] Data categorized into unconscious and conscious behaviors.

[0736] Step 4: Quantify behavior

[0737] server:

[0738] Based on the analysis results, the number of unconscious actions and time periods are quantified. Specifically, statistical information such as the frequency, average value, and fluctuation range of unconscious actions is generated. For example, the number of unconscious steps a user takes in a day is quantified as 5,000 steps, and the time period is defined as the morning.

[0739] input:

[0740] Data categorized into unconscious and conscious behaviors.

[0741] output:

[0742] Statistics on the number and time of unconscious behaviors.

[0743] Step 5: Emotion Recognition and Integration

[0744] server:

[0745] An emotion engine is used to recognize emotional states from facial expression data. Specifically, an emotion-identifying CNN model analyzes the facial expression data and classifies them into expressions such as "stress," "happiness," and "surprise." Subconscious behavior and emotion data are then integrated to generate statistical information relating unconscious behavior to emotional fluctuations. For example, if "stress" is recognized while walking at a specific time of day, it can be associated with unconscious behavior.

[0746] input:

[0747] Facial expression data, statistical information on unconscious behavior.

[0748] output:

[0749] Statistics of unconscious behavior integrated with emotion data.

[0750] Step 6: Advice Generation

[0751] server:

[0752] Based on the generated statistics and the results of the emotion engine, personalized advice is generated for the user. This advice includes lifestyle improvements, reminders, and motivational messages. For example, specific advice such as "You've been feeling stressed lately, so try doing yoga every night" is generated. Natural language generation technology (NLG) is used to present advice in a format that is easy for the user to understand.

[0753] input:

[0754] Statistics of unconscious behavior integrated with emotion data.

[0755] output:

[0756] Personalized advice.

[0757] Step 7: Notification

[0758] server:

[0759] The generated advice is notified to the user via a communication method (e.g., LINE, SMS, email). For example, the advice content is sent to the user's smartphone using the LINE Messaging API.

[0760] User:

[0761] Receive notifications and take action based on the advice, for example, walking 30 minutes every day as advised.

[0762] input:

[0763] Personalized advice.

[0764] output:

[0765] Advice given to users.

[0766] Step 8: Ongoing support

[0767] server:

[0768] It continuously collects data, tracks changes in user behavior and emotions, and re-analyzes the new data to generate new personalized advice, allowing users to continuously manage their health and improve their lifestyle.

[0769] input:

[0770] New behavioral and emotional data.

[0771] output:

[0772] Updated personalized advice.

[0773] (Application example 2)

[0774] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0775] There is a need to analyze users' unconscious behaviors and emotional states and provide personalized advice for improving their health. However, conventional systems generate advice by analyzing only user behavioral data, and therefore are unable to provide specific advice that takes into account the user's emotional state. Furthermore, even in physical store shopping experiences, it is difficult to analyze customers' unconscious behaviors and emotional states and provide personalized services in real time. To address these challenges, a new system is needed to provide more effective and personalized advice and services.

[0776] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for collecting user behavioral data, including heart rate, step count, location information, and facial expression data, from the smart device; means for preprocessing the collected behavioral data and removing outliers and missing values; means for constructing a generative model that distinguishes between unconscious and conscious behavior using the preprocessed behavioral data; means for quantifying unconscious behavior and generating statistical information based on the generative model; means for analyzing the user's emotional state and integrating it with the behavioral data; and means for generating personalized advice for the user based on the generated statistical information and emotional data and notifying the user via communication means. This makes it possible to analyze the user's unconscious behavior and emotional state and provide specific, personalized advice for improving health. Furthermore, it is possible to analyze customers' unconscious behavior and emotional state and provide personalized services in real time even during shopping experiences in physical stores.

[0777] A "smart device" is a wearable device for collecting user behavioral data and biometric information, and specifically includes devices such as smart watches and smart glasses.

[0778] "Behavioral data" refers to data related to a user's daily movements and physiological responses, such as the user's heart rate, number of steps, location information, and facial expression data.

[0779] "Preprocessing" is the process of removing outliers and missing values ​​from collected behavioral data and converting the data into a format that is easier to analyze.

[0780] A "generative model" is an algorithm or mathematical model that uses preprocessed behavioral data to distinguish between unconscious and conscious behavior.

[0781] "Unconscious behavior" refers to everyday actions and movements that users perform without being particularly conscious of them, and is the behavior that can be analyzed using a generative model.

[0782] "Statistical information" is data generated based on the quantified results of unconscious behavior, and includes statistical characteristics such as frequency, average value, and range of variation.

[0783] "Emotional state" is information that indicates the psychological state of the user, which is analyzed using data such as facial expressions of the user.

[0784] "Personalized advice" refers to specific instructions or suggestions tailored to a user's specific needs, based on their behavioral data and emotional state.

[0785] "Communication methods" are systems used to send advice and notifications to users, including messaging applications and email.

[0786] The present invention relates to a system that analyzes a user's unconscious behavior and emotional state and provides personalized advice for improving their health. This system uses smart devices to collect and analyze behavioral data and provide advice to the user. The specific hardware and software used are shown below.

[0787] Hardware Configuration

[0788] 1. Smart devices: Wearable devices such as smartwatches and smart glasses. These devices collect users' heart rate, steps, location, and facial expression data in real time.

[0789] 2. Server: Preprocesses and analyzes data, builds generative models, and generates advice.

[0790] 3. User terminal: A communication device, such as a smartphone, through which the user receives advice.

[0791] Software Configuration

[0792] 1. Data preprocessing software: Using the Python library scikit-learn, outliers and missing values ​​are removed from the collected behavioral data, and the data is standardized.

[0793] 2. Generative model: A deep learning model for emotion analysis. For example, EmotionRecognitionModel(tentative).

[0794] 3. Communication Software: We use messaging applications and email as a means of communication to send notifications.

[0795] Data processing and calculation

[0796] The server receives data collected from smart devices in real time, removes outliers and missing values, and then feeds the standardized data into a generative model to distinguish between unconscious and conscious behavior. It also analyzes the user's emotional state using an emotion engine and generates statistical information that integrates behavioral and emotional data.

[0797] Specific examples

[0798] For example, the system records the number of steps a user takes throughout the day, fluctuations in heart rate during that time, and emotional data determined from facial expressions, and identifies unconscious behavior from walking patterns and heart rate fluctuations during specific times of the day. If a user walks a specific route every day between 6:00 and 8:00 p.m. and their emotional state during that time indicates "stress," the system will integrate this as unconscious behavior and emotional data and generate and notify the user with specific advice such as, "You seem to be feeling stressed recently. Try making time to refresh yourself on the weekend."

[0799] Prompt Sentence Examples

[0800] "We've detected an elevated heart rate in your customer. They're likely interested in your product, so issue them a special discount coupon."

[0801] This invention enables detailed analysis of users' unconscious behavior and emotional state, and provides specific and personalized advice for improving their health. Furthermore, by providing personalized services in real time in brick-and-mortar stores, it is expected that customer satisfaction and sales will increase.

[0802] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0803] Step 1:

[0804] The server collects user behavioral data from smart devices (smartwatches and smart glasses), including heart rate, steps, location information, and facial expression data. The collected data is sent to the server in real time. The input is raw data from the smart devices, and the output is unprocessed behavioral data stored on the server.

[0805] Step 2:

[0806] The server preprocesses the collected behavioral data. Specifically, it removes outliers and missing values ​​and standardizes the data. For this purpose, it uses libraries such as StandardScaler from scikit-learn. The input is the raw behavioral data, and the output is the preprocessed behavioral data.

[0807] Step 3:

[0808] The server feeds the preprocessed behavioral data into a generative model to build a generative model that distinguishes between unconscious and conscious behaviors. This uses a deep learning model, where the input is the preprocessed behavioral data and the output is labeled data that distinguishes between unconscious and conscious behaviors.

[0809] Step 4:

[0810] The server quantifies unconscious behaviors based on the generative model and generates statistical information, including the number of unconscious behaviors, time periods, frequencies, average values, and fluctuation ranges. The input is labeled data, and the output is quantified unconscious behaviors and statistical information based on them.

[0811] Step 5:

[0812] The server uses an emotion engine to analyze the user's emotional state and integrate it with behavioral data. Specifically, it recognizes the user's emotions from facial expression data and combines them with statistical information on unconscious behavior to generate an integrated dataset. The input is facial expression data and statistical information on unconscious behavior, and the output is the integrated dataset.

[0813] Step 6:

[0814] The server generates personalized advice for users based on the generated integrated data. Specific advice content includes lifestyle improvement tips, reminders, and motivational messages. For example, it generates a message such as, "You seem to be feeling stressed lately. Try to make time to refresh yourself on the weekend." The input is the integrated dataset, and the output is the advice content.

[0815] Step 7:

[0816] The server notifies the user of the generated advice via a communication method (messaging application or email). The input is the advice content, and the output is the notification sent to the user's terminal.

[0817] The above processing steps enable detailed analysis of a user's unconscious behavior and emotional state, enabling the provision of specific and personalized advice for improving health. This also enables real-time personalized services to be provided in physical stores.

[0818] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0819] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0820] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0821] [Third embodiment]

[0822] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0823] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0824] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0825] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0826] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0827] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0828] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0829] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0830] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0831] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0832] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0833] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0834] This system quantifies the user's unconscious behavior and provides personalized advice for improving their health. Below, we will explain the program functions and processing of this system in natural language.

[0835] Data collection

[0836] Device:

[0837] Smart devices worn by users (smartwatches and fitness trackers) measure behavioral data such as heart rate, number of steps, and location information in real time. For example, they record the number of steps a user takes throughout the day and their heart rate fluctuations during that time.

[0838] The device sends the measurement data to the server in real time, using communication infrastructure such as LINE.

[0839] Data Preprocessing

[0840] server:

[0841] The received behavioral data is stored in a database, and outliers and missing values ​​are removed. The data stored in the database is then formatted into a format that is easier to analyze.

[0842] For example, if a user's data contains an abnormally high heart rate, it can be removed and the other data normalized.

[0843] Data analysis

[0844] server:

[0845] The preprocessed data is analyzed using a generative model (deep learning model). This model is designed to distinguish between unconscious and conscious behavior. Unconscious behavior is identified from fluctuations in walking patterns and heart rate during specific time periods.

[0846] For example, if a user walks a specific route every day between 6:00 p.m. and 8:00 p.m., this is determined to be an unconscious activity such as commuting or shopping.

[0847] Quantifying behavior

[0848] server:

[0849] The number of unconscious behaviors and time periods are quantified to generate statistical information, including the frequency, average value, and range of variation of unconscious behaviors.

[0850] For example, it may be shown that the user's daily unconscious walking count is 5,000 steps, and that this occurs mainly in the morning.

[0851] Advice Generation

[0852] server:

[0853] Based on the statistics generated, personalized advice is created for the user, which comes in the form of specific suggestions, reminders and motivational messages to improve their health.

[0854] For example, advice such as, "You unconsciously spend a lot of time sitting. Standing up and doing some light exercise every hour can have a positive effect on your health." is generated.

[0855] notification

[0856] The server generates advice and notifies the user via LINE or other communication methods.

[0857] User:

[0858] Users receive notifications and take action based on the advice, which allows them to change their unconscious behaviors and improve their health.

[0859] Ongoing support

[0860] The server continuously collects data and tracks changes in the user's behavior, generating new statistics and advice in a timely manner. This continuous process allows users to manage their health over the long term.

[0861] This system allows users to accurately understand unconscious behavior and take specific measures to improve it, contributing to the prevention and management of lifestyle-related diseases.

[0862] The processing flow will be explained below.

[0863] Step 1:

[0864] The device begins to measure the user's activity data. The smart device's sensors measure heart rate, steps, location, etc. As the user begins their daily activities, these sensors record data in real time.

[0865] Step 2:

[0866] The device sends the measurement data to the server via LINE. Using the LINE API, the device periodically uploads the data to the server. This process is done automatically, without any burden on the user.

[0867] Step 3:

[0868] The behavioral data received by the server is first stored in a database, where it is tagged and organized for each user.

[0869] Step 4:

[0870] The server performs data cleaning, detecting outliers and missing values ​​from the collected data and removing or filling them in. For example, it removes extremely abnormal heart rate data and fills in missing data points using statistical methods.

[0871] Step 5:

[0872] The server uses the preprocessed data to prepare it for feeding into the generative model. The data is converted into a time series and into a format optimized for analysis.

[0873] Step 6:

[0874] The server uses a generative model (deep learning model) to distinguish between unconscious and conscious behavior. The model is fed with behavioral data and uses it to identify unconscious behavior. For example, it can identify unconscious behavior based on fluctuations in the number of steps taken or heart rate at a specific time and place.

[0875] Step 7:

[0876] Based on the results of the generative model, the server quantifies the number of unconscious behaviors and time periods, and generates statistical information, including the frequency, time allocation, and fluctuation patterns of unconscious behaviors.

[0877] Step 8:

[0878] The server uses the generated statistics to create personalized advice for the user, including lifestyle changes, reminders, and motivational messages.

[0879] Step 9:

[0880] The server notifies the user of the generated advice via LINE. The notification is sent to the user's device using the LINE API. For example, a message such as "Getting up and doing some light exercise every hour can have health benefits" can be sent.

[0881] Step 10:

[0882] The user receives a notification and takes action based on the advice, for example, walking 30 minutes every day.

[0883] Step 11:

[0884] The server continually tracks changes in user behavior, collecting new data, then reanalyzes it based on the new behavioral data to generate new personalized advice, and the process continues.

[0885] These detailed steps clarify the overall processing flow of the system, allowing users to understand their own unconscious behaviors and take concrete actions to improve them.

[0886] Example 1

[0887] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0888] In modern life, many people unconsciously engage in unhealthy behaviors. However, there is a lack of systems that accurately identify these unconscious behaviors and implement specific measures to improve them. Furthermore, existing health management systems often lack the functionality to not only track specific behaviors but also provide users with personalized advice. Therefore, there is a need for a system that can quantify users' unconscious behaviors and provide specific, personalized advice for improving their health based on that information.

[0889] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0890] In this invention, the server includes means for collecting biometric data including heart rate, step count, and location information from the smart device, means for preprocessing the collected biometric data and removing outliers and missing values, means for using the preprocessed biometric data to build a generative model that distinguishes between unconscious and conscious behavior, means for quantifying unconscious behavior and generating statistical information based on the generative model, means for generating personalized advice for the user based on the generated statistical information and notifying the user via communication means, and means for continuously providing the generated advice to the user and tracking changes in behavior. This makes it possible to accurately understand the user's unconscious behavior and take specific improvement measures, thereby promoting improvement of the user's health condition.

[0891] A "smart device" is a device worn by a user to measure biometric data such as heart rate, number of steps, and location information.

[0892] "Biometric data" refers to data related to a user's body, such as heart rate, number of steps, and location information.

[0893] "Means of collection" refers to the method of acquiring biometric data using a smart device and transferring it to a server.

[0894] "Preprocessing means" refers to a method for removing outliers and missing values ​​from collected biometric data and standardizing the data.

[0895] A "generative model" is a mathematical model, such as a deep learning model, that distinguishes between unconscious and conscious behavior from user behavior data.

[0896] "Unconscious behavior" refers to behavior that a user performs without being aware of it, such as walking naturally during commuting or breaks.

[0897] "Conscious behavior" refers to behavior that a user intentionally performs, such as exercise or a specific activity.

[0898] "Means of quantifying" refers to methods of expressing the number of times or time periods of unconscious behavior as specific numbers.

[0899] "Statistical information" refers to statistical data such as the frequency, average value, and range of variation of quantified unconscious behavior.

[0900] "Personalized advice" refers to individually tailored health improvement suggestions and reminders generated based on a user's specific subconscious behaviors and patterns.

[0901] "Communication Method" refers to the communications infrastructure or protocol used to provide generated advice or other notifications to users, such as LINE or Bluetooth.

[0902] "Means for tracking changes in behavior" refers to methods for continuously collecting data on users' unconscious behavior and analyzing and updating statistical information.

[0903] In the embodiment of the present invention, a system for quantifying a user's unconscious behavior and providing personalized advice for improving health will be described in detail. A specific implementation method of this system will be described below.

[0904] Data collection

[0905] Device:

[0906] Smart devices worn by users (e.g., smartwatches and fitness trackers) measure biometric data such as heart rate, number of steps, and location information in real time. For example, they record the number of steps a user takes throughout the day, heart rate fluctuations during that time, and the amount of time spent in a specific location. The collected data is automatically sent to a server. Communication is achieved using technologies such as LINE and Bluetooth.

[0907] Data Preprocessing

[0908] server:

[0909] The server stores the received data in a database. Next, the data is cleansed to remove outliers (e.g., a heart rate of 1000 bpm) and missing values ​​(data that has not been measured). Standardization unifies the units of data, making subsequent analysis easier.

[0910] For example, the server inserts new data into the database, a data cleansing algorithm filters out outliers, and a standardization function is applied to convert the data into a uniform format.

[0911] Data analysis

[0912] server:

[0913] The preprocessed data is then analyzed using a generative AI model (deep learning model), which is designed to distinguish between unconscious and conscious behaviors. For example, continuous walking during a specific time period can be determined to be unconscious behavior.

[0914] As a specific example of how it works, the server inputs data into a deep learning model, and the model analyzes the data to distinguish between unconscious and conscious behavior.

[0915] Quantifying behavior

[0916] server:

[0917] The number of unconscious actions and the time periods are quantified to generate statistical information. This statistical information includes the frequency, average value, and fluctuation range of unconscious actions. For example, it shows the total number of unconscious steps a user takes in a day and the average time period during which they occur.

[0918] As a specific example of operation, the server aggregates the analysis results, and a digitization algorithm generates statistical information.

[0919] Advice Generation

[0920] server:

[0921] Based on the generated statistics, the system creates personalized advice for users. These advice come in the form of specific suggestions, reminders, and motivational messages for improving their health. For example, the system might generate advice like, "You spend a lot of time sitting without realizing it. Standing up and doing some light exercise every hour can help improve your health."

[0922] As a specific example of operation, the server executes an advice generation algorithm based on the statistical information of each user and formats the generated advice as text.

[0923] notification

[0924] server:

[0925] The generated advice will be sent to the user via LINE or other communication means.

[0926] user:

[0927] Users receive notifications and take action based on the advice, which allows them to change their unconscious behaviors and improve their health.

[0928] Specifically, the server creates a notification message and sends it via the LINE API. The user then checks the notification on their smartphone or other device and adjusts their actions accordingly.

[0929] Ongoing support

[0930] server:

[0931] It continuously collects data and tracks changes in user behavior, generating new statistics and advice in a timely manner, allowing users to manage their health over the long term.

[0932] As a specific example of operation, the server periodically receives new behavioral data, repeats preprocessing and analysis, and updates advice at regular intervals to keep notifying the user.

[0933] This system allows users to accurately understand unconscious behavior and take specific measures to improve it, contributing to the prevention and management of lifestyle-related diseases.

[0934] Prompt Sentence Examples

[0935] Sample prompt 1: "Analyze the user's walking data over a specified period of time and generate personalized health advice."

[0936] Sample prompt 2: "Remove outliers and use the preprocessed data to generate statistics on unconscious behavior."

[0937] Based on these prompts, the generative AI model analyzes the user's behavioral data and generates appropriate advice, thus realizing a system that effectively supports users in managing their health.

[0938] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0939] Step 1: Data collection

[0940] Device: A smart device worn by the user measures biometric data such as heart rate, number of steps, and location information in real time. The collected data is automatically sent to a server. Specifically, sensors detect heart rate and number of steps, and location information is measured using GPS. Communication is via LINE or Bluetooth. The input is real-time biometric data, and the output is biometric data sent to the server.

[0941] Specific behavior:

[0942] Sensors measure heart rate, steps, and location.

[0943] The measured data is temporarily stored in the smart device.

[0944] The smart device sends data to the server using LINE or Bluetooth.

[0945] Step 2: Data Preprocessing

[0946] Server: The server stores the data it receives in a database. Next, it cleanses the data to remove outliers (for example, a heart rate of 1000 bpm) and missing values ​​(data that has not been measured). It standardizes the data units and converts them into a format that is easy to analyze. The input is the biometric data sent to the server, and the output is the cleansed, standardized data.

[0947] Specific behavior:

[0948] The server inserts new data into the database.

[0949] Data cleansing algorithms filter out outliers and missing values.

[0950] A standardization function is applied to transform the data into a uniform format.

[0951] Step 3: Data analysis

[0952] Server: The preprocessed data is analyzed using a generative AI model (deep learning model). This model is designed to distinguish between unconscious and conscious behavior. As a result of the analysis, unconscious and conscious behavior are identified. The input is the preprocessed data, and the output is the classification result of unconscious and conscious behavior.

[0953] Specific behavior:

[0954] The server inputs the preprocessed data into the deep learning model.

[0955] The model analyzes the data and distinguishes between unconscious and conscious behavior.

[0956] The analysis results are saved on the server.

[0957] Step 4: Quantify behavior

[0958] Server: Quantifies the number of unconscious behaviors and time periods and generates statistical information. This statistical information includes the frequency, average value, and fluctuation range of unconscious behavior. The input is the classification result of unconscious behavior and conscious behavior, and the output is statistical information.

[0959] Specific behavior:

[0960] The server aggregates the behavioral data based on the analysis results.

[0961] A numerical algorithm calculates the frequency and time period of unconscious behavior as a number.

[0962] Statistics are generated and stored in a database.

[0963] Step 5: Advice Generation

[0964] Server: Creates personalized advice for users based on the generated statistics. These advice come in the form of specific suggestions, reminders, and motivational messages to improve their health. The input is statistics, and the output is personalized advice.

[0965] Specific behavior:

[0966] The server runs an advice generation algorithm based on each user's statistical information.

[0967] Format the generated advice into text.

[0968] Step 6: Notification

[0969] Server: Notifies the user of the generated advice via LINE or other communication methods.

[0970] User: Receives notifications and takes action based on the advice. The input is personalized advice, and the output is a change in user behavior.

[0971] Specific behavior:

[0972] The server creates a notification message and sends it via the LINE API.

[0973] The user checks the notification on their smartphone or other device and follows the advice.

[0974] Step 7: Ongoing support

[0975] Server: Continuously collects data and keeps track of changes in user behavior. New statistics and advice are generated and provided to the user in a timely manner. The input is new biometric data, and the output is updated advice.

[0976] Specific behavior:

[0977] The server periodically receives new behavioral data and repeats the preprocessing and analysis.

[0978] It updates the advice at regular intervals to keep the user informed.

[0979] In this way, users can accurately understand their unconscious behavior and take concrete measures to improve it.

[0980] (Application example 1)

[0981] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0982] Maximizing the operating efficiency of factory robots and reducing unnecessary maintenance are important challenges for improving factory productivity. However, current systems struggle to centrally manage everything from collecting and analyzing robot operation data to predicting and notifying maintenance, and many systems use individual approaches. As a result, maintenance timing is often misjudged, and resources are wasted due to excessive maintenance. There is a need to build a system that solves this problem, maximizes the uptime of factory robots, and provides optimal maintenance schedules.

[0983] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0984] In this invention, the server includes means for collecting user behavioral data, including heart rate, step count, and location information, means for preprocessing the collected behavioral data and removing outliers and missing values, means for using the preprocessed behavioral data to build a generative model that distinguishes between unconscious and conscious behavior, means for quantifying the unconscious behavior based on the generative model and generating statistical information, means for generating personalized advice for the user based on the generated statistical information and notifying the user via a communication means, means for collecting operation data using sensors mounted on the factory robot and measuring operating time and power consumption, means for removing outliers and missing values ​​from the collected operation data and formatting the data into an easily analyzable format, means for analyzing unconscious and conscious behavior using the preprocessed operation data and generating statistical information, and means for predicting factory robot maintenance based on the generated statistical information and notifying a manager. This improves the operating efficiency of the factory robot and enables the provision of an optimal maintenance schedule.

[0985] A "smart device" is a device that can collect user behavioral data (heart rate, number of steps, location information, etc.).

[0986] "Behavioral data" refers to data that includes the operating status and behavior patterns of users and factory robots.

[0987] "Preprocessing" is the process of removing outliers and missing values ​​from collected data and formatting it into a format that is easy to analyze.

[0988] A "generative model" is a deep learning model designed to distinguish between unconscious and conscious behavior.

[0989] "Unconscious behavior" refers to actions that users or factory robots automatically repeat, and refers to routine operations that occur during specific periods of time.

[0990] "Intentional behavior" refers to actions taken by users or factory robots with a specific purpose, including responding to abnormalities and maintenance activities.

[0991] "Statistical information" is information that quantifies and summarizes the frequency and time period of unconscious and conscious behavior, as well as other related data.

[0992] "Personalized advice" is specific, individualized advice tailored to the situation of the user or factory robot, based on collected data and statistical information.

[0993] A "sensor" is a device that is installed on a factory robot and measures operational data (operating time, power consumption, operating route, etc.).

[0994] "Maintenance prediction" refers to predicting when factory robots will require maintenance and the specific work required, based on the generated statistical information.

[0995] This invention provides a system that maximizes the operational efficiency of factory robots and enables the provision of optimal maintenance schedules. The system collects data from sensors installed in smart devices and factory robots, preprocesses and analyzes the data, and provides personalized advice and maintenance predictions.

[0996] Data collection

[0997] The server collects operational data (such as operating time, power consumption, and operating route) in real time from sensors installed in factory robots. Smart devices also collect behavioral data such as the user's heart rate, number of steps, and location information, and send it to the server. This data transfer uses the factory's network infrastructure (Wi-Fi, LAN).

[0998] Data Preprocessing

[0999] The server stores the collected motion and behavior data in a database (MySQL), removes outliers and missing values, and formats the preprocessed data into a format that is easy to analyze.

[1000] Data analysis

[1001] The server uses a TensorFlow-based generative model to analyze the preprocessed data. This generative model is designed to distinguish between unconscious and conscious behavior, identifying unconscious behavior from fluctuations in operating patterns and power consumption during specific time periods.

[1002] Quantification of movement

[1003] The server quantifies unconscious behavior based on the generative model and generates statistical information. This statistical information includes the frequency, average value, and fluctuation range of unconscious behavior. For example, it may show that a certain robot operates unconsciously for 8 hours a day, mainly during the daytime.

[1004] Maintenance prediction and advice generation

[1005] Based on the generated statistical information, the server predicts when factory robots will need maintenance and the specific tasks to be performed. Additionally, it creates messages for users, including advice on lifestyle improvements, daily reminders, and motivational messages. These advice and predictions are then sent to administrators and users via communication methods such as LINE using the Line Messaging API.

[1006] Specific examples

[1007] Here is an example prompt:

[1008] By collecting and analyzing the operation data of your factory robots in real time, we can predict when maintenance will be required and the specific work to be done, reducing wasted time and allowing you to operate your robots efficiently.

[1009] This system is expected to improve factory production efficiency and extend the lifespan of robots. It also allows users to understand their own unconscious behavior and take specific measures to improve it, contributing to the prevention and management of lifestyle-related diseases.

[1010] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1011] Step 1: Data collection

[1012] The terminal measures operational data (operating time, power consumption, operating route, etc.) in real time from sensors installed on factory robots and sends this data to a server. The input is raw data from the sensors, and the output is raw data stored on the server. Specifically, the sensors collect operational data at regular intervals and send it via Wi-Fi or LAN.

[1013] Step 2: Data Preprocessing

[1014] The server stores the received operational data in a database (MySQL) and removes outliers and missing values. The input is raw data, and the output is preprocessed clean data. Specific operations include inserting data into the database, filtering outliers, and completing or removing missing values.

[1015] Step 3: Data analysis

[1016] The server analyzes the preprocessed data through a generative AI model (a deep learning model using TensorFlow). The input is clean data, and the output is analyzed data that distinguishes between unconscious and conscious behavior. Specifically, the data is input into the deep learning model and pattern recognition is performed.

[1017] Step 4: Quantifying behavior

[1018] The server generates statistical information by quantifying the frequency and time period of unconscious behavior based on the data analyzed using the generative model. The input is the analyzed data, and the output is statistical information. Specific operations include frequency calculation, data aggregation by time period, and calculation of the range of variation.

[1019] Step 5: Maintenance prediction and advice generation

[1020] Based on the generated statistical information, the server predicts when factory robot maintenance will be required and the specific work to be done, and generates advice. The input is statistical information, and the output is maintenance prediction data and specific advice. Specific operations include generating a maintenance schedule based on threshold judgments and creating advice for users.

[1021] Step 6: Notification

[1022] The server uses the Line Messaging API to notify administrators and users of the generated maintenance forecast data and advice. The input is the maintenance forecast data and advice, and the output is a notification message. The specific operation involves sending a message using the API.

[1023] Step 7: Ongoing support

[1024] The server continuously collects, analyzes, and notifies the robots to improve their operational efficiency. The input is real-time operational data, and the output is updated analysis data and maintenance predictions. Specifically, the server collects and analyzes data periodically, and then provides timely maintenance predictions and notifications.

[1025] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1026] This invention combines a system that quantifies a user's unconscious behavior and provides personalized advice for improving their health with an emotion engine that identifies the user's emotions. Below, we will explain the program functions and processing of this system in natural language.

[1027] Data collection

[1028] Device:

[1029] Smart devices worn by users (smartwatches and fitness trackers) measure behavioral data in real time, such as heart rate, number of steps, location information, and facial expression data for emotion recognition. For example, they record the number of steps a user takes throughout the day, heart rate fluctuations during that time, and emotional data determined from facial expressions.

[1030] The device sends the measurement data to the server in real time, using communication infrastructure such as LINE.

[1031] Data Preprocessing

[1032] server:

[1033] The received behavioral data is stored in a database, and outliers and missing values ​​are removed. The data stored in the database is then formatted into a format that is easier to analyze.

[1034] For example, if a user's data contains an abnormally high heart rate, it can be removed and the other data normalized.

[1035] Data analysis

[1036] server:

[1037] The preprocessed data is analyzed using a generative model (deep learning model). This model is designed to distinguish between unconscious and conscious behavior. Unconscious behavior is identified from fluctuations in walking patterns and heart rate during specific time periods.

[1038] For example, if a user walks a specific route every day between 6:00 p.m. and 8:00 p.m., this is determined to be an unconscious activity such as commuting or shopping.

[1039] Quantifying behavior

[1040] server:

[1041] The number of unconscious behaviors and time periods are quantified to generate statistical information, including the frequency, average value, and range of variation of unconscious behaviors.

[1042] For example, it may be shown that the user's daily unconscious walking count is 5,000 steps, and that this occurs mainly in the morning.

[1043] Emotion Recognition and Integration

[1044] server:

[1045] The emotional engine recognizes the user's emotional state. It integrates behavioral and emotional data to generate statistical information that correlates unconscious behavior with emotional fluctuations. For example, if the emotional state while walking at a specific time indicates "stress," this can be associated with unconscious behavior.

[1046] Emotion-based advice generation

[1047] server:

[1048] Based on the results of the emotion engine, personalized advice is created for each user, including lifestyle improvements, reminders, and motivational messages, with a particular emphasis on advice tailored to the user's emotional state.

[1049] For example, advice such as "You seem to be feeling stressed lately. Try making time to refresh yourself on the weekend" may be generated.

[1050] notification

[1051] The server generates advice and notifies the user via LINE or other communication methods.

[1052] User:

[1053] Receive notifications and take action based on the advice, for example, walking 30 minutes every day.

[1054] Ongoing support

[1055] The server continuously collects data, tracking changes in user behavior and emotions, and then re-analyzes the new behavioral and emotional data to generate new personalized advice. This process is repeated continuously.

[1056] This system allows users to understand not only their unconscious behaviors but also their emotional state and take specific actions to improve it, resulting in more effective health management and prevention of lifestyle-related diseases.

[1057] The processing flow will be explained below.

[1058] Step 1:

[1059] The device begins to measure the user's behavioral data. The smart device's sensors measure heart rate, steps, location information, and facial expression data for emotion recognition. As the user begins their daily activities, these sensors record data in real time. For example, they collect information on how many steps the user took during the day, heart rate fluctuations during that time, and emotional state from facial expressions.

[1060] Step 2:

[1061] The device sends measurement data to the server via LINE. Using the LINE API, the device periodically uploads data to the server. This process is automatic, and behavioral data is sent to the server without placing any burden on the user. For example, the latest measurement data is sent to the server every hour.

[1062] Step 3:

[1063] The server stores the received behavioral data in a database, which is categorized by user.

[1064] Step 4:

[1065] The server performs data cleaning, detecting outliers and missing values ​​from the collected data and removing or filling them in. For example, extremely high heart rates or extremely high step counts are removed as outliers, and missing values ​​are filled in by guessing values ​​based on the data before and after.

[1066] Step 5:

[1067] The server uses the preprocessed data to prepare it for feeding into the generative model. The data is converted into a time series and into a format optimized for analysis.

[1068] Step 6:

[1069] The server uses a generative model (deep learning model) to distinguish between unconscious and conscious behavior. The model is fed with behavioral data, and unconscious behavior is identified based on that data. For example, unconscious behavior can be identified based on fluctuations in the number of steps taken and heart rate at a specific time and location. Walking a specific route during a specific time period (for example, from 6:00 PM to 8:00 PM) is determined to be a routine and regular unconscious behavior.

[1070] Step 7:

[1071] Based on the results of the generative model, the server quantifies the number of unconscious behaviors and time periods, and generates statistical information. This includes the frequency of unconscious behavior, time allocation, and fluctuation patterns. For example, statistics are generated on how much time a user spends walking unconsciously each day.

[1072] Step 8:

[1073] The server uses an emotion engine to recognize the user's emotional state. It integrates behavioral and emotional data to generate statistical information that correlates unconscious behavior with emotional fluctuations. For example, if the emotional state while walking at a specific time indicates "stress," it can be associated with unconscious behavior.

[1074] Step 9:

[1075] The server uses the generated statistics to create personalized advice for users. The advice includes lifestyle improvement tips, reminders, and motivational messages. The system emphasizes advice tailored to the user's emotional state. For example, the system might generate advice like, "You've been feeling stressed lately, so try taking a walk on the weekend to refresh yourself."

[1076] Step 10:

[1077] The server generates advice and notifies the user via LINE or other communication methods. Advice is sent to the user using the LINE API. For example, at 9:00 a.m., a message saying, "Try to walk for 30 minutes today" is sent.

[1078] Step 11:

[1079] The user receives a notification and takes action based on the advice. For example, they follow the notification and take a walk, recording how they felt during the walk. By following the advice, the user is expected to make effective behavioral improvements.

[1080] Step 12:

[1081] The server continuously tracks changes in the user's behavior and emotions, collecting new data, and then re-analyzing the new behavioral and emotional data to generate new personalized advice. This process is repeated continuously.

[1082] These detailed steps clarify the overall processing flow of the system, allowing users to understand their own unconscious behaviors and emotional states and take concrete measures to improve them, leading to more effective health management and the prevention of lifestyle-related diseases.

[1083] Example 2

[1084] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1085] Conventional health management systems can collect and analyze user behavioral data, but it is difficult to provide personalized advice that takes into account the user's emotional state. As a result, the effectiveness of advice is limited, making it difficult to maintain user motivation or lead to actual lifestyle improvements. In addition, there are insufficient ways to handle outliers and missing values, leading to problems with the accuracy and reliability of the data.

[1086] The identification processing by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for collecting user behavioral data from the smart device, including heart rate, number of steps, location information, and facial expression data; means for preprocessing the collected behavioral data and removing outliers and missing values; means for constructing a generative model that distinguishes between unconscious behavior and conscious behavior using the preprocessed behavioral data; means for quantifying unconscious behavior and generating statistical information based on the generative model; means for integrating the generated statistical information with emotional states and generating statistical information that associates unconscious behavior with emotional fluctuations; and means for generating personalized advice for the user based on the generated statistical information and the results of the emotion engine and notifying the user via communication means. This allows the user to receive highly accurate personalized advice based on their behavioral data and emotional data, enabling more effective health management and lifestyle improvement.

[1087] 1. A "smart device" is a wearable device that can collect behavioral data such as heart rate, number of steps, location information, and facial expression data.

[1088] 2. "Behavioral Data" means data collected by smart devices, such as a user's heart rate, number of steps, location information, and facial expression data.

[1089] 3. "Preprocessing" refers to the process of removing outliers and missing values ​​from collected behavioral data and formatting it into a format suitable for analysis.

[1090] 4. "Outliers" are abnormally high or low values ​​in the collected data and values ​​that should be removed for the system to perform accurate analysis.

[1091] 5. A "missing value" is a value that has a missing portion in the collected data.

[1092] 6. A "generative model" is a deep learning model that is built to distinguish between unconscious and conscious behaviors using preprocessed behavioral data.

[1093] 7. "Unconscious behavior" refers to behavior that a user performs unconsciously, such as everyday walking or commuting.

[1094] 8. "Conscious behavior" refers to behavior that a user consciously engages in, such as exercise or consciously performing a specific task.

[1095] 9. “Statistical information” refers to statistical data such as the frequency and time period of unconscious behavior quantified based on a generative model.

[1096] 10. "Emotion Engine" means software or algorithms that analyze a user's facial expression data to identify their emotional state.

[1097] 11. “Personalized Advice” means advice that is individually tailored to a user based on their behavioral and emotional data.

[1098] 12. "Communication Means" means an interface for transmitting information from the Server to the User, including, for example, an Internet connection, a mobile network, or a messaging service.

[1099] This invention is a system that analyzes a user's unconscious behavior and emotional state and provides personalized advice. To implement this system, the following specific hardware and software are used.

[1100] Device:

[1101] A user wears a smart device (e.g., a smartwatch or fitness tracker). The smart device measures behavioral data such as heart rate, steps, location information, and facial expression data in real time. The data is sent to a server via Bluetooth or Wi-Fi. For example, a smartwatch measures heart rate every second and sends the data to a server every hour.

[1102] server:

[1103] The server stores the behavioral data sent from the smart devices in a database. First, the data is preprocessed to remove outliers and missing values. Specifically, a Python script is used to remove outliers with heart rates above 300 and normalize the data. This preprocessing improves the reliability of the data and the accuracy of the analysis.

[1104] Next, the preprocessed data is analyzed using a deep learning model. This generative model (using a deep learning framework such as TensorFlow or PyTorch) distinguishes between unconscious and conscious behavior. For example, if a user walks a specific route between 6:00 PM and 8:00 PM every day, this is recognized as unconscious behavior. The model analyzes walking data and heart rate fluctuations to identify behavioral patterns.

[1105] The number of unconscious actions and the time periods are then quantified to generate statistical information. For example, the number of unconscious steps a user takes in a day is tallied and the time period is quantified (e.g., 5,000 steps in the morning). This statistical information forms the basis for effective advice based on the user's behavior.

[1106] Furthermore, an emotion engine is used to recognize the user's emotional state from facial expression data. Specifically, an emotion-identifying CNN model is used to analyze the facial expression data and classify the user's emotional state into categories such as "stress," "joy," and "surprise." The emotion data is then integrated with behavioral data to generate statistical information relating unconscious behavior to emotional fluctuations. For example, if the emotional state while walking at a specific time indicates "stress," this is associated with unconscious behavior.

[1107] Based on the generated statistics and the results of the emotion engine, personalized advice is generated for the user. This advice includes lifestyle improvements, reminders, and motivational messages. For example, specific advice may be generated such as, "Your stress level has been high lately, so try doing some yoga to relax." This process uses natural language generation (NLG) technology.

[1108] The server notifies the user of the generated advice via LINE or other communication methods (e.g., SMS or email). Upon receiving the notification, the user takes action based on the advice. The behavioral data is then collected again, and continuous data analysis and advice generation are repeated.

[1109] Prompt Sentence Examples

[1110] "A user uses a smartwatch to collect daily activity data, including heart rate, steps, location, and facial expression data. Analyze this data to assess the user's stress level and generate personalized health improvement advice."

[1111] This system enables users to receive highly accurate personalized advice based on their behavioral and emotional data, enabling them to effectively manage their health and improve their lifestyles. Furthermore, by processing outliers and missing values, the system can improve the reliability of data and the accuracy of analysis.

[1112] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1113] Step 1: Data collection

[1114] Device:

[1115] The user wears a smart device, which measures heart rate, steps, location information, and facial expression data in real time. These measurement data are stored in the internal memory at regular intervals (e.g., every second). The stored data is then periodically sent to a server via Bluetooth or Wi-Fi. For example, a smartwatch records heart rate every second and sends it to a server every hour.

[1116] input:

[1117] Your heart rate, steps, location, and facial expression data.

[1118] output:

[1119] The raw data sent to the server.

[1120] Step 2: Data Preprocessing

[1121] server:

[1122] The server stores the received raw data in a database. First, it performs preprocessing to remove outliers and missing values. Specifically, a Python script is used to remove data such as abnormally high heart rates and normalize the data. For example, outliers with heart rates above 300 are removed, and the entire dataset is normalized to a mean of 0 and a standard deviation of 1.

[1123] input:

[1124] Raw data sent from smart devices.

[1125] output:

[1126] Preprocessed data, normalized to remove outliers and missing values.

[1127] Step 3: Data analysis

[1128] server:

[1129] The preprocessed data is then analyzed using a deep learning model. Frameworks such as TensorFlow and PyTorch are used to distinguish between unconscious and conscious behavior using a trained generative model. This analysis detects patterns of walking data and heart rate fluctuations during specific time periods to identify user behavior. For example, walking data from 6:00 PM to 8:00 PM every day is analyzed and recognized as unconscious commuting behavior.

[1130] input:

[1131] Normalized preprocessed data.

[1132] output:

[1133] Data categorized into unconscious and conscious behaviors.

[1134] Step 4: Quantify behavior

[1135] server:

[1136] Based on the analysis results, the number of unconscious actions and time periods are quantified. Specifically, statistical information such as the frequency, average value, and fluctuation range of unconscious actions is generated. For example, the number of unconscious steps a user takes in a day is quantified as 5,000 steps, and the time period is defined as the morning.

[1137] input:

[1138] Data categorized into unconscious and conscious behaviors.

[1139] output:

[1140] Statistics on the number and time of unconscious behaviors.

[1141] Step 5: Emotion Recognition and Integration

[1142] server:

[1143] An emotion engine is used to recognize emotional states from facial expression data. Specifically, an emotion-identifying CNN model analyzes the facial expression data and classifies them into expressions such as "stress," "happiness," and "surprise." Subconscious behavior and emotion data are then integrated to generate statistical information relating unconscious behavior to emotional fluctuations. For example, if "stress" is recognized while walking at a specific time of day, it can be associated with unconscious behavior.

[1144] input:

[1145] Facial expression data, statistical information on unconscious behavior.

[1146] output:

[1147] Statistics of unconscious behavior integrated with emotion data.

[1148] Step 6: Advice Generation

[1149] server:

[1150] Based on the generated statistics and the results of the emotion engine, personalized advice is generated for the user. This advice includes lifestyle improvements, reminders, and motivational messages. For example, specific advice such as "You've been feeling stressed lately, so try doing yoga every night" is generated. Natural language generation technology (NLG) is used to present advice in a format that is easy for the user to understand.

[1151] input:

[1152] Statistics of unconscious behavior integrated with emotion data.

[1153] output:

[1154] Personalized advice.

[1155] Step 7: Notification

[1156] server:

[1157] The generated advice is notified to the user via a communication method (e.g., LINE, SMS, email). For example, the advice content is sent to the user's smartphone using the LINE Messaging API.

[1158] User:

[1159] Receive notifications and take action based on the advice, for example, walking 30 minutes every day as advised.

[1160] input:

[1161] Personalized advice.

[1162] output:

[1163] Advice given to users.

[1164] Step 8: Ongoing support

[1165] server:

[1166] It continuously collects data, tracks changes in user behavior and emotions, and re-analyzes the new data to generate new personalized advice, allowing users to continuously manage their health and improve their lifestyle.

[1167] input:

[1168] New behavioral and emotional data.

[1169] output:

[1170] Updated personalized advice.

[1171] (Application example 2)

[1172] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1173] There is a need to analyze users' unconscious behaviors and emotional states and provide personalized advice for improving their health. However, conventional systems generate advice by analyzing only user behavioral data, and therefore are unable to provide specific advice that takes into account the user's emotional state. Furthermore, even in physical store shopping experiences, it is difficult to analyze customers' unconscious behaviors and emotional states and provide personalized services in real time. To address these challenges, a new system is needed to provide more effective and personalized advice and services.

[1174] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for collecting user behavioral data, including heart rate, step count, location information, and facial expression data, from the smart device; means for preprocessing the collected behavioral data and removing outliers and missing values; means for constructing a generative model that distinguishes between unconscious and conscious behavior using the preprocessed behavioral data; means for quantifying unconscious behavior and generating statistical information based on the generative model; means for analyzing the user's emotional state and integrating it with the behavioral data; and means for generating personalized advice for the user based on the generated statistical information and emotional data and notifying the user via communication means. This makes it possible to analyze the user's unconscious behavior and emotional state and provide specific, personalized advice for improving health. Furthermore, it is possible to analyze customers' unconscious behavior and emotional state and provide personalized services in real time even during shopping experiences in physical stores.

[1175] A "smart device" is a wearable device for collecting user behavioral data and biometric information, and specifically includes devices such as smart watches and smart glasses.

[1176] "Behavioral data" refers to data related to a user's daily movements and physiological responses, such as the user's heart rate, number of steps, location information, and facial expression data.

[1177] "Preprocessing" is the process of removing outliers and missing values ​​from collected behavioral data and converting the data into a format that is easier to analyze.

[1178] A "generative model" is an algorithm or mathematical model that uses preprocessed behavioral data to distinguish between unconscious and conscious behavior.

[1179] "Unconscious behavior" refers to everyday actions and movements that users perform without being particularly conscious of them, and is the behavior that can be analyzed using a generative model.

[1180] "Statistical information" is data generated based on the quantified results of unconscious behavior, and includes statistical characteristics such as frequency, average value, and range of variation.

[1181] "Emotional state" is information that indicates the psychological state of the user, which is analyzed using data such as facial expressions of the user.

[1182] "Personalized advice" refers to specific instructions or suggestions tailored to a user's specific needs, based on their behavioral data and emotional state.

[1183] "Communication methods" are systems used to send advice and notifications to users, including messaging applications and email.

[1184] The present invention relates to a system that analyzes a user's unconscious behavior and emotional state and provides personalized advice for improving their health. This system uses smart devices to collect and analyze behavioral data and provide advice to the user. The specific hardware and software used are shown below.

[1185] Hardware Configuration

[1186] 1. Smart devices: Wearable devices such as smartwatches and smart glasses. These devices collect users' heart rate, steps, location, and facial expression data in real time.

[1187] 2. Server: Preprocesses and analyzes data, builds generative models, and generates advice.

[1188] 3. User terminal: A communication device, such as a smartphone, through which the user receives advice.

[1189] Software Configuration

[1190] 1. Data preprocessing software: Using the Python library scikit-learn, outliers and missing values ​​are removed from the collected behavioral data, and the data is standardized.

[1191] 2. Generative model: A deep learning model for emotion analysis. For example, EmotionRecognitionModel(tentative).

[1192] 3. Communication Software: We use messaging applications and email as a means of communication to send notifications.

[1193] Data processing and calculation

[1194] The server receives data collected from smart devices in real time, removes outliers and missing values, and then feeds the standardized data into a generative model to distinguish between unconscious and conscious behavior. It also analyzes the user's emotional state using an emotion engine and generates statistical information that integrates behavioral and emotional data.

[1195] Specific examples

[1196] For example, the system records the number of steps a user takes throughout the day, fluctuations in heart rate during that time, and emotional data determined from facial expressions, and identifies unconscious behavior from walking patterns and heart rate fluctuations during specific times of the day. If a user walks a specific route every day between 6:00 and 8:00 p.m. and their emotional state during that time indicates "stress," the system will integrate this as unconscious behavior and emotional data and generate and notify the user with specific advice such as, "You seem to be feeling stressed recently. Try making time to refresh yourself on the weekend."

[1197] Prompt Sentence Examples

[1198] "We've detected an elevated heart rate in your customer. They're likely interested in your product, so issue them a special discount coupon."

[1199] This invention enables detailed analysis of users' unconscious behavior and emotional state, and provides specific and personalized advice for improving their health. Furthermore, by providing personalized services in real time in brick-and-mortar stores, it is expected that customer satisfaction and sales will increase.

[1200] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1201] Step 1:

[1202] The server collects user behavioral data from smart devices (smartwatches and smart glasses), including heart rate, steps, location information, and facial expression data. The collected data is sent to the server in real time. The input is raw data from the smart devices, and the output is unprocessed behavioral data stored on the server.

[1203] Step 2:

[1204] The server preprocesses the collected behavioral data. Specifically, it removes outliers and missing values ​​and standardizes the data. For this purpose, it uses libraries such as StandardScaler from scikit-learn. The input is the raw behavioral data, and the output is the preprocessed behavioral data.

[1205] Step 3:

[1206] The server feeds the preprocessed behavioral data into a generative model to build a generative model that distinguishes between unconscious and conscious behaviors. This uses a deep learning model, where the input is the preprocessed behavioral data and the output is labeled data that distinguishes between unconscious and conscious behaviors.

[1207] Step 4:

[1208] The server quantifies unconscious behaviors based on the generative model and generates statistical information, including the number of unconscious behaviors, time periods, frequencies, average values, and fluctuation ranges. The input is labeled data, and the output is quantified unconscious behaviors and statistical information based on them.

[1209] Step 5:

[1210] The server uses an emotion engine to analyze the user's emotional state and integrate it with behavioral data. Specifically, it recognizes the user's emotions from facial expression data and combines them with statistical information on unconscious behavior to generate an integrated dataset. The input is facial expression data and statistical information on unconscious behavior, and the output is the integrated dataset.

[1211] Step 6:

[1212] The server generates personalized advice for users based on the generated integrated data. Specific advice content includes lifestyle improvement tips, reminders, and motivational messages. For example, it generates a message such as, "You seem to be feeling stressed lately. Try to make time to refresh yourself on the weekend." The input is the integrated dataset, and the output is the advice content.

[1213] Step 7:

[1214] The server notifies the user of the generated advice via a communication method (messaging application or email). The input is the advice content, and the output is the notification sent to the user's terminal.

[1215] The above processing steps enable detailed analysis of a user's unconscious behavior and emotional state, enabling the provision of specific and personalized advice for improving health. This also enables real-time personalized services to be provided in physical stores.

[1216] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1217] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1218] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1219] [Fourth embodiment]

[1220] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1221] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1222] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1223] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1224] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1225] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1226] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1227] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1228] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1229] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1230] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1231] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1232] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1233] This system quantifies the user's unconscious behavior and provides personalized advice for improving their health. Below, we will explain the program functions and processing of this system in natural language.

[1234] Data collection

[1235] Device:

[1236] Smart devices worn by users (smartwatches and fitness trackers) measure behavioral data such as heart rate, number of steps, and location information in real time. For example, they record the number of steps a user takes throughout the day and their heart rate fluctuations during that time.

[1237] The device sends the measurement data to the server in real time, using communication infrastructure such as LINE.

[1238] Data Preprocessing

[1239] server:

[1240] The received behavioral data is stored in a database, and outliers and missing values ​​are removed. The data stored in the database is then formatted into a format that is easier to analyze.

[1241] For example, if a user's data contains an abnormally high heart rate, it can be removed and the other data normalized.

[1242] Data analysis

[1243] server:

[1244] The preprocessed data is analyzed using a generative model (deep learning model). This model is designed to distinguish between unconscious and conscious behavior. Unconscious behavior is identified from fluctuations in walking patterns and heart rate during specific time periods.

[1245] For example, if a user walks a specific route every day between 6:00 p.m. and 8:00 p.m., this is determined to be an unconscious activity such as commuting or shopping.

[1246] Quantifying behavior

[1247] server:

[1248] The number of unconscious behaviors and time periods are quantified to generate statistical information, including the frequency, average value, and range of variation of unconscious behaviors.

[1249] For example, it may be shown that the user's daily unconscious walking count is 5,000 steps, and that this occurs mainly in the morning.

[1250] Advice Generation

[1251] server:

[1252] Based on the statistics generated, personalized advice is created for the user, which comes in the form of specific suggestions, reminders and motivational messages to improve their health.

[1253] For example, advice such as, "You unconsciously spend a lot of time sitting. Standing up and doing some light exercise every hour can have a positive effect on your health." is generated.

[1254] notification

[1255] The server generates advice and notifies the user via LINE or other communication methods.

[1256] User:

[1257] Users receive notifications and take action based on the advice, which allows them to change their unconscious behaviors and improve their health.

[1258] Ongoing support

[1259] The server continuously collects data and tracks changes in the user's behavior, generating new statistics and advice in a timely manner. This continuous process allows users to manage their health over the long term.

[1260] This system allows users to accurately understand unconscious behavior and take specific measures to improve it, contributing to the prevention and management of lifestyle-related diseases.

[1261] The processing flow will be explained below.

[1262] Step 1:

[1263] The device begins to measure the user's activity data. The smart device's sensors measure heart rate, steps, location, etc. As the user begins their daily activities, these sensors record data in real time.

[1264] Step 2:

[1265] The device sends the measurement data to the server via LINE. Using the LINE API, the device periodically uploads the data to the server. This process is done automatically, without any burden on the user.

[1266] Step 3:

[1267] The behavioral data received by the server is first stored in a database, where it is tagged and organized for each user.

[1268] Step 4:

[1269] The server performs data cleaning, detecting outliers and missing values ​​from the collected data and removing or filling them in. For example, it removes extremely abnormal heart rate data and fills in missing data points using statistical methods.

[1270] Step 5:

[1271] The server uses the preprocessed data to prepare it for feeding into the generative model. The data is converted into a time series and into a format optimized for analysis.

[1272] Step 6:

[1273] The server uses a generative model (deep learning model) to distinguish between unconscious and conscious behavior. The model is fed with behavioral data and uses it to identify unconscious behavior. For example, it can identify unconscious behavior based on fluctuations in the number of steps taken or heart rate at a specific time and place.

[1274] Step 7:

[1275] Based on the results of the generative model, the server quantifies the number of unconscious behaviors and time periods, and generates statistical information, including the frequency, time allocation, and fluctuation patterns of unconscious behaviors.

[1276] Step 8:

[1277] The server uses the generated statistics to create personalized advice for the user, including lifestyle changes, reminders, and motivational messages.

[1278] Step 9:

[1279] The server notifies the user of the generated advice via LINE. The notification is sent to the user's device using the LINE API. For example, a message such as "Getting up and doing some light exercise every hour can have health benefits" can be sent.

[1280] Step 10:

[1281] The user receives a notification and takes action based on the advice, for example, walking 30 minutes every day.

[1282] Step 11:

[1283] The server continually tracks changes in user behavior, collecting new data, then reanalyzes it based on the new behavioral data to generate new personalized advice, and the process continues.

[1284] These detailed steps clarify the overall processing flow of the system, allowing users to understand their own unconscious behaviors and take concrete actions to improve them.

[1285] Example 1

[1286] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1287] In modern life, many people unconsciously engage in unhealthy behaviors. However, there is a lack of systems that accurately identify these unconscious behaviors and implement specific measures to improve them. Furthermore, existing health management systems often lack the functionality to not only track specific behaviors but also provide users with personalized advice. Therefore, there is a need for a system that can quantify users' unconscious behaviors and provide specific, personalized advice for improving their health based on that information.

[1288] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1289] In this invention, the server includes means for collecting biometric data including heart rate, step count, and location information from the smart device, means for preprocessing the collected biometric data and removing outliers and missing values, means for using the preprocessed biometric data to build a generative model that distinguishes between unconscious and conscious behavior, means for quantifying unconscious behavior and generating statistical information based on the generative model, means for generating personalized advice for the user based on the generated statistical information and notifying the user via communication means, and means for continuously providing the generated advice to the user and tracking changes in behavior. This makes it possible to accurately understand the user's unconscious behavior and take specific improvement measures, thereby promoting improvement of the user's health condition.

[1290] A "smart device" is a device worn by a user to measure biometric data such as heart rate, number of steps, and location information.

[1291] "Biometric data" refers to data related to a user's body, such as heart rate, number of steps, and location information.

[1292] "Means of collection" refers to the method of acquiring biometric data using a smart device and transferring it to a server.

[1293] "Preprocessing means" refers to a method for removing outliers and missing values ​​from collected biometric data and standardizing the data.

[1294] A "generative model" is a mathematical model, such as a deep learning model, that distinguishes between unconscious and conscious behavior from user behavior data.

[1295] "Unconscious behavior" refers to behavior that a user performs without being aware of it, such as walking naturally during commuting or breaks.

[1296] "Conscious behavior" refers to behavior that a user intentionally performs, such as exercise or a specific activity.

[1297] "Means of quantifying" refers to methods of expressing the number of times or time periods of unconscious behavior as specific numbers.

[1298] "Statistical information" refers to statistical data such as the frequency, average value, and range of variation of quantified unconscious behavior.

[1299] "Personalized advice" refers to individually tailored health improvement suggestions and reminders generated based on a user's specific subconscious behaviors and patterns.

[1300] "Communication Method" refers to the communications infrastructure or protocol used to provide generated advice or other notifications to users, such as LINE or Bluetooth.

[1301] "Means for tracking changes in behavior" refers to methods for continuously collecting data on users' unconscious behavior and analyzing and updating statistical information.

[1302] In the embodiment of the present invention, a system for quantifying a user's unconscious behavior and providing personalized advice for improving health will be described in detail. A specific implementation method of this system will be described below.

[1303] Data collection

[1304] Device:

[1305] Smart devices worn by users (e.g., smartwatches and fitness trackers) measure biometric data such as heart rate, number of steps, and location information in real time. For example, they record the number of steps a user takes throughout the day, heart rate fluctuations during that time, and the amount of time spent in a specific location. The collected data is automatically sent to a server. Communication is achieved using technologies such as LINE and Bluetooth.

[1306] Data Preprocessing

[1307] server:

[1308] The server stores the received data in a database. Next, the data is cleansed to remove outliers (e.g., a heart rate of 1000 bpm) and missing values ​​(data that has not been measured). Standardization unifies the units of data, making subsequent analysis easier.

[1309] For example, the server inserts new data into the database, a data cleansing algorithm filters out outliers, and a standardization function is applied to convert the data into a uniform format.

[1310] Data analysis

[1311] server:

[1312] The preprocessed data is then analyzed using a generative AI model (deep learning model), which is designed to distinguish between unconscious and conscious behaviors. For example, continuous walking during a specific time period can be determined to be unconscious behavior.

[1313] As a specific example of how it works, the server inputs data into a deep learning model, and the model analyzes the data to distinguish between unconscious and conscious behavior.

[1314] Quantifying behavior

[1315] server:

[1316] The number of unconscious actions and the time periods are quantified to generate statistical information. This statistical information includes the frequency, average value, and fluctuation range of unconscious actions. For example, it shows the total number of unconscious steps a user takes in a day and the average time period during which they occur.

[1317] As a specific example of operation, the server aggregates the analysis results, and a digitization algorithm generates statistical information.

[1318] Advice Generation

[1319] server:

[1320] Based on the generated statistics, the system creates personalized advice for users. These advice come in the form of specific suggestions, reminders, and motivational messages for improving their health. For example, the system might generate advice like, "You spend a lot of time sitting without realizing it. Standing up and doing some light exercise every hour can help improve your health."

[1321] As a specific example of operation, the server executes an advice generation algorithm based on the statistical information of each user and formats the generated advice as text.

[1322] notification

[1323] server:

[1324] The generated advice will be sent to the user via LINE or other communication means.

[1325] user:

[1326] Users receive notifications and take action based on the advice, which allows them to change their unconscious behaviors and improve their health.

[1327] Specifically, the server creates a notification message and sends it via the LINE API. The user then checks the notification on their smartphone or other device and adjusts their actions accordingly.

[1328] Ongoing support

[1329] server:

[1330] It continuously collects data and tracks changes in user behavior, generating new statistics and advice in a timely manner, allowing users to manage their health over the long term.

[1331] As a specific example of operation, the server periodically receives new behavioral data, repeats preprocessing and analysis, and updates advice at regular intervals to keep notifying the user.

[1332] This system allows users to accurately understand unconscious behavior and take specific measures to improve it, contributing to the prevention and management of lifestyle-related diseases.

[1333] Prompt Sentence Examples

[1334] Sample prompt 1: "Analyze the user's walking data over a specified period of time and generate personalized health advice."

[1335] Sample prompt 2: "Remove outliers and use the preprocessed data to generate statistics on unconscious behavior."

[1336] Based on these prompts, the generative AI model analyzes the user's behavioral data and generates appropriate advice, thus realizing a system that effectively supports users in managing their health.

[1337] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1338] Step 1: Data collection

[1339] Device: A smart device worn by the user measures biometric data such as heart rate, number of steps, and location information in real time. The collected data is automatically sent to a server. Specifically, sensors detect heart rate and number of steps, and location information is measured using GPS. Communication is via LINE or Bluetooth. The input is real-time biometric data, and the output is biometric data sent to the server.

[1340] Specific behavior:

[1341] Sensors measure heart rate, steps, and location.

[1342] The measured data is temporarily stored in the smart device.

[1343] The smart device sends data to the server using LINE or Bluetooth.

[1344] Step 2: Data Preprocessing

[1345] Server: The server stores the data it receives in a database. Next, it cleanses the data to remove outliers (for example, a heart rate of 1000 bpm) and missing values ​​(data that has not been measured). It standardizes the data units and converts them into a format that is easy to analyze. The input is the biometric data sent to the server, and the output is the cleansed, standardized data.

[1346] Specific behavior:

[1347] The server inserts new data into the database.

[1348] Data cleansing algorithms filter out outliers and missing values.

[1349] A standardization function is applied to transform the data into a uniform format.

[1350] Step 3: Data analysis

[1351] Server: The preprocessed data is analyzed using a generative AI model (deep learning model). This model is designed to distinguish between unconscious and conscious behavior. As a result of the analysis, unconscious and conscious behavior are identified. The input is the preprocessed data, and the output is the classification result of unconscious and conscious behavior.

[1352] Specific behavior:

[1353] The server inputs the preprocessed data into the deep learning model.

[1354] The model analyzes the data and distinguishes between unconscious and conscious behavior.

[1355] The analysis results are saved on the server.

[1356] Step 4: Quantify behavior

[1357] Server: Quantifies the number of unconscious behaviors and time periods and generates statistical information. This statistical information includes the frequency, average value, and fluctuation range of unconscious behavior. The input is the classification result of unconscious behavior and conscious behavior, and the output is statistical information.

[1358] Specific behavior:

[1359] The server aggregates the behavioral data based on the analysis results.

[1360] A numerical algorithm calculates the frequency and time period of unconscious behavior as a number.

[1361] Statistics are generated and stored in a database.

[1362] Step 5: Advice Generation

[1363] Server: Creates personalized advice for users based on the generated statistics. These advice come in the form of specific suggestions, reminders, and motivational messages to improve their health. The input is statistics, and the output is personalized advice.

[1364] Specific behavior:

[1365] The server runs an advice generation algorithm based on each user's statistical information.

[1366] Format the generated advice into text.

[1367] Step 6: Notification

[1368] Server: Notifies the user of the generated advice via LINE or other communication methods.

[1369] User: Receives notifications and takes action based on the advice. The input is personalized advice, and the output is a change in user behavior.

[1370] Specific behavior:

[1371] The server creates a notification message and sends it via the LINE API.

[1372] The user checks the notification on their smartphone or other device and follows the advice.

[1373] Step 7: Ongoing support

[1374] Server: Continuously collects data and keeps track of changes in user behavior. New statistics and advice are generated and provided to the user in a timely manner. The input is new biometric data, and the output is updated advice.

[1375] Specific behavior:

[1376] The server periodically receives new behavioral data and repeats the preprocessing and analysis.

[1377] It updates the advice at regular intervals to keep the user informed.

[1378] In this way, users can accurately understand their unconscious behavior and take concrete measures to improve it.

[1379] (Application example 1)

[1380] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1381] Maximizing the operating efficiency of factory robots and reducing unnecessary maintenance are important challenges for improving factory productivity. However, current systems struggle to centrally manage everything from collecting and analyzing robot operation data to predicting and notifying maintenance, and many systems use individual approaches. As a result, maintenance timing is often misjudged, and resources are wasted due to excessive maintenance. There is a need to build a system that solves this problem, maximizes the uptime of factory robots, and provides optimal maintenance schedules.

[1382] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1383] In this invention, the server includes means for collecting user behavioral data, including heart rate, step count, and location information, means for preprocessing the collected behavioral data and removing outliers and missing values, means for using the preprocessed behavioral data to build a generative model that distinguishes between unconscious and conscious behavior, means for quantifying the unconscious behavior based on the generative model and generating statistical information, means for generating personalized advice for the user based on the generated statistical information and notifying the user via a communication means, means for collecting operation data using sensors mounted on the factory robot and measuring operating time and power consumption, means for removing outliers and missing values ​​from the collected operation data and formatting the data into an easily analyzable format, means for analyzing unconscious and conscious behavior using the preprocessed operation data and generating statistical information, and means for predicting factory robot maintenance based on the generated statistical information and notifying a manager. This improves the operating efficiency of the factory robot and enables the provision of an optimal maintenance schedule.

[1384] A "smart device" is a device that can collect user behavioral data (heart rate, number of steps, location information, etc.).

[1385] "Behavioral data" refers to data that includes the operating status and behavior patterns of users and factory robots.

[1386] "Preprocessing" is the process of removing outliers and missing values ​​from collected data and formatting it into a format that is easy to analyze.

[1387] A "generative model" is a deep learning model designed to distinguish between unconscious and conscious behavior.

[1388] "Unconscious behavior" refers to actions that users or factory robots automatically repeat, and refers to routine operations that occur during specific periods of time.

[1389] "Intentional behavior" refers to actions taken by users or factory robots with a specific purpose, including responding to abnormalities and maintenance activities.

[1390] "Statistical information" is information that quantifies and summarizes the frequency and time period of unconscious and conscious behavior, as well as other related data.

[1391] "Personalized advice" is specific, individualized advice tailored to the situation of the user or factory robot, based on collected data and statistical information.

[1392] A "sensor" is a device that is installed on a factory robot and measures operational data (operating time, power consumption, operating route, etc.).

[1393] "Maintenance prediction" refers to predicting when factory robots will require maintenance and the specific work required, based on the generated statistical information.

[1394] This invention provides a system that maximizes the operational efficiency of factory robots and enables the provision of optimal maintenance schedules. The system collects data from sensors installed in smart devices and factory robots, preprocesses and analyzes the data, and provides personalized advice and maintenance predictions.

[1395] Data collection

[1396] The server collects operational data (such as operating time, power consumption, and operating route) in real time from sensors installed in factory robots. Smart devices also collect behavioral data such as the user's heart rate, number of steps, and location information, and send it to the server. This data transfer uses the factory's network infrastructure (Wi-Fi, LAN).

[1397] Data Preprocessing

[1398] The server stores the collected motion and behavior data in a database (MySQL), removes outliers and missing values, and formats the preprocessed data into a format that is easy to analyze.

[1399] Data analysis

[1400] The server uses a TensorFlow-based generative model to analyze the preprocessed data. This generative model is designed to distinguish between unconscious and conscious behavior, identifying unconscious behavior from fluctuations in operating patterns and power consumption during specific time periods.

[1401] Quantification of movement

[1402] The server quantifies unconscious behavior based on the generative model and generates statistical information. This statistical information includes the frequency, average value, and fluctuation range of unconscious behavior. For example, it may show that a certain robot operates unconsciously for 8 hours a day, mainly during the daytime.

[1403] Maintenance prediction and advice generation

[1404] Based on the generated statistical information, the server predicts when factory robots will need maintenance and the specific tasks to be performed. Additionally, it creates messages for users, including advice on lifestyle improvements, daily reminders, and motivational messages. These advice and predictions are then sent to administrators and users via communication methods such as LINE using the Line Messaging API.

[1405] Specific examples

[1406] Here is an example prompt:

[1407] By collecting and analyzing the operation data of your factory robots in real time, we can predict when maintenance will be required and the specific work to be done, reducing wasted time and allowing you to operate your robots efficiently.

[1408] This system is expected to improve factory production efficiency and extend the lifespan of robots. It also allows users to understand their own unconscious behavior and take specific measures to improve it, contributing to the prevention and management of lifestyle-related diseases.

[1409] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1410] Step 1: Data collection

[1411] The terminal measures operational data (operating time, power consumption, operating route, etc.) in real time from sensors installed on factory robots and sends this data to a server. The input is raw data from the sensors, and the output is raw data stored on the server. Specifically, the sensors collect operational data at regular intervals and send it via Wi-Fi or LAN.

[1412] Step 2: Data Preprocessing

[1413] The server stores the received operational data in a database (MySQL) and removes outliers and missing values. The input is raw data, and the output is preprocessed clean data. Specific operations include inserting data into the database, filtering outliers, and completing or removing missing values.

[1414] Step 3: Data analysis

[1415] The server analyzes the preprocessed data through a generative AI model (a deep learning model using TensorFlow). The input is clean data, and the output is analyzed data that distinguishes between unconscious and conscious behavior. Specifically, the data is input into the deep learning model and pattern recognition is performed.

[1416] Step 4: Quantifying behavior

[1417] The server generates statistical information by quantifying the frequency and time period of unconscious behavior based on the data analyzed using the generative model. The input is the analyzed data, and the output is statistical information. Specific operations include frequency calculation, data aggregation by time period, and calculation of the range of variation.

[1418] Step 5: Maintenance prediction and advice generation

[1419] Based on the generated statistical information, the server predicts when factory robot maintenance will be required and the specific work to be done, and generates advice. The input is statistical information, and the output is maintenance prediction data and specific advice. Specific operations include generating a maintenance schedule based on threshold judgments and creating advice for users.

[1420] Step 6: Notification

[1421] The server uses the Line Messaging API to notify administrators and users of the generated maintenance forecast data and advice. The input is the maintenance forecast data and advice, and the output is a notification message. The specific operation involves sending a message using the API.

[1422] Step 7: Ongoing support

[1423] The server continuously collects, analyzes, and notifies the robots to improve their operational efficiency. The input is real-time operational data, and the output is updated analysis data and maintenance predictions. Specifically, the server collects and analyzes data periodically, and then provides timely maintenance predictions and notifications.

[1424] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1425] This invention combines a system that quantifies a user's unconscious behavior and provides personalized advice for improving their health with an emotion engine that identifies the user's emotions. Below, we will explain the program functions and processing of this system in natural language.

[1426] Data collection

[1427] Device:

[1428] Smart devices worn by users (smartwatches and fitness trackers) measure behavioral data in real time, such as heart rate, number of steps, location information, and facial expression data for emotion recognition. For example, they record the number of steps a user takes throughout the day, heart rate fluctuations during that time, and emotional data determined from facial expressions.

[1429] The device sends the measurement data to the server in real time, using communication infrastructure such as LINE.

[1430] Data Preprocessing

[1431] server:

[1432] The received behavioral data is stored in a database, and outliers and missing values ​​are removed. The data stored in the database is then formatted into a format that is easier to analyze.

[1433] For example, if a user's data contains an abnormally high heart rate, it can be removed and the other data normalized.

[1434] Data analysis

[1435] server:

[1436] The preprocessed data is analyzed using a generative model (deep learning model). This model is designed to distinguish between unconscious and conscious behavior. Unconscious behavior is identified from fluctuations in walking patterns and heart rate during specific time periods.

[1437] For example, if a user walks a specific route every day between 6:00 p.m. and 8:00 p.m., this is determined to be an unconscious activity such as commuting or shopping.

[1438] Quantifying behavior

[1439] server:

[1440] The number of unconscious behaviors and time periods are quantified to generate statistical information, including the frequency, average value, and range of variation of unconscious behaviors.

[1441] For example, it may be shown that the user's daily unconscious walking count is 5,000 steps, and that this occurs mainly in the morning.

[1442] Emotion Recognition and Integration

[1443] server:

[1444] The emotional engine recognizes the user's emotional state. It integrates behavioral and emotional data to generate statistical information that correlates unconscious behavior with emotional fluctuations. For example, if the emotional state while walking at a specific time indicates "stress," this can be associated with unconscious behavior.

[1445] Emotion-based advice generation

[1446] server:

[1447] Based on the results of the emotion engine, personalized advice is created for each user, including lifestyle improvements, reminders, and motivational messages, with a particular emphasis on advice tailored to the user's emotional state.

[1448] For example, advice such as "You seem to be feeling stressed lately. Try making time to refresh yourself on the weekend" may be generated.

[1449] notification

[1450] The server generates advice and notifies the user via LINE or other communication methods.

[1451] User:

[1452] Receive notifications and take action based on the advice, for example, walking 30 minutes every day.

[1453] Ongoing support

[1454] The server continuously collects data, tracking changes in user behavior and emotions, and then re-analyzes the new behavioral and emotional data to generate new personalized advice. This process is repeated continuously.

[1455] This system allows users to understand not only their unconscious behaviors but also their emotional state and take specific actions to improve it, resulting in more effective health management and prevention of lifestyle-related diseases.

[1456] The processing flow will be explained below.

[1457] Step 1:

[1458] The device begins to measure the user's behavioral data. The smart device's sensors measure heart rate, steps, location information, and facial expression data for emotion recognition. As the user begins their daily activities, these sensors record data in real time. For example, they collect information on how many steps the user took during the day, heart rate fluctuations during that time, and emotional state from facial expressions.

[1459] Step 2:

[1460] The device sends measurement data to the server via LINE. Using the LINE API, the device periodically uploads data to the server. This process is automatic, and behavioral data is sent to the server without placing any burden on the user. For example, the latest measurement data is sent to the server every hour.

[1461] Step 3:

[1462] The server stores the received behavioral data in a database, which is categorized by user.

[1463] Step 4:

[1464] The server performs data cleaning, detecting outliers and missing values ​​from the collected data and removing or filling them in. For example, extremely high heart rates or extremely high step counts are removed as outliers, and missing values ​​are filled in by guessing values ​​based on the data before and after.

[1465] Step 5:

[1466] The server uses the preprocessed data to prepare it for feeding into the generative model. The data is converted into a time series and into a format optimized for analysis.

[1467] Step 6:

[1468] The server uses a generative model (deep learning model) to distinguish between unconscious and conscious behavior. The model is fed with behavioral data, and unconscious behavior is identified based on that data. For example, unconscious behavior can be identified based on fluctuations in the number of steps taken and heart rate at a specific time and location. Walking a specific route during a specific time period (for example, from 6:00 PM to 8:00 PM) is determined to be a routine and regular unconscious behavior.

[1469] Step 7:

[1470] Based on the results of the generative model, the server quantifies the number of unconscious behaviors and time periods, and generates statistical information. This includes the frequency of unconscious behavior, time allocation, and fluctuation patterns. For example, statistics are generated on how much time a user spends walking unconsciously each day.

[1471] Step 8:

[1472] The server uses an emotion engine to recognize the user's emotional state. It integrates behavioral and emotional data to generate statistical information that correlates unconscious behavior with emotional fluctuations. For example, if the emotional state while walking at a specific time indicates "stress," it can be associated with unconscious behavior.

[1473] Step 9:

[1474] The server uses the generated statistics to create personalized advice for users. The advice includes lifestyle improvement tips, reminders, and motivational messages. The system emphasizes advice tailored to the user's emotional state. For example, the system might generate advice like, "You've been feeling stressed lately, so try taking a walk on the weekend to refresh yourself."

[1475] Step 10:

[1476] The server generates advice and notifies the user via LINE or other communication methods. Advice is sent to the user using the LINE API. For example, at 9:00 a.m., a message saying, "Try to walk for 30 minutes today" is sent.

[1477] Step 11:

[1478] The user receives a notification and takes action based on the advice. For example, they follow the notification and take a walk, recording how they felt during the walk. By following the advice, the user is expected to make effective behavioral improvements.

[1479] Step 12:

[1480] The server continuously tracks changes in the user's behavior and emotions, collecting new data, and then re-analyzing the new behavioral and emotional data to generate new personalized advice. This process is repeated continuously.

[1481] These detailed steps clarify the overall processing flow of the system, allowing users to understand their own unconscious behaviors and emotional states and take concrete measures to improve them, leading to more effective health management and the prevention of lifestyle-related diseases.

[1482] Example 2

[1483] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1484] Conventional health management systems can collect and analyze user behavioral data, but it is difficult to provide personalized advice that takes into account the user's emotional state. As a result, the effectiveness of advice is limited, making it difficult to maintain user motivation or lead to actual lifestyle improvements. In addition, there are insufficient ways to handle outliers and missing values, leading to problems with the accuracy and reliability of the data.

[1485] The identification processing by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for collecting user behavioral data from the smart device, including heart rate, number of steps, location information, and facial expression data; means for preprocessing the collected behavioral data and removing outliers and missing values; means for constructing a generative model that distinguishes between unconscious behavior and conscious behavior using the preprocessed behavioral data; means for quantifying unconscious behavior and generating statistical information based on the generative model; means for integrating the generated statistical information with emotional states and generating statistical information that associates unconscious behavior with emotional fluctuations; and means for generating personalized advice for the user based on the generated statistical information and the results of the emotion engine and notifying the user via communication means. This allows the user to receive highly accurate personalized advice based on their behavioral data and emotional data, enabling more effective health management and lifestyle improvement.

[1486] 1. A "smart device" is a wearable device that can collect behavioral data such as heart rate, number of steps, location information, and facial expression data.

[1487] 2. "Behavioral Data" means data collected by smart devices, such as a user's heart rate, number of steps, location information, and facial expression data.

[1488] 3. "Preprocessing" refers to the process of removing outliers and missing values ​​from collected behavioral data and formatting it into a format suitable for analysis.

[1489] 4. "Outliers" are abnormally high or low values ​​in the collected data and values ​​that should be removed for the system to perform accurate analysis.

[1490] 5. A "missing value" is a value that has a missing portion in the collected data.

[1491] 6. A "generative model" is a deep learning model that is built to distinguish between unconscious and conscious behaviors using preprocessed behavioral data.

[1492] 7. "Unconscious behavior" refers to behavior that a user performs unconsciously, such as everyday walking or commuting.

[1493] 8. "Conscious behavior" refers to behavior that a user consciously engages in, such as exercise or consciously performing a specific task.

[1494] 9. “Statistical information” refers to statistical data such as the frequency and time period of unconscious behavior quantified based on a generative model.

[1495] 10. "Emotion Engine" means software or algorithms that analyze a user's facial expression data to identify their emotional state.

[1496] 11. “Personalized Advice” means advice that is individually tailored to a user based on their behavioral and emotional data.

[1497] 12. "Communication Means" means an interface for transmitting information from the Server to the User, including, for example, an Internet connection, a mobile network, or a messaging service.

[1498] This invention is a system that analyzes a user's unconscious behavior and emotional state and provides personalized advice. To implement this system, the following specific hardware and software are used.

[1499] Device:

[1500] A user wears a smart device (e.g., a smartwatch or fitness tracker). The smart device measures behavioral data such as heart rate, steps, location information, and facial expression data in real time. The data is sent to a server via Bluetooth or Wi-Fi. For example, a smartwatch measures heart rate every second and sends the data to a server every hour.

[1501] server:

[1502] The server stores the behavioral data sent from the smart devices in a database. First, the data is preprocessed to remove outliers and missing values. Specifically, a Python script is used to remove outliers with heart rates above 300 and normalize the data. This preprocessing improves the reliability of the data and the accuracy of the analysis.

[1503] Next, the preprocessed data is analyzed using a deep learning model. This generative model (using a deep learning framework such as TensorFlow or PyTorch) distinguishes between unconscious and conscious behavior. For example, if a user walks a specific route between 6:00 PM and 8:00 PM every day, this is recognized as unconscious behavior. The model analyzes walking data and heart rate fluctuations to identify behavioral patterns.

[1504] The number of unconscious actions and the time periods are then quantified to generate statistical information. For example, the number of unconscious steps a user takes in a day is tallied and the time period is quantified (e.g., 5,000 steps in the morning). This statistical information forms the basis for effective advice based on the user's behavior.

[1505] Furthermore, an emotion engine is used to recognize the user's emotional state from facial expression data. Specifically, an emotion-identifying CNN model is used to analyze the facial expression data and classify the user's emotional state into categories such as "stress," "joy," and "surprise." The emotion data is then integrated with behavioral data to generate statistical information relating unconscious behavior to emotional fluctuations. For example, if the emotional state while walking at a specific time indicates "stress," this is associated with unconscious behavior.

[1506] Based on the generated statistics and the results of the emotion engine, personalized advice is generated for the user. This advice includes lifestyle improvements, reminders, and motivational messages. For example, specific advice may be generated such as, "Your stress level has been high lately, so try doing some yoga to relax." This process uses natural language generation (NLG) technology.

[1507] The server notifies the user of the generated advice via LINE or other communication methods (e.g., SMS or email). Upon receiving the notification, the user takes action based on the advice. The behavioral data is then collected again, and continuous data analysis and advice generation are repeated.

[1508] Prompt Sentence Examples

[1509] "A user uses a smartwatch to collect daily activity data, including heart rate, steps, location, and facial expression data. Analyze this data to assess the user's stress level and generate personalized health improvement advice."

[1510] This system enables users to receive highly accurate personalized advice based on their behavioral and emotional data, enabling them to effectively manage their health and improve their lifestyles. Furthermore, by processing outliers and missing values, the system can improve the reliability of data and the accuracy of analysis.

[1511] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1512] Step 1: Data collection

[1513] Device:

[1514] The user wears a smart device, which measures heart rate, steps, location information, and facial expression data in real time. These measurement data are stored in the internal memory at regular intervals (e.g., every second). The stored data is then periodically sent to a server via Bluetooth or Wi-Fi. For example, a smartwatch records heart rate every second and sends it to a server every hour.

[1515] input:

[1516] Your heart rate, steps, location, and facial expression data.

[1517] output:

[1518] The raw data sent to the server.

[1519] Step 2: Data Preprocessing

[1520] server:

[1521] The server stores the received raw data in a database. First, it performs preprocessing to remove outliers and missing values. Specifically, a Python script is used to remove data such as abnormally high heart rates and normalize the data. For example, outliers with heart rates above 300 are removed, and the entire dataset is normalized to a mean of 0 and a standard deviation of 1.

[1522] input:

[1523] Raw data sent from smart devices.

[1524] output:

[1525] Preprocessed data, normalized to remove outliers and missing values.

[1526] Step 3: Data analysis

[1527] server:

[1528] The preprocessed data is then analyzed using a deep learning model. Frameworks such as TensorFlow and PyTorch are used to distinguish between unconscious and conscious behavior using a trained generative model. This analysis detects patterns of walking data and heart rate fluctuations during specific time periods to identify user behavior. For example, walking data from 6:00 PM to 8:00 PM every day is analyzed and recognized as unconscious commuting behavior.

[1529] input:

[1530] Normalized preprocessed data.

[1531] output:

[1532] Data categorized into unconscious and conscious behaviors.

[1533] Step 4: Quantify behavior

[1534] server:

[1535] Based on the analysis results, the number of unconscious actions and time periods are quantified. Specifically, statistical information such as the frequency, average value, and fluctuation range of unconscious actions is generated. For example, the number of unconscious steps a user takes in a day is quantified as 5,000 steps, and the time period is defined as the morning.

[1536] input:

[1537] Data categorized into unconscious and conscious behaviors.

[1538] output:

[1539] Statistics on the number and time of unconscious behaviors.

[1540] Step 5: Emotion Recognition and Integration

[1541] server:

[1542] An emotion engine is used to recognize emotional states from facial expression data. Specifically, an emotion-identifying CNN model analyzes the facial expression data and classifies them into expressions such as "stress," "happiness," and "surprise." Subconscious behavior and emotion data are then integrated to generate statistical information relating unconscious behavior to emotional fluctuations. For example, if "stress" is recognized while walking at a specific time of day, it can be associated with unconscious behavior.

[1543] input:

[1544] Facial expression data, statistical information on unconscious behavior.

[1545] output:

[1546] Statistics of unconscious behavior integrated with emotion data.

[1547] Step 6: Advice Generation

[1548] server:

[1549] Based on the generated statistics and the results of the emotion engine, personalized advice is generated for the user. This advice includes lifestyle improvements, reminders, and motivational messages. For example, specific advice such as "You've been feeling stressed lately, so try doing yoga every night" is generated. Natural language generation technology (NLG) is used to present advice in a format that is easy for the user to understand.

[1550] input:

[1551] Statistics of unconscious behavior integrated with emotion data.

[1552] output:

[1553] Personalized advice.

[1554] Step 7: Notification

[1555] server:

[1556] The generated advice is notified to the user via a communication method (e.g., LINE, SMS, email). For example, the advice content is sent to the user's smartphone using the LINE Messaging API.

[1557] User:

[1558] Receive notifications and take action based on the advice, for example, walking 30 minutes every day as advised.

[1559] input:

[1560] Personalized advice.

[1561] output:

[1562] Advice given to users.

[1563] Step 8: Ongoing support

[1564] server:

[1565] It continuously collects data, tracks changes in user behavior and emotions, and re-analyzes the new data to generate new personalized advice, allowing users to continuously manage their health and improve their lifestyle.

[1566] input:

[1567] New behavioral and emotional data.

[1568] output:

[1569] Updated personalized advice.

[1570] (Application example 2)

[1571] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1572] There is a need to analyze users' unconscious behaviors and emotional states and provide personalized advice for improving their health. However, conventional systems generate advice by analyzing only user behavioral data, and therefore are unable to provide specific advice that takes into account the user's emotional state. Furthermore, even in physical store shopping experiences, it is difficult to analyze customers' unconscious behaviors and emotional states and provide personalized services in real time. To address these challenges, a new system is needed to provide more effective and personalized advice and services.

[1573] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for collecting user behavioral data, including heart rate, step count, location information, and facial expression data, from the smart device; means for preprocessing the collected behavioral data and removing outliers and missing values; means for constructing a generative model that distinguishes between unconscious and conscious behavior using the preprocessed behavioral data; means for quantifying unconscious behavior and generating statistical information based on the generative model; means for analyzing the user's emotional state and integrating it with the behavioral data; and means for generating personalized advice for the user based on the generated statistical information and emotional data and notifying the user via communication means. This makes it possible to analyze the user's unconscious behavior and emotional state and provide specific, personalized advice for improving health. Furthermore, it is possible to analyze customers' unconscious behavior and emotional state and provide personalized services in real time even during shopping experiences in physical stores.

[1574] A "smart device" is a wearable device for collecting user behavioral data and biometric information, and specifically includes devices such as smart watches and smart glasses.

[1575] "Behavioral data" refers to data related to a user's daily movements and physiological responses, such as the user's heart rate, number of steps, location information, and facial expression data.

[1576] "Preprocessing" is the process of removing outliers and missing values ​​from collected behavioral data and converting the data into a format that is easier to analyze.

[1577] A "generative model" is an algorithm or mathematical model that uses preprocessed behavioral data to distinguish between unconscious and conscious behavior.

[1578] "Unconscious behavior" refers to everyday actions and movements that users perform without being particularly conscious of them, and is the behavior that can be analyzed using a generative model.

[1579] "Statistical information" is data generated based on the quantified results of unconscious behavior, and includes statistical characteristics such as frequency, average value, and range of variation.

[1580] "Emotional state" is information that indicates the psychological state of the user, which is analyzed using data such as facial expressions of the user.

[1581] "Personalized advice" refers to specific instructions or suggestions tailored to a user's specific needs, based on their behavioral data and emotional state.

[1582] "Communication methods" are systems used to send advice and notifications to users, including messaging applications and email.

[1583] The present invention relates to a system that analyzes a user's unconscious behavior and emotional state and provides personalized advice for improving their health. This system uses smart devices to collect and analyze behavioral data and provide advice to the user. The specific hardware and software used are shown below.

[1584] Hardware Configuration

[1585] 1. Smart devices: Wearable devices such as smartwatches and smart glasses. These devices collect users' heart rate, steps, location, and facial expression data in real time.

[1586] 2. Server: Preprocesses and analyzes data, builds generative models, and generates advice.

[1587] 3. User terminal: A communication device, such as a smartphone, through which the user receives advice.

[1588] Software Configuration

[1589] 1. Data preprocessing software: Using the Python library scikit-learn, outliers and missing values ​​are removed from the collected behavioral data, and the data is standardized.

[1590] 2. Generative model: A deep learning model for emotion analysis. For example, EmotionRecognitionModel(tentative).

[1591] 3. Communication Software: We use messaging applications and email as a means of communication to send notifications.

[1592] Data processing and calculation

[1593] The server receives data collected from smart devices in real time, removes outliers and missing values, and then feeds the standardized data into a generative model to distinguish between unconscious and conscious behavior. It also analyzes the user's emotional state using an emotion engine and generates statistical information that integrates behavioral and emotional data.

[1594] Specific examples

[1595] For example, the system records the number of steps a user takes throughout the day, fluctuations in heart rate during that time, and emotional data determined from facial expressions, and identifies unconscious behavior from walking patterns and heart rate fluctuations during specific times of the day. If a user walks a specific route every day between 6:00 and 8:00 p.m. and their emotional state during that time indicates "stress," the system will integrate this as unconscious behavior and emotional data and generate and notify the user with specific advice such as, "You seem to be feeling stressed recently. Try making time to refresh yourself on the weekend."

[1596] Prompt Sentence Examples

[1597] "We've detected an elevated heart rate in your customer. They're likely interested in your product, so issue them a special discount coupon."

[1598] This invention enables detailed analysis of users' unconscious behavior and emotional state, and provides specific and personalized advice for improving their health. Furthermore, by providing personalized services in real time in brick-and-mortar stores, it is expected that customer satisfaction and sales will increase.

[1599] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1600] Step 1:

[1601] The server collects user behavioral data from smart devices (smartwatches and smart glasses), including heart rate, steps, location information, and facial expression data. The collected data is sent to the server in real time. The input is raw data from the smart devices, and the output is unprocessed behavioral data stored on the server.

[1602] Step 2:

[1603] The server preprocesses the collected behavioral data. Specifically, it removes outliers and missing values ​​and standardizes the data. For this purpose, it uses libraries such as StandardScaler from scikit-learn. The input is the raw behavioral data, and the output is the preprocessed behavioral data.

[1604] Step 3:

[1605] The server feeds the preprocessed behavioral data into a generative model to build a generative model that distinguishes between unconscious and conscious behaviors. This uses a deep learning model, where the input is the preprocessed behavioral data and the output is labeled data that distinguishes between unconscious and conscious behaviors.

[1606] Step 4:

[1607] The server quantifies unconscious behaviors based on the generative model and generates statistical information, including the number of unconscious behaviors, time periods, frequencies, average values, and fluctuation ranges. The input is labeled data, and the output is quantified unconscious behaviors and statistical information based on them.

[1608] Step 5:

[1609] The server uses an emotion engine to analyze the user's emotional state and integrate it with behavioral data. Specifically, it recognizes the user's emotions from facial expression data and combines them with statistical information on unconscious behavior to generate an integrated dataset. The input is facial expression data and statistical information on unconscious behavior, and the output is the integrated dataset.

[1610] Step 6:

[1611] The server generates personalized advice for users based on the generated integrated data. Specific advice content includes lifestyle improvement tips, reminders, and motivational messages. For example, it generates a message such as, "You seem to be feeling stressed lately. Try to make time to refresh yourself on the weekend." The input is the integrated dataset, and the output is the advice content.

[1612] Step 7:

[1613] The server notifies the user of the generated advice via a communication method (messaging application or email). The input is the advice content, and the output is the notification sent to the user's terminal.

[1614] The above processing steps enable detailed analysis of a user's unconscious behavior and emotional state, enabling the provision of specific and personalized advice for improving health. This also enables real-time personalized services to be provided in physical stores.

[1615] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1616] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1617] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1618] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1619] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1620] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1621] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1622] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1623] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1624] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1625] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1626] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1627] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1628] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1629] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1630] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1631] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1632] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1633] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1634] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1635] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1636] The following is further disclosed regarding the above embodiment.

[1637] (Claim 1)

[1638] A means for collecting user behavior data, including heart rate, step count, and location information, from smart devices;

[1639] A means of preprocessing the collected behavioral data to remove outliers and missing values;

[1640] A means of using the preprocessed behavioral data to build a generative model that distinguishes between unconscious and conscious behavior;

[1641] A means for quantifying unconscious behavior and generating statistical information based on the generative model;

[1642] The system includes a means for generating personalized advice for a user based on the generated statistical information and notifying the user via a communication means.

[1643] (Claim 2)

[1644] The system according to claim 1, further comprising means for analyzing the user's behavioral patterns from the preprocessed behavioral data and quantifying the number of unconscious behaviors, time periods, and correlations with other behaviors.

[1645] (Claim 3)

[1646] The system of claim 1 further comprising a means for generating specific advice including lifestyle improvements, daily reminders, and motivational messages based on the generated statistical information, and notifying the user of the advice via a communication means.

[1647] "Example 1"

[1648] (Claim 1)

[1649] A means for collecting biometric data, including heart rate, step count, and location information, from a smart device;

[1650] a means for preprocessing the collected biometric data to remove outliers and missing values;

[1651] A means of constructing a generative model that distinguishes between unconscious and conscious behaviors using preprocessed biometric data; and

[1652] A means for quantifying unconscious behavior and generating statistical information based on the generative model;

[1653] a means for generating personalized advice for a user based on the generated statistical information and notifying the user via a communication means;

[1654] A means to continually provide generated advice to users and track changes in behavior

[1655] A system including:

[1656] (Claim 2)

[1657] The system according to claim 1, further comprising means for analyzing the user's behavioral patterns from the preprocessed biometric data and quantifying the number of unconscious behaviors, time periods, and correlations with other behaviors.

[1658] (Claim 3)

[1659] The system of claim 1 further comprises a means for generating specific advice including lifestyle improvement tips, daily reminders, and motivational messages based on the generated statistical information, and notifying the user of this advice via a communication means.

[1660] "Application Example 1"

[1661] (Claim 1)

[1662] A means for collecting user behavior data, including heart rate, step count, and location information, from smart devices;

[1663] A means of preprocessing the collected behavioral data to remove outliers and missing values;

[1664] A means of using the preprocessed behavioral data to build a generative model that distinguishes between unconscious and conscious behavior;

[1665] A means for quantifying unconscious behavior and generating statistical information based on the generative model;

[1666] means for generating personalized advice for the user based on the generated statistical information and notifying the user via a communication means;

[1667] A means of collecting operational data using sensors installed on factory robots and measuring operating time and power consumption,

[1668] A means of removing outliers and missing values ​​from the collected operational data and formatting it into an easy-to-analyze format;

[1669] a means for analyzing unconscious and conscious behaviors using the preprocessed motion data to generate statistical information;

[1670] A means for predicting maintenance of the factory robot based on the generated statistical information and notifying the manager;

[1671] A system including:

[1672] (Claim 2)

[1673] The system according to claim 1, further comprising means for analyzing the user's behavioral patterns from the preprocessed behavioral data and quantifying the number of unconscious behaviors, time periods, and correlations with other behaviors.

[1674] (Claim 3)

[1675] The system of claim 1 further comprising a means for generating specific advice including lifestyle improvements, daily reminders, and motivational messages based on the generated statistical information, and notifying the user of the advice via a communication means.

[1676] "Example 2: Combining Emotion Engines"

[1677] (Claim 1)

[1678] A means for collecting user behavior data, including heart rate, step count, location information, and facial expression data, from a smart device;

[1679] A means of preprocessing the collected behavioral data to remove outliers and missing values;

[1680] A means of using the preprocessed behavioral data to build a generative model that distinguishes between unconscious and conscious behavior;

[1681] A means for quantifying unconscious behavior and generating statistical information based on the generative model;

[1682] a means for integrating the generated statistical information with the emotional state to generate statistical information relating fluctuations in unconscious behavior and emotions;

[1683] The system includes a means for generating personalized advice for a user based on the generated statistical information and the results of the emotion engine, and notifying the user via a communication means.

[1684] (Claim 2)

[1685] The system according to claim 1, further comprising means for analyzing the user's behavioral patterns from the preprocessed behavioral data and quantifying the number of unconscious behaviors, time periods, and correlations with other behaviors.

[1686] (Claim 3)

[1687] The system of claim 1 further comprises means for generating specific advice including lifestyle improvements, daily reminders, and motivational messages based on the generated statistical information and emotional data, and notifying the user of the advice via a communication means.

[1688] "Application example 2 when combining emotion engines"

[1689] (Claim 1)

[1690] A means for collecting user behavior data, including heart rate, step count, location information, and facial expression data, from a smart device;

[1691] A means of preprocessing the collected behavioral data to remove outliers and missing values;

[1692] A means of using the preprocessed behavioral data to build a generative model that distinguishes between unconscious and conscious behavior;

[1693] A means for quantifying unconscious behavior and generating statistical information based on the generative model;

[1694] A means of analyzing the user's emotional state and integrating it with behavioral data;

[1695] The system includes a means for generating personalized advice for a user based on the generated statistical information and emotional data and notifying the user via a communication means.

[1696] (Claim 2)

[1697] The system according to claim 1, further comprising means for analyzing the user's behavioral patterns from the preprocessed behavioral data and quantifying the number of unconscious behaviors, time periods, and correlations with other behaviors.

[1698] (Claim 3)

[1699] The system of claim 1 further comprising means for generating specific advice including lifestyle improvements, daily reminders, and motivational messages based on the generated statistical information and emotional data, and notifying the user of the advice via a communication means. [Explanation of symbols]

[1700] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. A means for collecting user behavior data, including heart rate, step count, and location information, from smart devices; A means of preprocessing the collected behavioral data to remove outliers and missing values; A means of using the preprocessed behavioral data to build a generative model that distinguishes between unconscious and conscious behavior; A means for quantifying unconscious behavior and generating statistical information based on the generative model; The system includes a means for generating personalized advice for a user based on the generated statistical information and notifying the user via a communication means.

2. The system according to claim 1 , further comprising means for analyzing the user's behavioral patterns from the preprocessed behavioral data and quantifying the number of unconscious behaviors, time periods, and correlations with other behaviors.

3. The system of claim 1 further comprises a means for generating specific advice including lifestyle improvements, daily reminders, and motivational messages based on the generated statistical information, and notifying the user of this advice via a communication means.

Images