system

Abstract

We provide the system. [Solution] A means for collecting user activity data from a sensor device, A means for integrating the aforementioned activity data and user emotions and analyzing them using a generative artificial intelligence model, Based on the aforementioned analysis results, a means for generating hobby programs optimized for the user's interests, A means of providing the hobby program to the user in real time via a communication means, A means for collecting feedback on the execution of the hobby program and improving the generative artificial intelligence model, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, the method including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a character of the chatbot, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance as a response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] It is difficult for existing systems to find hobbies and activities suitable for individual users, and there is also a problem that it is difficult to maintain motivation for discovering new hobbies and deepening existing hobbies. Furthermore, conventional methods lack personalized proposals based on users' interests and feelings, and the lack of opportunities for users to enjoy new experiences is also a problem.

Means for Solving the Problems

[0005] This invention provides a means for collecting user activity data using a sensor device, integrating that activity data with emotions, and analyzing it using a generative artificial intelligence model. Based on the analysis results, it generates a hobby program optimized for the user and provides it in real time via communication means. Furthermore, by collecting feedback during the execution of the hobby program and continuously improving the generative artificial intelligence model, the invention aims to solve the above-mentioned problems by providing a means for providing even more accurate services to the user.

[0006] "Activity data" refers to information about a user's physical or online behavior, collected through sensor devices and applications.

[0007] A "sensor device" is a hardware device used to collect user activity data in real time or periodically.

[0008] "Emotions" refer to information that indicates a user's psychological state, and this data is obtained through surveys, behavioral analysis, and other means.

[0009] A "generative artificial intelligence model" refers to an algorithm that analyzes user activity data and emotional data to recognize patterns and trends.

[0010] A "hobby program" refers to activity plans and suggestions created based on the user's interests and preferences, and is intended to deepen the user's existing hobbies or help them develop new ones.

[0011] "Communication means" refers to methods that enable the sending and receiving of information between a system and a user, using the internet or other network protocols.

[0012] "Feedback" refers to information about the user's experience and is important data used to improve the system.

[0013] A "content library" is a collection of digital content used to provide users with relevant information and entertainment.

[0014] "Virtual or augmented reality technologies" are digital technologies used to provide users with immersive experiences that complement or simulate real-world environments. [Brief explanation of the drawing]

[0015] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Example 2 when a sentiment engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when a sentiment engine is combined.

Mode for Carrying Out the Invention

[0016] Hereinafter, an example of an embodiment of the system according to the technology of the present disclosure will be described according to the accompanying drawings.

[0017] First, the terms used in the following description will be explained.

[0018] In the following embodiments, the numbered processor (hereinafter simply referred to as "processor") may be one arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be one type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0019] In the following embodiments, the numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0020] In the following embodiments, the numbered storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, etc.

[0021] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0022] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0023] [First Embodiment]

[0024] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0025] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0026] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0027] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0028] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0029] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0030] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0031] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0032] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0033] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0034] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0035] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0036] This invention is a system that provides hobby programs optimized based on the individual interests and emotions of the user. This system utilizes advanced generative artificial intelligence technology to support users in discovering new hobbies and deepening their existing ones.

[0037] Data collection and analysis

[0038] First, the device collects user activity data from sensor equipment. This includes movement information, exercise levels, and activity logs from digital channels. Furthermore, data on the user's emotions and psychological state is also obtained through questionnaires and biosensors. This data is transmitted to a server using secure communication methods.

[0039] The server integrates and preprocesses the received activity and emotion data. Next, it uses a generative artificial intelligence model to analyze the data and perform pattern recognition based on the user's interests and emotional responses. This analysis extracts the most suitable hobby activities and new experiences for the user.

[0040] Generating personalized hobby programs

[0041] Based on the analysis results, the server generates a personalized hobby program for the user. This program suggests specific hobbies and activities according to the user's interests, and provides information on how to participate and related resources. For example, a user interested in nature photography might be provided with information on nearby scenic spots and photography workshops.

[0042] Real-time AI coaching

[0043] The generated hobby program is sent to the device via communication and the user is notified. On the device, an AI coaching function provides real-time support and advice as the user engages in hobby activities. This allows the user to proceed at their own pace and maintain continuous motivation.

[0044] Content delivery and continuous improvement

[0045] The server uses a content library to provide information and guides related to the user's interests. It can also utilize virtual or augmented reality technologies to provide users with immersive experiences. User feedback is collected through the device, and the server uses this data to improve the generated artificial intelligence model, thereby increasing the accuracy of future programs.

[0046] These elements allow the system to provide users with a personalized and valuable hobby experience, enabling them to feel new emotions and inspiration.

[0047] The following describes the processing flow.

[0048] Step 1:

[0049] The device collects user activity data. This is done by wearable devices and applications on smartphones, and includes GPS data, heart rate, movement history, and app usage logs. It also collects emotional data by conducting short surveys with users.

[0050] Step 2:

[0051] The device sends the collected data to the server at regular intervals. During this process, a secure protocol (e.g., TLS) is used to ensure data privacy and security.

[0052] Step 3:

[0053] The server integrates the received data. The integrated data undergoes a data cleaning process to impute missing values ​​and remove outliers. This preprocessing creates a dataset suitable for analysis.

[0054] Step 4:

[0055] The server uses a generated artificial intelligence model to analyze the integrated data. The model analyzes the user's interests, activity patterns, and emotional responses, identifying specific patterns and tendencies. This allows for the discovery of hobbies and potential new activities that are suited to the user.

[0056] Step 5:

[0057] Based on the analysis results, the server generates a hobby program optimized for the user. This program includes hobby suggestions, specific activity instructions, and information on necessary resources.

[0058] Step 6:

[0059] The device receives the generated hobby program and notifies the user. The notification is sent via a messaging app, providing the user with guidance and advice on the next steps.

[0060] Step 7:

[0061] Users participate in hobby programs and record their experiences and feedback through the app. This feedback helps maintain motivation and plan future activities.

[0062] Step 8:

[0063] The server collects user feedback and uses it to retrain the generated artificial intelligence model. This process allows the model to more accurately predict user interests and preferences, and to more effectively optimize the next hobby program.

[0064] (Example 1)

[0065] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0066] In modern society, many people are too busy to find opportunities to take up new hobbies. Furthermore, finding a hobby that aligns with one's interests and emotions can be challenging. Therefore, there is a need for a system that can suggest optimal hobbies based on individual user preferences and emotions, thereby enhancing the experience of engaging in hobby activities.

[0067] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0068] In this invention, the server includes means for collecting user activity information from a detection device, means for integrating the activity information and the user's emotional state and analyzing it using a generative machine learning model, and means for constructing hobby activities optimized to the user's preferences based on the analysis results. This makes it possible for users to easily find personalized hobby activities that match their interests and emotions and enjoy a richer life experience.

[0069] A "user" refers to an individual who uses the system to receive suggestions and experiences related to their hobbies.

[0070] "Activity information" refers to data collected by detection devices, such as user movement information, exercise levels, and activity logs from digital channels.

[0071] "Emotional state" refers to data that represents the user's psychological state, obtained through their heart rate and surveys.

[0072] A "detection device" refers to sensors or devices used to collect user activity information.

[0073] A "generative machine learning model" refers to an algorithm or framework used to analyze collected activity information and emotional states to suggest the most suitable hobby activities for the user.

[0074] "Hobby activities" refer to activities and resources suggested based on the user's interests and preferences.

[0075] "Analysis" refers to the process of processing activity information and emotional states using generative machine learning models to recognize patterns based on user interests.

[0076] "Communication technology" refers to data transmission methods used to present hobby activities to users in real time.

[0077] "Response" refers to feedback and evaluations that arise as a result of a user engaging in a suggested hobby activity.

[0078] This invention is a system that suggests personalized hobby activities to users. The system mainly consists of terminals and servers, and collects data using external detection devices and performs analysis using generative machine learning models.

[0079] The device is responsible for collecting user activity information. Specifically, it acquires movement information using a GPS sensor and measures physical activity using an accelerometer. It also records activity logs on digital channels. The user's emotional state is also collected using questionnaire forms and a heart rate sensor. The collected activity information and emotional state are securely transmitted to the server using encryption technology.

[0080] The server integrates and preprocesses the received data, making it suitable for machine learning. Next, it analyzes the data using a generative AI model. Specifically, it uses machine learning frameworks such as TENSORFLOW® and PyTorch to recognize patterns that match the user's interests and preferences. In this process, prompts such as "Suggest a new hobby based on the user's movements and emotional data" are used.

[0081] Based on the analysis, the server generates hobby activities optimized for the user. For example, for a user who enjoys outdoor activities, it recommends nearby hiking trails and provides information on necessary equipment and events. These hobby activities are transmitted to the terminal in real time using communication technology, and the user is notified.

[0082] Users receive notifications via their devices and can receive real-time support and advice on suggested hobby activities through the AI ​​coaching function. This allows users to pursue their hobbies at their own pace and maintain their motivation.

[0083] Finally, the device collects user feedback and sends it back to the server, allowing the generated AI model to be continuously improved. This feedback loop ensures that the system can always provide new suggestions tailored to the user, thereby improving the quality of the user experience.

[0084] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0085] Step 1:

[0086] The device collects user activity information from detection devices. Inputs include GPS sensor location data, accelerometer motion data, and activity logs from digital channels. This data is collected in real time to record the user's activity patterns. The output is user activity information, a combination of this data.

[0087] Step 2:

[0088] The device collects the user's emotional state. Specifically, it collects text data through a survey form and measures physiological responses through a heart rate sensor. The input consists of the user's responses and biosensor data, and the output is a numerical emotional index.

[0089] Step 3:

[0090] The device securely transmits activity information and emotional states collected by the device to the server using encryption technology. The input is raw data collected by the device, and the output is an encrypted data stream. This ensures that the data is transmitted to the server without being leaked to third parties.

[0091] Step 4:

[0092] The server integrates and preprocesses the received data. The data input consists of activity information and emotional states transmitted from the terminal. The server normalizes the data and removes noise to convert it into an easily analyzable format. The output is a preprocessed dataset.

[0093] Step 5:

[0094] The server uses a generated AI model to analyze pre-processed data. In this step, a machine learning algorithm derives the user's interests and preferences based on the input data. The prompt is "Suggest new hobbies based on the user's behavior and emotional data." The output of this analysis is a pattern of hobbies optimized for the user.

[0095] Step 6:

[0096] The server builds hobby activities optimized for the user based on the analysis results. The input is the analysis results, and the output is a hobby suggestion tailored to the user's interests. For example, for a user interested in nature photography, the server will create a suggestion that includes information on suitable locations for landscape photography in the vicinity and photography workshops.

[0097] Step 7:

[0098] The server transmits generated hobby activities to the terminal in real time using communication technology. The input is a proposal from the server, and the output is an activity guide notified to the user. This allows the user to start the activity immediately.

[0099] Step 8:

[0100] Users engage in hobby activities and receive support from an AI coaching function on their device. Inputs include user engagement and feedback during the activity. Outputs include real-time advice and suggestions on how to proceed with the activity.

[0101] Step 9:

[0102] The device collects user feedback and sends it back to the server. The input is user ratings and feedback data. The output is training data to improve the generative AI model. This feedback cycle improves the accuracy of subsequent suggestions.

[0103] (Application Example 1)

[0104] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0105] Modern content delivery technologies struggle to provide users with personalized experiences based on their interests and hobbies, particularly lacking real-time optimization and immersive experiences. Furthermore, the lack of methods for recommending content that aligns with users' emotions and interests often leads to a degraded user experience.

[0106] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0107] In this invention, the server includes means for collecting user activity information from sensor devices, means for integrating the activity information and the user's emotions and analyzing them using a generated artificial intelligence model, means for generating a hobby plan optimized for the user's interests based on the analysis results, means for providing the hobby plan to the user in real time via communication means, means for collecting feedback on the execution of the hobby plan and improving the generated artificial intelligence model, means for recommending relevant visual and musical information according to the user's interests and emotions, and means for providing an interactive experience using augmented reality technology. This enables the provision of personalized content to the user in real time, allowing for a more deeply immersive and interactive hobby experience.

[0108] "User activity information" refers to information that indicates the user's physical movements and actions, collected by sensor devices.

[0109] A "generated artificial intelligence model" is an algorithm that analyzes users' interests and emotions based on accumulated data and provides them with the most suitable content.

[0110] A "hobby plan" is a plan that includes suggestions for activities and content optimized for each user's individual interests.

[0111] "Communication methods" refer to the infrastructure and protocols used to send and receive data.

[0112] "Feedback" refers to additional opinions and impressions based on the user's experience with their hobby plan.

[0113] "Visual and musical information" refers to visual and audio content that is recommended in response to the user's interests and emotions.

[0114] Augmented reality technology is a technology that provides users with new experiences by overlaying digital information onto the real world.

[0115] An "interactive experience" is an experience designed to allow users to actively participate and interact with the content and environment through their actions.

[0116] The system for carrying out this invention mainly includes a sensor device for collecting user activity information, a server for analysis, and a terminal for providing information to the user. The sensor device is embedded in a smartphone or wearable device and is responsible for detecting the user's physical movements and behavioral data and transmitting it to the server.

[0117] The server integrates activity information and emotions based on the received data and performs analysis using a generated artificial intelligence model. Here, the advanced generative AI model identifies the user's interest and emotional patterns and generates a personalized hobby plan. This hobby plan is optimized to capture the user's interest by selecting visual and musical information according to the user's specific preferences. This process uses Python and machine learning libraries to perform data processing and analysis.

[0118] The device receives hobby plans provided by the server via communication and notifies the user in real time. It can also provide interactive experiences using augmented reality technology, overlaying digital information onto the real world through the smartphone's camera, allowing users to experience things like forest walks or museum exhibits with a strong sense of realism. The user's feedback on this experience is sent to the server and used to further improve the generated artificial intelligence model. This entire process provides users with highly customized content, enabling them to enjoy a rich experience.

[0119] As a concrete example, when a user is relaxed, the application can recommend documentary videos related to forests and nature. In this case, an example of a prompt for the generative AI model would be: "Suggest personalized content for the user to enjoy a new nature experience. His mood is relaxed." This allows the system to select the most appropriate information and experience for the user.

[0120] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0121] Step 1:

[0122] Users wear sensor devices while performing their daily activities. The sensor devices detect the user's physical movements and behavioral data in real time and collect it as activity information. This activity information includes steps taken, travel routes, current location, and digital device usage.

[0123] Step 2:

[0124] The terminal transmits activity information collected from sensor devices to a server. The transmitted data is integrated with the user's emotional data on the server. This emotional data is obtained through user input and biosensors and indicates the user's current mood and psychological state.

[0125] Step 3:

[0126] The server inputs integrated activity and sentiment data into a generating AI model. The AI ​​model then analyzes the data and performs specific pattern recognition. The model identifies the user's interests and emotional tendencies and generates an optimized hobby plan. Machine learning algorithms are used in this analysis.

[0127] Step 4:

[0128] The server sends the generated hobby plan to the terminal. This hobby plan includes visual and musical information selected based on the user's interests. The terminal notifies the user in real time and displays the necessary information on the screen.

[0129] Step 5:

[0130] Users engage in hobby activities using hobby plans provided by the device. The device can also provide related augmented reality experiences, allowing users to overlay digital information onto the real world through the camera.

[0131] Step 6:

[0132] After completing a hobby activity, users enter feedback about the experience into their device. This feedback includes how beneficial the hobby plan was and areas where further improvement is needed.

[0133] Step 7:

[0134] The device sends the collected feedback to the server. The server adds this feedback to the dataset of the generating AI model and uses it to improve the next hobby plan generation. This gradually improves the AI ​​model and increases the accuracy of the content it provides.

[0135] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0136] This invention relates to a system for analyzing a user's activity data and emotional state to provide a personalized hobby program. In particular, it uses an emotion engine to collect emotional data and utilize it for analysis, thereby achieving personalization of the user's hobby experience.

[0137] Data collection and emotion recognition

[0138] The device collects user activity data using wearable devices and smartphones. This activity data includes location information, indicators of physical activity, and application usage history. Furthermore, an emotion engine built into the device analyzes the user's facial expressions and voice tone through the camera and microphone, acquiring emotion data in real time.

[0139] Data Integration and Analysis

[0140] The server receives activity and emotion data transmitted from the terminal and integrates them. The integrated data is preprocessed and then analyzed by a generative artificial intelligence model. The model analyzes the user's interests, emotional changes, and behavioral patterns to design a hobby program optimized for the user.

[0141] Generating and providing personalized hobby programs

[0142] Based on the analysis results, the server generates a hobby program that takes into account the user's current emotional state. The generated program includes suggestions for specific hobby activities, detailed steps for those activities, and related resources. Furthermore, by utilizing emotional data, the program is designed to provide the user with the most satisfying experience.

[0143] Real-time AI coaching and feedback collection

[0144] The device notifies the user of a generated hobby program and encourages them to begin activities based on it. While the activities are being carried out, the emotion engine continuously monitors the user's emotions and sends that data to the device. The device uses this data to provide advice and feedback in real time to increase the user's motivation.

[0145] For example, if a user is interested in learning a new musical instrument, the device analyzes the user's emotional data during practice and sends encouraging messages at the appropriate time. By adapting to the user's emotional state in this way, more effective learning and enjoyment of the hobby can be achieved.

[0146] Continuous model improvement

[0147] The server collects feedback even after the activity is completed and continuously improves the generated artificial intelligence model based on that data. This process ensures that subsequent hobby programs are more accurate and tailored to each individual user.

[0148] The present invention aims to provide users with personalized hobby programs that are adjusted in real time by an emotion engine, enabling them to discover new interests and emotions.

[0149] The following describes the processing flow.

[0150] Step 1:

[0151] The device activates its on-board sensor and emotion engine to collect user activity and emotion data. The sensor records the user's location, activity level, and app usage, while the emotion engine analyzes the user's facial expressions and voice to obtain real-time emotion data.

[0152] Step 2:

[0153] The device sends the collected data to the server. The data is sent in batches at regular time intervals, and encrypted communication protocols are used to protect privacy.

[0154] Step 3:

[0155] The server preprocesses the received data and prepares it for analysis. This includes data cleaning, missing value imputation, outlier removal, and data normalization.

[0156] Step 4:

[0157] The server uses an artificial intelligence model to analyze pre-processed data. The model interprets the user's activity patterns and real-time changes in emotions, generating foundational information for a hobby program optimized for the user.

[0158] Step 5:

[0159] Based on the analysis results, the server generates a personalized hobby program for the user. This program includes activity suggestions that take into account the user's current emotional state, along with the steps and resources needed to perform those activities.

[0160] Step 6:

[0161] The device notifies the user of any generated hobby programs. These notifications are delivered through applications the user uses regularly, providing the user with detailed information about the program and recommended steps.

[0162] Step 7:

[0163] The user runs a hobby program, and during execution, the device continuously collects emotional data using an emotion engine. The device analyzes this data in real time and provides advice and positive feedback to support the activity.

[0164] Step 8:

[0165] After the user completes an activity, feedback on the experience obtained through the device is sent to the server. The server then analyzes the feedback and uses it as data to improve the model.

[0166] Step 9:

[0167] The server updates the generated artificial intelligence model based on feedback. This allows for more user-friendly recommendations in the next generation of hobby programs.

[0168] (Example 2)

[0169] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0170] In recent years, there has been a growing demand for systems that provide customized experiences based on individual interests and emotions. Traditional systems struggle to fully utilize users' personal data, resulting in ineffective delivery of programs tailored to individual needs. This has led to a persistent situation where user satisfaction remains low.

[0171] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0172] In this invention, the server includes means for collecting information from a device that acquires the user's physical data and location data; means for integrating the information and the user's emotional state using an analysis device and analyzing it using a generation algorithm model; and means for generating an activity program optimized for the user's interests and emotional state based on the analysis results. This makes it possible to provide a recreation experience optimized for each individual user and improve user satisfaction.

[0173] "User physical data" refers to information indicating the user's physical activity, including steps taken, heart rate, and exercise level.

[0174] "Location data" refers to information about the location of a device or user, including geographical location data such as latitude and longitude.

[0175] An "analysis device" is a device that integrates acquired data and has the function of analyzing users' emotions and behavioral patterns.

[0176] A "generative algorithm model" is an artificial intelligence-based algorithm used to analyze acquired data and generate optimized suggestions and programs for the user.

[0177] An "activity program" is a plan that includes suggestions and procedures for carrying out specific hobbies and recreational activities, generated based on the user's interests and emotional state.

[0178] A "recreational experience" is an experience aimed at enjoyment and relaxation, obtained by users engaging in activities or hobbies suggested to them.

[0179] This invention is a technology system for providing users with personalized recreational experiences. It primarily utilizes the user's physical and location data to generate activity programs optimized for the user's hobbies and interests. The implementation method of this system will be described in detail below.

[0180] The device utilizes sensors built into wearable devices and smartphones to collect the user's physical data. This allows for the acquisition of data such as the user's steps, heart rate, and location. The device also uses its built-in camera and microphone to analyze the user's facial expressions and voice, determining their emotional state in real time.

[0181] The acquired data is sent to a server, which uses an analysis device to integrate this data. The integrated data is then subjected to a generative algorithm model and analyzed based on the user's interests and emotional state. As a result, an activity program optimized for the user is constructed. This program includes suggestions for specific hobby activities, execution steps, and relevant online resources.

[0182] For example, if a user expresses interest in learning a new instrument, the device can provide links to practice programs and online lessons. It can also send encouraging messages and progress-based advice in real time, depending on the user's emotional state.

[0183] This system collects feedback even after an activity is completed and uses that information to improve the generation algorithm model. This process will enable the provision of more effective and smarter programs in the future.

[0184] (Example of a prompt message)

[0185] "How can we analyze user activity and emotional data to suggest the most suitable hobby program?"

[0186] "Please suggest ways for users to maintain motivation while learning a new instrument."

[0187] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0188] Step 1:

[0189] The device collects the user's physical and location data. Specifically, it uses sensors in wearable devices and smartphones to obtain data such as steps taken, heart rate, and location information. The input is this raw data, and the output is structured data that organizes it chronologically.

[0190] Step 2:

[0191] The device uses its internal camera and microphone to analyze the user's facial expressions and voice tone in real time, thereby determining the user's emotional state. The input is collected video and audio data, and the generated output is data indicating the user's specific emotional tendencies.

[0192] Step 3:

[0193] The device sends the physical, location, and emotional data obtained in the previous step to the server. The server receives this data and stores it in its database. The input is the various data sent, and the output is the stored integrated data.

[0194] Step 4:

[0195] The server preprocesses the integrated data and inputs it into the generating AI model. Specifically, it performs noise reduction and imputation of missing data. The input is the integrated raw data, and the output is data formatted in a format suitable for analysis.

[0196] Step 5:

[0197] The server uses a generative AI model to analyze user behavior patterns and emotional tendencies. This allows it to design activity programs optimized for the user. The input is pre-processed data, and the generated output is a specific activity program.

[0198] Step 6:

[0199] The server sends the generated activity program to the terminal, and the terminal notifies the user. The terminal displays the program's contents on the screen, prompting the user to begin the activity. The input is the generated activity program, and the output is the activity start notification received by the user.

[0200] Step 7:

[0201] As the user engages in activities, the device continues to collect emotional data and provide real-time feedback. For example, if progress is good, it sends an encouraging message. The input is emotional data during the activity, and the output is specific feedback to the user.

[0202] Step 8:

[0203] Once the activity is complete, the device collects user feedback and sends it to the server. The server uses this feedback to update the generated AI model and improve the program for the next time. The input is the collected feedback information, and the output is the improved model.

[0204] (Application Example 2)

[0205] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0206] In modern life, there is a demand for personalized content delivery based on an individual's emotional state and interests. However, conventional systems have struggled to provide accurate hobby activities and content that reflect emotional states in real time. This has led to decreased user satisfaction and the inability to obtain an optimal content experience.

[0207] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0208] In this invention, the server includes means for collecting user activity data from a measuring device, means for integrating the activity data and the user's emotions and analyzing them using a generative artificial intelligence model, means for generating hobby activities optimized to the user's interests based on the analysis results, means for providing the hobby activities to the user in real time via communication means, means for collecting feedback on the performance of the hobby activities and improving the generative artificial intelligence model, and means for selecting and providing appropriate video content based on the user's emotional state. This makes it possible to provide a real-time and optimized content experience based on the user's emotional state.

[0209] A "user" refers to an individual who uses the system to engage in hobby activities or receive content.

[0210] "Activity data" refers to data that includes information such as the user's location, indicators of physical activity, and device usage history.

[0211] "Measurement device" refers to sensor devices or equipment used to collect user activity data.

[0212] "Emotions" refer to the psychological state judged from the user's facial expressions, tone of voice, and other factors.

[0213] A "generative artificial intelligence model" refers to a set of algorithms and programs that analyze activity data and emotional data to provide user-optimized suggestions.

[0214] "Hobby activities" refer to recreational and learning activities identified based on the user's interests.

[0215] "Communication methods" refer to the technologies and protocols used to send and receive data and programs between a system and a user.

[0216] "Feedback" refers to information collected from users about their impressions and reactions while engaging in hobby activities.

[0217] "Video content" refers to information in the form of videos or images that is selected based on the user's emotional state.

[0218] The system for implementing this invention is comprised of various technical elements combined to collect and analyze user activity and emotional data. At the heart of this system is a generative artificial intelligence model that provides users with optimized hobby activities and video content in real time. The server implements the invention through the following procedure.

[0219] First, a measuring device worn by the user, such as a smartphone or wearable device, collects activity data. This device incorporates sensors that record location information, indicators of physical activity, and application usage history. In addition, an emotion engine is used to analyze the user's facial expressions and tone of voice to understand their emotional state in real time.

[0220] Next, the server receives data sent from the user's terminal and integrates and analyzes this data using a raw artificial intelligence model. This model, developed using programming languages ​​such as Python, analyzes the user's interests and emotions in detail and generates an optimal hobby activity program. It also selects appropriate video content based on the user's emotions.

[0221] For example, if a user wants to relax after work, the server can recognize that emotional state and notify the device of music videos with relaxing effects.

[0222] Finally, feedback is collected during the execution of the generated hobby activities and used to improve the model's performance. This will improve the accuracy of future hobby activity and content selection.

[0223] Examples of prompts for a generative AI model include:

[0224] "Analyze the user's facial expression data and suggest relaxing content that matches his current emotional state."

[0225] "Please find a program you'll enjoy based on the emotional evaluation derived from this voice tone analysis."

[0226] These are some examples.

[0227] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0228] Step 1:

[0229] The device collects activity data through a measuring device worn by the user. This data includes location information, indicators of physical activity, and application usage history. The input is data signals from sensors, and the output is a structured activity dataset. This activity data is later sent to a server for analysis.

[0230] Step 2:

[0231] The device analyzes facial expressions and voice tone through the camera and microphone to acquire user emotion data using an emotion engine. This input is real-time image and audio data, and the output identifies the user's emotional state. The emotion engine then executes machine learning algorithms to classify the emotion.

[0232] Step 3:

[0233] The server receives activity and emotion data transmitted from the terminal and integrates them. The input is a dataset of activity data and emotion states, and the output is an integrated user characteristics profile. This profile is stored in a database and analyzed using a raw artificial intelligence model.

[0234] Step 4:

[0235] The server uses a generative artificial intelligence model to analyze an integrated profile and generate hobby activities and video content optimized for the user. The input is the user profile, and the output is a recommended hobby activity program and selected content. This process involves the AI ​​model identifying data patterns and recommendation algorithms.

[0236] Step 5:

[0237] The server provides understood hobby activity programs and video content to the terminal in real time via communication means. The input is information about the generated programs and content, and the output is executable instructions sent via the user interface. The user can then take action based on these instructions.

[0238] Step 6:

[0239] The terminal collects feedback while the provided program is running and sends this data to the server. The input is user feedback information, and the output is feedback data for model improvement. The server uses this feedback to continuously learn and improve the raw artificial intelligence model.

[0240] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0241] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0242] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0243] [Second Embodiment]

[0244] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0245] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0246] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0247] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0248] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0249] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0250] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0251] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0252] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0253] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0254] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0255] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0256] This invention is a system that provides hobby programs optimized based on the individual interests and emotions of the user. This system utilizes advanced generative artificial intelligence technology to support users in discovering new hobbies and deepening their existing ones.

[0257] Data collection and analysis

[0258] First, the device collects user activity data from sensor equipment. This includes movement information, exercise levels, and activity logs from digital channels. Furthermore, data on the user's emotions and psychological state is also obtained through questionnaires and biosensors. This data is transmitted to a server using secure communication methods.

[0259] The server integrates and preprocesses the received activity and emotion data. Next, it uses a generative artificial intelligence model to analyze the data and perform pattern recognition based on the user's interests and emotional responses. This analysis extracts the most suitable hobby activities and new experiences for the user.

[0260] Generating personalized hobby programs

[0261] Based on the analysis results, the server generates a personalized hobby program for the user. This program suggests specific hobbies and activities according to the user's interests, and provides information on how to participate and related resources. For example, a user interested in nature photography might be provided with information on nearby scenic spots and photography workshops.

[0262] Real-time AI coaching

[0263] The generated hobby program is sent to the device via communication and the user is notified. On the device, an AI coaching function provides real-time support and advice as the user engages in hobby activities. This allows the user to proceed at their own pace and maintain continuous motivation.

[0264] Content delivery and continuous improvement

[0265] The server uses a content library to provide information and guides related to the user's interests. It can also utilize virtual or augmented reality technologies to provide users with immersive experiences. User feedback is collected through the device, and the server uses this data to improve the generated artificial intelligence model, thereby increasing the accuracy of future programs.

[0266] These elements allow the system to provide users with a personalized and valuable hobby experience, enabling them to feel new emotions and inspiration.

[0267] The following describes the processing flow.

[0268] Step 1:

[0269] The device collects user activity data. This is done by wearable devices and applications on smartphones, and includes GPS data, heart rate, movement history, and app usage logs. It also collects emotional data by conducting short surveys with users.

[0270] Step 2:

[0271] The device sends the collected data to the server at regular intervals. During this process, a secure protocol (e.g., TLS) is used to ensure data privacy and security.

[0272] Step 3:

[0273] The server integrates the received data. The integrated data undergoes a data cleaning process to impute missing values ​​and remove outliers. This preprocessing creates a dataset suitable for analysis.

[0274] Step 4:

[0275] The server analyzes the integrated data using the generated artificial intelligence model. The model analyzes the user's interests, activity patterns, and emotional reactions to identify specific patterns and trends. This enables the discovery of hobbies suitable for the user and potential new activities.

[0276] Step 5:

[0277] Based on the analysis results, the server generates a hobby program optimized for the user. This program includes hobby suggestions, specific procedures for activities, and necessary resource information.

[0278] Step 6:

[0279] The terminal receives the generated hobby program and notifies the user. The notification is carried out via the messaging app, providing the user with guidance and advice for the next steps.

[0280] Step 7:

[0281] The user executes the hobby program and records the experiences and feedback obtained during the process through the app. This feedback helps maintain motivation and plan for future activities.

[0282] Step 8:

[0283] The server collects the user's feedback and retrains the generated artificial intelligence model. This process allows the model to more accurately predict the user's interests and preferences, enabling more effective optimization of the next hobby program.

[0284] (Example 1)

[0285] Next, Example 1 will be described. In the following description, the data processing device 12 is referred to as the "server", and the smart glasses 214 are referred to as the "terminal".

[0286] In modern society, many people do not have the opportunity to have new hobbies due to busyness. Furthermore, it can be difficult to find hobbies that match one's interests and feelings. Therefore, there is a need for a system that can propose optimal hobbies based on users' individual preferences and feelings and improve the experience of hobby activities.

[0287] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0288] In this invention, the server includes means for collecting users' activity information from a detection device, means for integrating the activity information and the users' emotional states and analyzing them using a generated machine learning model, and means for constructing hobby activities optimized for users' preferences based on the analysis results. As a result, it becomes possible for users to easily find personalized hobby activities according to their interests and feelings and enjoy a rich life experience.

[0289] "User" refers to an individual who receives proposals and experiences of hobby activities using the system.

[0290] "Activity information" refers to data collected by a detection device, such as a user's movement information, amount of exercise, and activity logs on digital channels. <opposite

[0291] "Emotional state" refers to data representing the psychological state obtained through a user's heart rate and questionnaire.

[0292] "Detection device" refers to sensors and devices used to collect users' activity information.

[0293] "Generated machine learning model" refers to algorithms and frameworks used to analyze the collected activity information and emotional states and propose optimal hobby activities for users.

[0294] "Hobby activity" refers to activities and resources proposed based on users' interests and preferences.

[0295] "Analysis" refers to the process of processing activity information and emotional states using generative machine learning models to recognize patterns based on user interests.

[0296] "Communication technology" refers to data transmission methods used to present hobby activities to users in real time.

[0297] "Response" refers to feedback and evaluations that arise as a result of a user engaging in a suggested hobby activity.

[0298] This invention is a system that suggests personalized hobby activities to users. The system mainly consists of terminals and servers, and collects data using external detection devices and performs analysis using generative machine learning models.

[0299] The device is responsible for collecting user activity information. Specifically, it acquires movement information using a GPS sensor and measures physical activity using an accelerometer. It also records activity logs on digital channels. The user's emotional state is also collected using questionnaire forms and a heart rate sensor. The collected activity information and emotional state are securely transmitted to the server using encryption technology.

[0300] The server integrates and preprocesses the received data, making it suitable for machine learning. Next, it analyzes the data using a generative AI model. Specifically, it uses machine learning frameworks such as TensorFlow and PyTorch to recognize patterns that match the user's interests and preferences. In this process, prompts such as "Suggest a new hobby based on the user's movements and emotional data" are used.

[0301] As a result of the analysis, the server generates hobby activities optimized for the user. For example, for a user who likes outdoor activities, it recommends nearby hiking trails and provides necessary equipment and event information. This hobby activity is transmitted to the terminal in real time using communication technology and notified to the user.

[0302] The user receives the notification via the terminal and can receive real-time support and advice on the proposed hobby activity through the AI coaching function. This enables the user to proceed with the hobby activity at their own pace and maintain motivation.

[0303] Finally, the terminal collects the reaction from the user and transmits it back to the server, and the generated AI model is sequentially improved. Through this feedback loop, the system can always generate new proposals suitable for the user, improving the quality of the user experience.

[0304] The flow of the specific process in Example 1 will be described using FIG. 11.

[0305] Step 1:

[0306] The terminal collects the user's activity information from the detection device. The inputs include the position data of the GPS sensor, the motion data of the acceleration sensor, and the activity log on the digital channel. These data are collected in real time to record the user's activity pattern. The output is the user's activity information combining these data.

[0307] Step 2:

[0308] The terminal collects the user's emotional state. Specifically, in addition to inputting text data in the questionnaire form, it measures the physiological reaction through the heart rate sensor. The inputs are the user's answers and biometric sensor data, and the output is the quantified emotional index.

[0309] Step 3:

[0310] The device securely transmits activity information and emotional states collected by the device to the server using encryption technology. The input is raw data collected by the device, and the output is an encrypted data stream. This ensures that the data is transmitted to the server without being leaked to third parties.

[0311] Step 4:

[0312] The server integrates and preprocesses the received data. The data input consists of activity information and emotional states transmitted from the terminal. The server normalizes the data and removes noise to convert it into an easily analyzable format. The output is a preprocessed dataset.

[0313] Step 5:

[0314] The server uses a generated AI model to analyze pre-processed data. In this step, a machine learning algorithm derives the user's interests and preferences based on the input data. The prompt is "Suggest new hobbies based on the user's behavior and emotional data." The output of this analysis is a pattern of hobbies optimized for the user.

[0315] Step 6:

[0316] The server builds hobby activities optimized for the user based on the analysis results. The input is the analysis results, and the output is a hobby suggestion tailored to the user's interests. For example, for a user interested in nature photography, the server will create a suggestion that includes information on suitable locations for landscape photography in the vicinity and photography workshops.

[0317] Step 7:

[0318] The server transmits generated hobby activities to the terminal in real time using communication technology. The input is a proposal from the server, and the output is an activity guide notified to the user. This allows the user to start the activity immediately.

[0319] Step 8:

[0320] Users engage in hobby activities and receive support from an AI coaching function on their device. Inputs include user engagement and feedback during the activity. Outputs include real-time advice and suggestions on how to proceed with the activity.

[0321] Step 9:

[0322] The device collects user feedback and sends it back to the server. The input is user ratings and feedback data. The output is training data to improve the generative AI model. This feedback cycle improves the accuracy of subsequent suggestions.

[0323] (Application Example 1)

[0324] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0325] Modern content delivery technologies struggle to provide users with personalized experiences based on their interests and hobbies, particularly lacking real-time optimization and immersive experiences. Furthermore, the lack of methods for recommending content that aligns with users' emotions and interests often leads to a degraded user experience.

[0326] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0327] In this invention, the server includes means for collecting user activity information from sensor devices, means for integrating the activity information and the user's emotions and analyzing them using a generated artificial intelligence model, means for generating a hobby plan optimized for the user's interests based on the analysis results, means for providing the hobby plan to the user in real time via communication means, means for collecting feedback on the execution of the hobby plan and improving the generated artificial intelligence model, means for recommending relevant visual and musical information according to the user's interests and emotions, and means for providing an interactive experience using augmented reality technology. This enables the provision of personalized content to the user in real time, allowing for a more deeply immersive and interactive hobby experience.

[0328] "User activity information" refers to information that indicates the user's physical movements and actions, collected by sensor devices.

[0329] A "generated artificial intelligence model" is an algorithm that analyzes users' interests and emotions based on accumulated data and provides them with the most suitable content.

[0330] A "hobby plan" is a plan that includes suggestions for activities and content optimized for each user's individual interests.

[0331] "Communication methods" refer to the infrastructure and protocols used to send and receive data.

[0332] "Feedback" refers to additional opinions and impressions based on the user's experience with their hobby plan.

[0333] "Visual and musical information" refers to visual and audio content that is recommended in response to the user's interests and emotions.

[0334] Augmented reality technology is a technology that provides users with new experiences by overlaying digital information onto the real world.

[0335] An "interactive experience" is an experience designed to allow users to actively participate and interact with the content and environment through their actions.

[0336] The system for carrying out this invention mainly includes a sensor device for collecting user activity information, a server for analysis, and a terminal for providing information to the user. The sensor device is embedded in a smartphone or wearable device and is responsible for detecting the user's physical movements and behavioral data and transmitting it to the server.

[0337] The server integrates activity information and emotions based on the received data and performs analysis using a generated artificial intelligence model. Here, the advanced generative AI model identifies the user's interest and emotional patterns and generates a personalized hobby plan. This hobby plan is optimized to capture the user's interest by selecting visual and musical information according to the user's specific preferences. This process uses Python and machine learning libraries to perform data processing and analysis.

[0338] The device receives hobby plans provided by the server via communication and notifies the user in real time. It can also provide interactive experiences using augmented reality technology, overlaying digital information onto the real world through the smartphone's camera, allowing users to experience things like forest walks or museum exhibits with a strong sense of realism. The user's feedback on this experience is sent to the server and used to further improve the generated artificial intelligence model. This entire process provides users with highly customized content, enabling them to enjoy a rich experience.

[0339] As a concrete example, when a user is relaxed, the application can recommend documentary videos related to forests and nature. In this case, an example of a prompt for the generative AI model would be: "Suggest personalized content for the user to enjoy a new nature experience. His mood is relaxed." This allows the system to select the most appropriate information and experience for the user.

[0340] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0341] Step 1:

[0342] Users wear sensor devices while performing their daily activities. The sensor devices detect the user's physical movements and behavioral data in real time and collect it as activity information. This activity information includes steps taken, travel routes, current location, and digital device usage.

[0343] Step 2:

[0344] The terminal transmits activity information collected from sensor devices to a server. The transmitted data is integrated with the user's emotional data on the server. This emotional data is obtained through user input and biosensors and indicates the user's current mood and psychological state.

[0345] Step 3:

[0346] The server inputs integrated activity and sentiment data into a generating AI model. The AI ​​model then analyzes the data and performs specific pattern recognition. The model identifies the user's interests and emotional tendencies and generates an optimized hobby plan. Machine learning algorithms are used in this analysis.

[0347] Step 4:

[0348] The server sends the generated hobby plan to the terminal. This hobby plan includes visual and musical information selected based on the user's interests. The terminal notifies the user in real time and displays the necessary information on the screen.

[0349] Step 5:

[0350] Users engage in hobby activities using hobby plans provided by the device. The device can also provide related augmented reality experiences, allowing users to overlay digital information onto the real world through the camera.

[0351] Step 6:

[0352] After completing a hobby activity, users enter feedback about the experience into their device. This feedback includes how beneficial the hobby plan was and areas where further improvement is needed.

[0353] Step 7:

[0354] The device sends the collected feedback to the server. The server adds this feedback to the dataset of the generating AI model and uses it to improve the next hobby plan generation. This gradually improves the AI ​​model and increases the accuracy of the content it provides.

[0355] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0356] This invention relates to a system for analyzing a user's activity data and emotional state to provide a personalized hobby program. In particular, it uses an emotion engine to collect emotional data and utilize it for analysis, thereby achieving personalization of the user's hobby experience.

[0357] Data collection and emotion recognition

[0358] The device collects user activity data using wearable devices and smartphones. This activity data includes location information, indicators of physical activity, and application usage history. Furthermore, an emotion engine built into the device analyzes the user's facial expressions and voice tone through the camera and microphone, acquiring emotion data in real time.

[0359] Data Integration and Analysis

[0360] The server receives activity and emotion data transmitted from the terminal and integrates them. The integrated data is preprocessed and then analyzed by a generative artificial intelligence model. The model analyzes the user's interests, emotional changes, and behavioral patterns to design a hobby program optimized for the user.

[0361] Generating and providing personalized hobby programs

[0362] Based on the analysis results, the server generates a hobby program that takes into account the user's current emotional state. The generated program includes suggestions for specific hobby activities, detailed steps for those activities, and related resources. Furthermore, by utilizing emotional data, the program is designed to provide the user with the most satisfying experience.

[0363] Real-time AI coaching and feedback collection

[0364] The device notifies the user of a generated hobby program and encourages them to begin activities based on it. While the activities are being carried out, the emotion engine continuously monitors the user's emotions and sends that data to the device. The device uses this data to provide advice and feedback in real time to increase the user's motivation.

[0365] For example, if a user is interested in learning a new musical instrument, the device analyzes the user's emotional data during practice and sends encouraging messages at the appropriate time. By adapting to the user's emotional state in this way, more effective learning and enjoyment of the hobby can be achieved.

[0366] Continuous model improvement

[0367] The server collects feedback even after the activity is completed and continuously improves the generated artificial intelligence model based on that data. This process ensures that subsequent hobby programs are more accurate and tailored to each individual user.

[0368] The present invention aims to provide users with personalized hobby programs that are adjusted in real time by an emotion engine, enabling them to discover new interests and emotions.

[0369] The following describes the processing flow.

[0370] Step 1:

[0371] The device activates its on-board sensor and emotion engine to collect user activity and emotion data. The sensor records the user's location, activity level, and app usage, while the emotion engine analyzes the user's facial expressions and voice to obtain real-time emotion data.

[0372] Step 2:

[0373] The device sends the collected data to the server. The data is sent in batches at regular time intervals, and encrypted communication protocols are used to protect privacy.

[0374] Step 3:

[0375] The server preprocesses the received data and prepares it for analysis. This includes data cleaning, missing value imputation, outlier removal, and data normalization.

[0376] Step 4:

[0377] The server uses an artificial intelligence model to analyze pre-processed data. The model interprets the user's activity patterns and real-time changes in emotions, generating foundational information for a hobby program optimized for the user.

[0378] Step 5:

[0379] Based on the analysis results, the server generates a personalized hobby program for the user. This program includes activity suggestions that take into account the user's current emotional state, along with the steps and resources needed to perform those activities.

[0380] Step 6:

[0381] The device notifies the user of any generated hobby programs. These notifications are delivered through applications the user uses regularly, providing the user with detailed information about the program and recommended steps.

[0382] Step 7:

[0383] The user runs a hobby program, and during execution, the device continuously collects emotional data using an emotion engine. The device analyzes this data in real time and provides advice and positive feedback to support the activity.

[0384] Step 8:

[0385] After the user completes an activity, feedback on the experience obtained through the device is sent to the server. The server then analyzes the feedback and uses it as data to improve the model.

[0386] Step 9:

[0387] The server updates the generated artificial intelligence model based on feedback. This allows for more user-friendly recommendations in the next generation of hobby programs.

[0388] (Example 2)

[0389] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0390] In recent years, there has been a growing demand for systems that provide customized experiences based on individual interests and emotions. Traditional systems struggle to fully utilize users' personal data, resulting in ineffective delivery of programs tailored to individual needs. This has led to a persistent situation where user satisfaction remains low.

[0391] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0392] In this invention, the server includes means for collecting information from a device that acquires the user's physical data and location data; means for integrating the information and the user's emotional state using an analysis device and analyzing it using a generation algorithm model; and means for generating an activity program optimized for the user's interests and emotional state based on the analysis results. This makes it possible to provide a recreation experience optimized for each individual user and improve user satisfaction.

[0393] "User physical data" refers to information indicating the user's physical activity, including steps taken, heart rate, and exercise level.

[0394] "Location data" refers to information about the location of a device or user, including geographical location data such as latitude and longitude.

[0395] An "analysis device" is a device that integrates acquired data and has the function of analyzing users' emotions and behavioral patterns.

[0396] A "generative algorithm model" is an artificial intelligence-based algorithm used to analyze acquired data and generate optimized suggestions and programs for the user.

[0397] An "activity program" is a plan that includes suggestions and procedures for carrying out specific hobbies and recreational activities, generated based on the user's interests and emotional state.

[0398] A "recreational experience" is an experience aimed at enjoyment and relaxation, obtained by users engaging in activities or hobbies suggested to them.

[0399] This invention is a technology system for providing users with personalized recreational experiences. It primarily utilizes the user's physical and location data to generate activity programs optimized for the user's hobbies and interests. The implementation method of this system will be described in detail below.

[0400] The device utilizes sensors built into wearable devices and smartphones to collect the user's physical data. This allows for the acquisition of data such as the user's steps, heart rate, and location. The device also uses its built-in camera and microphone to analyze the user's facial expressions and voice, determining their emotional state in real time.

[0401] The acquired data is sent to a server, which uses an analysis device to integrate this data. The integrated data is then subjected to a generative algorithm model and analyzed based on the user's interests and emotional state. As a result, an activity program optimized for the user is constructed. This program includes suggestions for specific hobby activities, execution steps, and relevant online resources.

[0402] For example, if a user expresses interest in learning a new instrument, the device can provide links to practice programs and online lessons. It can also send encouraging messages and progress-based advice in real time, depending on the user's emotional state.

[0403] This system collects feedback even after an activity is completed and uses that information to improve the generation algorithm model. This process will enable the provision of more effective and smarter programs in the future.

[0404] (Example of a prompt message)

[0405] "How can we analyze user activity and emotional data to suggest the most suitable hobby program?"

[0406] "Please suggest ways for users to maintain motivation while learning a new instrument."

[0407] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0408] Step 1:

[0409] The device collects the user's physical and location data. Specifically, it uses sensors in wearable devices and smartphones to obtain data such as steps taken, heart rate, and location information. The input is this raw data, and the output is structured data that organizes it chronologically.

[0410] Step 2:

[0411] The device uses its internal camera and microphone to analyze the user's facial expressions and voice tone in real time, thereby determining the user's emotional state. The input is collected video and audio data, and the generated output is data indicating the user's specific emotional tendencies.

[0412] Step 3:

[0413] The device sends the physical, location, and emotional data obtained in the previous step to the server. The server receives this data and stores it in its database. The input is the various data sent, and the output is the stored integrated data.

[0414] Step 4:

[0415] The server preprocesses the integrated data and inputs it into the generating AI model. Specifically, it performs noise reduction and imputation of missing data. The input is the integrated raw data, and the output is data formatted in a format suitable for analysis.

[0416] Step 5:

[0417] The server uses a generative AI model to analyze user behavior patterns and emotional tendencies. This allows it to design activity programs optimized for the user. The input is pre-processed data, and the generated output is a specific activity program.

[0418] Step 6:

[0419] The server sends the generated activity program to the terminal, and the terminal notifies the user. The terminal displays the program's contents on the screen, prompting the user to begin the activity. The input is the generated activity program, and the output is the activity start notification received by the user.

[0420] Step 7:

[0421] As the user engages in activities, the device continues to collect emotional data and provide real-time feedback. For example, if progress is good, it sends an encouraging message. The input is emotional data during the activity, and the output is specific feedback to the user.

[0422] Step 8:

[0423] Once the activity is complete, the device collects user feedback and sends it to the server. The server uses this feedback to update the generated AI model and improve the program for the next time. The input is the collected feedback information, and the output is the improved model.

[0424] (Application Example 2)

[0425] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0426] In modern life, there is a demand for personalized content delivery based on an individual's emotional state and interests. However, conventional systems have struggled to provide accurate hobby activities and content that reflect emotional states in real time. This has led to decreased user satisfaction and the inability to obtain an optimal content experience.

[0427] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0428] In this invention, the server includes means for collecting user activity data from a measuring device, means for integrating the activity data and the user's emotions and analyzing them using a generative artificial intelligence model, means for generating hobby activities optimized to the user's interests based on the analysis results, means for providing the hobby activities to the user in real time via communication means, means for collecting feedback on the performance of the hobby activities and improving the generative artificial intelligence model, and means for selecting and providing appropriate video content based on the user's emotional state. This makes it possible to provide a real-time and optimized content experience based on the user's emotional state.

[0429] A "user" refers to an individual who uses the system to engage in hobby activities or receive content.

[0430] "Activity data" refers to data that includes information such as the user's location, indicators of physical activity, and device usage history.

[0431] "Measurement device" refers to sensor devices or equipment used to collect user activity data.

[0432] "Emotions" refer to the psychological state judged from the user's facial expressions, tone of voice, and other factors.

[0433] A "generative artificial intelligence model" refers to a set of algorithms and programs that analyze activity data and emotional data to provide user-optimized suggestions.

[0434] "Hobby activities" refer to recreational and learning activities identified based on the user's interests.

[0435] "Communication methods" refer to the technologies and protocols used to send and receive data and programs between a system and a user.

[0436] "Feedback" refers to information collected from users about their impressions and reactions while engaging in hobby activities.

[0437] "Video content" refers to information in the form of videos or images that is selected based on the user's emotional state.

[0438] The system for implementing this invention is comprised of various technical elements combined to collect and analyze user activity and emotional data. At the heart of this system is a generative artificial intelligence model that provides users with optimized hobby activities and video content in real time. The server implements the invention through the following procedure.

[0439] First, a measuring device worn by the user, such as a smartphone or wearable device, collects activity data. This device incorporates sensors that record location information, indicators of physical activity, and application usage history. In addition, an emotion engine is used to analyze the user's facial expressions and tone of voice to understand their emotional state in real time.

[0440] Next, the server receives data sent from the user's terminal and integrates and analyzes this data using a raw artificial intelligence model. This model, developed using programming languages ​​such as Python, analyzes the user's interests and emotions in detail and generates an optimal hobby activity program. It also selects appropriate video content based on the user's emotions.

[0441] For example, if a user wants to relax after work, the server can recognize that emotional state and notify the device of music videos with relaxing effects.

[0442] Finally, feedback is collected during the execution of the generated hobby activities and used to improve the model's performance. This will improve the accuracy of future hobby activity and content selection.

[0443] Examples of prompts for a generative AI model include:

[0444] "Analyze the user's facial expression data and suggest relaxing content that matches his current emotional state."

[0445] "Please find a program you'll enjoy based on the emotional evaluation derived from this voice tone analysis."

[0446] These are some examples.

[0447] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0448] Step 1:

[0449] The device collects activity data through a measuring device worn by the user. This data includes location information, indicators of physical activity, and application usage history. The input is data signals from sensors, and the output is a structured activity dataset. This activity data is later sent to a server for analysis.

[0450] Step 2:

[0451] The device analyzes facial expressions and voice tone through the camera and microphone to acquire user emotion data using an emotion engine. This input is real-time image and audio data, and the output identifies the user's emotional state. The emotion engine then executes machine learning algorithms to classify the emotion.

[0452] Step 3:

[0453] The server receives activity and emotion data transmitted from the terminal and integrates them. The input is a dataset of activity data and emotion states, and the output is an integrated user characteristics profile. This profile is stored in a database and analyzed using a raw artificial intelligence model.

[0454] Step 4:

[0455] The server uses a generative artificial intelligence model to analyze an integrated profile and generate hobby activities and video content optimized for the user. The input is the user profile, and the output is a recommended hobby activity program and selected content. This process involves the AI ​​model identifying data patterns and recommendation algorithms.

[0456] Step 5:

[0457] The server provides understood hobby activity programs and video content to the terminal in real time via communication means. The input is information about the generated programs and content, and the output is executable instructions sent via the user interface. The user can then take action based on these instructions.

[0458] Step 6:

[0459] The terminal collects feedback while the provided program is running and sends this data to the server. The input is user feedback information, and the output is feedback data for model improvement. The server uses this feedback to continuously learn and improve the raw artificial intelligence model.

[0460] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0461] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0462] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0463] [Third Embodiment]

[0464] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0465] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0466] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0467] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0468] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0469] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0470] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0471] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0472] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0473] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0474] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0475] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0476] This invention is a system that provides hobby programs optimized based on the individual interests and emotions of the user. This system utilizes advanced generative artificial intelligence technology to support users in discovering new hobbies and deepening their existing ones.

[0477] Data collection and analysis

[0478] First, the device collects user activity data from sensor equipment. This includes movement information, exercise levels, and activity logs from digital channels. Furthermore, data on the user's emotions and psychological state is also obtained through questionnaires and biosensors. This data is transmitted to a server using secure communication methods.

[0479] The server integrates and preprocesses the received activity and emotion data. Next, it uses a generative artificial intelligence model to analyze the data and perform pattern recognition based on the user's interests and emotional responses. This analysis extracts the most suitable hobby activities and new experiences for the user.

[0480] Generating personalized hobby programs

[0481] Based on the analysis results, the server generates a personalized hobby program for the user. This program suggests specific hobbies and activities according to the user's interests, and provides information on how to participate and related resources. For example, a user interested in nature photography might be provided with information on nearby scenic spots and photography workshops.

[0482] Real-time AI coaching

[0483] The generated hobby program is sent to the device via communication and the user is notified. On the device, an AI coaching function provides real-time support and advice as the user engages in hobby activities. This allows the user to proceed at their own pace and maintain continuous motivation.

[0484] Content delivery and continuous improvement

[0485] The server uses a content library to provide information and guides related to the user's interests. It can also utilize virtual or augmented reality technologies to provide users with immersive experiences. User feedback is collected through the device, and the server uses this data to improve the generated artificial intelligence model, thereby increasing the accuracy of future programs.

[0486] These elements allow the system to provide users with a personalized and valuable hobby experience, enabling them to feel new emotions and inspiration.

[0487] The following describes the processing flow.

[0488] Step 1:

[0489] The device collects user activity data. This is done by wearable devices and applications on smartphones, and includes GPS data, heart rate, movement history, and app usage logs. It also collects emotional data by conducting short surveys with users.

[0490] Step 2:

[0491] The device sends the collected data to the server at regular intervals. During this process, a secure protocol (e.g., TLS) is used to ensure data privacy and security.

[0492] Step 3:

[0493] The server integrates the received data. The integrated data undergoes a data cleaning process to impute missing values ​​and remove outliers. This preprocessing creates a dataset suitable for analysis.

[0494] Step 4:

[0495] The server uses a generated artificial intelligence model to analyze the integrated data. The model analyzes the user's interests, activity patterns, and emotional responses, identifying specific patterns and tendencies. This allows for the discovery of hobbies and potential new activities that are suited to the user.

[0496] Step 5:

[0497] Based on the analysis results, the server generates a hobby program optimized for the user. This program includes hobby suggestions, specific activity instructions, and information on necessary resources.

[0498] Step 6:

[0499] The device receives the generated hobby program and notifies the user. The notification is sent via a messaging app, providing the user with guidance and advice on the next steps.

[0500] Step 7:

[0501] Users participate in hobby programs and record their experiences and feedback through the app. This feedback helps maintain motivation and plan future activities.

[0502] Step 8:

[0503] The server collects user feedback and uses it to retrain the generated artificial intelligence model. This process allows the model to more accurately predict user interests and preferences, and to more effectively optimize the next hobby program.

[0504] (Example 1)

[0505] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0506] In modern society, many people are too busy to find opportunities to take up new hobbies. Furthermore, finding a hobby that aligns with one's interests and emotions can be challenging. Therefore, there is a need for a system that can suggest optimal hobbies based on individual user preferences and emotions, thereby enhancing the experience of engaging in hobby activities.

[0507] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0508] In this invention, the server includes means for collecting user activity information from a detection device, means for integrating the activity information and the user's emotional state and analyzing it using a generative machine learning model, and means for constructing hobby activities optimized to the user's preferences based on the analysis results. This makes it possible for users to easily find personalized hobby activities that match their interests and emotions and enjoy a richer life experience.

[0509] A "user" refers to an individual who uses the system to receive suggestions and experiences related to their hobbies.

[0510] "Activity information" refers to data collected by detection devices, such as user movement information, exercise levels, and activity logs from digital channels.

[0511] "Emotional state" refers to data that represents the user's psychological state, obtained through their heart rate and surveys.

[0512] A "detection device" refers to sensors or devices used to collect user activity information.

[0513] A "generative machine learning model" refers to an algorithm or framework used to analyze collected activity information and emotional states to suggest the most suitable hobby activities for the user.

[0514] "Hobby activities" refer to activities and resources suggested based on the user's interests and preferences.

[0515] "Analysis" refers to the process of processing activity information and emotional states using generative machine learning models to recognize patterns based on user interests.

[0516] "Communication technology" refers to data transmission methods used to present hobby activities to users in real time.

[0517] "Response" refers to feedback and evaluations that arise as a result of a user engaging in a suggested hobby activity.

[0518] This invention is a system that suggests personalized hobby activities to users. The system mainly consists of terminals and servers, and collects data using external detection devices and performs analysis using generative machine learning models.

[0519] The device is responsible for collecting user activity information. Specifically, it acquires movement information using a GPS sensor and measures physical activity using an accelerometer. It also records activity logs on digital channels. The user's emotional state is also collected using questionnaire forms and a heart rate sensor. The collected activity information and emotional state are securely transmitted to the server using encryption technology.

[0520] The server integrates and preprocesses the received data, making it suitable for machine learning. Next, it analyzes the data using a generative AI model. Specifically, it uses machine learning frameworks such as TensorFlow and PyTorch to recognize patterns that match the user's interests and preferences. In this process, prompts such as "Suggest a new hobby based on the user's movements and emotional data" are used.

[0521] Based on the analysis, the server generates hobby activities optimized for the user. For example, for a user who enjoys outdoor activities, it recommends nearby hiking trails and provides information on necessary equipment and events. These hobby activities are transmitted to the terminal in real time using communication technology, and the user is notified.

[0522] Users receive notifications via their devices and can receive real-time support and advice on suggested hobby activities through the AI ​​coaching function. This allows users to pursue their hobbies at their own pace and maintain their motivation.

[0523] Finally, the device collects user feedback and sends it back to the server, allowing the generated AI model to be continuously improved. This feedback loop ensures that the system can always provide new suggestions tailored to the user, thereby improving the quality of the user experience.

[0524] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0525] Step 1:

[0526] The device collects user activity information from detection devices. Inputs include GPS sensor location data, accelerometer motion data, and activity logs from digital channels. This data is collected in real time to record the user's activity patterns. The output is user activity information, a combination of this data.

[0527] Step 2:

[0528] The device collects the user's emotional state. Specifically, it collects text data through a survey form and measures physiological responses through a heart rate sensor. The input consists of the user's responses and biosensor data, and the output is a numerical emotional index.

[0529] Step 3:

[0530] The device securely transmits activity information and emotional states collected by the device to the server using encryption technology. The input is raw data collected by the device, and the output is an encrypted data stream. This ensures that the data is transmitted to the server without being leaked to third parties.

[0531] Step 4:

[0532] The server integrates and preprocesses the received data. The data input consists of activity information and emotional states transmitted from the terminal. The server normalizes the data and removes noise to convert it into an easily analyzable format. The output is a preprocessed dataset.

[0533] Step 5:

[0534] The server uses a generated AI model to analyze pre-processed data. In this step, a machine learning algorithm derives the user's interests and preferences based on the input data. The prompt is "Suggest new hobbies based on the user's behavior and emotional data." The output of this analysis is a pattern of hobbies optimized for the user.

[0535] Step 6:

[0536] The server builds hobby activities optimized for the user based on the analysis results. The input is the analysis results, and the output is a hobby suggestion tailored to the user's interests. For example, for a user interested in nature photography, the server will create a suggestion that includes information on suitable locations for landscape photography in the vicinity and photography workshops.

[0537] Step 7:

[0538] The server transmits generated hobby activities to the terminal in real time using communication technology. The input is a proposal from the server, and the output is an activity guide notified to the user. This allows the user to start the activity immediately.

[0539] Step 8:

[0540] Users engage in hobby activities and receive support from an AI coaching function on their device. Inputs include user engagement and feedback during the activity. Outputs include real-time advice and suggestions on how to proceed with the activity.

[0541] Step 9:

[0542] The device collects user feedback and sends it back to the server. The input is user ratings and feedback data. The output is training data to improve the generative AI model. This feedback cycle improves the accuracy of subsequent suggestions.

[0543] (Application Example 1)

[0544] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0545] Modern content delivery technologies struggle to provide users with personalized experiences based on their interests and hobbies, particularly lacking real-time optimization and immersive experiences. Furthermore, the lack of methods for recommending content that aligns with users' emotions and interests often leads to a degraded user experience.

[0546] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0547] In this invention, the server includes means for collecting user activity information from sensor devices, means for integrating the activity information and the user's emotions and analyzing them using a generated artificial intelligence model, means for generating a hobby plan optimized for the user's interests based on the analysis results, means for providing the hobby plan to the user in real time via communication means, means for collecting feedback on the execution of the hobby plan and improving the generated artificial intelligence model, means for recommending relevant visual and musical information according to the user's interests and emotions, and means for providing an interactive experience using augmented reality technology. This enables the provision of personalized content to the user in real time, allowing for a more deeply immersive and interactive hobby experience.

[0548] "User activity information" refers to information that indicates the user's physical movements and actions, collected by sensor devices.

[0549] A "generated artificial intelligence model" is an algorithm that analyzes users' interests and emotions based on accumulated data and provides them with the most suitable content.

[0550] A "hobby plan" is a plan that includes suggestions for activities and content optimized for each user's individual interests.

[0551] "Communication methods" refer to the infrastructure and protocols used to send and receive data.

[0552] "Feedback" refers to additional opinions and impressions based on the user's experience with their hobby plan.

[0553] "Visual and musical information" refers to visual and audio content that is recommended in response to the user's interests and emotions.

[0554] Augmented reality technology is a technology that provides users with new experiences by overlaying digital information onto the real world.

[0555] An "interactive experience" is an experience designed to allow users to actively participate and interact with the content and environment through their actions.

[0556] The system for carrying out this invention mainly includes a sensor device for collecting user activity information, a server for analysis, and a terminal for providing information to the user. The sensor device is embedded in a smartphone or wearable device and is responsible for detecting the user's physical movements and behavioral data and transmitting it to the server.

[0557] The server integrates activity information and emotions based on the received data and performs analysis using a generated artificial intelligence model. Here, the advanced generative AI model identifies the user's interest and emotional patterns and generates a personalized hobby plan. This hobby plan is optimized to capture the user's interest by selecting visual and musical information according to the user's specific preferences. This process uses Python and machine learning libraries to perform data processing and analysis.

[0558] The device receives hobby plans provided by the server via communication and notifies the user in real time. It can also provide interactive experiences using augmented reality technology, overlaying digital information onto the real world through the smartphone's camera, allowing users to experience things like forest walks or museum exhibits with a strong sense of realism. The user's feedback on this experience is sent to the server and used to further improve the generated artificial intelligence model. This entire process provides users with highly customized content, enabling them to enjoy a rich experience.

[0559] As a concrete example, when a user is relaxed, the application can recommend documentary videos related to forests and nature. In this case, an example of a prompt for the generative AI model would be: "Suggest personalized content for the user to enjoy a new nature experience. His mood is relaxed." This allows the system to select the most appropriate information and experience for the user.

[0560] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0561] Step 1:

[0562] Users wear sensor devices while performing their daily activities. The sensor devices detect the user's physical movements and behavioral data in real time and collect it as activity information. This activity information includes steps taken, travel routes, current location, and digital device usage.

[0563] Step 2:

[0564] The terminal transmits activity information collected from sensor devices to a server. The transmitted data is integrated with the user's emotional data on the server. This emotional data is obtained through user input and biosensors and indicates the user's current mood and psychological state.

[0565] Step 3:

[0566] The server inputs integrated activity and sentiment data into a generating AI model. The AI ​​model then analyzes the data and performs specific pattern recognition. The model identifies the user's interests and emotional tendencies and generates an optimized hobby plan. Machine learning algorithms are used in this analysis.

[0567] Step 4:

[0568] The server sends the generated hobby plan to the terminal. This hobby plan includes visual and musical information selected based on the user's interests. The terminal notifies the user in real time and displays the necessary information on the screen.

[0569] Step 5:

[0570] Users engage in hobby activities using hobby plans provided by the device. The device can also provide related augmented reality experiences, allowing users to overlay digital information onto the real world through the camera.

[0571] Step 6:

[0572] After completing a hobby activity, users enter feedback about the experience into their device. This feedback includes how beneficial the hobby plan was and areas where further improvement is needed.

[0573] Step 7:

[0574] The device sends the collected feedback to the server. The server adds this feedback to the dataset of the generating AI model and uses it to improve the next hobby plan generation. This gradually improves the AI ​​model and increases the accuracy of the content it provides.

[0575] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0576] This invention relates to a system for analyzing a user's activity data and emotional state to provide a personalized hobby program. In particular, it uses an emotion engine to collect emotional data and utilize it for analysis, thereby achieving personalization of the user's hobby experience.

[0577] Data collection and emotion recognition

[0578] The device collects user activity data using wearable devices and smartphones. This activity data includes location information, indicators of physical activity, and application usage history. Furthermore, an emotion engine built into the device analyzes the user's facial expressions and voice tone through the camera and microphone, acquiring emotion data in real time.

[0579] Data Integration and Analysis

[0580] The server receives activity and emotion data transmitted from the terminal and integrates them. The integrated data is preprocessed and then analyzed by a generative artificial intelligence model. The model analyzes the user's interests, emotional changes, and behavioral patterns to design a hobby program optimized for the user.

[0581] Generating and providing personalized hobby programs

[0582] Based on the analysis results, the server generates a hobby program that takes into account the user's current emotional state. The generated program includes suggestions for specific hobby activities, detailed steps for those activities, and related resources. Furthermore, by utilizing emotional data, the program is designed to provide the user with the most satisfying experience.

[0583] Real-time AI coaching and feedback collection

[0584] The device notifies the user of a generated hobby program and encourages them to begin activities based on it. While the activities are being carried out, the emotion engine continuously monitors the user's emotions and sends that data to the device. The device uses this data to provide advice and feedback in real time to increase the user's motivation.

[0585] For example, if a user is interested in learning a new musical instrument, the device analyzes the user's emotional data during practice and sends encouraging messages at the appropriate time. By adapting to the user's emotional state in this way, more effective learning and enjoyment of the hobby can be achieved.

[0586] Continuous model improvement

[0587] The server collects feedback even after the activity is completed and continuously improves the generated artificial intelligence model based on that data. This process ensures that subsequent hobby programs are more accurate and tailored to each individual user.

[0588] The present invention aims to provide users with personalized hobby programs that are adjusted in real time by an emotion engine, enabling them to discover new interests and emotions.

[0589] The following describes the processing flow.

[0590] Step 1:

[0591] The device activates its on-board sensor and emotion engine to collect user activity and emotion data. The sensor records the user's location, activity level, and app usage, while the emotion engine analyzes the user's facial expressions and voice to obtain real-time emotion data.

[0592] Step 2:

[0593] The device sends the collected data to the server. The data is sent in batches at regular time intervals, and encrypted communication protocols are used to protect privacy.

[0594] Step 3:

[0595] The server preprocesses the received data and prepares it for analysis. This includes data cleaning, missing value imputation, outlier removal, and data normalization.

[0596] Step 4:

[0597] The server uses an artificial intelligence model to analyze pre-processed data. The model interprets the user's activity patterns and real-time changes in emotions, generating foundational information for a hobby program optimized for the user.

[0598] Step 5:

[0599] Based on the analysis results, the server generates a personalized hobby program for the user. This program includes activity suggestions that take into account the user's current emotional state, along with the steps and resources needed to perform those activities.

[0600] Step 6:

[0601] The device notifies the user of any generated hobby programs. These notifications are delivered through applications the user uses regularly, providing the user with detailed information about the program and recommended steps.

[0602] Step 7:

[0603] The user runs a hobby program, and during execution, the device continuously collects emotional data using an emotion engine. The device analyzes this data in real time and provides advice and positive feedback to support the activity.

[0604] Step 8:

[0605] After the user completes an activity, feedback on the experience obtained through the device is sent to the server. The server then analyzes the feedback and uses it as data to improve the model.

[0606] Step 9:

[0607] The server updates the generated artificial intelligence model based on feedback. This allows for more user-friendly recommendations in the next generation of hobby programs.

[0608] (Example 2)

[0609] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0610] In recent years, there has been a growing demand for systems that provide customized experiences based on individual interests and emotions. Traditional systems struggle to fully utilize users' personal data, resulting in ineffective delivery of programs tailored to individual needs. This has led to a persistent situation where user satisfaction remains low.

[0611] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0612] In this invention, the server includes means for collecting information from a device that acquires the user's physical data and location data; means for integrating the information and the user's emotional state using an analysis device and analyzing it using a generation algorithm model; and means for generating an activity program optimized for the user's interests and emotional state based on the analysis results. This makes it possible to provide a recreation experience optimized for each individual user and improve user satisfaction.

[0613] "User physical data" refers to information indicating the user's physical activity, including steps taken, heart rate, and exercise level.

[0614] "Location data" refers to information about the location of a device or user, including geographical location data such as latitude and longitude.

[0615] An "analysis device" is a device that integrates acquired data and has the function of analyzing users' emotions and behavioral patterns.

[0616] A "generative algorithm model" is an artificial intelligence-based algorithm used to analyze acquired data and generate optimized suggestions and programs for the user.

[0617] An "activity program" is a plan that includes suggestions and procedures for carrying out specific hobbies and recreational activities, generated based on the user's interests and emotional state.

[0618] A "recreational experience" is an experience aimed at enjoyment and relaxation, obtained by users engaging in activities or hobbies suggested to them.

[0619] This invention is a technology system for providing users with personalized recreational experiences. It primarily utilizes the user's physical and location data to generate activity programs optimized for the user's hobbies and interests. The implementation method of this system will be described in detail below.

[0620] The device utilizes sensors built into wearable devices and smartphones to collect the user's physical data. This allows for the acquisition of data such as the user's steps, heart rate, and location. The device also uses its built-in camera and microphone to analyze the user's facial expressions and voice, determining their emotional state in real time.

[0621] The acquired data is sent to a server, which uses an analysis device to integrate this data. The integrated data is then subjected to a generative algorithm model and analyzed based on the user's interests and emotional state. As a result, an activity program optimized for the user is constructed. This program includes suggestions for specific hobby activities, execution steps, and relevant online resources.

[0622] For example, if a user expresses interest in learning a new instrument, the device can provide links to practice programs and online lessons. It can also send encouraging messages and progress-based advice in real time, depending on the user's emotional state.

[0623] This system collects feedback even after an activity is completed and uses that information to improve the generation algorithm model. This process will enable the provision of more effective and smarter programs in the future.

[0624] (Example of a prompt message)

[0625] "How can we analyze user activity and emotional data to suggest the most suitable hobby program?"

[0626] "Please suggest ways for users to maintain motivation while learning a new instrument."

[0627] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0628] Step 1:

[0629] The device collects the user's physical and location data. Specifically, it uses sensors in wearable devices and smartphones to obtain data such as steps taken, heart rate, and location information. The input is this raw data, and the output is structured data that organizes it chronologically.

[0630] Step 2:

[0631] The device uses its internal camera and microphone to analyze the user's facial expressions and voice tone in real time, thereby determining the user's emotional state. The input is collected video and audio data, and the generated output is data indicating the user's specific emotional tendencies.

[0632] Step 3:

[0633] The device sends the physical, location, and emotional data obtained in the previous step to the server. The server receives this data and stores it in its database. The input is the various data sent, and the output is the stored integrated data.

[0634] Step 4:

[0635] The server preprocesses the integrated data and inputs it into the generating AI model. Specifically, it performs noise reduction and imputation of missing data. The input is the integrated raw data, and the output is data formatted in a format suitable for analysis.

[0636] Step 5:

[0637] The server uses a generative AI model to analyze user behavior patterns and emotional tendencies. This allows it to design activity programs optimized for the user. The input is pre-processed data, and the generated output is a specific activity program.

[0638] Step 6:

[0639] The server sends the generated activity program to the terminal, and the terminal notifies the user. The terminal displays the program's contents on the screen, prompting the user to begin the activity. The input is the generated activity program, and the output is the activity start notification received by the user.

[0640] Step 7:

[0641] As the user engages in activities, the device continues to collect emotional data and provide real-time feedback. For example, if progress is good, it sends an encouraging message. The input is emotional data during the activity, and the output is specific feedback to the user.

[0642] Step 8:

[0643] Once the activity is complete, the device collects user feedback and sends it to the server. The server uses this feedback to update the generated AI model and improve the program for the next time. The input is the collected feedback information, and the output is the improved model.

[0644] (Application Example 2)

[0645] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0646] In modern life, there is a demand for personalized content delivery based on an individual's emotional state and interests. However, conventional systems have struggled to provide accurate hobby activities and content that reflect emotional states in real time. This has led to decreased user satisfaction and the inability to obtain an optimal content experience.

[0647] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0648] In this invention, the server includes means for collecting user activity data from a measuring device, means for integrating the activity data and the user's emotions and analyzing them using a generative artificial intelligence model, means for generating hobby activities optimized to the user's interests based on the analysis results, means for providing the hobby activities to the user in real time via communication means, means for collecting feedback on the performance of the hobby activities and improving the generative artificial intelligence model, and means for selecting and providing appropriate video content based on the user's emotional state. This makes it possible to provide a real-time and optimized content experience based on the user's emotional state.

[0649] A "user" refers to an individual who uses the system to engage in hobby activities or receive content.

[0650] "Activity data" refers to data that includes information such as the user's location, indicators of physical activity, and device usage history.

[0651] "Measurement device" refers to sensor devices or equipment used to collect user activity data.

[0652] "Emotions" refer to the psychological state judged from the user's facial expressions, tone of voice, and other factors.

[0653] A "generative artificial intelligence model" refers to a set of algorithms and programs that analyze activity data and emotional data to provide user-optimized suggestions.

[0654] "Hobby activities" refer to recreational and learning activities identified based on the user's interests.

[0655] "Communication methods" refer to the technologies and protocols used to send and receive data and programs between a system and a user.

[0656] "Feedback" refers to information collected from users about their impressions and reactions while engaging in hobby activities.

[0657] "Video content" refers to information in the form of videos or images that is selected based on the user's emotional state.

[0658] The system for implementing this invention is comprised of various technical elements combined to collect and analyze user activity and emotional data. At the heart of this system is a generative artificial intelligence model that provides users with optimized hobby activities and video content in real time. The server implements the invention through the following procedure.

[0659] First, a measuring device worn by the user, such as a smartphone or wearable device, collects activity data. This device incorporates sensors that record location information, indicators of physical activity, and application usage history. In addition, an emotion engine is used to analyze the user's facial expressions and tone of voice to understand their emotional state in real time.

[0660] Next, the server receives data sent from the user's terminal and integrates and analyzes this data using a raw artificial intelligence model. This model, developed using programming languages ​​such as Python, analyzes the user's interests and emotions in detail and generates an optimal hobby activity program. It also selects appropriate video content based on the user's emotions.

[0661] For example, if a user wants to relax after work, the server can recognize that emotional state and notify the device of music videos with relaxing effects.

[0662] Finally, feedback is collected during the execution of the generated hobby activities and used to improve the model's performance. This will improve the accuracy of future hobby activity and content selection.

[0663] Examples of prompts for a generative AI model include:

[0664] "Analyze the user's facial expression data and suggest relaxing content that matches his current emotional state."

[0665] "Please find a program you'll enjoy based on the emotional evaluation derived from this voice tone analysis."

[0666] These are some examples.

[0667] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0668] Step 1:

[0669] The device collects activity data through a measuring device worn by the user. This data includes location information, indicators of physical activity, and application usage history. The input is data signals from sensors, and the output is a structured activity dataset. This activity data is later sent to a server for analysis.

[0670] Step 2:

[0671] The device analyzes facial expressions and voice tone through the camera and microphone to acquire user emotion data using an emotion engine. This input is real-time image and audio data, and the output identifies the user's emotional state. The emotion engine then executes machine learning algorithms to classify the emotion.

[0672] Step 3:

[0673] The server receives activity and emotion data transmitted from the terminal and integrates them. The input is a dataset of activity data and emotion states, and the output is an integrated user characteristics profile. This profile is stored in a database and analyzed using a raw artificial intelligence model.

[0674] Step 4:

[0675] The server uses a generative artificial intelligence model to analyze an integrated profile and generate hobby activities and video content optimized for the user. The input is the user profile, and the output is a recommended hobby activity program and selected content. This process involves the AI ​​model identifying data patterns and recommendation algorithms.

[0676] Step 5:

[0677] The server provides understood hobby activity programs and video content to the terminal in real time via communication means. The input is information about the generated programs and content, and the output is executable instructions sent via the user interface. The user can then take action based on these instructions.

[0678] Step 6:

[0679] The terminal collects feedback while the provided program is running and sends this data to the server. The input is user feedback information, and the output is feedback data for model improvement. The server uses this feedback to continuously learn and improve the raw artificial intelligence model.

[0680] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0681] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0682] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0683] [Fourth Embodiment]

[0684] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0685] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0686] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0687] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0688] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0689] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0690] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0691] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0692] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0693] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0694] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0695] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0696] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0697] This invention is a system that provides hobby programs optimized based on the individual interests and emotions of the user. This system utilizes advanced generative artificial intelligence technology to support users in discovering new hobbies and deepening their existing ones.

[0698] Data collection and analysis

[0699] First, the device collects user activity data from sensor equipment. This includes movement information, exercise levels, and activity logs from digital channels. Furthermore, data on the user's emotions and psychological state is also obtained through questionnaires and biosensors. This data is transmitted to a server using secure communication methods.

[0700] The server integrates and preprocesses the received activity and emotion data. Next, it uses a generative artificial intelligence model to analyze the data and perform pattern recognition based on the user's interests and emotional responses. This analysis extracts the most suitable hobby activities and new experiences for the user.

[0701] Generating personalized hobby programs

[0702] Based on the analysis results, the server generates a personalized hobby program for the user. This program suggests specific hobbies and activities according to the user's interests, and provides information on how to participate and related resources. For example, a user interested in nature photography might be provided with information on nearby scenic spots and photography workshops.

[0703] Real-time AI coaching

[0704] The generated hobby program is sent to the device via communication and the user is notified. On the device, an AI coaching function provides real-time support and advice as the user engages in hobby activities. This allows the user to proceed at their own pace and maintain continuous motivation.

[0705] Content delivery and continuous improvement

[0706] The server uses a content library to provide information and guides related to the user's interests. It can also utilize virtual or augmented reality technologies to provide users with immersive experiences. User feedback is collected through the device, and the server uses this data to improve the generated artificial intelligence model, thereby increasing the accuracy of future programs.

[0707] These elements allow the system to provide users with a personalized and valuable hobby experience, enabling them to feel new emotions and inspiration.

[0708] The following describes the processing flow.

[0709] Step 1:

[0710] The device collects user activity data. This is done by wearable devices and applications on smartphones, and includes GPS data, heart rate, movement history, and app usage logs. It also collects emotional data by conducting short surveys with users.

[0711] Step 2:

[0712] The device sends the collected data to the server at regular intervals. During this process, a secure protocol (e.g., TLS) is used to ensure data privacy and security.

[0713] Step 3:

[0714] The server integrates the received data. The integrated data undergoes a data cleaning process to impute missing values ​​and remove outliers. This preprocessing creates a dataset suitable for analysis.

[0715] Step 4:

[0716] The server uses a generated artificial intelligence model to analyze the integrated data. The model analyzes the user's interests, activity patterns, and emotional responses, identifying specific patterns and tendencies. This allows for the discovery of hobbies and potential new activities that are suited to the user.

[0717] Step 5:

[0718] Based on the analysis results, the server generates a hobby program optimized for the user. This program includes hobby suggestions, specific activity instructions, and information on necessary resources.

[0719] Step 6:

[0720] The device receives the generated hobby program and notifies the user. The notification is sent via a messaging app, providing the user with guidance and advice on the next steps.

[0721] Step 7:

[0722] Users participate in hobby programs and record their experiences and feedback through the app. This feedback helps maintain motivation and plan future activities.

[0723] Step 8:

[0724] The server collects user feedback and uses it to retrain the generated artificial intelligence model. This process allows the model to more accurately predict user interests and preferences, and to more effectively optimize the next hobby program.

[0725] (Example 1)

[0726] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0727] In modern society, many people are too busy to find opportunities to take up new hobbies. Furthermore, finding a hobby that aligns with one's interests and emotions can be challenging. Therefore, there is a need for a system that can suggest optimal hobbies based on individual user preferences and emotions, thereby enhancing the experience of engaging in hobby activities.

[0728] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0729] In this invention, the server includes means for collecting user activity information from a detection device, means for integrating the activity information and the user's emotional state and analyzing it using a generative machine learning model, and means for constructing hobby activities optimized to the user's preferences based on the analysis results. This makes it possible for users to easily find personalized hobby activities that match their interests and emotions and enjoy a richer life experience.

[0730] A "user" refers to an individual who uses the system to receive suggestions and experiences related to their hobbies.

[0731] "Activity information" refers to data collected by detection devices, such as user movement information, exercise levels, and activity logs from digital channels.

[0732] "Emotional state" refers to data that represents the user's psychological state, obtained through their heart rate and surveys.

[0733] A "detection device" refers to sensors or devices used to collect user activity information.

[0734] A "generative machine learning model" refers to an algorithm or framework used to analyze collected activity information and emotional states to suggest the most suitable hobby activities for the user.

[0735] "Hobby activities" refer to activities and resources suggested based on the user's interests and preferences.

[0736] "Analysis" refers to the process of processing activity information and emotional states using generative machine learning models to recognize patterns based on user interests.

[0737] "Communication technology" refers to data transmission methods used to present hobby activities to users in real time.

[0738] "Response" refers to feedback and evaluations that arise as a result of a user engaging in a suggested hobby activity.

[0739] This invention is a system that suggests personalized hobby activities to users. The system mainly consists of terminals and servers, and collects data using external detection devices and performs analysis using generative machine learning models.

[0740] The device is responsible for collecting user activity information. Specifically, it acquires movement information using a GPS sensor and measures physical activity using an accelerometer. It also records activity logs on digital channels. The user's emotional state is also collected using questionnaire forms and a heart rate sensor. The collected activity information and emotional state are securely transmitted to the server using encryption technology.

[0741] The server integrates and preprocesses the received data, making it suitable for machine learning. Next, it analyzes the data using a generative AI model. Specifically, it uses machine learning frameworks such as TensorFlow and PyTorch to recognize patterns that match the user's interests and preferences. In this process, prompts such as "Suggest a new hobby based on the user's movements and emotional data" are used.

[0742] Based on the analysis, the server generates hobby activities optimized for the user. For example, for a user who enjoys outdoor activities, it recommends nearby hiking trails and provides information on necessary equipment and events. These hobby activities are transmitted to the terminal in real time using communication technology, and the user is notified.

[0743] Users receive notifications via their devices and can receive real-time support and advice on suggested hobby activities through the AI ​​coaching function. This allows users to pursue their hobbies at their own pace and maintain their motivation.

[0744] Finally, the device collects user feedback and sends it back to the server, allowing the generated AI model to be continuously improved. This feedback loop ensures that the system can always provide new suggestions tailored to the user, thereby improving the quality of the user experience.

[0745] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0746] Step 1:

[0747] The device collects user activity information from detection devices. Inputs include GPS sensor location data, accelerometer motion data, and activity logs from digital channels. This data is collected in real time to record the user's activity patterns. The output is user activity information, a combination of this data.

[0748] Step 2:

[0749] The device collects the user's emotional state. Specifically, it collects text data through a survey form and measures physiological responses through a heart rate sensor. The input consists of the user's responses and biosensor data, and the output is a numerical emotional index.

[0750] Step 3:

[0751] The device securely transmits activity information and emotional states collected by the device to the server using encryption technology. The input is raw data collected by the device, and the output is an encrypted data stream. This ensures that the data is transmitted to the server without being leaked to third parties.

[0752] Step 4:

[0753] The server integrates and preprocesses the received data. The data input consists of activity information and emotional states transmitted from the terminal. The server normalizes the data and removes noise to convert it into an easily analyzable format. The output is a preprocessed dataset.

[0754] Step 5:

[0755] The server uses a generated AI model to analyze pre-processed data. In this step, a machine learning algorithm derives the user's interests and preferences based on the input data. The prompt is "Suggest new hobbies based on the user's behavior and emotional data." The output of this analysis is a pattern of hobbies optimized for the user.

[0756] Step 6:

[0757] The server builds hobby activities optimized for the user based on the analysis results. The input is the analysis results, and the output is a hobby suggestion tailored to the user's interests. For example, for a user interested in nature photography, the server will create a suggestion that includes information on suitable locations for landscape photography in the vicinity and photography workshops.

[0758] Step 7:

[0759] The server transmits generated hobby activities to the terminal in real time using communication technology. The input is a proposal from the server, and the output is an activity guide notified to the user. This allows the user to start the activity immediately.

[0760] Step 8:

[0761] Users engage in hobby activities and receive support from an AI coaching function on their device. Inputs include user engagement and feedback during the activity. Outputs include real-time advice and suggestions on how to proceed with the activity.

[0762] Step 9:

[0763] The device collects user feedback and sends it back to the server. The input is user ratings and feedback data. The output is training data to improve the generative AI model. This feedback cycle improves the accuracy of subsequent suggestions.

[0764] (Application Example 1)

[0765] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0766] Modern content delivery technologies struggle to provide users with personalized experiences based on their interests and hobbies, particularly lacking real-time optimization and immersive experiences. Furthermore, the lack of methods for recommending content that aligns with users' emotions and interests often leads to a degraded user experience.

[0767] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0768] In this invention, the server includes means for collecting user activity information from sensor devices, means for integrating the activity information and the user's emotions and analyzing them using a generated artificial intelligence model, means for generating a hobby plan optimized for the user's interests based on the analysis results, means for providing the hobby plan to the user in real time via communication means, means for collecting feedback on the execution of the hobby plan and improving the generated artificial intelligence model, means for recommending relevant visual and musical information according to the user's interests and emotions, and means for providing an interactive experience using augmented reality technology. This enables the provision of personalized content to the user in real time, allowing for a more deeply immersive and interactive hobby experience.

[0769] "User activity information" refers to information that indicates the user's physical movements and actions, collected by sensor devices.

[0770] A "generated artificial intelligence model" is an algorithm that analyzes users' interests and emotions based on accumulated data and provides them with the most suitable content.

[0771] A "hobby plan" is a plan that includes suggestions for activities and content optimized for each user's individual interests.

[0772] "Communication methods" refer to the infrastructure and protocols used to send and receive data.

[0773] "Feedback" refers to additional opinions and impressions based on the user's experience with their hobby plan.

[0774] "Visual and musical information" refers to visual and audio content that is recommended in response to the user's interests and emotions.

[0775] Augmented reality technology is a technology that provides users with new experiences by overlaying digital information onto the real world.

[0776] An "interactive experience" is an experience designed to allow users to actively participate and interact with the content and environment through their actions.

[0777] The system for carrying out this invention mainly includes a sensor device for collecting user activity information, a server for analysis, and a terminal for providing information to the user. The sensor device is embedded in a smartphone or wearable device and is responsible for detecting the user's physical movements and behavioral data and transmitting it to the server.

[0778] The server integrates activity information and emotions based on the received data and performs analysis using a generated artificial intelligence model. Here, the advanced generative AI model identifies the user's interest and emotional patterns and generates a personalized hobby plan. This hobby plan is optimized to capture the user's interest by selecting visual and musical information according to the user's specific preferences. This process uses Python and machine learning libraries to perform data processing and analysis.

[0779] The device receives hobby plans provided by the server via communication and notifies the user in real time. It can also provide interactive experiences using augmented reality technology, overlaying digital information onto the real world through the smartphone's camera, allowing users to experience things like forest walks or museum exhibits with a strong sense of realism. The user's feedback on this experience is sent to the server and used to further improve the generated artificial intelligence model. This entire process provides users with highly customized content, enabling them to enjoy a rich experience.

[0780] As a concrete example, when a user is relaxed, the application can recommend documentary videos related to forests and nature. In this case, an example of a prompt for the generative AI model would be: "Suggest personalized content for the user to enjoy a new nature experience. His mood is relaxed." This allows the system to select the most appropriate information and experience for the user.

[0781] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0782] Step 1:

[0783] Users wear sensor devices while performing their daily activities. The sensor devices detect the user's physical movements and behavioral data in real time and collect it as activity information. This activity information includes steps taken, travel routes, current location, and digital device usage.

[0784] Step 2:

[0785] The terminal transmits activity information collected from sensor devices to a server. The transmitted data is integrated with the user's emotional data on the server. This emotional data is obtained through user input and biosensors and indicates the user's current mood and psychological state.

[0786] Step 3:

[0787] The server inputs integrated activity and sentiment data into a generating AI model. The AI ​​model then analyzes the data and performs specific pattern recognition. The model identifies the user's interests and emotional tendencies and generates an optimized hobby plan. Machine learning algorithms are used in this analysis.

[0788] Step 4:

[0789] The server sends the generated hobby plan to the terminal. This hobby plan includes visual and musical information selected based on the user's interests. The terminal notifies the user in real time and displays the necessary information on the screen.

[0790] Step 5:

[0791] Users engage in hobby activities using hobby plans provided by the device. The device can also provide related augmented reality experiences, allowing users to overlay digital information onto the real world through the camera.

[0792] Step 6:

[0793] After completing a hobby activity, users enter feedback about the experience into their device. This feedback includes how beneficial the hobby plan was and areas where further improvement is needed.

[0794] Step 7:

[0795] The device sends the collected feedback to the server. The server adds this feedback to the dataset of the generating AI model and uses it to improve the next hobby plan generation. This gradually improves the AI ​​model and increases the accuracy of the content it provides.

[0796] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0797] This invention relates to a system for analyzing a user's activity data and emotional state to provide a personalized hobby program. In particular, it uses an emotion engine to collect emotional data and utilize it for analysis, thereby achieving personalization of the user's hobby experience.

[0798] Data collection and emotion recognition

[0799] The device collects user activity data using wearable devices and smartphones. This activity data includes location information, indicators of physical activity, and application usage history. Furthermore, an emotion engine built into the device analyzes the user's facial expressions and voice tone through the camera and microphone, acquiring emotion data in real time.

[0800] Data Integration and Analysis

[0801] The server receives activity and emotion data transmitted from the terminal and integrates them. The integrated data is preprocessed and then analyzed by a generative artificial intelligence model. The model analyzes the user's interests, emotional changes, and behavioral patterns to design a hobby program optimized for the user.

[0802] Generating and providing personalized hobby programs

[0803] Based on the analysis results, the server generates a hobby program that takes into account the user's current emotional state. The generated program includes suggestions for specific hobby activities, detailed steps for those activities, and related resources. Furthermore, by utilizing emotional data, the program is designed to provide the user with the most satisfying experience.

[0804] Real-time AI coaching and feedback collection

[0805] The device notifies the user of a generated hobby program and encourages them to begin activities based on it. While the activities are being carried out, the emotion engine continuously monitors the user's emotions and sends that data to the device. The device uses this data to provide advice and feedback in real time to increase the user's motivation.

[0806] For example, if a user is interested in learning a new musical instrument, the device analyzes the user's emotional data during practice and sends encouraging messages at the appropriate time. By adapting to the user's emotional state in this way, more effective learning and enjoyment of the hobby can be achieved.

[0807] Continuous model improvement

[0808] The server collects feedback even after the activity is completed and continuously improves the generated artificial intelligence model based on that data. This process ensures that subsequent hobby programs are more accurate and tailored to each individual user.

[0809] The present invention aims to provide users with personalized hobby programs that are adjusted in real time by an emotion engine, enabling them to discover new interests and emotions.

[0810] The following describes the processing flow.

[0811] Step 1:

[0812] The device activates its on-board sensor and emotion engine to collect user activity and emotion data. The sensor records the user's location, activity level, and app usage, while the emotion engine analyzes the user's facial expressions and voice to obtain real-time emotion data.

[0813] Step 2:

[0814] The device sends the collected data to the server. The data is sent in batches at regular time intervals, and encrypted communication protocols are used to protect privacy.

[0815] Step 3:

[0816] The server preprocesses the received data and prepares it for analysis. This includes data cleaning, missing value imputation, outlier removal, and data normalization.

[0817] Step 4:

[0818] The server uses an artificial intelligence model to analyze pre-processed data. The model interprets the user's activity patterns and real-time changes in emotions, generating foundational information for a hobby program optimized for the user.

[0819] Step 5:

[0820] Based on the analysis results, the server generates a personalized hobby program for the user. This program includes activity suggestions that take into account the user's current emotional state, along with the steps and resources needed to perform those activities.

[0821] Step 6:

[0822] The device notifies the user of any generated hobby programs. These notifications are delivered through applications the user uses regularly, providing the user with detailed information about the program and recommended steps.

[0823] Step 7:

[0824] The user runs a hobby program, and during execution, the device continuously collects emotional data using an emotion engine. The device analyzes this data in real time and provides advice and positive feedback to support the activity.

[0825] Step 8:

[0826] After the user completes an activity, feedback on the experience obtained through the device is sent to the server. The server then analyzes the feedback and uses it as data to improve the model.

[0827] Step 9:

[0828] The server updates the generated artificial intelligence model based on feedback. This allows for more user-friendly recommendations in the next generation of hobby programs.

[0829] (Example 2)

[0830] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0831] In recent years, there has been a growing demand for systems that provide customized experiences based on individual interests and emotions. Traditional systems struggle to fully utilize users' personal data, resulting in ineffective delivery of programs tailored to individual needs. This has led to a persistent situation where user satisfaction remains low.

[0832] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0833] In this invention, the server includes means for collecting information from a device that acquires the user's physical data and location data; means for integrating the information and the user's emotional state using an analysis device and analyzing it using a generation algorithm model; and means for generating an activity program optimized for the user's interests and emotional state based on the analysis results. This makes it possible to provide a recreation experience optimized for each individual user and improve user satisfaction.

[0834] "User physical data" refers to information indicating the user's physical activity, including steps taken, heart rate, and exercise level.

[0835] "Location data" refers to information about the location of a device or user, including geographical location data such as latitude and longitude.

[0836] An "analysis device" is a device that integrates acquired data and has the function of analyzing users' emotions and behavioral patterns.

[0837] A "generative algorithm model" is an artificial intelligence-based algorithm used to analyze acquired data and generate optimized suggestions and programs for the user.

[0838] An "activity program" is a plan that includes suggestions and procedures for carrying out specific hobbies and recreational activities, generated based on the user's interests and emotional state.

[0839] A "recreational experience" is an experience aimed at enjoyment and relaxation, obtained by users engaging in activities or hobbies suggested to them.

[0840] This invention is a technology system for providing users with personalized recreational experiences. It primarily utilizes the user's physical and location data to generate activity programs optimized for the user's hobbies and interests. The implementation method of this system will be described in detail below.

[0841] The device utilizes sensors built into wearable devices and smartphones to collect the user's physical data. This allows for the acquisition of data such as the user's steps, heart rate, and location. The device also uses its built-in camera and microphone to analyze the user's facial expressions and voice, determining their emotional state in real time.

[0842] The acquired data is sent to a server, which uses an analysis device to integrate this data. The integrated data is then subjected to a generative algorithm model and analyzed based on the user's interests and emotional state. As a result, an activity program optimized for the user is constructed. This program includes suggestions for specific hobby activities, execution steps, and relevant online resources.

[0843] For example, if a user expresses interest in learning a new instrument, the device can provide links to practice programs and online lessons. It can also send encouraging messages and progress-based advice in real time, depending on the user's emotional state.

[0844] This system collects feedback even after an activity is completed and uses that information to improve the generation algorithm model. This process will enable the provision of more effective and smarter programs in the future.

[0845] (Example of a prompt message)

[0846] "How can we analyze user activity and emotional data to suggest the most suitable hobby program?"

[0847] "Please suggest ways for users to maintain motivation while learning a new instrument."

[0848] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0849] Step 1:

[0850] The device collects the user's physical and location data. Specifically, it uses sensors in wearable devices and smartphones to obtain data such as steps taken, heart rate, and location information. The input is this raw data, and the output is structured data that organizes it chronologically.

[0851] Step 2:

[0852] The device uses its internal camera and microphone to analyze the user's facial expressions and voice tone in real time, thereby determining the user's emotional state. The input is collected video and audio data, and the generated output is data indicating the user's specific emotional tendencies.

[0853] Step 3:

[0854] The device sends the physical, location, and emotional data obtained in the previous step to the server. The server receives this data and stores it in its database. The input is the various data sent, and the output is the stored integrated data.

[0855] Step 4:

[0856] The server preprocesses the integrated data and inputs it into the generating AI model. Specifically, it performs noise reduction and imputation of missing data. The input is the integrated raw data, and the output is data formatted in a format suitable for analysis.

[0857] Step 5:

[0858] The server uses a generative AI model to analyze user behavior patterns and emotional tendencies. This allows it to design activity programs optimized for the user. The input is pre-processed data, and the generated output is a specific activity program.

[0859] Step 6:

[0860] The server sends the generated activity program to the terminal, and the terminal notifies the user. The terminal displays the program's contents on the screen, prompting the user to begin the activity. The input is the generated activity program, and the output is the activity start notification received by the user.

[0861] Step 7:

[0862] As the user engages in activities, the device continues to collect emotional data and provide real-time feedback. For example, if progress is good, it sends an encouraging message. The input is emotional data during the activity, and the output is specific feedback to the user.

[0863] Step 8:

[0864] Once the activity is complete, the device collects user feedback and sends it to the server. The server uses this feedback to update the generated AI model and improve the program for the next time. The input is the collected feedback information, and the output is the improved model.

[0865] (Application Example 2)

[0866] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0867] In modern life, there is a demand for personalized content delivery based on an individual's emotional state and interests. However, conventional systems have struggled to provide accurate hobby activities and content that reflect emotional states in real time. This has led to decreased user satisfaction and the inability to obtain an optimal content experience.

[0868] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0869] In this invention, the server includes means for collecting user activity data from a measuring device, means for integrating the activity data and the user's emotions and analyzing them using a generative artificial intelligence model, means for generating hobby activities optimized to the user's interests based on the analysis results, means for providing the hobby activities to the user in real time via communication means, means for collecting feedback on the performance of the hobby activities and improving the generative artificial intelligence model, and means for selecting and providing appropriate video content based on the user's emotional state. This makes it possible to provide a real-time and optimized content experience based on the user's emotional state.

[0870] A "user" refers to an individual who uses the system to engage in hobby activities or receive content.

[0871] "Activity data" refers to data that includes information such as the user's location, indicators of physical activity, and device usage history.

[0872] "Measurement device" refers to sensor devices or equipment used to collect user activity data.

[0873] "Emotions" refer to the psychological state judged from the user's facial expressions, tone of voice, and other factors.

[0874] A "generative artificial intelligence model" refers to a set of algorithms and programs that analyze activity data and emotional data to provide user-optimized suggestions.

[0875] "Hobby activities" refer to recreational and learning activities identified based on the user's interests.

[0876] "Communication methods" refer to the technologies and protocols used to send and receive data and programs between a system and a user.

[0877] "Feedback" refers to information collected from users about their impressions and reactions while engaging in hobby activities.

[0878] "Video content" refers to information in the form of videos or images that is selected based on the user's emotional state.

[0879] The system for implementing this invention is comprised of various technical elements combined to collect and analyze user activity and emotional data. At the heart of this system is a generative artificial intelligence model that provides users with optimized hobby activities and video content in real time. The server implements the invention through the following procedure.

[0880] First, a measuring device worn by the user, such as a smartphone or wearable device, collects activity data. This device incorporates sensors that record location information, indicators of physical activity, and application usage history. In addition, an emotion engine is used to analyze the user's facial expressions and tone of voice to understand their emotional state in real time.

[0881] Next, the server receives data sent from the user's terminal and integrates and analyzes this data using a raw artificial intelligence model. This model, developed using programming languages ​​such as Python, analyzes the user's interests and emotions in detail and generates an optimal hobby activity program. It also selects appropriate video content based on the user's emotions.

[0882] For example, if a user wants to relax after work, the server can recognize that emotional state and notify the device of music videos with relaxing effects.

[0883] Finally, feedback is collected during the execution of the generated hobby activities and used to improve the model's performance. This will improve the accuracy of future hobby activity and content selection.

[0884] Examples of prompts for a generative AI model include:

[0885] "Analyze the user's facial expression data and suggest relaxing content that matches his current emotional state."

[0886] "Please find a program you'll enjoy based on the emotional evaluation derived from this voice tone analysis."

[0887] These are some examples.

[0888] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0889] Step 1:

[0890] The device collects activity data through a measuring device worn by the user. This data includes location information, indicators of physical activity, and application usage history. The input is data signals from sensors, and the output is a structured activity dataset. This activity data is later sent to a server for analysis.

[0891] Step 2:

[0892] The device analyzes facial expressions and voice tone through the camera and microphone to acquire user emotion data using an emotion engine. This input is real-time image and audio data, and the output identifies the user's emotional state. The emotion engine then executes machine learning algorithms to classify the emotion.

[0893] Step 3:

[0894] The server receives activity and emotion data transmitted from the terminal and integrates them. The input is a dataset of activity data and emotion states, and the output is an integrated user characteristics profile. This profile is stored in a database and analyzed using a raw artificial intelligence model.

[0895] Step 4:

[0896] The server uses a generative artificial intelligence model to analyze an integrated profile and generate hobby activities and video content optimized for the user. The input is the user profile, and the output is a recommended hobby activity program and selected content. This process involves the AI ​​model identifying data patterns and recommendation algorithms.

[0897] Step 5:

[0898] The server provides understood hobby activity programs and video content to the terminal in real time via communication means. The input is information about the generated programs and content, and the output is executable instructions sent via the user interface. The user can then take action based on these instructions.

[0899] Step 6:

[0900] The terminal collects feedback while the provided program is running and sends this data to the server. The input is user feedback information, and the output is feedback data for model improvement. The server uses this feedback to continuously learn and improve the raw artificial intelligence model.

[0901] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0902] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0903] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0904] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0905] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0906] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0907] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0908] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0909] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0910] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0911] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0912] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0913] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0914] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0915] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0916] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0917] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0918] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0919] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0920] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0921] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0922] The following is further disclosed regarding the embodiments described above.

[0923] (Claim 1)

[0924] A means for collecting user activity data from a sensor device,

[0925] A means for integrating the aforementioned activity data and user emotions and analyzing them using a generative artificial intelligence model,

[0926] Based on the aforementioned analysis results, a means for generating hobby programs optimized for the user's interests,

[0927] A means of providing the hobby program to the user in real time via a communication means,

[0928] A means for collecting feedback on the execution of the hobby program and improving the generative artificial intelligence model,

[0929] A system that includes this.

[0930] (Claim 2)

[0931] The system according to claim 1, further comprising means of providing hobby-related content from a content library that stores information related to the user, thereby improving the user experience.

[0932] (Claim 3)

[0933] The system according to claim 1, further comprising means for providing an immersive experience to a user using virtual or augmented reality technology.

[0934] "Example 1"

[0935] (Claim 1)

[0936] A means for collecting user activity information from a detection device,

[0937] A means for integrating the aforementioned activity information and the user's emotional state and analyzing it using a generative machine learning model,

[0938] Based on the aforementioned analysis results, a means for constructing hobby activities optimized for the user's preferences,

[0939] A means of presenting the aforementioned hobby activities to the user in real time via communication technology,

[0940] A means for collecting responses related to the performance of the aforementioned hobby activities and for improving the aforementioned generative machine learning model,

[0941] A system that includes this.

[0942] (Claim 2)

[0943] The system according to claim 1, further comprising means for providing hobby-related content from a collection of information storing data related to the user, thereby enhancing the user experience.

[0944] (Claim 3)

[0945] The system according to claim 1, further comprising means for providing an immersive experience to a user using virtual or augmented reality technology.

[0946] "Application Example 1"

[0947] (Claim 1)

[0948] A means for collecting user activity information from sensor devices,

[0949] A means for integrating the aforementioned activity information and user emotions and analyzing them using a generated artificial intelligence model,

[0950] Based on the aforementioned analysis results, a means for generating a hobby plan optimized for the user's interests,

[0951] A means of providing the user with the hobby plan in real time via a communication means,

[0952] A means for collecting feedback on the execution of the hobby plan and improving the generated artificial intelligence model,

[0953] A means of recommending relevant visual and musical information based on the user's interests and emotions,

[0954] Means of providing interactive experiences using augmented reality technology,

[0955] A system that includes this.

[0956] (Claim 2)

[0957] The system according to claim 1, further comprising means for providing hobby content from an information library that stores information related to the user, thereby improving the user experience.

[0958] (Claim 3)

[0959] The system according to claim 1, further comprising means for providing a user with an immersive experience using virtual reality or augmented reality technology and enabling virtual exploration via a smart device.

[0960] "Example 2 of combining an emotion engine"

[0961] (Claim 1)

[0962] A means for collecting information from a device that acquires the user's physical data and location data,

[0963] A means for integrating the aforementioned information and the user's emotional state using an analysis device and analyzing it using a generative algorithm model,

[0964] Based on the aforementioned analysis results, a means for generating an activity program optimized for the user's interests and emotional state,

[0965] A means for providing the activity program to the user in real time via an electronic communication device,

[0966] means for collecting responses regarding the execution of the activity program and for improving the generation algorithm model,

[0967] A system that includes this.

[0968] (Claim 2)

[0969] The system according to claim 1, further comprising means for providing recreational content from a digital library storing user-related information to improve the user experience.

[0970] (Claim 3)

[0971] The system according to claim 1, further comprising means for providing an immersive experience to a user using virtual reality technology or augmented reality technology.

[0972] "Application example 2 when combining with an emotional engine"

[0973] (Claim 1)

[0974] A means of collecting user activity data from a measuring device,

[0975] A means for integrating the aforementioned activity data and user emotions and analyzing them using a generative artificial intelligence model,

[0976] Based on the aforementioned analysis results, a means for generating hobby activities optimized for the user's interests,

[0977] A means of providing the user with the aforementioned hobby activities in real time via a communication means,

[0978] A means for collecting feedback on the performance of the aforementioned hobby activities and for improving the generative artificial intelligence model,

[0979] A means of selecting and providing appropriate video content based on the user's emotional state,

[0980] A system that includes this.

[0981] (Claim 2)

[0982] The system according to claim 1, further comprising means of providing hobby content from an information warehouse that stores information related to the user, thereby improving the user experience.

[0983] (Claim 3)

[0984] The system according to claim 1, further comprising means for providing an immersive experience to a user using virtual or augmented reality technology. [Explanation of symbols]

[0985] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means for collecting user activity data from a sensor device, A means for integrating the aforementioned activity data and user emotions and analyzing them using a generative artificial intelligence model, Based on the aforementioned analysis results, a means for generating hobby programs optimized for the user's interests, A means of providing the hobby program to the user in real time via a communication means, A means for collecting feedback on the execution of the hobby program and improving the generative artificial intelligence model, A system that includes this.

2. The system according to claim 1, further comprising means for providing hobby-related content from a content library that stores information related to the user, thereby improving the user experience.

3. The system according to claim 1, further comprising means for providing an immersive experience to a user using virtual or augmented reality technology.

Images