system

Abstract

We provide the system. [Solution] Means of collecting user behavior data, A means for analyzing the aforementioned behavioral data to identify user behavior patterns, A means for generating a user interface optimized for each user based on the aforementioned behavioral patterns, A system including means for providing the generated user interface to a user.

Description

Technical Field

[0005]

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, the method including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a character of the chatbot, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance as a response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In conventional systems, the main goal was to provide a user interface (UI) that was somewhat user-friendly for many users, but it was difficult to provide an optimal UI for individual users. In particular, there is a need for technology that provides a more personalized experience by taking into account the behavior characteristics and preferences of diverse users. Against this background, there is a need for a system that automatically generates and provides a UI optimized for each user.

Means for Solving the Problems

[0005] The present invention provides a system that includes means for collecting user behavior data, means for analyzing the behavior data to identify user behavior patterns, means for generating a user interface (UI) optimized for each user based on the behavior patterns, and means for providing the generated user interface to the user. Furthermore, the present invention includes means for collecting the usage status of the provided user interface as feedback data, and by continuously improving the user interface using the feedback data, a higher quality user experience is achieved.

[0006] "User behavior data" refers to records of digital actions taken by users when using a system, such as operations, selections, browsing history, and click patterns.

[0007] "Behavioral patterns" refer to a series of regularities and tendencies analyzed from user behavior data, reflecting users' habits and preferences.

[0008] "User interface (UI)" refers to the screen design, navigation, and interaction elements that allow users to directly interact with a system.

[0009] "Optimized for each user" means that the product or service has been customized to be the most user-friendly for a particular user, based on their individual behavior patterns.

[0010] "Means of generation" refers to the processes and technologies used by the system to create the optimal UI based on user behavior patterns.

[0011] "Means of delivery" refers to the processes and technologies used to display the generated UI in a way that is usable by the user.

[0012] "Feedback data" refers to data that records user usage and reactions to the provided UI.

[0013] "Continuous improvement" means using collected feedback data to regularly review the system's UI and evolve it to become even more user-friendly. [Brief explanation of the drawing]

[0014] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of the data processing device and smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] This is a sequence diagram showing the processing flow of the data processing system in Example 2, which incorporates an emotion engine. [Figure 14]A sequence diagram showing the processing flow of a data processing system in Application Example 2 when combined with an emotion engine.

Embodiments for Carrying Out the Invention

[0015] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0016] First, the terms used in the following description will be explained.

[0017] In the following embodiments, a labeled processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0018] In the following embodiments, a labeled RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0019] In the following embodiments, a labeled storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, etc.

[0020] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0021] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0022] [First Embodiment]

[0023] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0024] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0025] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0026] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0027] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0028] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0029] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0030] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0031] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0032] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0033] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0034] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0035] The present invention includes a program for collecting and analyzing user behavior patterns and generating, providing, and improving an optimized user interface (UI). The program's processing will be explained below in natural language, with specific examples.

[0036] Program processing flow

[0037] 1. Data Collection

[0038] Terminal: This collects data on user actions and behaviors when using the system. For example, this includes actions such as users viewing a specific product page, using the search function, or clicking on product details.

[0039] Terminal: Sends collected data to the server in real time.

[0040] 2. Data Analysis

[0041] Server: The server receives data and saves it to the database. When saving, it verifies that the data is accurate and performs data cleansing as needed.

[0042] Server: Extracts behavioral data related to specific users from the database and prepares it for analysis.

[0043] Generative AI Model: This model uses extracted behavioral data as input to analyze user behavior patterns. For example, it can identify when users access the site most frequently, which product categories they prefer, and which features they use most often.

[0044] 3. UI generation

[0045] Generative AI Model: Automatically generates an optimized UI based on user behavior patterns. Specifically, it determines the placement of navigation menus, button positions, and the display order of recommended content.

[0046] Server: Prepares the generated UI layout and necessary resources (images, stylesheets, scripts, etc.) and gets them ready to be delivered to the user.

[0047] 4. UI provided

[0048] Server: Sends an optimized UI to the user's device.

[0049] Terminal: Displays the received UI within a browser or application and provides it to the user.

[0050] 5. Gathering feedback and continuous improvement

[0051] Terminal: Monitors the usage of the provided UI and collects user reactions and operation logs.

[0052] Server: Stores collected feedback data in a database and uses it as new data for analysis.

[0053] Generative AI model: Based on feedback data, it retrains the UI and performs further optimization.

[0054] Specific example

[0055] Example 1: In the case of an e-commerce site

[0056] User A is using an e-commerce site. The default UI displays a general layout for all users. The system detects that User A has viewed products in the "Electronics" category multiple times, and that they frequently view pages related to "Smartphones."

[0057] Data collection: User A's browsing data, specifically the pages related to "smartphones," is sent from the device to the server.

[0058] Data Analysis: The server collects and stores this data, and a generating AI model analyzes user A's behavioral patterns. It is determined that user A is interested in "smartphones".

[0059] UI Generation: The generation AI model generates the optimal UI for user A. In this UI, a "Smartphone" feature section is displayed at the top of the homepage, and products relevant to user A are displayed preferentially.

[0060] UI delivery: The server sends this customized UI to user A's device, and the device displays the UI.

[0061] Feedback Collection: User A begins using the new UI, and operation logs and click data are collected again. For example, this may include a scene where User A purchases a smartphone from the featured section.

[0062] Continuous improvement: Further analyze the collected feedback data and adjust the UI as needed.

[0063] This provides user A with an intuitive interface that is easy to use. This is expected to improve the quality of the user experience and, as a result, increase site usage and purchase rates.

[0064] The system of this invention operates in this manner, improving user satisfaction by providing a user interface tailored to each individual user.

[0065] The following describes the processing flow.

[0066] Step 1: Start data collection

[0067] Terminal: When a user uses the system, it begins monitoring and recording their operations and behavioral data. This includes actions such as viewing a specific product page or using the search function.

[0068] Terminal: Sends recorded behavioral data to the server in real time.

[0069] Step 2: Data reception and storage

[0070] Server: Receives behavioral data sent from the terminal. Verifyes that the data is accurate and complete.

[0071] Server: Receives data and stores it in a database, assigning labels to each category (e.g., page views, searches, clicks).

[0072] Step 3: Data Cleansing

[0073] Server: Cleanses behavioral data in the database, removing duplicate and incomplete data. This improves the accuracy of the analysis.

[0074] Step 4: Data Analysis

[0075] Server: Extracts behavioral data related to a specific user from a cleansed database.

[0076] Server: Inputs the extracted data into a generating AI model to analyze user behavior patterns. For example, it identifies whether a user frequently views products in a particular category.

[0077] Step 5: Generate an optimized UI

[0078] Generative AI Model: Based on analyzed behavioral patterns, it generates a UI layout optimized for that user. Specifically, this includes adjusting the position of navigation menus and customizing product recommendation sections.

[0079] Server: Prepares the generated UI layout information and necessary resources (images, stylesheets, scripts, etc.) and prepares for delivery.

[0080] Step 6: Provide the UI

[0081] Server: Sends the optimized UI to the user's device.

[0082] Terminal: Displays the received UI and provides the user with a new interface.

[0083] Step 7: Gathering Feedback

[0084] Device: Monitors the usage of the provided UI and collects feedback data such as click data and scroll data.

[0085] Terminal: Sends collected feedback data to the server in real time.

[0086] Step 8: Analysis of Feedback Data

[0087] Server: Stores feedback data sent from terminals in a database and prepares it for re-analysis.

[0088] Generative AI Model: Based on feedback data, the UI design model is retrained and continuously improved.

[0089] Step 9: Regenerate and serve the UI

[0090] Server: Uses the retrained model to generate a newly optimized UI and sends it to the user's device.

[0091] Terminal: Displays and provides the user with the regenerated new UI.

[0092] Through these steps, a dynamic and optimized UI is provided that adapts to user behavior. This improves the user experience and maximizes the efficiency of service utilization.

[0093] (Example 1)

[0094] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0095] Traditional user interfaces (UIs) have the problem of providing the same layout and functionality to all users, meaning they are not optimized for individual user preferences and behavioral patterns. As a result, the quality of the user experience suffers, leading to challenges such as low site usage frequency and low purchase rates. Furthermore, even when user behavior data is collected, there is a lack of systems to properly analyze and utilize it, which often slows down improvements to the user interface.

[0096] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0097] In this invention, the server includes means for collecting and transmitting user behavior data in real time, means for storing and analyzing the behavior data in a database, means for analyzing user behavior patterns using a generative AI model, means for monitoring the usage of the provided user interface, means for collecting the usage data as feedback data and storing it in a database, and means for retraining the user interface using the feedback data and continuously optimizing it. This enables the provision of a UI optimized for the behavior patterns of individual users, improving the quality of the user experience and increasing site usage frequency and purchase rates.

[0098] "User behavior data" refers to data about the operations and actions that users perform when using a system.

[0099] A "server" is a computer system that stores, processes, and analyzes data received from users.

[0100] A "database" is an information aggregation system designed to efficiently store data and enable searching and manipulation of that data.

[0101] A "generative AI model" is an artificial intelligence model that has been trained to perform a specific task using machine learning algorithms.

[0102] "Analysis of behavioral patterns" is the process of analyzing collected behavioral data to understand the behavioral tendencies and preferences of specific users.

[0103] A "user interface (UI)" refers to the screens and operating methods used by a user, enabling interaction with the system.

[0104] "Real-time" means that data collection, transmission, and processing are performed instantly without delay.

[0105] "Feedback data" refers to the reactions and operation logs generated by users using the provided user interface.

[0106] "Retraining" is the process of retraining an existing model based on new data to improve its performance.

[0107] "Optimization" means adjusting a system or process to achieve maximum performance or effectiveness under specific conditions.

[0108] The present invention includes a program for collecting and analyzing user behavior data and generating, providing, and improving an optimized user interface (UI). Specific embodiments of this system will be described below.

[0109] System Configuration

[0110] 1. Hardware

[0111] The system uses the following hardware:

[0112] Device: A device used by a user to access information (e.g., personal computer, smartphone, tablet).

[0113] Server: A high-performance computer system used for collecting, storing, and analyzing data.

[0114] Database: An information aggregation system for efficiently storing, searching, and manipulating data (e.g., MySQL, PostgreSQL).

[0115] 2. Software

[0116] The system uses the following software:

[0117] Generative AI Model: An artificial intelligence model that uses machine learning algorithms to analyze user behavior patterns and generate the optimal UI.

[0118] Data cleansing tools: Tools used to verify the accuracy of collected data and to remove any missing or redundant parts.

[0119] Web server software: Software used to deliver appropriate resources in response to user requests (e.g., Apache, Nginx).

[0120] System Embodiment

[0121] 1. Data Collection

[0122] User: Users search for products and view specific product pages through the system.

[0123] Terminal: Records and collects user activity data in real time. For example, clicked links, entered search queries, pages viewed, etc.

[0124] Terminal: Sends collected data to the server in real time using HTTP requests.

[0125] 2. Data Analysis

[0126] Server: Receives data, saves it to a database, and performs analysis. During the saving process, it verifies the accuracy of the data and performs data cleansing.

[0127] Server: Extracts behavioral data related to a specific user from the database. For example, it filters relevant data using the user ID as the key.

[0128] Generative AI model: Inputs extracted data to analyze user behavior patterns and identify areas of interest and behavioral trends.

[0129] 3. UI generation

[0130] Generative AI Model: Generates the optimal UI based on user behavior patterns. Specifically, it determines the position of navigation menus and the display order of recommended products.

[0131] Server: Prepares the resources required for the generated UI layout (image files, CSS stylesheets, JavaScript scripts, etc.).

[0132] 4. UI provided

[0133] Server: Sends an optimized UI to the user's device. Sends HTML, CSS, and JavaScript together as an HTTP response.

[0134] Terminal: Displays the received UI within a browser or application and provides it to the user.

[0135] 5. Gathering feedback and continuous improvement

[0136] Terminal: Monitors the user's use of the provided UI and collects operation logs.

[0137] Device: Sends UI usage data to the server as feedback data.

[0138] Server: Saves feedback data to a database and uses it as new analysis data.

[0139] Generative AI Model: The model is retrained based on feedback data to further optimize the UI.

[0140] Specific example

[0141] Example 1: In the case of an e-commerce site

[0142] Let's assume that user A uses an e-commerce site and frequently browses pages in the "Smartphone" category.

[0143] Data collection: User A's operation data is collected in real time and sent to the server.

[0144] Data analysis: The server collects and stores data, and the generated AI model analyzes user A's behavioral patterns.

[0145] UI Generation: The generation AI model generates the optimal UI for user A and places a "Smartphone" special section at the top of the homepage.

[0146] UI delivery: The server sends the new UI to user A's device, and the browser displays that UI.

[0147] Feedback collection: User A's operation logs and click data are collected again and sent to the server.

[0148] Continuous improvement: Further optimize the UI based on collected feedback data.

[0149] Example of a prompt:

[0150] "If an e-commerce site user frequently browses a specific product category (e.g., smartphones), please explain the process for generating and providing an optimized UI specifically tailored to that user."

[0151] In this way, the system of the present invention improves the quality of the user experience by providing a UI optimized based on the behavioral patterns of individual users.

[0152] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0153] Step 1:

[0154] Collection of operational data:

[0155] When users use the system, they perform operations such as viewing product pages and using the search function.

[0156] The device records this operation data in real time. Specifically, it collects information such as clicked links, entered search queries, and viewed pages.

[0157] Input: User operation data

[0158] Output: Collected real-time operation data

[0159] Step 2:

[0160] Sending data:

[0161] The device sends the collected data to the server in real time using HTTP requests.

[0162] Input: Collected operational data

[0163] Output: Operation data sent to the server

[0164] Step 3:

[0165] Data storage and cleansing:

[0166] The server saves the received operation data to the database.

[0167] Before the server saves the data, it verifies its accuracy and cleanses any unnecessary data. For example, it removes duplicate data and corrects incomplete data.

[0168] Input: Sent operation data

[0169] Output: Saved operation data after cleansing

[0170] Step 4:

[0171] User data extraction:

[0172] The server extracts behavioral data related to a specific user from the database. For example, it filters the data using the user ID as the key.

[0173] Input: Operational data in the database

[0174] Output: Extracted behavioral data

[0175] Step 5:

[0176] Analysis of behavioral patterns:

[0177] The generative AI model receives extracted behavioral data as input and analyzes user behavior patterns. It uses machine learning algorithms to identify user areas of interest and behavioral trends.

[0178] Input: Extracted behavioral data

[0179] Output: Analyzed behavioral patterns

[0180] Step 6:

[0181] UI generation:

[0182] The generation AI model generates the optimal UI based on user behavior patterns. Specifically, it determines the position of navigation menus and the display order of recommended products.

[0183] Input: Analyzed behavioral patterns

[0184] Output: Generated UI layout

[0185] Step 7:

[0186] Resource preparation:

[0187] The server prepares the necessary resources (image files, CSS stylesheets, JavaScript scripts, etc.) for the generated UI layout.

[0188] Input: Generated UI layout

[0189] Output: Preparation of necessary resources

[0190] Step 8:

[0191] UI submission:

[0192] The server sends an optimized UI to the user's device. The HTTP response includes HTML, CSS, and JavaScript files.

[0193] Input: Optimized UI and resources

[0194] Output: UI sent to the user's device

[0195] Step 9:

[0196] UI display:

[0197] The UI received by the device is displayed within the browser or application. The browser engine parses the HTML and displays it on the screen.

[0198] Input: Submitted UI

[0199] Output: Displayed UI

[0200] Step 10:

[0201] Monitoring UI usage:

[0202] The device monitors how users are using the new UI, recording click logs, time spent on the device, and other metrics.

[0203] Input: User operation data

[0204] Output: Collected feedback data

[0205] Step 11:

[0206] Collecting and sending feedback data:

[0207] The device sends the collected feedback data to the server.

[0208] Input: Collected feedback data

[0209] Output: Feedback data sent to the server

[0210] Step 12:

[0211] Storage and analysis of feedback data:

[0212] The server saves the feedback data to the database.

[0213] The generative AI model retrains itself based on feedback data to further optimize the UI.

[0214] Input: Submitted feedback data

[0215] Output: Optimized UI model after retraining

[0216] In this way, the entire system can continuously provide an optimized UI based on the behavioral patterns of individual users.

[0217] (Application Example 1)

[0218] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0219] Conventional autonomous driving systems have a problem in that the information provided to the driver and passengers is uniform and not optimized to the individual user's preferences and behavioral patterns. Specifically, important information is not displayed in the optimal position within the driver's field of vision while driving, causing the driver's attention to be scattered when acquiring information, resulting in problems of reduced safety and comfort. To solve these problems, a system is needed that dynamically optimizes information display based on the driver's real-time behavioral data.

[0220] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0221] In this invention, the server includes means for collecting user behavior data, means for analyzing the behavior data to identify user behavior patterns, means for generating a user interface (UI) optimized for each user based on the behavior patterns, means for providing the generated user interface to the user, a display device for tracking the driver's gaze position and head direction to acquire data, and means for changing the optimal information placement on the display device in real time. As a result, necessary information based on the driver's behavior patterns is displayed in the optimal position in the field of view, improving safety and comfort.

[0222] "User behavior data" refers to data about the operations and actions that users perform when using a system.

[0223] "Behavioral patterns" refer to the tendencies and habits of a user's series of actions.

[0224] A "user interface (UI)" is an interface that includes screens and operability for interaction between the user and the system.

[0225] "Tracking" is a technology that uses sensors to track the user's gaze position, head movements, and other similar information.

[0226] A "display device" is a device that can track eye position and head direction, such as smart glasses or head-mounted displays.

[0227] "Information arrangement" refers to the position and order of information displayed within one's field of vision.

[0228] "Real-time" means that the system responds instantly to user actions.

[0229] This invention relates to a system that tracks the driver's gaze position and head direction and dynamically optimizes the display of information. The system includes a program for collecting and analyzing user behavior data and generating and providing an optimized user interface (UI).

[0230] Details of the program's processing

[0231] Data collection

[0232] The device (such as smart glasses) tracks the driver's gaze position and head movements in real time. Sensors (such as eye-tracking sensors and motion sensors) are used to acquire this data. The acquired data is transmitted in real time to the vehicle's central server.

[0233] Data Analysis

[0234] The server stores the received data in a database and performs data cleansing as needed. Then, it uses a generative AI model (e.g., TensorFlow or PyTorch) to analyze the driver's behavior patterns. This analysis identifies which information the user frequently checks and which locations they focus their attention on.

[0235] UI generation

[0236] The generating AI model creates the optimal information layout for the driver based on analyzed behavioral patterns. Specifically, it places information that the driver frequently checks in the center of their field of vision, and other information in the periphery. A central server prepares this generated UI layout for transmission to the smart glasses in the vehicle.

[0237] UI provided

[0238] The device (such as smart glasses) displays information on its screen based on the received UI layout. This allows the driver to see optimized information within their field of vision.

[0239] Gathering feedback and continuous improvement

[0240] The device monitors the usage of the provided UI and collects feedback data. The collected feedback data is sent back to the server and stored in a database. The server analyzes this feedback data to retrain the generative AI model and further optimize the UI. This cyclical process ensures continuous improvement.

[0241] Specific example

[0242] Let's say driver A is driving on a highway and is using smart glasses. Driver A frequently checks speed information and road sign information. In this case, an optimized UI would place the speed information in the center of the driver's line of sight, with the road sign information displayed below it. Also, if the driver is using voice navigation, that information would also be displayed near their line of sight.

[0243] Examples of prompts for generative AI models

[0244] Analyze user behavior data to identify frequently referenced information and optimize its placement within the smart glasses' field of view. Specifically, if speed information and road sign information are frequently referenced, position them closer to the center of the user's line of sight.

[0245] This allows drivers to quickly and intuitively obtain necessary information, improving driving safety and comfort.

[0246] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0247] Step 1:

[0248] The device (smart glasses) tracks the driver's gaze position and head movements in real time. Specifically, it uses gaze tracking sensors and motion sensors to acquire data on where the driver is looking and in which direction their head is facing. This data is transmitted in real time to the vehicle's central server.

[0249] Step 2:

[0250] The server receives gaze position and head movement data transmitted from the terminal and stores it in a database. Simultaneously, it performs data cleansing to remove noise and errors and improve data accuracy. Here, the input is gaze and head movement data, and the output is the cleansed data. Specifically, data formatting is standardized and missing values ​​are imputed.

[0251] Step 3:

[0252] The server inputs the cleansed data into a generating AI model. The generating AI model analyzes this data to identify the driver's behavior patterns. Specifically, the analysis identifies information that the driver frequently checks and its location from the accumulated data. The input is cleansed data, and the output is data indicating the driver's behavior patterns.

[0253] Step 4:

[0254] The generative AI model generates the optimal information layout for the driver based on analyzed behavioral patterns. Specifically, it places information that the driver frequently checks (e.g., speed information and road sign information) in the center of the field of view, and other information in the periphery. The input is behavioral pattern data, and the output is an optimized UI layout.

[0255] Step 5:

[0256] The server sends the generated UI layout to the smart glasses in the vehicle. The smart glasses display the received UI layout on their screen. This allows the driver to see optimized information within their field of vision. The input is UI layout data, and the output is visual information on the smart glasses' display.

[0257] Step 6:

[0258] The device (smart glasses) monitors the usage of the provided UI and collects feedback data such as operation logs and click data. Specifically, it records what information the driver checks and how often. This data is then sent back to the server. The input is usage data, and the output is the collected feedback data.

[0259] Step 7:

[0260] The server stores the collected feedback data in a database and inputs it into a generative AI model. The generative AI model analyzes the feedback data and retrains and optimizes the UI. Specifically, it fine-tunes the placement based on the driver's reactions to the previous UI placement. The input is the feedback data, and the output is the improved UI layout.

[0261] In this way, drivers will always be able to obtain optimized information, which is expected to improve safety and comfort.

[0262] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0263] This invention combines an emotion engine with a system that generates, provides, and improves user interfaces (UIs) optimized based on user behavior patterns. This system collects and analyzes not only user behavior data but also emotion data to provide the most suitable UI for each individual user.

[0264] Program processing flow

[0265] 1. Data Collection

[0266] Terminal: This collects data on user actions and behaviors when using the system. For example, this includes actions such as users viewing a specific product page, using the search function, or clicking on product details. It also analyzes and records emotional data in real time from the user's facial expressions and voice using an emotion engine.

[0267] Terminal: Sends collected behavioral and emotional data to the server in real time.

[0268] 2. Data reception and storage

[0269] Server: Receives behavioral and sentiment data sent from terminals. Verifyes that the data is accurate and complete.

[0270] Server: Stores received data in a database and assigns labels to each category (e.g., page views, searches, clicks, sentiment).

[0271] 3. Data Cleansing

[0272] Server: Cleanses behavioral and sentiment data in the database, removing duplicate and incomplete data. This improves the accuracy of the analysis.

[0273] 4. Data Analysis

[0274] Server: Extracts behavior data and sentiment data related to a specific user from the cleansed database.

[0275] Server: Inputs the extracted data into the generative AI model to analyze the user's behavior pattern and sentiment pattern. For example, it identifies whether the user frequently browses products in a specific category and what kind of sentiment state the user is in during use.

[0276] 5. Generation of Optimized UI

[0277] Generative AI model: Based on the analyzed behavior pattern and sentiment pattern, generates a UI layout optimized for that user. Specifically, it includes adjusting the position of the navigation menu, customizing the product recommendation section, and making design changes considering the sentiment state.

[0278] Server: Prepares the generated UI layout information and necessary resources (images, style sheets, scripts, etc.) and makes preparations for distribution.

[0279] 6. Provision of UI

[0280] Server: Sends the optimized UI to the user's terminal.

[0281] Terminal: Displays the received UI and provides the user with a new interface.

[0282] 7. Feedback Collection and Continuous Improvement

[0283] Terminal: Monitors the usage status of the provided UI and collects click data, scroll data, and sentiment data.

[0284] Terminal: Sends the collected feedback data to the server in real time.

[0285] Server: Save the feedback data sent from the terminal to the database and prepare it for re-analysis.

[0286] Generative AI Model: Based on the feedback data, re-learn the UI design model and continuously improve it.

[0287] Specific Example

[0288] Example 1: In the case of an e-commerce site

[0289] User B is using an e-commerce site. In the initial UI settings, a general layout for all users is displayed. The system detects that User B has browsed products in the "Fashion" category multiple times and frequently views pages related to "Shoes" among them. Also, the emotion engine detects that User B shows interest such as smiling during browsing.

[0290] Data Collection: The behavioral data of User B browsing the "Shoes" related pages and the emotion data such as smiling during browsing are sent from the terminal to the server.

[0291] Data Analysis: The server collects and saves these data, and the generative AI model analyzes the behavioral pattern and emotion pattern of User B. It is found that User B is interested in "Shoes" and it is understood that User B perceives this favorably.

[0292] UI Generation: The generative AI model generates an optimal UI for User B. In this UI, a special section on "Shoes" is displayed at the top of the homepage, and products related to User B are preferentially listed. Also, if the emotion engine continues to detect a favorable reaction from the user, the colors and layout are adjusted to give a positive impression.

[0293] UI Provision: The server sends this customized UI to User B's terminal, and the terminal displays the UI.

[0294] Feedback Collection: User B begins using the new UI, and operation logs, click data, and sentiment data are collected again. For example, this may include a scene where User B purchases shoes from the featured section.

[0295] Continuous improvement: Further analyze the collected feedback data and adjust the UI as needed.

[0296] In this manner, the system of the present invention operates, improving the quality of the user experience by providing an optimal UI for each individual user based on behavioral and emotional data.

[0297] The following describes the processing flow.

[0298] Step 1: Start data collection

[0299] Terminal: When a user uses the system, it monitors and starts recording their operations and behavioral data. This includes actions such as viewing a specific product page, using the search function, or clicking on product details. It also uses an emotion engine to analyze and record emotional data in real time from the user's facial expressions and voice.

[0300] Step 2: Data transmission

[0301] Terminal: Sends recorded behavioral and emotional data to the server in real time.

[0302] Step 3: Data reception and storage

[0303] Server: Receives behavioral and sentiment data sent from terminals. Verifyes that the data is accurate and complete.

[0304] Server: Stores received data in a database and assigns labels to each category (e.g., page views, searches, clicks, sentiment).

[0305] Step 4: Data Cleansing

[0306] Server: Cleans the action data and sentiment data in the database, and removes duplicate data and incomplete data. This improves the analysis accuracy.

[0307] Step 5: Data Analysis

[0308] Server: Extracts the action data and sentiment data related to a specific user from the cleansed database.

[0309] Server: Inputs the extracted data into the generated AI model to analyze the user's action pattern and sentiment pattern. For example, it identifies whether the user frequently browses products in a specific category and what kind of emotional state the user is in during use.

[0310] Step 6: Generation of Optimized UI

[0311] Generated AI model: Generates a UI layout optimized for that user based on the analyzed action pattern and sentiment pattern. Specifically, it includes adjusting the position of the navigation menu, customizing the product recommendation section, and making design changes considering the emotional state.

[0312] Server: Prepares the generated UI layout information and necessary resources (images, style sheets, scripts, etc.) and makes preparations for distribution.

[0313] Step 7: Provision of UI<00009^{87}

[0314] Server: Sends the optimized UI to the user's terminal.

[0315] Terminal: Displays the received UI and provides the user with a new interface.

[0316] Step 8: Feedback Collection

[0317] Device: Monitors the usage of the provided UI and collects click data, scroll data, and sentiment data.

[0318] Terminal: Sends collected feedback data to the server in real time.

[0319] Step 9: Analysis of Feedback Data

[0320] Server: Stores feedback data sent from terminals in a database and prepares it for re-analysis.

[0321] Generative AI Model: Based on feedback data, the UI design model is retrained and continuously improved.

[0322] Step 10: Regenerate and serve the UI

[0323] Server: Uses the retrained model to generate a newly optimized UI and sends it to the user's device.

[0324] Terminal: Displays and provides the user with the regenerated new UI.

[0325] Through the steps described above, a dynamic and optimal UI is provided based on user behavior and emotional data. This improves the user experience and maximizes the efficiency of service utilization. For example, by detecting when a user frequently views a particular product and the emotions they show interest in it, it is possible to provide product recommendations based on browsing history and adjust the UI based on those emotions.

[0326] (Example 2)

[0327] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0328] Traditional user interfaces (UIs) have been optimized solely based on user behavior data, but this approach fails to take into account user emotional states and positive responses, resulting in insufficient optimization accuracy. Furthermore, feedback collection and analysis for continuously improving the effectiveness of a UI once it has been provided are often inadequate. The objective of this invention is to solve these problems and provide a more intuitive and effective user experience.

[0329] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0330] In this invention,

[0331] The server has means for collecting user behavior data and emotional data,

[0332] A means for analyzing the aforementioned behavioral data and emotional data to identify the user's behavioral patterns and emotional patterns,

[0333] A means for generating a user interface optimized for each user based on the aforementioned behavioral and emotional patterns,

[0334] Means for monitoring the usage of the provided user interface and collecting feedback data,

[0335] A means for analyzing user behavior patterns and emotional patterns using a generative AI model and generating the user interface,

[0336] This includes means for continuously improving the user interface using the collected feedback data.

[0337] This allows for the incorporation of emotional data in addition to behavioral data, and enables continuous improvement of the user interface based on collected feedback data. This, in turn, allows for the delivery of a more personalized, intuitive, and effective user experience.

[0338] "Behavioral data" refers to records of operations and actions performed by users when using a system.

[0339] "Emotional data" refers to information about a user's emotional state, analyzed from their facial expressions and voice.

[0340] "Behavioral patterns" refer to the consistent behavioral patterns of users, based on collected behavioral data.

[0341] An "emotional pattern" is a consistent emotional state of a user, captured based on collected emotional data.

[0342] "User interface (UI)" is a general term for the screens and input methods that users use to operate a system.

[0343] A "generative AI model" is an artificial intelligence model that analyzes user behavior patterns and emotional patterns to generate the optimal user interface.

[0344] "Feedback data" refers to data about the usage and effectiveness of the provided user interface, including click data and scroll data.

[0345] An "optimized user interface" is a user interface that is individually tailored based on the user's behavioral and emotional patterns.

[0346] "Monitoring" refers to the process of monitoring user interface usage in real time and collecting data.

[0347] This invention relates to a system that analyzes user behavior and emotional patterns to provide and improve an optimized user interface (UI). This system collects and analyzes user behavior and emotional data, and improves the quality of the user experience by providing the most suitable UI for each individual user based on the results.

[0348] The system consists of terminals, servers, and generative AI models.

[0349] Data collection

[0350] The device collects data on user actions and behaviors as they use the system. This data includes activities such as browsing web pages, using search functions, and clicking on product details. Furthermore, it uses an emotion engine to acquire real-time emotional data from the user's facial expressions and voice. The collected behavioral and emotional data is transmitted to the server in real time. Encryption technologies such as SSL / TLS are used for data transmission to ensure security.

[0351] Data reception and storage

[0352] The server receives behavioral and emotional data transmitted from the terminal. It performs CRC checks and data integrity checks to verify the accuracy and completeness of the received data. The received data is then stored in a database and labeled according to category (e.g., page views, searches, clicks, emotional state).

[0353] Data cleansing

[0354] The server validates behavioral and sentiment data in the database, removing incomplete and duplicate data. Data integrity tools and scripts are used for this purpose. After a clean dataset is generated, the data is formatted and normalized to improve analysis accuracy.

[0355] Data Analysis

[0356] The server extracts behavioral and emotional data related to a specific user from a cleansed database. This is done using SQL queries and data extraction tools. The extracted data is input into a generative AI model, where the user's behavioral and emotional patterns are analyzed. For example, it might be identified that the user frequently browses products in the "shoes" category and smiles while browsing. Deep learning and machine learning algorithms are used for this analysis.

[0357] Generating an optimized UI

[0358] The generative AI model generates the optimal UI layout for the user based on analyzed behavioral and emotional patterns. Specifically, this includes adjusting the position of navigation menus, customizing product recommendation sections, and making design changes that take emotional states into account. The server prepares the generated UI layout information and necessary resources (images, stylesheets, scripts, etc.) and gets ready to send the optimized UI.

[0359] UI provision

[0360] The server sends an optimized UI to the user's device. This is done using the HTTP protocol. The device displays the received UI in a browser or app, providing the user with a new interface. The display is updated in real time, and user actions are reflected smoothly.

[0361] Feedback gathering and continuous improvement

[0362] The device monitors the usage of the provided UI and collects click data, scroll data, and sentiment data. This data is sent to the server in real time as feedback data. As before, encryption technology is used for transmission to ensure security. The server stores the received feedback data in a database and prepares it for reanalysis. This feedback data is used to retrain the generative AI model, enabling continuous UI improvement.

[0363] Specific example

[0364] Example 1: In the case of an e-commerce site

[0365] User B is using an e-commerce site. The default UI displays a general layout for all users. The system detects that User B repeatedly browses products in the "Fashion" category, and frequently views the "Shoes" page. The emotion engine also detects that User B smiles while browsing. Based on this data, a generative AI model generates an optimal UI for User B, and a special "Shoes" section is displayed at the top of the homepage. Once User B starts using the new UI, their operation logs, click data, and emotion data are collected again, and the system uses this to continuously improve the UI.

[0366] Example of a prompt

[0367] "Analyze the user's behavioral and emotional patterns, and generate a UI that is optimal for that user's preferences. The user has recently been viewing many 'shoe' related products and has shown interest by smiling while doing so."

[0368] Thus, the system of the present invention improves the quality of the user experience by combining and analyzing behavioral data and emotional data, and by providing an optimal and personalized UI for each individual user.

[0369] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0370] Step 1: Data Collection

[0371] Input: User action data, user sentiment data

[0372] Operation and output:

[0373] The device collects behavioral data from users as they use the system. Specifically, this includes web page browsing history, search terms, and click history.

[0374] The device utilizes an emotion engine to collect emotional data in real time from the user's facial expressions and voice. For example, it can detect the moment the user smiles or the tone of their voice.

[0375] The device sends this data to the server in real time. Encryption technologies such as SSL / TLS are used for data transmission to ensure secure transfer of data to the server.

[0376] Step 2: Data reception and storage

[0377] Input: Behavioral data and emotional data sent from the device.

[0378] Operation and output:

[0379] The server receives behavioral and emotional data transmitted from the terminal. To maintain data integrity, CRC checks and data integrity checks are performed.

[0380] The server stores the received data in a database. Each piece of data in the database is assigned a category label (e.g., page views, searches, clicks, sentiment).

[0381] Step 3: Data Cleansing

[0382] Input: Stored behavioral data, emotional data

[0383] Operation and output:

[0384] The server validates behavioral and sentiment data in the database, removing incomplete and duplicate data. This is done using data integrity tools and scripts.

[0385] The server generates a clean dataset and performs data format conversion and normalization to improve analysis accuracy.

[0386] The output consists of cleansed behavioral and emotional data.

[0387] Step 4: Data Analysis

[0388] Input: Cleansed behavioral data, emotional data

[0389] Operation and output:

[0390] The server extracts behavioral and sentiment data related to a specific user from the cleansed database. This is done using SQL queries and data extraction tools.

[0391] The server inputs the extracted data into a generating AI model to analyze the user's behavioral and emotional patterns. For example, it might identify that the user frequently browses products in the "shoes" category and smiles while browsing.

[0392] The output consists of each user's behavioral and emotional patterns.

[0393] Step 5: Generate an optimized UI

[0394] Input: Behavioral patterns, emotional patterns

[0395] Operation and output:

[0396] The generative AI model generates the optimal UI layout for each user based on analyzed behavioral and emotional patterns. Specific examples include adjusting the position of navigation menus and customizing product recommendation sections.

[0397] The server prepares the generated UI layout information and necessary resources (images, stylesheets, scripts, etc.) and gets ready to send the optimized UI.

[0398] The output consists of a user-optimized UI layout design and the necessary resources.

[0399] Step 6: Provide the UI

[0400] Input: Optimized UI layout, resources

[0401] Operation and output:

[0402] The server sends an optimized UI to the user's device. The HTTP protocol is used for this purpose.

[0403] The device displays the received UI in a browser or app, providing the user with a new interface. The display is updated in real time, and user actions are reflected smoothly.

[0404] The output is an optimized UI provided to the user.

[0405] Step 7: Gathering Feedback and Continuous Improvement

[0406] Input: Usage data of the provided UI (click data, scroll data, sentiment data)

[0407] Operation and output:

[0408] The device monitors the usage of the provided UI and collects click data, scroll data, and sentiment data. Specific examples include the frequency of clicks and scroll depth on a particular product page.

[0409] The device sends the collected feedback data to the server in real time.

[0410] The server saves the received feedback data to a database and prepares it for reanalysis.

[0411] The generative AI model retrains its UI design model based on feedback data, enabling continuous improvement.

[0412] The output is an improved user interface for the next version.

[0413] (Application Example 2)

[0414] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0415] Conventional user interface (UI) optimization systems generated UIs based solely on user behavior data, failing to reflect user emotions or real-time states. Consequently, the quality of the user experience was not always high, and user-specific optimization was insufficient. This invention aims to improve the user experience by utilizing both user behavior data and emotional data to provide each user with a highly personalized UI.

[0416] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0417] In this invention, the server includes means for collecting user behavior data and emotional data, means for analyzing the behavior data and emotional data to identify the user's behavior patterns and emotional patterns, and means for generating a user interface (UI) optimized for each user based on the behavior patterns and emotional patterns. This makes it possible to generate and provide an optimal UI from both the user's behavior and emotions, thereby providing a more personalized user experience suited to individual users.

[0418] "User behavior data" refers to data about a user's actions and behavior when using an online system or application, and specifically includes page viewing history, click count, search history, etc.

[0419] "Emotional data" refers to data related to emotions obtained from the user's facial expressions, voice, gestures, etc., such as data representing the user's reactions like smiling, anger, or surprise.

[0420] "Behavioral patterns" refer to a series of trends and characteristics of user behavior obtained by analyzing user behavior data, such as frequently viewing products in a specific category or accessing the site at a specific time of day.

[0421] An "emotional pattern" refers to a series of tendencies and characteristics of emotions obtained by analyzing emotional data, and describes patterns of how a user exhibits emotions in specific situations.

[0422] An "optimized user interface (UI)" is an interface customized based on user behavior and emotional patterns, resulting in an interface that is most user-friendly and appealing to each individual user, with a well-designed and structured interface.

[0423] "Feedback data" refers to data about the usage of the provided user interface, such as what actions the user performed and what emotions they expressed in which parts of the interface.

[0424] "Methods for continuous improvement" refer to a process of re-evaluating the user interface based on feedback data, repeatedly analyzing it, and optimizing it.

[0425] A "generative AI model" is a software model that includes machine learning and deep learning algorithms to generate the optimal user interface based on user behavior data and emotional data.

[0426] This invention is a system that analyzes user behavior data and emotional data to provide a user interface (UI) optimized for each individual user. The specific method for realizing this system is described below.

[0427] Hardware and software used

[0428] 1. Hardware

[0429] The hardware used will consist of terminals (smartphones, tablets, PCs, etc.) and servers for collecting user behavioral and emotional data. Ideally, the terminals should be equipped with a camera and microphone, which will allow for the collection of user facial and voice data.

[0430] 2. Software

[0431] The following software may be used:

[0432] OpenCV: A library for acquiring and processing camera images.

[0433] SpeechRecognition Library: A library for collecting and analyzing speech.

[0434] Requests library: A library for sending and receiving data using REST APIs.

[0435] Generative AI model: A software model that includes machine learning and deep learning algorithms to generate the optimal UI based on user behavior and emotional data.

[0436] Data collection and analysis

[0437] The device collects user behavior data (e.g., page viewing history, click count, search history, etc.) as the user interacts with it. It also uses the camera and microphone to capture the user's facial expressions and voice in real time, collecting emotional data. This data is transmitted from the device to the server in real time and stored in a database.

[0438] The server analyzes the received behavioral and emotional data to identify the user's behavioral and emotional patterns. Specifically, it analyzes whether the user frequently views products in certain categories and their emotional state while using the products, based on the behavioral data.

[0439] UI optimization and delivery

[0440] The generative AI model generates a user-optimized UI based on analyzed behavioral and emotional patterns. This UI specifically includes adjusting the position of navigation menus, customizing product recommendation sections, and making design changes that take emotional states into account.

[0441] The server sends the generated UI layout information and necessary resources (images, stylesheets, scripts, etc.) to the device, which then displays the new UI based on this data.

[0442] Feedback gathering and continuous improvement

[0443] The device monitors the usage of the provided UI, collecting click data, scroll data, and sentiment data in real time. The feedback data is sent back to the server, and the generative AI model uses this data to retrain the UI design model and make continuous improvements.

[0444] Examples of specific cases and prompt statements

[0445] For example, when a user is browsing "shoes" in the fashion category, if their facial expression is smiling, the emotion data "positive" is recorded. Based on this behavioral and emotion data, a generative AI model displays recommended products related to "shoes" on the home screen.

[0446] Example of a prompt:

[0447] When user A is browsing "shoes," if the user's facial expression is smiling, the emotion data "positive" is recorded and analyzed together with behavioral data. As a result, the user will be shown more recommended products related to "shoes."

[0448] Thus, the system of the present invention can integrate user behavior data and emotional data to provide a highly personalized UI.

[0449] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0450] Step 1:

[0451] Data collection

[0452] The device collects user behavior data (page viewing history, click count, search history, etc.) during user interaction. It also uses a camera and microphone to capture user facial expressions and voice in real time, collecting emotional data. Input here consists of user actions and visual / audio information, while output is behavioral data and emotional data. Specifically, camera images are acquired frame by frame using OpenCV, and audio data is collected using the SpeechRecognition library.

[0453] Step 2:

[0454] Data transmission

[0455] The device sends collected behavioral and emotional data to the server in real time. The input here is the collected behavioral and emotional data, and the output is the transmitted data. Specifically, the Requests library is used to send the behavioral and emotional data to the server.

[0456] Step 3:

[0457] Data reception and storage

[0458] The server receives behavioral and emotional data transmitted from the terminal and stores it in a database. The input here is the transmitted data, and the output is the data stored in the database. Specifically, after receiving the data, it is stored in the database in an appropriate format.

[0459] Step 4:

[0460] Data cleansing

[0461] The server cleanses behavioral and sentiment data in the database, removing duplicates and incomplete data. The input is the raw data stored in the database, and the output is the cleansed data. Specifically, it applies algorithms for data consistency checking and deduplication.

[0462] Step 5:

[0463] Data Analysis

[0464] The server extracts behavioral and emotional data related to a specific user from the cleansed data and inputs it into a generative AI model. The generative AI model analyzes the user's behavioral and emotional patterns. Here, the input is the cleansed data, and the output is the behavioral and emotional patterns. Specifically, it performs data analysis using a machine learning model.

[0465] Step 6:

[0466] UI optimization

[0467] The generative AI model generates a user-optimized UI based on analyzed behavioral and emotional patterns. The input here is behavioral and emotional patterns, and the output is an optimized UI layout. Specific actions include adjusting the position of navigation menus and customizing product recommendation sections.

[0468] Step 7:

[0469] UI provided

[0470] The server sends the generated UI layout information and necessary resources to the device. The device then displays the new UI based on this data. The input here is the UI layout information and resources sent from the server, and the output is the displayed new UI. Specifically, the UI layout information and resources are applied to the user's device.

[0471] Step 8:

[0472] Feedback Collection

[0473] The device monitors the usage of the provided UI and collects click data, scroll data, and sentiment data. The inputs here are user interaction data and visual / audio information, while the output is feedback data. Specifically, it prepares to send user interaction data to the server while continuously logging it.

[0474] Step 9:

[0475] Continuous improvement

[0476] The server stores feedback data sent from the terminal in a database, and the generated AI model retrains the UI design model based on this data, continuously improving it. Here, the input is feedback data, and the output is the updated UI design model. Specifically, it performs a retraining process using feedback data to continuously optimize the UI.

[0477] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0478] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0479] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0480] [Second Embodiment]

[0481] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0482] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0483] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0484] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0485] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0486] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0487] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0488] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0489] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0490] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0491] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0492] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0493] The present invention includes a program for collecting and analyzing user behavior patterns and generating, providing, and improving an optimized user interface (UI). The program's processing will be explained below in natural language, with specific examples.

[0494] Program processing flow

[0495] 1. Data Collection

[0496] Terminal: This collects data on user actions and behaviors when using the system. For example, this includes actions such as users viewing a specific product page, using the search function, or clicking on product details.

[0497] Terminal: Sends collected data to the server in real time.

[0498] 2. Data Analysis

[0499] Server: The server receives data and saves it to the database. When saving, it verifies that the data is accurate and performs data cleansing as needed.

[0500] Server: Extracts behavioral data related to specific users from the database and prepares it for analysis.

[0501] Generative AI Model: This model uses extracted behavioral data as input to analyze user behavior patterns. For example, it can identify when users access the site most frequently, which product categories they prefer, and which features they use most often.

[0502] 3. UI generation

[0503] Generative AI Model: Automatically generates an optimized UI based on user behavior patterns. Specifically, it determines the placement of navigation menus, button positions, and the display order of recommended content.

[0504] Server: Prepares the generated UI layout and necessary resources (images, stylesheets, scripts, etc.) and gets them ready to be delivered to the user.

[0505] 4. UI provided

[0506] Server: Sends an optimized UI to the user's device.

[0507] Terminal: Displays the received UI within a browser or application and provides it to the user.

[0508] 5. Gathering feedback and continuous improvement

[0509] Terminal: Monitors the usage of the provided UI and collects user reactions and operation logs.

[0510] Server: Stores collected feedback data in a database and uses it as new data for analysis.

[0511] Generative AI model: Based on feedback data, it retrains the UI and performs further optimization.

[0512] Specific example

[0513] Example 1: In the case of an e-commerce site

[0514] User A is using an e-commerce site. The default UI displays a general layout for all users. The system detects that User A has viewed products in the "Electronics" category multiple times, and that they frequently view pages related to "Smartphones."

[0515] Data collection: User A's browsing data, specifically the pages related to "smartphones," is sent from the device to the server.

[0516] Data Analysis: The server collects and stores this data, and a generating AI model analyzes user A's behavioral patterns. It is determined that user A is interested in "smartphones".

[0517] UI Generation: The generation AI model generates the optimal UI for user A. In this UI, a "Smartphone" feature section is displayed at the top of the homepage, and products relevant to user A are displayed preferentially.

[0518] UI delivery: The server sends this customized UI to user A's device, and the device displays the UI.

[0519] Feedback Collection: User A begins using the new UI, and operation logs and click data are collected again. For example, this may include a scene where User A purchases a smartphone from the featured section.

[0520] Continuous improvement: Further analyze the collected feedback data and adjust the UI as needed.

[0521] This provides user A with an intuitive interface that is easy to use. This is expected to improve the quality of the user experience and, as a result, increase site usage and purchase rates.

[0522] The system of this invention operates in this manner, improving user satisfaction by providing a user interface tailored to each individual user.

[0523] The following describes the processing flow.

[0524] Step 1: Start data collection

[0525] Terminal: When a user uses the system, it begins monitoring and recording their operations and behavioral data. This includes actions such as viewing a specific product page or using the search function.

[0526] Terminal: Sends recorded behavioral data to the server in real time.

[0527] Step 2: Data reception and storage

[0528] Server: Receives behavioral data sent from the terminal. Verifyes that the data is accurate and complete.

[0529] Server: Receives data and stores it in a database, assigning labels to each category (e.g., page views, searches, clicks).

[0530] Step 3: Data Cleansing

[0531] Server: Cleanses behavioral data in the database, removing duplicate and incomplete data. This improves the accuracy of the analysis.

[0532] Step 4: Data Analysis

[0533] Server: Extracts behavioral data related to a specific user from a cleansed database.

[0534] Server: Inputs the extracted data into a generating AI model to analyze user behavior patterns. For example, it identifies whether a user frequently views products in a particular category.

[0535] Step 5: Generate an optimized UI

[0536] Generative AI Model: Based on analyzed behavioral patterns, it generates a UI layout optimized for that user. Specifically, this includes adjusting the position of navigation menus and customizing product recommendation sections.

[0537] Server: Prepares the generated UI layout information and necessary resources (images, stylesheets, scripts, etc.) and prepares for delivery.

[0538] Step 6: Provide the UI

[0539] Server: Sends the optimized UI to the user's device.

[0540] Terminal: Displays the received UI and provides the user with a new interface.

[0541] Step 7: Gathering Feedback

[0542] Device: Monitors the usage of the provided UI and collects feedback data such as click data and scroll data.

[0543] Terminal: Sends collected feedback data to the server in real time.

[0544] Step 8: Analysis of Feedback Data

[0545] Server: Stores feedback data sent from terminals in a database and prepares it for re-analysis.

[0546] Generative AI Model: Based on feedback data, the UI design model is retrained and continuously improved.

[0547] Step 9: Regenerate and serve the UI

[0548] Server: Uses the retrained model to generate a newly optimized UI and sends it to the user's device.

[0549] Terminal: Displays and provides the user with the regenerated new UI.

[0550] Through these steps, a dynamic and optimized UI is provided that adapts to user behavior. This improves the user experience and maximizes the efficiency of service utilization.

[0551] (Example 1)

[0552] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0553] Traditional user interfaces (UIs) have the problem of providing the same layout and functionality to all users, meaning they are not optimized for individual user preferences and behavioral patterns. As a result, the quality of the user experience suffers, leading to challenges such as low site usage frequency and low purchase rates. Furthermore, even when user behavior data is collected, there is a lack of systems to properly analyze and utilize it, which often slows down improvements to the user interface.

[0554] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0555] In this invention, the server includes means for collecting and transmitting user behavior data in real time, means for storing and analyzing the behavior data in a database, means for analyzing user behavior patterns using a generative AI model, means for monitoring the usage of the provided user interface, means for collecting the usage data as feedback data and storing it in a database, and means for retraining the user interface using the feedback data and continuously optimizing it. This enables the provision of a UI optimized for the behavior patterns of individual users, improving the quality of the user experience and increasing site usage frequency and purchase rates.

[0556] "User behavior data" refers to data about the operations and actions that users perform when using a system.

[0557] A "server" is a computer system that stores, processes, and analyzes data received from users.

[0558] A "database" is an information aggregation system designed to efficiently store data and enable searching and manipulation of that data.

[0559] A "generative AI model" is an artificial intelligence model that has been trained to perform a specific task using machine learning algorithms.

[0560] "Analysis of behavioral patterns" is the process of analyzing collected behavioral data to understand the behavioral tendencies and preferences of specific users.

[0561] A "user interface (UI)" refers to the screens and operating methods used by a user, enabling interaction with the system.

[0562] "Real-time" means that data collection, transmission, and processing are performed instantly without delay.

[0563] "Feedback data" refers to the reactions and operation logs generated by users using the provided user interface.

[0564] "Retraining" is the process of retraining an existing model based on new data to improve its performance.

[0565] "Optimization" means adjusting a system or process to achieve maximum performance or effectiveness under specific conditions.

[0566] The present invention includes a program for collecting and analyzing user behavior data and generating, providing, and improving an optimized user interface (UI). Specific embodiments of this system will be described below.

[0567] System Configuration

[0568] 1. Hardware

[0569] The system uses the following hardware:

[0570] Device: A device used by a user to access information (e.g., personal computer, smartphone, tablet).

[0571] Server: A high-performance computer system used for collecting, storing, and analyzing data.

[0572] Database: An information aggregation system for efficiently storing, searching, and manipulating data (e.g., MySQL, PostgreSQL).

[0573] 2. Software

[0574] The system uses the following software:

[0575] Generative AI Model: An artificial intelligence model that uses machine learning algorithms to analyze user behavior patterns and generate the optimal UI.

[0576] Data cleansing tools: Tools used to verify the accuracy of collected data and to remove any missing or redundant parts.

[0577] Web server software: Software used to deliver appropriate resources in response to user requests (e.g., Apache, Nginx).

[0578] System Embodiment

[0579] 1. Data Collection

[0580] User: Users search for products and view specific product pages through the system.

[0581] Terminal: Records and collects user activity data in real time. For example, clicked links, entered search queries, pages viewed, etc.

[0582] Terminal: Sends collected data to the server in real time using HTTP requests.

[0583] 2. Data Analysis

[0584] Server: Receives data, saves it to a database, and performs analysis. During the saving process, it verifies the accuracy of the data and performs data cleansing.

[0585] Server: Extracts behavioral data related to a specific user from the database. For example, it filters relevant data using the user ID as the key.

[0586] Generative AI model: Inputs extracted data to analyze user behavior patterns and identify areas of interest and behavioral trends.

[0587] 3. UI generation

[0588] Generative AI Model: Generates the optimal UI based on user behavior patterns. Specifically, it determines the position of navigation menus and the display order of recommended products.

[0589] Server: Prepares the resources required for the generated UI layout (image files, CSS stylesheets, JavaScript scripts, etc.).

[0590] 4. UI provided

[0591] Server: Sends an optimized UI to the user's device. Sends HTML, CSS, and JavaScript together as an HTTP response.

[0592] Terminal: Displays the received UI within a browser or application and provides it to the user.

[0593] 5. Gathering feedback and continuous improvement

[0594] Terminal: Monitors the user's use of the provided UI and collects operation logs.

[0595] Device: Sends UI usage data to the server as feedback data.

[0596] Server: Saves feedback data to a database and uses it as new analysis data.

[0597] Generative AI Model: The model is retrained based on feedback data to further optimize the UI.

[0598] Specific example

[0599] Example 1: In the case of an e-commerce site

[0600] Let's assume that user A uses an e-commerce site and frequently browses pages in the "Smartphone" category.

[0601] Data collection: User A's operation data is collected in real time and sent to the server.

[0602] Data analysis: The server collects and stores data, and the generated AI model analyzes user A's behavioral patterns.

[0603] UI Generation: The generation AI model generates the optimal UI for user A and places a "Smartphone" special section at the top of the homepage.

[0604] UI delivery: The server sends the new UI to user A's device, and the browser displays that UI.

[0605] Feedback collection: User A's operation logs and click data are collected again and sent to the server.

[0606] Continuous improvement: Further optimize the UI based on collected feedback data.

[0607] Example of a prompt:

[0608] "If an e-commerce site user frequently browses a specific product category (e.g., smartphones), please explain the process for generating and providing an optimized UI specifically tailored to that user."

[0609] In this way, the system of the present invention improves the quality of the user experience by providing a UI optimized based on the behavioral patterns of individual users.

[0610] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0611] Step 1:

[0612] Collection of operational data:

[0613] When users use the system, they perform operations such as viewing product pages and using the search function.

[0614] The device records this operation data in real time. Specifically, it collects information such as clicked links, entered search queries, and viewed pages.

[0615] Input: User operation data

[0616] Output: Collected real-time operation data

[0617] Step 2:

[0618] Sending data:

[0619] The device sends the collected data to the server in real time using HTTP requests.

[0620] Input: Collected operational data

[0621] Output: Operation data sent to the server

[0622] Step 3:

[0623] Data storage and cleansing:

[0624] The server saves the received operation data to the database.

[0625] Before the server saves the data, it verifies its accuracy and cleanses any unnecessary data. For example, it removes duplicate data and corrects incomplete data.

[0626] Input: Sent operation data

[0627] Output: Saved operation data after cleansing

[0628] Step 4:

[0629] User data extraction:

[0630] The server extracts behavioral data related to a specific user from the database. For example, it filters the data using the user ID as the key.

[0631] Input: Operational data in the database

[0632] Output: Extracted behavioral data

[0633] Step 5:

[0634] Analysis of behavioral patterns:

[0635] The generative AI model receives extracted behavioral data as input and analyzes user behavior patterns. It uses machine learning algorithms to identify user areas of interest and behavioral trends.

[0636] Input: Extracted behavioral data

[0637] Output: Analyzed behavioral patterns

[0638] Step 6:

[0639] UI generation:

[0640] The generation AI model generates the optimal UI based on user behavior patterns. Specifically, it determines the position of navigation menus and the display order of recommended products.

[0641] Input: Analyzed behavioral patterns

[0642] Output: Generated UI layout

[0643] Step 7:

[0644] Resource preparation:

[0645] The server prepares the necessary resources (image files, CSS stylesheets, JavaScript scripts, etc.) for the generated UI layout.

[0646] Input: Generated UI layout

[0647] Output: Preparation of necessary resources

[0648] Step 8:

[0649] UI submission:

[0650] The server sends an optimized UI to the user's device. The HTTP response includes HTML, CSS, and JavaScript files.

[0651] Input: Optimized UI and resources

[0652] Output: UI sent to the user's device

[0653] Step 9:

[0654] UI display:

[0655] The UI received by the device is displayed within the browser or application. The browser engine parses the HTML and displays it on the screen.

[0656] Input: Submitted UI

[0657] Output: Displayed UI

[0658] Step 10:

[0659] Monitoring UI usage:

[0660] The device monitors how users are using the new UI, recording click logs, time spent on the device, and other metrics.

[0661] Input: User operation data

[0662] Output: Collected feedback data

[0663] Step 11:

[0664] Collecting and sending feedback data:

[0665] The device sends the collected feedback data to the server.

[0666] Input: Collected feedback data

[0667] Output: Feedback data sent to the server

[0668] Step 12:

[0669] Storage and analysis of feedback data:

[0670] The server saves the feedback data to the database.

[0671] The generative AI model retrains itself based on feedback data to further optimize the UI.

[0672] Input: Submitted feedback data

[0673] Output: Optimized UI model after retraining

[0674] In this way, the entire system can continuously provide an optimized UI based on the behavioral patterns of individual users.

[0675] (Application Example 1)

[0676] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0677] Conventional autonomous driving systems have a problem in that the information provided to the driver and passengers is uniform and not optimized to the individual user's preferences and behavioral patterns. Specifically, important information is not displayed in the optimal position within the driver's field of vision while driving, causing the driver's attention to be scattered when acquiring information, resulting in problems of reduced safety and comfort. To solve these problems, a system is needed that dynamically optimizes information display based on the driver's real-time behavioral data.

[0678] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0679] In this invention, the server includes means for collecting user behavior data, means for analyzing the behavior data to identify user behavior patterns, means for generating a user interface (UI) optimized for each user based on the behavior patterns, means for providing the generated user interface to the user, a display device for tracking the driver's gaze position and head direction to acquire data, and means for changing the optimal information placement on the display device in real time. As a result, necessary information based on the driver's behavior patterns is displayed in the optimal position in the field of view, improving safety and comfort.

[0680] "User behavior data" refers to data about the operations and actions that users perform when using a system.

[0681] "Behavioral patterns" refer to the tendencies and habits of a user's series of actions.

[0682] A "user interface (UI)" is an interface that includes screens and operability for interaction between the user and the system.

[0683] "Tracking" is a technology that uses sensors to track the user's gaze position, head movements, and other similar information.

[0684] A "display device" is a device that can track eye position and head direction, such as smart glasses or head-mounted displays.

[0685] "Information arrangement" refers to the position and order of information displayed within one's field of vision.

[0686] "Real-time" means that the system responds instantly to user actions.

[0687] This invention relates to a system that tracks the driver's gaze position and head direction and dynamically optimizes the display of information. The system includes a program for collecting and analyzing user behavior data and generating and providing an optimized user interface (UI).

[0688] Details of the program's processing

[0689] Data collection

[0690] The device (such as smart glasses) tracks the driver's gaze position and head movements in real time. Sensors (such as eye-tracking sensors and motion sensors) are used to acquire this data. The acquired data is transmitted in real time to the vehicle's central server.

[0691] Data Analysis

[0692] The server stores the received data in a database and performs data cleansing as needed. Then, it uses a generative AI model (e.g., TensorFlow or PyTorch) to analyze the driver's behavior patterns. This analysis identifies which information the user frequently checks and which locations they focus their attention on.

[0693] UI generation

[0694] The generating AI model creates the optimal information layout for the driver based on analyzed behavioral patterns. Specifically, it places information that the driver frequently checks in the center of their field of vision, and other information in the periphery. A central server prepares this generated UI layout for transmission to the smart glasses in the vehicle.

[0695] UI provided

[0696] The device (such as smart glasses) displays information on its screen based on the received UI layout. This allows the driver to see optimized information within their field of vision.

[0697] Gathering feedback and continuous improvement

[0698] The device monitors the usage of the provided UI and collects feedback data. The collected feedback data is sent back to the server and stored in a database. The server analyzes this feedback data to retrain the generative AI model and further optimize the UI. This cyclical process ensures continuous improvement.

[0699] Specific example

[0700] Let's say driver A is driving on a highway and is using smart glasses. Driver A frequently checks speed information and road sign information. In this case, an optimized UI would place the speed information in the center of the driver's line of sight, with the road sign information displayed below it. Also, if the driver is using voice navigation, that information would also be displayed near their line of sight.

[0701] Examples of prompts for generative AI models

[0702] Analyze user behavior data to identify frequently referenced information and optimize its placement within the smart glasses' field of view. Specifically, if speed information and road sign information are frequently referenced, position them closer to the center of the user's line of sight.

[0703] This allows drivers to quickly and intuitively obtain necessary information, improving driving safety and comfort.

[0704] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0705] Step 1:

[0706] The device (smart glasses) tracks the driver's gaze position and head movements in real time. Specifically, it uses gaze tracking sensors and motion sensors to acquire data on where the driver is looking and in which direction their head is facing. This data is transmitted in real time to the vehicle's central server.

[0707] Step 2:

[0708] The server receives gaze position and head movement data transmitted from the terminal and stores it in a database. Simultaneously, it performs data cleansing to remove noise and errors and improve data accuracy. Here, the input is gaze and head movement data, and the output is the cleansed data. Specifically, data formatting is standardized and missing values ​​are imputed.

[0709] Step 3:

[0710] The server inputs the cleansed data into a generating AI model. The generating AI model analyzes this data to identify the driver's behavior patterns. Specifically, the analysis identifies information that the driver frequently checks and its location from the accumulated data. The input is cleansed data, and the output is data indicating the driver's behavior patterns.

[0711] Step 4:

[0712] The generative AI model generates the optimal information layout for the driver based on analyzed behavioral patterns. Specifically, it places information that the driver frequently checks (e.g., speed information and road sign information) in the center of the field of view, and other information in the periphery. The input is behavioral pattern data, and the output is an optimized UI layout.

[0713] Step 5:

[0714] The server sends the generated UI layout to the smart glasses in the vehicle. The smart glasses display the received UI layout on their screen. This allows the driver to see optimized information within their field of vision. The input is UI layout data, and the output is visual information on the smart glasses' display.

[0715] Step 6:

[0716] The device (smart glasses) monitors the usage of the provided UI and collects feedback data such as operation logs and click data. Specifically, it records what information the driver checks and how often. This data is then sent back to the server. The input is usage data, and the output is the collected feedback data.

[0717] Step 7:

[0718] The server stores the collected feedback data in a database and inputs it into a generative AI model. The generative AI model analyzes the feedback data and retrains and optimizes the UI. Specifically, it fine-tunes the placement based on the driver's reactions to the previous UI placement. The input is the feedback data, and the output is the improved UI layout.

[0719] In this way, drivers will always be able to obtain optimized information, which is expected to improve safety and comfort.

[0720] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0721] This invention combines an emotion engine with a system that generates, provides, and improves user interfaces (UIs) optimized based on user behavior patterns. This system collects and analyzes not only user behavior data but also emotion data to provide the most suitable UI for each individual user.

[0722] Program processing flow

[0723] 1. Data Collection

[0724] Terminal: This collects data on user actions and behaviors when using the system. For example, this includes actions such as users viewing a specific product page, using the search function, or clicking on product details. It also analyzes and records emotional data in real time from the user's facial expressions and voice using an emotion engine.

[0725] Terminal: Sends collected behavioral and emotional data to the server in real time.

[0726] 2. Data reception and storage

[0727] Server: Receives behavioral and sentiment data sent from terminals. Verifyes that the data is accurate and complete.

[0728] Server: Stores received data in a database and assigns labels to each category (e.g., page views, searches, clicks, sentiment).

[0729] 3. Data Cleansing

[0730] Server: Cleanses behavioral and sentiment data in the database, removing duplicate and incomplete data. This improves the accuracy of the analysis.

[0731] 4. Data Analysis

[0732] Server: Extracts behavioral and sentiment data related to specific users from the cleansed database.

[0733] Server: The extracted data is input into a generating AI model to analyze user behavior patterns and emotional patterns. For example, it identifies whether a user frequently views products in a particular category and what emotional state they are in while using the product.

[0734] 5. Generating an optimized UI

[0735] Generative AI Model: Based on analyzed behavioral and emotional patterns, it generates a UI layout optimized for that user. Specifically, this includes adjusting the position of navigation menus, customizing product recommendation sections, and making design changes that take emotional states into consideration.

[0736] Server: Prepares the generated UI layout information and necessary resources (images, stylesheets, scripts, etc.) and prepares for delivery.

[0737] 6. Providing a UI

[0738] Server: Sends the optimized UI to the user's device.

[0739] Terminal: Displays the received UI and provides the user with a new interface.

[0740] 7. Gathering feedback and continuous improvement

[0741] Device: Monitors the usage of the provided UI and collects click data, scroll data, and sentiment data.

[0742] Terminal: Sends collected feedback data to the server in real time.

[0743] Server: Stores feedback data sent from terminals in a database and prepares it for re-analysis.

[0744] Generative AI Model: Based on feedback data, the UI design model is retrained and continuously improved.

[0745] Specific example

[0746] Example 1: In the case of an e-commerce site

[0747] User B is using an e-commerce site. The default UI displays a general layout for all users. The system detects that User B has viewed products in the "Fashion" category multiple times, and that they frequently view pages related to "Shoes." The emotion engine also detects that User B has shown interest, such as smiling, while browsing.

[0748] Data Collection: User B's browsing data, including their behavior when viewing "shoes" related pages, and emotional data such as their smile during browsing, are sent from the device to the server.

[0749] Data Analysis: The server collects and stores this data, and the generating AI model analyzes user B's behavioral and emotional patterns. It is determined that user B is interested in "shoes" and that they view this interest favorably.

[0750] UI Generation: The generation AI model generates the optimal UI for user B. In this UI, a "Shoes" feature section is displayed at the top of the homepage, and products relevant to user B are prioritized. Additionally, if the user continues to respond positively, the emotion engine adjusts the colors and layout to create a positive impression.

[0751] UI delivery: The server sends this customized UI to user B's device, and the device displays the UI.

[0752] Feedback Collection: User B begins using the new UI, and operation logs, click data, and sentiment data are collected again. For example, this may include a scene where User B purchases shoes from the featured section.

[0753] Continuous improvement: Further analyze the collected feedback data and adjust the UI as needed.

[0754] In this manner, the system of the present invention operates, improving the quality of the user experience by providing an optimal UI for each individual user based on behavioral and emotional data.

[0755] The following describes the processing flow.

[0756] Step 1: Start data collection

[0757] Terminal: When a user uses the system, it monitors and starts recording their operations and behavioral data. This includes actions such as viewing a specific product page, using the search function, or clicking on product details. It also uses an emotion engine to analyze and record emotional data in real time from the user's facial expressions and voice.

[0758] Step 2: Data transmission

[0759] Terminal: Sends recorded behavioral and emotional data to the server in real time.

[0760] Step 3: Data reception and storage

[0761] Server: Receives behavioral and sentiment data sent from terminals. Verifyes that the data is accurate and complete.

[0762] Server: Stores received data in a database and assigns labels to each category (e.g., page views, searches, clicks, sentiment).

[0763] Step 4: Data Cleansing

[0764] Server: Cleanses behavioral and sentiment data in the database, removing duplicate and incomplete data. This improves the accuracy of the analysis.

[0765] Step 5: Data Analysis

[0766] Server: Extracts behavioral and sentiment data related to specific users from the cleansed database.

[0767] Server: The extracted data is input into a generating AI model to analyze user behavior patterns and emotional patterns. For example, it identifies whether a user frequently views products in a particular category and what emotional state they are in while using the product.

[0768] Step 6: Generate an optimized UI

[0769] Generative AI Model: Based on analyzed behavioral and emotional patterns, it generates a UI layout optimized for that user. Specifically, this includes adjusting the position of navigation menus, customizing product recommendation sections, and making design changes that take emotional states into consideration.

[0770] Server: Prepares the generated UI layout information and necessary resources (images, stylesheets, scripts, etc.) and prepares for delivery.

[0771] Step 7: Provide the UI

[0772] Server: Sends the optimized UI to the user's device.

[0773] Terminal: Displays the received UI and provides the user with a new interface.

[0774] Step 8: Gathering Feedback

[0775] Device: Monitors the usage of the provided UI and collects click data, scroll data, and sentiment data.

[0776] Terminal: Sends collected feedback data to the server in real time.

[0777] Step 9: Analysis of Feedback Data

[0778] Server: Stores feedback data sent from terminals in a database and prepares it for re-analysis.

[0779] Generative AI Model: Based on feedback data, the UI design model is retrained and continuously improved.

[0780] Step 10: Regenerate and serve the UI

[0781] Server: Uses the retrained model to generate a newly optimized UI and sends it to the user's device.

[0782] Terminal: Displays and provides the user with the regenerated new UI.

[0783] Through the steps described above, a dynamic and optimal UI is provided based on user behavior and emotional data. This improves the user experience and maximizes the efficiency of service utilization. For example, by detecting when a user frequently views a particular product and the emotions they show interest in it, it is possible to provide product recommendations based on browsing history and adjust the UI based on those emotions.

[0784] (Example 2)

[0785] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0786] Traditional user interfaces (UIs) have been optimized solely based on user behavior data, but this approach fails to take into account user emotional states and positive responses, resulting in insufficient optimization accuracy. Furthermore, feedback collection and analysis for continuously improving the effectiveness of a UI once it has been provided are often inadequate. The objective of this invention is to solve these problems and provide a more intuitive and effective user experience.

[0787] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0788] In this invention,

[0789] The server has means for collecting user behavior data and emotional data,

[0790] A means for analyzing the aforementioned behavioral data and emotional data to identify the user's behavioral patterns and emotional patterns,

[0791] A means for generating a user interface optimized for each user based on the aforementioned behavioral and emotional patterns,

[0792] Means for monitoring the usage of the provided user interface and collecting feedback data,

[0793] A means for analyzing user behavior patterns and emotional patterns using a generative AI model and generating the user interface,

[0794] This includes means for continuously improving the user interface using the collected feedback data.

[0795] This allows for the incorporation of emotional data in addition to behavioral data, and enables continuous improvement of the user interface based on collected feedback data. This, in turn, allows for the delivery of a more personalized, intuitive, and effective user experience.

[0796] "Behavioral data" refers to records of operations and actions performed by users when using a system.

[0797] "Emotional data" refers to information about a user's emotional state, analyzed from their facial expressions and voice.

[0798] "Behavioral patterns" refer to the consistent behavioral patterns of users, based on collected behavioral data.

[0799] An "emotional pattern" is a consistent emotional state of a user, captured based on collected emotional data.

[0800] "User interface (UI)" is a general term for the screens and input methods that users use to operate a system.

[0801] A "generative AI model" is an artificial intelligence model that analyzes user behavior patterns and emotional patterns to generate the optimal user interface.

[0802] "Feedback data" refers to data about the usage and effectiveness of the provided user interface, including click data and scroll data.

[0803] An "optimized user interface" is a user interface that is individually tailored based on the user's behavioral and emotional patterns.

[0804] "Monitoring" refers to the process of monitoring user interface usage in real time and collecting data.

[0805] This invention relates to a system that analyzes user behavior and emotional patterns to provide and improve an optimized user interface (UI). This system collects and analyzes user behavior and emotional data, and improves the quality of the user experience by providing the most suitable UI for each individual user based on the results.

[0806] The system consists of terminals, servers, and generative AI models.

[0807] Data collection

[0808] The device collects data on user actions and behaviors as they use the system. This data includes activities such as browsing web pages, using search functions, and clicking on product details. Furthermore, it uses an emotion engine to acquire real-time emotional data from the user's facial expressions and voice. The collected behavioral and emotional data is transmitted to the server in real time. Encryption technologies such as SSL / TLS are used for data transmission to ensure security.

[0809] Data reception and storage

[0810] The server receives behavioral and emotional data transmitted from the terminal. It performs CRC checks and data integrity checks to verify the accuracy and completeness of the received data. The received data is then stored in a database and labeled according to category (e.g., page views, searches, clicks, emotional state).

[0811] Data cleansing

[0812] The server validates behavioral and sentiment data in the database, removing incomplete and duplicate data. Data integrity tools and scripts are used for this purpose. After a clean dataset is generated, the data is formatted and normalized to improve analysis accuracy.

[0813] Data Analysis

[0814] The server extracts behavioral and emotional data related to a specific user from a cleansed database. This is done using SQL queries and data extraction tools. The extracted data is input into a generative AI model, where the user's behavioral and emotional patterns are analyzed. For example, it might be identified that the user frequently browses products in the "shoes" category and smiles while browsing. Deep learning and machine learning algorithms are used for this analysis.

[0815] Generating an optimized UI

[0816] The generative AI model generates the optimal UI layout for the user based on analyzed behavioral and emotional patterns. Specifically, this includes adjusting the position of navigation menus, customizing product recommendation sections, and making design changes that take emotional states into account. The server prepares the generated UI layout information and necessary resources (images, stylesheets, scripts, etc.) and gets ready to send the optimized UI.

[0817] UI provision

[0818] The server sends an optimized UI to the user's device. This is done using the HTTP protocol. The device displays the received UI in a browser or app, providing the user with a new interface. The display is updated in real time, and user actions are reflected smoothly.

[0819] Feedback gathering and continuous improvement

[0820] The device monitors the usage of the provided UI and collects click data, scroll data, and sentiment data. This data is sent to the server in real time as feedback data. As before, encryption technology is used for transmission to ensure security. The server stores the received feedback data in a database and prepares it for reanalysis. This feedback data is used to retrain the generative AI model, enabling continuous UI improvement.

[0821] Specific example

[0822] Example 1: In the case of an e-commerce site

[0823] User B is using an e-commerce site. The default UI displays a general layout for all users. The system detects that User B repeatedly browses products in the "Fashion" category, and frequently views the "Shoes" page. The emotion engine also detects that User B smiles while browsing. Based on this data, a generative AI model generates an optimal UI for User B, and a special "Shoes" section is displayed at the top of the homepage. Once User B starts using the new UI, their operation logs, click data, and emotion data are collected again, and the system uses this to continuously improve the UI.

[0824] Example of a prompt

[0825] "Analyze the user's behavioral and emotional patterns, and generate a UI that is optimal for that user's preferences. The user has recently been viewing many 'shoe' related products and has shown interest by smiling while doing so."

[0826] Thus, the system of the present invention improves the quality of the user experience by combining and analyzing behavioral data and emotional data, and by providing an optimal and personalized UI for each individual user.

[0827] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0828] Step 1: Data Collection

[0829] Input: User action data, user sentiment data

[0830] Operation and output:

[0831] The device collects behavioral data from users as they use the system. Specifically, this includes web page browsing history, search terms, and click history.

[0832] The device utilizes an emotion engine to collect emotional data in real time from the user's facial expressions and voice. For example, it can detect the moment the user smiles or the tone of their voice.

[0833] The device sends this data to the server in real time. Encryption technologies such as SSL / TLS are used for data transmission to ensure secure transfer of data to the server.

[0834] Step 2: Data reception and storage

[0835] Input: Behavioral data and emotional data sent from the device.

[0836] Operation and output:

[0837] The server receives behavioral and emotional data transmitted from the terminal. To maintain data integrity, CRC checks and data integrity checks are performed.

[0838] The server stores the received data in a database. Each piece of data in the database is assigned a category label (e.g., page views, searches, clicks, sentiment).

[0839] Step 3: Data Cleansing

[0840] Input: Stored behavioral data, emotional data

[0841] Operation and output:

[0842] The server validates behavioral and sentiment data in the database, removing incomplete and duplicate data. This is done using data integrity tools and scripts.

[0843] The server generates a clean dataset and performs data format conversion and normalization to improve analysis accuracy.

[0844] The output consists of cleansed behavioral and emotional data.

[0845] Step 4: Data Analysis

[0846] Input: Cleansed behavioral data, emotional data

[0847] Operation and output:

[0848] The server extracts behavioral and sentiment data related to a specific user from the cleansed database. This is done using SQL queries and data extraction tools.

[0849] The server inputs the extracted data into a generating AI model to analyze the user's behavioral and emotional patterns. For example, it might identify that the user frequently browses products in the "shoes" category and smiles while browsing.

[0850] The output consists of each user's behavioral and emotional patterns.

[0851] Step 5: Generate an optimized UI

[0852] Input: Behavioral patterns, emotional patterns

[0853] Operation and output:

[0854] The generative AI model generates the optimal UI layout for each user based on analyzed behavioral and emotional patterns. Specific examples include adjusting the position of navigation menus and customizing product recommendation sections.

[0855] The server prepares the generated UI layout information and necessary resources (images, stylesheets, scripts, etc.) and gets ready to send the optimized UI.

[0856] The output consists of a user-optimized UI layout design and the necessary resources.

[0857] Step 6: Provide the UI

[0858] Input: Optimized UI layout, resources

[0859] Operation and output:

[0860] The server sends an optimized UI to the user's device. The HTTP protocol is used for this purpose.

[0861] The device displays the received UI in a browser or app, providing the user with a new interface. The display is updated in real time, and user actions are reflected smoothly.

[0862] The output is an optimized UI provided to the user.

[0863] Step 7: Gathering Feedback and Continuous Improvement

[0864] Input: Usage data of the provided UI (click data, scroll data, sentiment data)

[0865] Operation and output:

[0866] The device monitors the usage of the provided UI and collects click data, scroll data, and sentiment data. Specific examples include the frequency of clicks and scroll depth on a particular product page.

[0867] The device sends the collected feedback data to the server in real time.

[0868] The server saves the received feedback data to a database and prepares it for reanalysis.

[0869] The generative AI model retrains its UI design model based on feedback data, enabling continuous improvement.

[0870] The output is an improved user interface for the next version.

[0871] (Application Example 2)

[0872] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0873] Conventional user interface (UI) optimization systems generated UIs based solely on user behavior data, failing to reflect user emotions or real-time states. Consequently, the quality of the user experience was not always high, and user-specific optimization was insufficient. This invention aims to improve the user experience by utilizing both user behavior data and emotional data to provide each user with a highly personalized UI.

[0874] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0875] In this invention, the server includes means for collecting user behavior data and emotional data, means for analyzing the behavior data and emotional data to identify the user's behavior patterns and emotional patterns, and means for generating a user interface (UI) optimized for each user based on the behavior patterns and emotional patterns. This makes it possible to generate and provide an optimal UI from both the user's behavior and emotions, thereby providing a more personalized user experience suited to individual users.

[0876] "User behavior data" refers to data about a user's actions and behavior when using an online system or application, and specifically includes page viewing history, click count, search history, etc.

[0877] "Emotional data" refers to data related to emotions obtained from the user's facial expressions, voice, gestures, etc., such as data representing the user's reactions like smiling, anger, or surprise.

[0878] "Behavioral patterns" refer to a series of trends and characteristics of user behavior obtained by analyzing user behavior data, such as frequently viewing products in a specific category or accessing the site at a specific time of day.

[0879] An "emotional pattern" refers to a series of tendencies and characteristics of emotions obtained by analyzing emotional data, and describes patterns of how a user exhibits emotions in specific situations.

[0880] An "optimized user interface (UI)" is an interface customized based on user behavior and emotional patterns, resulting in an interface that is most user-friendly and appealing to each individual user, with a well-designed and structured interface.

[0881] "Feedback data" refers to data about the usage of the provided user interface, such as what actions the user performed and what emotions they expressed in which parts of the interface.

[0882] "Methods for continuous improvement" refer to a process of re-evaluating the user interface based on feedback data, repeatedly analyzing it, and optimizing it.

[0883] A "generative AI model" is a software model that includes machine learning and deep learning algorithms to generate the optimal user interface based on user behavior data and emotional data.

[0884] This invention is a system that analyzes user behavior data and emotional data to provide a user interface (UI) optimized for each individual user. The specific method for realizing this system is described below.

[0885] Hardware and software used

[0886] 1. Hardware

[0887] The hardware used will consist of terminals (smartphones, tablets, PCs, etc.) and servers for collecting user behavioral and emotional data. Ideally, the terminals should be equipped with a camera and microphone, which will allow for the collection of user facial and voice data.

[0888] 2. Software

[0889] The following software may be used:

[0890] OpenCV: A library for acquiring and processing camera images.

[0891] SpeechRecognition Library: A library for collecting and analyzing speech.

[0892] Requests library: A library for sending and receiving data using REST APIs.

[0893] Generative AI model: A software model that includes machine learning and deep learning algorithms to generate the optimal UI based on user behavior and emotional data.

[0894] Data collection and analysis

[0895] The device collects user behavior data (e.g., page viewing history, click count, search history, etc.) as the user interacts with it. It also uses the camera and microphone to capture the user's facial expressions and voice in real time, collecting emotional data. This data is transmitted from the device to the server in real time and stored in a database.

[0896] The server analyzes the received behavioral and emotional data to identify the user's behavioral and emotional patterns. Specifically, it analyzes whether the user frequently views products in certain categories and their emotional state while using the products, based on the behavioral data.

[0897] UI optimization and delivery

[0898] The generative AI model generates a user-optimized UI based on analyzed behavioral and emotional patterns. This UI specifically includes adjusting the position of navigation menus, customizing product recommendation sections, and making design changes that take emotional states into account.

[0899] The server sends the generated UI layout information and necessary resources (images, stylesheets, scripts, etc.) to the device, which then displays the new UI based on this data.

[0900] Feedback gathering and continuous improvement

[0901] The device monitors the usage of the provided UI, collecting click data, scroll data, and sentiment data in real time. The feedback data is sent back to the server, and the generative AI model uses this data to retrain the UI design model and make continuous improvements.

[0902] Examples of specific cases and prompt statements

[0903] For example, when a user is browsing "shoes" in the fashion category, if their facial expression is smiling, the emotion data "positive" is recorded. Based on this behavioral and emotion data, a generative AI model displays recommended products related to "shoes" on the home screen.

[0904] Example of a prompt:

[0905] When user A is browsing "shoes," if the user's facial expression is smiling, the emotion data "positive" is recorded and analyzed together with behavioral data. As a result, the user will be shown more recommended products related to "shoes."

[0906] Thus, the system of the present invention can integrate user behavior data and emotional data to provide a highly personalized UI.

[0907] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0908] Step 1:

[0909] Data collection

[0910] The device collects user behavior data (page viewing history, click count, search history, etc.) during user interaction. It also uses a camera and microphone to capture user facial expressions and voice in real time, collecting emotional data. Input here consists of user actions and visual / audio information, while output is behavioral data and emotional data. Specifically, camera images are acquired frame by frame using OpenCV, and audio data is collected using the SpeechRecognition library.

[0911] Step 2:

[0912] Data transmission

[0913] The device sends collected behavioral and emotional data to the server in real time. The input here is the collected behavioral and emotional data, and the output is the transmitted data. Specifically, the Requests library is used to send the behavioral and emotional data to the server.

[0914] Step 3:

[0915] Data reception and storage

[0916] The server receives behavioral and emotional data transmitted from the terminal and stores it in a database. The input here is the transmitted data, and the output is the data stored in the database. Specifically, after receiving the data, it is stored in the database in an appropriate format.

[0917] Step 4:

[0918] Data cleansing

[0919] The server cleanses behavioral and sentiment data in the database, removing duplicates and incomplete data. The input is the raw data stored in the database, and the output is the cleansed data. Specifically, it applies algorithms for data consistency checking and deduplication.

[0920] Step 5:

[0921] Data Analysis

[0922] The server extracts behavioral and emotional data related to a specific user from the cleansed data and inputs it into a generative AI model. The generative AI model analyzes the user's behavioral and emotional patterns. Here, the input is the cleansed data, and the output is the behavioral and emotional patterns. Specifically, it performs data analysis using a machine learning model.

[0923] Step 6:

[0924] UI optimization

[0925] The generative AI model generates a user-optimized UI based on analyzed behavioral and emotional patterns. The input here is behavioral and emotional patterns, and the output is an optimized UI layout. Specific actions include adjusting the position of navigation menus and customizing product recommendation sections.

[0926] Step 7:

[0927] UI provided

[0928] The server sends the generated UI layout information and necessary resources to the device. The device then displays the new UI based on this data. The input here is the UI layout information and resources sent from the server, and the output is the displayed new UI. Specifically, the UI layout information and resources are applied to the user's device.

[0929] Step 8:

[0930] Feedback Collection

[0931] The device monitors the usage of the provided UI and collects click data, scroll data, and sentiment data. The inputs here are user interaction data and visual / audio information, while the output is feedback data. Specifically, it prepares to send user interaction data to the server while continuously logging it.

[0932] Step 9:

[0933] Continuous improvement

[0934] The server stores feedback data sent from the terminal in a database, and the generated AI model retrains the UI design model based on this data, continuously improving it. Here, the input is feedback data, and the output is the updated UI design model. Specifically, it performs a retraining process using feedback data to continuously optimize the UI.

[0935] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0936] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0937] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0938] [Third Embodiment]

[0939] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0940] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0941] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0942] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0943] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0944] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0945] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0946] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0947] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0948] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0949] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0950] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0951] The present invention includes a program for collecting and analyzing user behavior patterns and generating, providing, and improving an optimized user interface (UI). The program's processing will be explained below in natural language, with specific examples.

[0952] Program processing flow

[0953] 1. Data Collection

[0954] Terminal: This collects data on user actions and behaviors when using the system. For example, this includes actions such as users viewing a specific product page, using the search function, or clicking on product details.

[0955] Terminal: Sends collected data to the server in real time.

[0956] 2. Data Analysis

[0957] Server: The server receives data and saves it to the database. When saving, it verifies that the data is accurate and performs data cleansing as needed.

[0958] Server: Extracts behavioral data related to specific users from the database and prepares it for analysis.

[0959] Generative AI Model: This model uses extracted behavioral data as input to analyze user behavior patterns. For example, it can identify when users access the site most frequently, which product categories they prefer, and which features they use most often.

[0960] 3. UI generation

[0961] Generative AI Model: Automatically generates an optimized UI based on user behavior patterns. Specifically, it determines the placement of navigation menus, button positions, and the display order of recommended content.

[0962] Server: Prepares the generated UI layout and necessary resources (images, stylesheets, scripts, etc.) and gets them ready to be delivered to the user.

[0963] 4. UI provided

[0964] Server: Sends an optimized UI to the user's device.

[0965] Terminal: Displays the received UI within a browser or application and provides it to the user.

[0966] 5. Gathering feedback and continuous improvement

[0967] Terminal: Monitors the usage of the provided UI and collects user reactions and operation logs.

[0968] Server: Stores collected feedback data in a database and uses it as new data for analysis.

[0969] Generative AI model: Based on feedback data, it retrains the UI and performs further optimization.

[0970] Specific example

[0971] Example 1: In the case of an e-commerce site

[0972] User A is using an e-commerce site. The default UI displays a general layout for all users. The system detects that User A has viewed products in the "Electronics" category multiple times, and that they frequently view pages related to "Smartphones."

[0973] Data collection: User A's browsing data, specifically the pages related to "smartphones," is sent from the device to the server.

[0974] Data Analysis: The server collects and stores this data, and a generating AI model analyzes user A's behavioral patterns. It is determined that user A is interested in "smartphones".

[0975] UI Generation: The generation AI model generates the optimal UI for user A. In this UI, a "Smartphone" feature section is displayed at the top of the homepage, and products relevant to user A are displayed preferentially.

[0976] UI delivery: The server sends this customized UI to user A's device, and the device displays the UI.

[0977] Feedback Collection: User A begins using the new UI, and operation logs and click data are collected again. For example, this may include a scene where User A purchases a smartphone from the featured section.

[0978] Continuous improvement: Further analyze the collected feedback data and adjust the UI as needed.

[0979] This provides user A with an intuitive interface that is easy to use. This is expected to improve the quality of the user experience and, as a result, increase site usage and purchase rates.

[0980] The system of this invention operates in this manner, improving user satisfaction by providing a user interface tailored to each individual user.

[0981] The following describes the processing flow.

[0982] Step 1: Start data collection

[0983] Terminal: When a user uses the system, it begins monitoring and recording their operations and behavioral data. This includes actions such as viewing a specific product page or using the search function.

[0984] Terminal: Sends recorded behavioral data to the server in real time.

[0985] Step 2: Data reception and storage

[0986] Server: Receives behavioral data sent from the terminal. Verifyes that the data is accurate and complete.

[0987] Server: Receives data and stores it in a database, assigning labels to each category (e.g., page views, searches, clicks).

[0988] Step 3: Data Cleansing

[0989] Server: Cleanses behavioral data in the database, removing duplicate and incomplete data. This improves the accuracy of the analysis.

[0990] Step 4: Data Analysis

[0991] Server: Extracts behavioral data related to a specific user from a cleansed database.

[0992] Server: Inputs the extracted data into a generating AI model to analyze user behavior patterns. For example, it identifies whether a user frequently views products in a particular category.

[0993] Step 5: Generate an optimized UI

[0994] Generative AI Model: Based on analyzed behavioral patterns, it generates a UI layout optimized for that user. Specifically, this includes adjusting the position of navigation menus and customizing product recommendation sections.

[0995] Server: Prepares the generated UI layout information and necessary resources (images, stylesheets, scripts, etc.) and prepares for delivery.

[0996] Step 6: Provide the UI

[0997] Server: Sends the optimized UI to the user's device.

[0998] Terminal: Displays the received UI and provides the user with a new interface.

[0999] Step 7: Gathering Feedback

[1000] Device: Monitors the usage of the provided UI and collects feedback data such as click data and scroll data.

[1001] Terminal: Sends collected feedback data to the server in real time.

[1002] Step 8: Analysis of Feedback Data

[1003] Server: Stores feedback data sent from terminals in a database and prepares it for re-analysis.

[1004] Generative AI Model: Based on feedback data, the UI design model is retrained and continuously improved.

[1005] Step 9: Regenerate and serve the UI

[1006] Server: Uses the retrained model to generate a newly optimized UI and sends it to the user's device.

[1007] Terminal: Displays and provides the user with the regenerated new UI.

[1008] Through these steps, a dynamic and optimized UI is provided that adapts to user behavior. This improves the user experience and maximizes the efficiency of service utilization.

[1009] (Example 1)

[1010] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1011] Traditional user interfaces (UIs) have the problem of providing the same layout and functionality to all users, meaning they are not optimized for individual user preferences and behavioral patterns. As a result, the quality of the user experience suffers, leading to challenges such as low site usage frequency and low purchase rates. Furthermore, even when user behavior data is collected, there is a lack of systems to properly analyze and utilize it, which often slows down improvements to the user interface.

[1012] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1013] In this invention, the server includes means for collecting and transmitting user behavior data in real time, means for storing and analyzing the behavior data in a database, means for analyzing user behavior patterns using a generative AI model, means for monitoring the usage of the provided user interface, means for collecting the usage data as feedback data and storing it in a database, and means for retraining the user interface using the feedback data and continuously optimizing it. This enables the provision of a UI optimized for the behavior patterns of individual users, improving the quality of the user experience and increasing site usage frequency and purchase rates.

[1014] "User behavior data" refers to data about the operations and actions that users perform when using a system.

[1015] A "server" is a computer system that stores, processes, and analyzes data received from users.

[1016] A "database" is an information aggregation system designed to efficiently store data and enable searching and manipulation of that data.

[1017] A "generative AI model" is an artificial intelligence model that has been trained to perform a specific task using machine learning algorithms.

[1018] "Analysis of behavioral patterns" is the process of analyzing collected behavioral data to understand the behavioral tendencies and preferences of specific users.

[1019] A "user interface (UI)" refers to the screens and operating methods used by a user, enabling interaction with the system.

[1020] "Real-time" means that data collection, transmission, and processing are performed instantly without delay.

[1021] "Feedback data" refers to the reactions and operation logs generated by users using the provided user interface.

[1022] "Retraining" is the process of retraining an existing model based on new data to improve its performance.

[1023] "Optimization" means adjusting a system or process to achieve maximum performance or effectiveness under specific conditions.

[1024] The present invention includes a program for collecting and analyzing user behavior data and generating, providing, and improving an optimized user interface (UI). Specific embodiments of this system will be described below.

[1025] System Configuration

[1026] 1. Hardware

[1027] The system uses the following hardware:

[1028] Device: A device used by a user to access information (e.g., personal computer, smartphone, tablet).

[1029] Server: A high-performance computer system used for collecting, storing, and analyzing data.

[1030] Database: An information aggregation system for efficiently storing, searching, and manipulating data (e.g., MySQL, PostgreSQL).

[1031] 2. Software

[1032] The system uses the following software:

[1033] Generative AI Model: An artificial intelligence model that uses machine learning algorithms to analyze user behavior patterns and generate the optimal UI.

[1034] Data cleansing tools: Tools used to verify the accuracy of collected data and to remove any missing or redundant parts.

[1035] Web server software: Software used to deliver appropriate resources in response to user requests (e.g., Apache, Nginx).

[1036] System Embodiment

[1037] 1. Data Collection

[1038] User: Users search for products and view specific product pages through the system.

[1039] Terminal: Records and collects user activity data in real time. For example, clicked links, entered search queries, pages viewed, etc.

[1040] Terminal: Sends collected data to the server in real time using HTTP requests.

[1041] 2. Data Analysis

[1042] Server: Receives data, saves it to a database, and performs analysis. During the saving process, it verifies the accuracy of the data and performs data cleansing.

[1043] Server: Extracts behavioral data related to a specific user from the database. For example, it filters relevant data using the user ID as the key.

[1044] Generative AI model: Inputs extracted data to analyze user behavior patterns and identify areas of interest and behavioral trends.

[1045] 3. UI generation

[1046] Generative AI Model: Generates the optimal UI based on user behavior patterns. Specifically, it determines the position of navigation menus and the display order of recommended products.

[1047] Server: Prepares the resources required for the generated UI layout (image files, CSS stylesheets, JavaScript scripts, etc.).

[1048] 4. UI provided

[1049] Server: Sends an optimized UI to the user's device. Sends HTML, CSS, and JavaScript together as an HTTP response.

[1050] Terminal: Displays the received UI within a browser or application and provides it to the user.

[1051] 5. Gathering feedback and continuous improvement

[1052] Terminal: Monitors the user's use of the provided UI and collects operation logs.

[1053] Device: Sends UI usage data to the server as feedback data.

[1054] Server: Saves feedback data to a database and uses it as new analysis data.

[1055] Generative AI Model: The model is retrained based on feedback data to further optimize the UI.

[1056] Specific example

[1057] Example 1: In the case of an e-commerce site

[1058] Let's assume that user A uses an e-commerce site and frequently browses pages in the "Smartphone" category.

[1059] Data collection: User A's operation data is collected in real time and sent to the server.

[1060] Data analysis: The server collects and stores data, and the generated AI model analyzes user A's behavioral patterns.

[1061] UI Generation: The generation AI model generates the optimal UI for user A and places a "Smartphone" special section at the top of the homepage.

[1062] UI delivery: The server sends the new UI to user A's device, and the browser displays that UI.

[1063] Feedback collection: User A's operation logs and click data are collected again and sent to the server.

[1064] Continuous improvement: Further optimize the UI based on collected feedback data.

[1065] Example of a prompt:

[1066] "If an e-commerce site user frequently browses a specific product category (e.g., smartphones), please explain the process for generating and providing an optimized UI specifically tailored to that user."

[1067] In this way, the system of the present invention improves the quality of the user experience by providing a UI optimized based on the behavioral patterns of individual users.

[1068] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1069] Step 1:

[1070] Collection of operational data:

[1071] When users use the system, they perform operations such as viewing product pages and using the search function.

[1072] The device records this operation data in real time. Specifically, it collects information such as clicked links, entered search queries, and viewed pages.

[1073] Input: User operation data

[1074] Output: Collected real-time operation data

[1075] Step 2:

[1076] Sending data:

[1077] The device sends the collected data to the server in real time using HTTP requests.

[1078] Input: Collected operational data

[1079] Output: Operation data sent to the server

[1080] Step 3:

[1081] Data storage and cleansing:

[1082] The server saves the received operation data to the database.

[1083] Before the server saves the data, it verifies its accuracy and cleanses any unnecessary data. For example, it removes duplicate data and corrects incomplete data.

[1084] Input: Sent operation data

[1085] Output: Saved operation data after cleansing

[1086] Step 4:

[1087] User data extraction:

[1088] The server extracts behavioral data related to a specific user from the database. For example, it filters the data using the user ID as the key.

[1089] Input: Operational data in the database

[1090] Output: Extracted behavioral data

[1091] Step 5:

[1092] Analysis of behavioral patterns:

[1093] The generative AI model receives extracted behavioral data as input and analyzes user behavior patterns. It uses machine learning algorithms to identify user areas of interest and behavioral trends.

[1094] Input: Extracted behavioral data

[1095] Output: Analyzed behavioral patterns

[1096] Step 6:

[1097] UI generation:

[1098] The generation AI model generates the optimal UI based on user behavior patterns. Specifically, it determines the position of navigation menus and the display order of recommended products.

[1099] Input: Analyzed behavioral patterns

[1100] Output: Generated UI layout

[1101] Step 7:

[1102] Resource preparation:

[1103] The server prepares the necessary resources (image files, CSS stylesheets, JavaScript scripts, etc.) for the generated UI layout.

[1104] Input: Generated UI layout

[1105] Output: Preparation of necessary resources

[1106] Step 8:

[1107] UI submission:

[1108] The server sends an optimized UI to the user's device. The HTTP response includes HTML, CSS, and JavaScript files.

[1109] Input: Optimized UI and resources

[1110] Output: UI sent to the user's device

[1111] Step 9:

[1112] UI display:

[1113] The UI received by the device is displayed within the browser or application. The browser engine parses the HTML and displays it on the screen.

[1114] Input: Submitted UI

[1115] Output: Displayed UI

[1116] Step 10:

[1117] Monitoring UI usage:

[1118] The device monitors how users are using the new UI, recording click logs, time spent on the device, and other metrics.

[1119] Input: User operation data

[1120] Output: Collected feedback data

[1121] Step 11:

[1122] Collecting and sending feedback data:

[1123] The device sends the collected feedback data to the server.

[1124] Input: Collected feedback data

[1125] Output: Feedback data sent to the server

[1126] Step 12:

[1127] Storage and analysis of feedback data:

[1128] The server saves the feedback data to the database.

[1129] The generative AI model retrains itself based on feedback data to further optimize the UI.

[1130] Input: Submitted feedback data

[1131] Output: Optimized UI model after retraining

[1132] In this way, the entire system can continuously provide an optimized UI based on the behavioral patterns of individual users.

[1133] (Application Example 1)

[1134] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1135] Conventional autonomous driving systems have a problem in that the information provided to the driver and passengers is uniform and not optimized to the individual user's preferences and behavioral patterns. Specifically, important information is not displayed in the optimal position within the driver's field of vision while driving, causing the driver's attention to be scattered when acquiring information, resulting in problems of reduced safety and comfort. To solve these problems, a system is needed that dynamically optimizes information display based on the driver's real-time behavioral data.

[1136] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1137] In this invention, the server includes means for collecting user behavior data, means for analyzing the behavior data to identify user behavior patterns, means for generating a user interface (UI) optimized for each user based on the behavior patterns, means for providing the generated user interface to the user, a display device for tracking the driver's gaze position and head direction to acquire data, and means for changing the optimal information placement on the display device in real time. As a result, necessary information based on the driver's behavior patterns is displayed in the optimal position in the field of view, improving safety and comfort.

[1138] "User behavior data" refers to data about the operations and actions that users perform when using a system.

[1139] "Behavioral patterns" refer to the tendencies and habits of a user's series of actions.

[1140] A "user interface (UI)" is an interface that includes screens and operability for interaction between the user and the system.

[1141] "Tracking" is a technology that uses sensors to track the user's gaze position, head movements, and other similar information.

[1142] A "display device" is a device that can track eye position and head direction, such as smart glasses or head-mounted displays.

[1143] "Information arrangement" refers to the position and order of information displayed within one's field of vision.

[1144] "Real-time" means that the system responds instantly to user actions.

[1145] This invention relates to a system that tracks the driver's gaze position and head direction and dynamically optimizes the display of information. The system includes a program for collecting and analyzing user behavior data and generating and providing an optimized user interface (UI).

[1146] Details of the program's processing

[1147] Data collection

[1148] The device (such as smart glasses) tracks the driver's gaze position and head movements in real time. Sensors (such as eye-tracking sensors and motion sensors) are used to acquire this data. The acquired data is transmitted in real time to the vehicle's central server.

[1149] Data Analysis

[1150] The server stores the received data in a database and performs data cleansing as needed. Then, it uses a generative AI model (e.g., TensorFlow or PyTorch) to analyze the driver's behavior patterns. This analysis identifies which information the user frequently checks and which locations they focus their attention on.

[1151] UI generation

[1152] The generating AI model creates the optimal information layout for the driver based on analyzed behavioral patterns. Specifically, it places information that the driver frequently checks in the center of their field of vision, and other information in the periphery. A central server prepares this generated UI layout for transmission to the smart glasses in the vehicle.

[1153] UI provided

[1154] The device (such as smart glasses) displays information on its screen based on the received UI layout. This allows the driver to see optimized information within their field of vision.

[1155] Gathering feedback and continuous improvement

[1156] The device monitors the usage of the provided UI and collects feedback data. The collected feedback data is sent back to the server and stored in a database. The server analyzes this feedback data to retrain the generative AI model and further optimize the UI. This cyclical process ensures continuous improvement.

[1157] Specific example

[1158] Let's say driver A is driving on a highway and is using smart glasses. Driver A frequently checks speed information and road sign information. In this case, an optimized UI would place the speed information in the center of the driver's line of sight, with the road sign information displayed below it. Also, if the driver is using voice navigation, that information would also be displayed near their line of sight.

[1159] Examples of prompts for generative AI models

[1160] Analyze user behavior data to identify frequently referenced information and optimize its placement within the smart glasses' field of view. Specifically, if speed information and road sign information are frequently referenced, position them closer to the center of the user's line of sight.

[1161] This allows drivers to quickly and intuitively obtain necessary information, improving driving safety and comfort.

[1162] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1163] Step 1:

[1164] The device (smart glasses) tracks the driver's gaze position and head movements in real time. Specifically, it uses gaze tracking sensors and motion sensors to acquire data on where the driver is looking and in which direction their head is facing. This data is transmitted in real time to the vehicle's central server.

[1165] Step 2:

[1166] The server receives gaze position and head movement data transmitted from the terminal and stores it in a database. Simultaneously, it performs data cleansing to remove noise and errors and improve data accuracy. Here, the input is gaze and head movement data, and the output is the cleansed data. Specifically, data formatting is standardized and missing values ​​are imputed.

[1167] Step 3:

[1168] The server inputs the cleansed data into a generating AI model. The generating AI model analyzes this data to identify the driver's behavior patterns. Specifically, the analysis identifies information that the driver frequently checks and its location from the accumulated data. The input is cleansed data, and the output is data indicating the driver's behavior patterns.

[1169] Step 4:

[1170] The generative AI model generates the optimal information layout for the driver based on analyzed behavioral patterns. Specifically, it places information that the driver frequently checks (e.g., speed information and road sign information) in the center of the field of view, and other information in the periphery. The input is behavioral pattern data, and the output is an optimized UI layout.

[1171] Step 5:

[1172] The server sends the generated UI layout to the smart glasses in the vehicle. The smart glasses display the received UI layout on their screen. This allows the driver to see optimized information within their field of vision. The input is UI layout data, and the output is visual information on the smart glasses' display.

[1173] Step 6:

[1174] The device (smart glasses) monitors the usage of the provided UI and collects feedback data such as operation logs and click data. Specifically, it records what information the driver checks and how often. This data is then sent back to the server. The input is usage data, and the output is the collected feedback data.

[1175] Step 7:

[1176] The server stores the collected feedback data in a database and inputs it into a generative AI model. The generative AI model analyzes the feedback data and retrains and optimizes the UI. Specifically, it fine-tunes the placement based on the driver's reactions to the previous UI placement. The input is the feedback data, and the output is the improved UI layout.

[1177] In this way, drivers will always be able to obtain optimized information, which is expected to improve safety and comfort.

[1178] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1179] This invention combines an emotion engine with a system that generates, provides, and improves user interfaces (UIs) optimized based on user behavior patterns. This system collects and analyzes not only user behavior data but also emotion data to provide the most suitable UI for each individual user.

[1180] Program processing flow

[1181] 1. Data Collection

[1182] Terminal: This collects data on user actions and behaviors when using the system. For example, this includes actions such as users viewing a specific product page, using the search function, or clicking on product details. It also analyzes and records emotional data in real time from the user's facial expressions and voice using an emotion engine.

[1183] Terminal: Sends collected behavioral and emotional data to the server in real time.

[1184] 2. Data reception and storage

[1185] Server: Receives behavioral and sentiment data sent from terminals. Verifyes that the data is accurate and complete.

[1186] Server: Stores received data in a database and assigns labels to each category (e.g., page views, searches, clicks, sentiment).

[1187] 3. Data Cleansing

[1188] Server: Cleanses behavioral and sentiment data in the database, removing duplicate and incomplete data. This improves the accuracy of the analysis.

[1189] 4. Data Analysis

[1190] Server: Extracts behavioral and sentiment data related to specific users from the cleansed database.

[1191] Server: The extracted data is input into a generating AI model to analyze user behavior patterns and emotional patterns. For example, it identifies whether a user frequently views products in a particular category and what emotional state they are in while using the product.

[1192] 5. Generating an optimized UI

[1193] Generative AI Model: Based on analyzed behavioral and emotional patterns, it generates a UI layout optimized for that user. Specifically, this includes adjusting the position of navigation menus, customizing product recommendation sections, and making design changes that take emotional states into consideration.

[1194] Server: Prepares the generated UI layout information and necessary resources (images, stylesheets, scripts, etc.) and prepares for delivery.

[1195] 6. Providing a UI

[1196] Server: Sends the optimized UI to the user's device.

[1197] Terminal: Displays the received UI and provides the user with a new interface.

[1198] 7. Gathering feedback and continuous improvement

[1199] Device: Monitors the usage of the provided UI and collects click data, scroll data, and sentiment data.

[1200] Terminal: Sends collected feedback data to the server in real time.

[1201] Server: Stores feedback data sent from terminals in a database and prepares it for re-analysis.

[1202] Generative AI Model: Based on feedback data, the UI design model is retrained and continuously improved.

[1203] Specific example

[1204] Example 1: In the case of an e-commerce site

[1205] User B is using an e-commerce site. The default UI displays a general layout for all users. The system detects that User B has viewed products in the "Fashion" category multiple times, and that they frequently view pages related to "Shoes." The emotion engine also detects that User B has shown interest, such as smiling, while browsing.

[1206] Data Collection: User B's browsing data, including their behavior when viewing "shoes" related pages, and emotional data such as their smile during browsing, are sent from the device to the server.

[1207] Data Analysis: The server collects and stores this data, and the generating AI model analyzes user B's behavioral and emotional patterns. It is determined that user B is interested in "shoes" and that they view this interest favorably.

[1208] UI Generation: The generation AI model generates the optimal UI for user B. In this UI, a "Shoes" feature section is displayed at the top of the homepage, and products relevant to user B are prioritized. Additionally, if the user continues to respond positively, the emotion engine adjusts the colors and layout to create a positive impression.

[1209] UI delivery: The server sends this customized UI to user B's device, and the device displays the UI.

[1210] Feedback Collection: User B begins using the new UI, and operation logs, click data, and sentiment data are collected again. For example, this may include a scene where User B purchases shoes from the featured section.

[1211] Continuous improvement: Further analyze the collected feedback data and adjust the UI as needed.

[1212] In this manner, the system of the present invention operates, improving the quality of the user experience by providing an optimal UI for each individual user based on behavioral and emotional data.

[1213] The following describes the processing flow.

[1214] Step 1: Start data collection

[1215] Terminal: When a user uses the system, it monitors and starts recording their operations and behavioral data. This includes actions such as viewing a specific product page, using the search function, or clicking on product details. It also uses an emotion engine to analyze and record emotional data in real time from the user's facial expressions and voice.

[1216] Step 2: Data transmission

[1217] Terminal: Sends recorded behavioral and emotional data to the server in real time.

[1218] Step 3: Data reception and storage

[1219] Server: Receives behavioral and sentiment data sent from terminals. Verifyes that the data is accurate and complete.

[1220] Server: Stores received data in a database and assigns labels to each category (e.g., page views, searches, clicks, sentiment).

[1221] Step 4: Data Cleansing

[1222] Server: Cleanses behavioral and sentiment data in the database, removing duplicate and incomplete data. This improves the accuracy of the analysis.

[1223] Step 5: Data Analysis

[1224] Server: Extracts behavioral and sentiment data related to specific users from the cleansed database.

[1225] Server: The extracted data is input into a generating AI model to analyze user behavior patterns and emotional patterns. For example, it identifies whether a user frequently views products in a particular category and what emotional state they are in while using the product.

[1226] Step 6: Generate an optimized UI

[1227] Generative AI Model: Based on analyzed behavioral and emotional patterns, it generates a UI layout optimized for that user. Specifically, this includes adjusting the position of navigation menus, customizing product recommendation sections, and making design changes that take emotional states into consideration.

[1228] Server: Prepares the generated UI layout information and necessary resources (images, stylesheets, scripts, etc.) and prepares for delivery.

[1229] Step 7: Provide the UI

[1230] Server: Sends the optimized UI to the user's device.

[1231] Terminal: Displays the received UI and provides the user with a new interface.

[1232] Step 8: Gathering Feedback

[1233] Device: Monitors the usage of the provided UI and collects click data, scroll data, and sentiment data.

[1234] Terminal: Sends collected feedback data to the server in real time.

[1235] Step 9: Analysis of Feedback Data

[1236] Server: Stores feedback data sent from terminals in a database and prepares it for re-analysis.

[1237] Generative AI Model: Based on feedback data, the UI design model is retrained and continuously improved.

[1238] Step 10: Regenerate and serve the UI

[1239] Server: Uses the retrained model to generate a newly optimized UI and sends it to the user's device.

[1240] Terminal: Displays and provides the user with the regenerated new UI.

[1241] Through the steps described above, a dynamic and optimal UI is provided based on user behavior and emotional data. This improves the user experience and maximizes the efficiency of service utilization. For example, by detecting when a user frequently views a particular product and the emotions they show interest in it, it is possible to provide product recommendations based on browsing history and adjust the UI based on those emotions.

[1242] (Example 2)

[1243] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1244] Traditional user interfaces (UIs) have been optimized solely based on user behavior data, but this approach fails to take into account user emotional states and positive responses, resulting in insufficient optimization accuracy. Furthermore, feedback collection and analysis for continuously improving the effectiveness of a UI once it has been provided are often inadequate. The objective of this invention is to solve these problems and provide a more intuitive and effective user experience.

[1245] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1246] In this invention,

[1247] The server has means for collecting user behavior data and emotional data,

[1248] A means for analyzing the aforementioned behavioral data and emotional data to identify the user's behavioral patterns and emotional patterns,

[1249] A means for generating a user interface optimized for each user based on the aforementioned behavioral and emotional patterns,

[1250] Means for monitoring the usage of the provided user interface and collecting feedback data,

[1251] A means for analyzing user behavior patterns and emotional patterns using a generative AI model and generating the user interface,

[1252] This includes means for continuously improving the user interface using the collected feedback data.

[1253] This allows for the incorporation of emotional data in addition to behavioral data, and enables continuous improvement of the user interface based on collected feedback data. This, in turn, allows for the delivery of a more personalized, intuitive, and effective user experience.

[1254] "Behavioral data" refers to records of operations and actions performed by users when using a system.

[1255] "Emotional data" refers to information about a user's emotional state, analyzed from their facial expressions and voice.

[1256] "Behavioral patterns" refer to the consistent behavioral patterns of users, based on collected behavioral data.

[1257] An "emotional pattern" is a consistent emotional state of a user, captured based on collected emotional data.

[1258] "User interface (UI)" is a general term for the screens and input methods that users use to operate a system.

[1259] A "generative AI model" is an artificial intelligence model that analyzes user behavior patterns and emotional patterns to generate the optimal user interface.

[1260] "Feedback data" refers to data about the usage and effectiveness of the provided user interface, including click data and scroll data.

[1261] An "optimized user interface" is a user interface that is individually tailored based on the user's behavioral and emotional patterns.

[1262] "Monitoring" refers to the process of monitoring user interface usage in real time and collecting data.

[1263] This invention relates to a system that analyzes user behavior and emotional patterns to provide and improve an optimized user interface (UI). This system collects and analyzes user behavior and emotional data, and improves the quality of the user experience by providing the most suitable UI for each individual user based on the results.

[1264] The system consists of terminals, servers, and generative AI models.

[1265] Data collection

[1266] The device collects data on user actions and behaviors as they use the system. This data includes activities such as browsing web pages, using search functions, and clicking on product details. Furthermore, it uses an emotion engine to acquire real-time emotional data from the user's facial expressions and voice. The collected behavioral and emotional data is transmitted to the server in real time. Encryption technologies such as SSL / TLS are used for data transmission to ensure security.

[1267] Data reception and storage

[1268] The server receives behavioral and emotional data transmitted from the terminal. It performs CRC checks and data integrity checks to verify the accuracy and completeness of the received data. The received data is then stored in a database and labeled according to category (e.g., page views, searches, clicks, emotional state).

[1269] Data cleansing

[1270] The server validates behavioral and sentiment data in the database, removing incomplete and duplicate data. Data integrity tools and scripts are used for this purpose. After a clean dataset is generated, the data is formatted and normalized to improve analysis accuracy.

[1271] Data Analysis

[1272] The server extracts behavioral and emotional data related to a specific user from a cleansed database. This is done using SQL queries and data extraction tools. The extracted data is input into a generative AI model, where the user's behavioral and emotional patterns are analyzed. For example, it might be identified that the user frequently browses products in the "shoes" category and smiles while browsing. Deep learning and machine learning algorithms are used for this analysis.

[1273] Generating an optimized UI

[1274] The generative AI model generates the optimal UI layout for the user based on analyzed behavioral and emotional patterns. Specifically, this includes adjusting the position of navigation menus, customizing product recommendation sections, and making design changes that take emotional states into account. The server prepares the generated UI layout information and necessary resources (images, stylesheets, scripts, etc.) and gets ready to send the optimized UI.

[1275] UI provision

[1276] The server sends an optimized UI to the user's device. This is done using the HTTP protocol. The device displays the received UI in a browser or app, providing the user with a new interface. The display is updated in real time, and user actions are reflected smoothly.

[1277] Feedback gathering and continuous improvement

[1278] The device monitors the usage of the provided UI and collects click data, scroll data, and sentiment data. This data is sent to the server in real time as feedback data. As before, encryption technology is used for transmission to ensure security. The server stores the received feedback data in a database and prepares it for reanalysis. This feedback data is used to retrain the generative AI model, enabling continuous UI improvement.

[1279] Specific example

[1280] Example 1: In the case of an e-commerce site

[1281] User B is using an e-commerce site. The default UI displays a general layout for all users. The system detects that User B repeatedly browses products in the "Fashion" category, and frequently views the "Shoes" page. The emotion engine also detects that User B smiles while browsing. Based on this data, a generative AI model generates an optimal UI for User B, and a special "Shoes" section is displayed at the top of the homepage. Once User B starts using the new UI, their operation logs, click data, and emotion data are collected again, and the system uses this to continuously improve the UI.

[1282] Example of a prompt

[1283] "Analyze the user's behavioral and emotional patterns, and generate a UI that is optimal for that user's preferences. The user has recently been viewing many 'shoe' related products and has shown interest by smiling while doing so."

[1284] Thus, the system of the present invention improves the quality of the user experience by combining and analyzing behavioral data and emotional data, and by providing an optimal and personalized UI for each individual user.

[1285] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1286] Step 1: Data Collection

[1287] Input: User action data, user sentiment data

[1288] Operation and output:

[1289] The device collects behavioral data from users as they use the system. Specifically, this includes web page browsing history, search terms, and click history.

[1290] The device utilizes an emotion engine to collect emotional data in real time from the user's facial expressions and voice. For example, it can detect the moment the user smiles or the tone of their voice.

[1291] The device sends this data to the server in real time. Encryption technologies such as SSL / TLS are used for data transmission to ensure secure transfer of data to the server.

[1292] Step 2: Data reception and storage

[1293] Input: Behavioral data and emotional data sent from the device.

[1294] Operation and output:

[1295] The server receives behavioral and emotional data transmitted from the terminal. To maintain data integrity, CRC checks and data integrity checks are performed.

[1296] The server stores the received data in a database. Each piece of data in the database is assigned a category label (e.g., page views, searches, clicks, sentiment).

[1297] Step 3: Data Cleansing

[1298] Input: Stored behavioral data, emotional data

[1299] Operation and output:

[1300] The server validates behavioral and sentiment data in the database, removing incomplete and duplicate data. This is done using data integrity tools and scripts.

[1301] The server generates a clean dataset and performs data format conversion and normalization to improve analysis accuracy.

[1302] The output consists of cleansed behavioral and emotional data.

[1303] Step 4: Data Analysis

[1304] Input: Cleansed behavioral data, emotional data

[1305] Operation and output:

[1306] The server extracts behavioral and sentiment data related to a specific user from the cleansed database. This is done using SQL queries and data extraction tools.

[1307] The server inputs the extracted data into a generating AI model to analyze the user's behavioral and emotional patterns. For example, it might identify that the user frequently browses products in the "shoes" category and smiles while browsing.

[1308] The output consists of each user's behavioral and emotional patterns.

[1309] Step 5: Generate an optimized UI

[1310] Input: Behavioral patterns, emotional patterns

[1311] Operation and output:

[1312] The generative AI model generates the optimal UI layout for each user based on analyzed behavioral and emotional patterns. Specific examples include adjusting the position of navigation menus and customizing product recommendation sections.

[1313] The server prepares the generated UI layout information and necessary resources (images, stylesheets, scripts, etc.) and gets ready to send the optimized UI.

[1314] The output consists of a user-optimized UI layout design and the necessary resources.

[1315] Step 6: Provide the UI

[1316] Input: Optimized UI layout, resources

[1317] Operation and output:

[1318] The server sends an optimized UI to the user's device. The HTTP protocol is used for this purpose.

[1319] The device displays the received UI in a browser or app, providing the user with a new interface. The display is updated in real time, and user actions are reflected smoothly.

[1320] The output is an optimized UI provided to the user.

[1321] Step 7: Gathering Feedback and Continuous Improvement

[1322] Input: Usage data of the provided UI (click data, scroll data, sentiment data)

[1323] Operation and output:

[1324] The device monitors the usage of the provided UI and collects click data, scroll data, and sentiment data. Specific examples include the frequency of clicks and scroll depth on a particular product page.

[1325] The device sends the collected feedback data to the server in real time.

[1326] The server saves the received feedback data to a database and prepares it for reanalysis.

[1327] The generative AI model retrains its UI design model based on feedback data, enabling continuous improvement.

[1328] The output is an improved user interface for the next version.

[1329] (Application Example 2)

[1330] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1331] Conventional user interface (UI) optimization systems generated UIs based solely on user behavior data, failing to reflect user emotions or real-time states. Consequently, the quality of the user experience was not always high, and user-specific optimization was insufficient. This invention aims to improve the user experience by utilizing both user behavior data and emotional data to provide each user with a highly personalized UI.

[1332] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[1333] In this invention, the server includes means for collecting user behavior data and emotional data, means for analyzing the behavior data and emotional data to identify the user's behavior patterns and emotional patterns, and means for generating a user interface (UI) optimized for each user based on the behavior patterns and emotional patterns. This makes it possible to generate and provide an optimal UI from both the user's behavior and emotions, thereby providing a more personalized user experience suited to individual users.

[1334] "User behavior data" refers to data about a user's actions and behavior when using an online system or application, and specifically includes page viewing history, click count, search history, etc.

[1335] "Emotional data" refers to data related to emotions obtained from the user's facial expressions, voice, gestures, etc., such as data representing the user's reactions like smiling, anger, or surprise.

[1336] "Behavioral patterns" refer to a series of trends and characteristics of user behavior obtained by analyzing user behavior data, such as frequently viewing products in a specific category or accessing the site at a specific time of day.

[1337] An "emotional pattern" refers to a series of tendencies and characteristics of emotions obtained by analyzing emotional data, and describes patterns of how a user exhibits emotions in specific situations.

[1338] An "optimized user interface (UI)" is an interface customized based on user behavior and emotional patterns, resulting in an interface that is most user-friendly and appealing to each individual user, with a well-designed and structured interface.

[1339] "Feedback data" refers to data about the usage of the provided user interface, such as what actions the user performed and what emotions they expressed in which parts of the interface.

[1340] "Methods for continuous improvement" refer to a process of re-evaluating the user interface based on feedback data, repeatedly analyzing it, and optimizing it.

[1341] A "generative AI model" is a software model that includes machine learning and deep learning algorithms to generate the optimal user interface based on user behavior data and emotional data.

[1342] This invention is a system that analyzes user behavior data and emotional data to provide a user interface (UI) optimized for each individual user. The specific method for realizing this system is described below.

[1343] Hardware and software used

[1344] 1. Hardware

[1345] The hardware used will consist of terminals (smartphones, tablets, PCs, etc.) and servers for collecting user behavioral and emotional data. Ideally, the terminals should be equipped with a camera and microphone, which will allow for the collection of user facial and voice data.

[1346] 2. Software

[1347] The following software may be used:

[1348] OpenCV: A library for acquiring and processing camera images.

[1349] SpeechRecognition Library: A library for collecting and analyzing speech.

[1350] Requests library: A library for sending and receiving data using REST APIs.

[1351] Generative AI model: A software model that includes machine learning and deep learning algorithms to generate the optimal UI based on user behavior and emotional data.

[1352] Data collection and analysis

[1353] The device collects user behavior data (e.g., page viewing history, click count, search history, etc.) as the user interacts with it. It also uses the camera and microphone to capture the user's facial expressions and voice in real time, collecting emotional data. This data is transmitted from the device to the server in real time and stored in a database.

[1354] The server analyzes the received behavioral and emotional data to identify the user's behavioral and emotional patterns. Specifically, it analyzes whether the user frequently views products in certain categories and their emotional state while using the products, based on the behavioral data.

[1355] UI optimization and delivery

[1356] The generative AI model generates a user-optimized UI based on analyzed behavioral and emotional patterns. This UI specifically includes adjusting the position of navigation menus, customizing product recommendation sections, and making design changes that take emotional states into account.

[1357] The server sends the generated UI layout information and necessary resources (images, stylesheets, scripts, etc.) to the device, which then displays the new UI based on this data.

[1358] Feedback gathering and continuous improvement

[1359] The device monitors the usage of the provided UI, collecting click data, scroll data, and sentiment data in real time. The feedback data is sent back to the server, and the generative AI model uses this data to retrain the UI design model and make continuous improvements.

[1360] Examples of specific cases and prompt statements

[1361] For example, when a user is browsing "shoes" in the fashion category, if their facial expression is smiling, the emotion data "positive" is recorded. Based on this behavioral and emotion data, a generative AI model displays recommended products related to "shoes" on the home screen.

[1362] Example of a prompt:

[1363] When user A is browsing "shoes," if the user's facial expression is smiling, the emotion data "positive" is recorded and analyzed together with behavioral data. As a result, the user will be shown more recommended products related to "shoes."

[1364] Thus, the system of the present invention can integrate user behavior data and emotional data to provide a highly personalized UI.

[1365] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1366] Step 1:

[1367] Data collection

[1368] The device collects user behavior data (page viewing history, click count, search history, etc.) during user interaction. It also uses a camera and microphone to capture user facial expressions and voice in real time, collecting emotional data. Input here consists of user actions and visual / audio information, while output is behavioral data and emotional data. Specifically, camera images are acquired frame by frame using OpenCV, and audio data is collected using the SpeechRecognition library.

[1369] Step 2:

[1370] Data transmission

[1371] The device sends collected behavioral and emotional data to the server in real time. The input here is the collected behavioral and emotional data, and the output is the transmitted data. Specifically, the Requests library is used to send the behavioral and emotional data to the server.

[1372] Step 3:

[1373] Data reception and storage

[1374] The server receives behavioral and emotional data transmitted from the terminal and stores it in a database. The input here is the transmitted data, and the output is the data stored in the database. Specifically, after receiving the data, it is stored in the database in an appropriate format.

[1375] Step 4:

[1376] Data cleansing

[1377] The server cleanses behavioral and sentiment data in the database, removing duplicates and incomplete data. The input is the raw data stored in the database, and the output is the cleansed data. Specifically, it applies algorithms for data consistency checking and deduplication.

[1378] Step 5:

[1379] Data Analysis

[1380] The server extracts behavioral and emotional data related to a specific user from the cleansed data and inputs it into a generative AI model. The generative AI model analyzes the user's behavioral and emotional patterns. Here, the input is the cleansed data, and the output is the behavioral and emotional patterns. Specifically, it performs data analysis using a machine learning model.

[1381] Step 6:

[1382] UI optimization

[1383] The generative AI model generates a user-optimized UI based on analyzed behavioral and emotional patterns. The input here is behavioral and emotional patterns, and the output is an optimized UI layout. Specific actions include adjusting the position of navigation menus and customizing product recommendation sections.

[1384] Step 7:

[1385] UI provided

[1386] The server sends the generated UI layout information and necessary resources to the device. The device then displays the new UI based on this data. The input here is the UI layout information and resources sent from the server, and the output is the displayed new UI. Specifically, the UI layout information and resources are applied to the user's device.

[1387] Step 8:

[1388] Feedback Collection

[1389] The device monitors the usage of the provided UI and collects click data, scroll data, and sentiment data. The inputs here are user interaction data and visual / audio information, while the output is feedback data. Specifically, it prepares to send user interaction data to the server while continuously logging it.

[1390] Step 9:

[1391] Continuous improvement

[1392] The server stores feedback data sent from the terminal in a database, and the generated AI model retrains the UI design model based on this data, continuously improving it. Here, the input is feedback data, and the output is the updated UI design model. Specifically, it performs a retraining process using feedback data to continuously optimize the UI.

[1393] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1394] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1395] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[1396] [Fourth Embodiment]

[1397] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[1398] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1399] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1400] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[1401] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1402] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1403] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1404] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[1405] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1406] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1407] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1408] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1409] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1410] The present invention includes a program for collecting and analyzing user behavior patterns and generating, providing, and improving an optimized user interface (UI). The program's processing will be explained below in natural language, with specific examples.

[1411] Program processing flow

[1412] 1. Data Collection

[1413] Terminal: This collects data on user actions and behaviors when using the system. For example, this includes actions such as users viewing a specific product page, using the search function, or clicking on product details.

[1414] Terminal: Sends collected data to the server in real time.

[1415] 2. Data Analysis

[1416] Server: The server receives data and saves it to the database. When saving, it verifies that the data is accurate and performs data cleansing as needed.

[1417] Server: Extracts behavioral data related to specific users from the database and prepares it for analysis.

[1418] Generative AI Model: This model uses extracted behavioral data as input to analyze user behavior patterns. For example, it can identify when users access the site most frequently, which product categories they prefer, and which features they use most often.

[1419] 3. UI generation

[1420] Generative AI Model: Automatically generates an optimized UI based on user behavior patterns. Specifically, it determines the placement of navigation menus, button positions, and the display order of recommended content.

[1421] Server: Prepares the generated UI layout and necessary resources (images, stylesheets, scripts, etc.) and gets them ready to be delivered to the user.

[1422] 4. UI provided

[1423] Server: Sends an optimized UI to the user's device.

[1424] Terminal: Displays the received UI within a browser or application and provides it to the user.

[1425] 5. Gathering feedback and continuous improvement

[1426] Terminal: Monitors the usage of the provided UI and collects user reactions and operation logs.

[1427] Server: Stores collected feedback data in a database and uses it as new data for analysis.

[1428] Generative AI model: Based on feedback data, it retrains the UI and performs further optimization.

[1429] Specific example

[1430] Example 1: In the case of an e-commerce site

[1431] User A is using an e-commerce site. The default UI displays a general layout for all users. The system detects that User A has viewed products in the "Electronics" category multiple times, and that they frequently view pages related to "Smartphones."

[1432] Data collection: User A's browsing data, specifically the pages related to "smartphones," is sent from the device to the server.

[1433] Data Analysis: The server collects and stores this data, and a generating AI model analyzes user A's behavioral patterns. It is determined that user A is interested in "smartphones".

[1434] UI Generation: The generation AI model generates the optimal UI for user A. In this UI, a "Smartphone" feature section is displayed at the top of the homepage, and products relevant to user A are displayed preferentially.

[1435] UI delivery: The server sends this customized UI to user A's device, and the device displays the UI.

[1436] Feedback Collection: User A begins using the new UI, and operation logs and click data are collected again. For example, this may include a scene where User A purchases a smartphone from the featured section.

[1437] Continuous improvement: Further analyze the collected feedback data and adjust the UI as needed.

[1438] This provides user A with an intuitive interface that is easy to use. This is expected to improve the quality of the user experience and, as a result, increase site usage and purchase rates.

[1439] The system of this invention operates in this manner, improving user satisfaction by providing a user interface tailored to each individual user.

[1440] The following describes the processing flow.

[1441] Step 1: Start data collection

[1442] Terminal: When a user uses the system, it begins monitoring and recording their operations and behavioral data. This includes actions such as viewing a specific product page or using the search function.

[1443] Terminal: Sends recorded behavioral data to the server in real time.

[1444] Step 2: Data reception and storage

[1445] Server: Receives behavioral data sent from the terminal. Verifyes that the data is accurate and complete.

[1446] Server: Receives data and stores it in a database, assigning labels to each category (e.g., page views, searches, clicks).

[1447] Step 3: Data Cleansing

[1448] Server: Cleanses behavioral data in the database, removing duplicate and incomplete data. This improves the accuracy of the analysis.

[1449] Step 4: Data Analysis

[1450] Server: Extracts behavioral data related to a specific user from a cleansed database.

[1451] Server: Inputs the extracted data into a generating AI model to analyze user behavior patterns. For example, it identifies whether a user frequently views products in a particular category.

[1452] Step 5: Generate an optimized UI

[1453] Generative AI Model: Based on analyzed behavioral patterns, it generates a UI layout optimized for that user. Specifically, this includes adjusting the position of navigation menus and customizing product recommendation sections.

[1454] Server: Prepares the generated UI layout information and necessary resources (images, stylesheets, scripts, etc.) and prepares for delivery.

[1455] Step 6: Provide the UI

[1456] Server: Sends the optimized UI to the user's device.

[1457] Terminal: Displays the received UI and provides the user with a new interface.

[1458] Step 7: Gathering Feedback

[1459] Device: Monitors the usage of the provided UI and collects feedback data such as click data and scroll data.

[1460] Terminal: Sends collected feedback data to the server in real time.

[1461] Step 8: Analysis of Feedback Data

[1462] Server: Stores feedback data sent from terminals in a database and prepares it for re-analysis.

[1463] Generative AI Model: Based on feedback data, the UI design model is retrained and continuously improved.

[1464] Step 9: Regenerate and serve the UI

[1465] Server: Uses the retrained model to generate a newly optimized UI and sends it to the user's device.

[1466] Terminal: Displays and provides the user with the regenerated new UI.

[1467] Through these steps, a dynamic and optimized UI is provided that adapts to user behavior. This improves the user experience and maximizes the efficiency of service utilization.

[1468] (Example 1)

[1469] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1470] Traditional user interfaces (UIs) have the problem of providing the same layout and functionality to all users, meaning they are not optimized for individual user preferences and behavioral patterns. As a result, the quality of the user experience suffers, leading to challenges such as low site usage frequency and low purchase rates. Furthermore, even when user behavior data is collected, there is a lack of systems to properly analyze and utilize it, which often slows down improvements to the user interface.

[1471] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1472] In this invention, the server includes means for collecting and transmitting user behavior data in real time, means for storing and analyzing the behavior data in a database, means for analyzing user behavior patterns using a generative AI model, means for monitoring the usage of the provided user interface, means for collecting the usage data as feedback data and storing it in a database, and means for retraining the user interface using the feedback data and continuously optimizing it. This enables the provision of a UI optimized for the behavior patterns of individual users, improving the quality of the user experience and increasing site usage frequency and purchase rates.

[1473] "User behavior data" refers to data about the operations and actions that users perform when using a system.

[1474] A "server" is a computer system that stores, processes, and analyzes data received from users.

[1475] A "database" is an information aggregation system designed to efficiently store data and enable searching and manipulation of that data.

[1476] A "generative AI model" is an artificial intelligence model that has been trained to perform a specific task using machine learning algorithms.

[1477] "Analysis of behavioral patterns" is the process of analyzing collected behavioral data to understand the behavioral tendencies and preferences of specific users.

[1478] A "user interface (UI)" refers to the screens and operating methods used by a user, enabling interaction with the system.

[1479] "Real-time" means that data collection, transmission, and processing are performed instantly without delay.

[1480] "Feedback data" refers to the reactions and operation logs generated by users using the provided user interface.

[1481] "Retraining" is the process of retraining an existing model based on new data to improve its performance.

[1482] "Optimization" means adjusting a system or process to achieve maximum performance or effectiveness under specific conditions.

[1483] The present invention includes a program for collecting and analyzing user behavior data and generating, providing, and improving an optimized user interface (UI). Specific embodiments of this system will be described below.

[1484] System Configuration

[1485] 1. Hardware

[1486] The system uses the following hardware:

[1487] Device: A device used by a user to access information (e.g., personal computer, smartphone, tablet).

[1488] Server: A high-performance computer system used for collecting, storing, and analyzing data.

[1489] Database: An information aggregation system for efficiently storing, searching, and manipulating data (e.g., MySQL, PostgreSQL).

[1490] 2. Software

[1491] The system uses the following software:

[1492] Generative AI Model: An artificial intelligence model that uses machine learning algorithms to analyze user behavior patterns and generate the optimal UI.

[1493] Data cleansing tools: Tools used to verify the accuracy of collected data and to remove any missing or redundant parts.

[1494] Web server software: Software used to deliver appropriate resources in response to user requests (e.g., Apache, Nginx).

[1495] System Embodiment

[1496] 1. Data Collection

[1497] User: Users search for products and view specific product pages through the system.

[1498] Terminal: Records and collects user activity data in real time. For example, clicked links, entered search queries, pages viewed, etc.

[1499] Terminal: Sends collected data to the server in real time using HTTP requests.

[1500] 2. Data Analysis

[1501] Server: Receives data, saves it to a database, and performs analysis. During the saving process, it verifies the accuracy of the data and performs data cleansing.

[1502] Server: Extracts behavioral data related to a specific user from the database. For example, it filters relevant data using the user ID as the key.

[1503] Generative AI model: Inputs extracted data to analyze user behavior patterns and identify areas of interest and behavioral trends.

[1504] 3. UI generation

[1505] Generative AI Model: Generates the optimal UI based on user behavior patterns. Specifically, it determines the position of navigation menus and the display order of recommended products.

[1506] Server: Prepares the resources required for the generated UI layout (image files, CSS stylesheets, JavaScript scripts, etc.).

[1507] 4. UI provided

[1508] Server: Sends an optimized UI to the user's device. Sends HTML, CSS, and JavaScript together as an HTTP response.

[1509] Terminal: Displays the received UI within a browser or application and provides it to the user.

[1510] 5. Gathering feedback and continuous improvement

[1511] Terminal: Monitors the user's use of the provided UI and collects operation logs.

[1512] Device: Sends UI usage data to the server as feedback data.

[1513] Server: Saves feedback data to a database and uses it as new analysis data.

[1514] Generative AI Model: The model is retrained based on feedback data to further optimize the UI.

[1515] Specific example

[1516] Example 1: In the case of an e-commerce site

[1517] Let's assume that user A uses an e-commerce site and frequently browses pages in the "Smartphone" category.

[1518] Data collection: User A's operation data is collected in real time and sent to the server.

[1519] Data analysis: The server collects and stores data, and the generated AI model analyzes user A's behavioral patterns.

[1520] UI Generation: The generation AI model generates the optimal UI for user A and places a "Smartphone" special section at the top of the homepage.

[1521] UI delivery: The server sends the new UI to user A's device, and the browser displays that UI.

[1522] Feedback collection: User A's operation logs and click data are collected again and sent to the server.

[1523] Continuous improvement: Further optimize the UI based on collected feedback data.

[1524] Example of a prompt:

[1525] "If an e-commerce site user frequently browses a specific product category (e.g., smartphones), please explain the process for generating and providing an optimized UI specifically tailored to that user."

[1526] In this way, the system of the present invention improves the quality of the user experience by providing a UI optimized based on the behavioral patterns of individual users.

[1527] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1528] Step 1:

[1529] Collection of operational data:

[1530] When users use the system, they perform operations such as viewing product pages and using the search function.

[1531] The device records this operation data in real time. Specifically, it collects information such as clicked links, entered search queries, and viewed pages.

[1532] Input: User operation data

[1533] Output: Collected real-time operation data

[1534] Step 2:

[1535] Sending data:

[1536] The device sends the collected data to the server in real time using HTTP requests.

[1537] Input: Collected operational data

[1538] Output: Operation data sent to the server

[1539] Step 3:

[1540] Data storage and cleansing:

[1541] The server saves the received operation data to the database.

[1542] Before the server saves the data, it verifies its accuracy and cleanses any unnecessary data. For example, it removes duplicate data and corrects incomplete data.

[1543] Input: Sent operation data

[1544] Output: Saved operation data after cleansing

[1545] Step 4:

[1546] User data extraction:

[1547] The server extracts behavioral data related to a specific user from the database. For example, it filters the data using the user ID as the key.

[1548] Input: Operational data in the database

[1549] Output: Extracted behavioral data

[1550] Step 5:

[1551] Analysis of behavioral patterns:

[1552] The generative AI model receives extracted behavioral data as input and analyzes user behavior patterns. It uses machine learning algorithms to identify user areas of interest and behavioral trends.

[1553] Input: Extracted behavioral data

[1554] Output: Analyzed behavioral patterns

[1555] Step 6:

[1556] UI generation:

[1557] The generation AI model generates the optimal UI based on user behavior patterns. Specifically, it determines the position of navigation menus and the display order of recommended products.

[1558] Input: Analyzed behavioral patterns

[1559] Output: Generated UI layout

[1560] Step 7:

[1561] Resource preparation:

[1562] The server prepares the necessary resources (image files, CSS stylesheets, JavaScript scripts, etc.) for the generated UI layout.

[1563] Input: Generated UI layout

[1564] Output: Preparation of necessary resources

[1565] Step 8:

[1566] UI submission:

[1567] The server sends an optimized UI to the user's device. The HTTP response includes HTML, CSS, and JavaScript files.

[1568] Input: Optimized UI and resources

[1569] Output: UI sent to the user's device

[1570] Step 9:

[1571] UI display:

[1572] The UI received by the device is displayed within the browser or application. The browser engine parses the HTML and displays it on the screen.

[1573] Input: Submitted UI

[1574] Output: Displayed UI

[1575] Step 10:

[1576] Monitoring UI usage:

[1577] The device monitors how users are using the new UI, recording click logs, time spent on the device, and other metrics.

[1578] Input: User operation data

[1579] Output: Collected feedback data

[1580] Step 11:

[1581] Collecting and sending feedback data:

[1582] The device sends the collected feedback data to the server.

[1583] Input: Collected feedback data

[1584] Output: Feedback data sent to the server

[1585] Step 12:

[1586] Storage and analysis of feedback data:

[1587] The server saves the feedback data to the database.

[1588] The generative AI model retrains itself based on feedback data to further optimize the UI.

[1589] Input: Submitted feedback data

[1590] Output: Optimized UI model after retraining

[1591] In this way, the entire system can continuously provide an optimized UI based on the behavioral patterns of individual users.

[1592] (Application Example 1)

[1593] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1594] Conventional autonomous driving systems have a problem in that the information provided to the driver and passengers is uniform and not optimized to the individual user's preferences and behavioral patterns. Specifically, important information is not displayed in the optimal position within the driver's field of vision while driving, causing the driver's attention to be scattered when acquiring information, resulting in problems of reduced safety and comfort. To solve these problems, a system is needed that dynamically optimizes information display based on the driver's real-time behavioral data.

[1595] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1596] In this invention, the server includes means for collecting user behavior data, means for analyzing the behavior data to identify user behavior patterns, means for generating a user interface (UI) optimized for each user based on the behavior patterns, means for providing the generated user interface to the user, a display device for tracking the driver's gaze position and head direction to acquire data, and means for changing the optimal information placement on the display device in real time. As a result, necessary information based on the driver's behavior patterns is displayed in the optimal position in the field of view, improving safety and comfort.

[1597] "User behavior data" refers to data about the operations and actions that users perform when using a system.

[1598] "Behavioral patterns" refer to the tendencies and habits of a user's series of actions.

[1599] A "user interface (UI)" is an interface that includes screens and operability for interaction between the user and the system.

[1600] "Tracking" is a technology that uses sensors to track the user's gaze position, head movements, and other similar information.

[1601] A "display device" is a device that can track eye position and head direction, such as smart glasses or head-mounted displays.

[1602] "Information arrangement" refers to the position and order of information displayed within one's field of vision.

[1603] "Real-time" means that the system responds instantly to user actions.

[1604] This invention relates to a system that tracks the driver's gaze position and head direction and dynamically optimizes the display of information. The system includes a program for collecting and analyzing user behavior data and generating and providing an optimized user interface (UI).

[1605] Details of the program's processing

[1606] Data collection

[1607] The device (such as smart glasses) tracks the driver's gaze position and head movements in real time. Sensors (such as eye-tracking sensors and motion sensors) are used to acquire this data. The acquired data is transmitted in real time to the vehicle's central server.

[1608] Data Analysis

[1609] The server stores the received data in a database and performs data cleansing as needed. Then, it uses a generative AI model (e.g., TensorFlow or PyTorch) to analyze the driver's behavior patterns. This analysis identifies which information the user frequently checks and which locations they focus their attention on.

[1610] UI generation

[1611] The generating AI model creates the optimal information layout for the driver based on analyzed behavioral patterns. Specifically, it places information that the driver frequently checks in the center of their field of vision, and other information in the periphery. A central server prepares this generated UI layout for transmission to the smart glasses in the vehicle.

[1612] UI provided

[1613] The device (such as smart glasses) displays information on its screen based on the received UI layout. This allows the driver to see optimized information within their field of vision.

[1614] Gathering feedback and continuous improvement

[1615] The device monitors the usage of the provided UI and collects feedback data. The collected feedback data is sent back to the server and stored in a database. The server analyzes this feedback data to retrain the generative AI model and further optimize the UI. This cyclical process ensures continuous improvement.

[1616] Specific example

[1617] Let's say driver A is driving on a highway and is using smart glasses. Driver A frequently checks speed information and road sign information. In this case, an optimized UI would place the speed information in the center of the driver's line of sight, with the road sign information displayed below it. Also, if the driver is using voice navigation, that information would also be displayed near their line of sight.

[1618] Examples of prompts for generative AI models

[1619] Analyze user behavior data to identify frequently referenced information and optimize its placement within the smart glasses' field of view. Specifically, if speed information and road sign information are frequently referenced, position them closer to the center of the user's line of sight.

[1620] This allows drivers to quickly and intuitively obtain necessary information, improving driving safety and comfort.

[1621] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1622] Step 1:

[1623] The device (smart glasses) tracks the driver's gaze position and head movements in real time. Specifically, it uses gaze tracking sensors and motion sensors to acquire data on where the driver is looking and in which direction their head is facing. This data is transmitted in real time to the vehicle's central server.

[1624] Step 2:

[1625] The server receives gaze position and head movement data transmitted from the terminal and stores it in a database. Simultaneously, it performs data cleansing to remove noise and errors and improve data accuracy. Here, the input is gaze and head movement data, and the output is the cleansed data. Specifically, data formatting is standardized and missing values ​​are imputed.

[1626] Step 3:

[1627] The server inputs the cleansed data into a generating AI model. The generating AI model analyzes this data to identify the driver's behavior patterns. Specifically, the analysis identifies information that the driver frequently checks and its location from the accumulated data. The input is cleansed data, and the output is data indicating the driver's behavior patterns.

[1628] Step 4:

[1629] The generative AI model generates the optimal information layout for the driver based on analyzed behavioral patterns. Specifically, it places information that the driver frequently checks (e.g., speed information and road sign information) in the center of the field of view, and other information in the periphery. The input is behavioral pattern data, and the output is an optimized UI layout.

[1630] Step 5:

[1631] The server sends the generated UI layout to the smart glasses in the vehicle. The smart glasses display the received UI layout on their screen. This allows the driver to see optimized information within their field of vision. The input is UI layout data, and the output is visual information on the smart glasses' display.

[1632] Step 6:

[1633] The device (smart glasses) monitors the usage of the provided UI and collects feedback data such as operation logs and click data. Specifically, it records what information the driver checks and how often. This data is then sent back to the server. The input is usage data, and the output is the collected feedback data.

[1634] Step 7:

[1635] The server stores the collected feedback data in a database and inputs it into a generative AI model. The generative AI model analyzes the feedback data and retrains and optimizes the UI. Specifically, it fine-tunes the placement based on the driver's reactions to the previous UI placement. The input is the feedback data, and the output is the improved UI layout.

[1636] In this way, drivers will always be able to obtain optimized information, which is expected to improve safety and comfort.

[1637] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1638] This invention combines an emotion engine with a system that generates, provides, and improves user interfaces (UIs) optimized based on user behavior patterns. This system collects and analyzes not only user behavior data but also emotion data to provide the most suitable UI for each individual user.

[1639] Program processing flow

[1640] 1. Data Collection

[1641] Terminal: This collects data on user actions and behaviors when using the system. For example, this includes actions such as users viewing a specific product page, using the search function, or clicking on product details. It also analyzes and records emotional data in real time from the user's facial expressions and voice using an emotion engine.

[1642] Terminal: Sends collected behavioral and emotional data to the server in real time.

[1643] 2. Data reception and storage

[1644] Server: Receives behavioral and sentiment data sent from terminals. Verifyes that the data is accurate and complete.

[1645] Server: Stores received data in a database and assigns labels to each category (e.g., page views, searches, clicks, sentiment).

[1646] 3. Data Cleansing

[1647] Server: Cleanses behavioral and sentiment data in the database, removing duplicate and incomplete data. This improves the accuracy of the analysis.

[1648] 4. Data Analysis

[1649] Server: Extracts behavioral and sentiment data related to specific users from the cleansed database.

[1650] Server: The extracted data is input into a generating AI model to analyze user behavior patterns and emotional patterns. For example, it identifies whether a user frequently views products in a particular category and what emotional state they are in while using the product.

[1651] 5. Generating an optimized UI

[1652] Generative AI Model: Based on analyzed behavioral and emotional patterns, it generates a UI layout optimized for that user. Specifically, this includes adjusting the position of navigation menus, customizing product recommendation sections, and making design changes that take emotional states into consideration.

[1653] Server: Prepares the generated UI layout information and necessary resources (images, stylesheets, scripts, etc.) and prepares for delivery.

[1654] 6. Providing a UI

[1655] Server: Sends the optimized UI to the user's device.

[1656] Terminal: Displays the received UI and provides the user with a new interface.

[1657] 7. Gathering feedback and continuous improvement

[1658] Device: Monitors the usage of the provided UI and collects click data, scroll data, and sentiment data.

[1659] Terminal: Sends collected feedback data to the server in real time.

[1660] Server: Stores feedback data sent from terminals in a database and prepares it for re-analysis.

[1661] Generative AI Model: Based on feedback data, the UI design model is retrained and continuously improved.

[1662] Specific example

[1663] Example 1: In the case of an e-commerce site

[1664] User B is using an e-commerce site. The default UI displays a general layout for all users. The system detects that User B has viewed products in the "Fashion" category multiple times, and that they frequently view pages related to "Shoes." The emotion engine also detects that User B has shown interest, such as smiling, while browsing.

[1665] Data Collection: User B's browsing data, including their behavior when viewing "shoes" related pages, and emotional data such as their smile during browsing, are sent from the device to the server.

[1666] Data Analysis: The server collects and stores this data, and the generating AI model analyzes user B's behavioral and emotional patterns. It is determined that user B is interested in "shoes" and that they view this interest favorably.

[1667] UI Generation: The generation AI model generates the optimal UI for user B. In this UI, a "Shoes" feature section is displayed at the top of the homepage, and products relevant to user B are prioritized. Additionally, if the user continues to respond positively, the emotion engine adjusts the colors and layout to create a positive impression.

[1668] UI delivery: The server sends this customized UI to user B's device, and the device displays the UI.

[1669] Feedback Collection: User B begins using the new UI, and operation logs, click data, and sentiment data are collected again. For example, this may include a scene where User B purchases shoes from the featured section.

[1670] Continuous improvement: Further analyze the collected feedback data and adjust the UI as needed.

[1671] In this manner, the system of the present invention operates, improving the quality of the user experience by providing an optimal UI for each individual user based on behavioral and emotional data.

[1672] The following describes the processing flow.

[1673] Step 1: Start data collection

[1674] Terminal: When a user uses the system, it monitors and starts recording their operations and behavioral data. This includes actions such as viewing a specific product page, using the search function, or clicking on product details. It also uses an emotion engine to analyze and record emotional data in real time from the user's facial expressions and voice.

[1675] Step 2: Data transmission

[1676] Terminal: Sends recorded behavioral and emotional data to the server in real time.

[1677] Step 3: Data reception and storage

[1678] Server: Receives behavioral and sentiment data sent from terminals. Verifyes that the data is accurate and complete.

[1679] Server: Stores received data in a database and assigns labels to each category (e.g., page views, searches, clicks, sentiment).

[1680] Step 4: Data Cleansing

[1681] Server: Cleanses behavioral and sentiment data in the database, removing duplicate and incomplete data. This improves the accuracy of the analysis.

[1682] Step 5: Data Analysis

[1683] Server: Extracts behavioral and sentiment data related to specific users from the cleansed database.

[1684] Server: The extracted data is input into a generating AI model to analyze user behavior patterns and emotional patterns. For example, it identifies whether a user frequently views products in a particular category and what emotional state they are in while using the product.

[1685] Step 6: Generate an optimized UI

[1686] Generative AI Model: Based on analyzed behavioral and emotional patterns, it generates a UI layout optimized for that user. Specifically, this includes adjusting the position of navigation menus, customizing product recommendation sections, and making design changes that take emotional states into consideration.

[1687] Server: Prepares the generated UI layout information and necessary resources (images, stylesheets, scripts, etc.) and prepares for delivery.

[1688] Step 7: Provide the UI

[1689] Server: Sends the optimized UI to the user's device.

[1690] Terminal: Displays the received UI and provides the user with a new interface.

[1691] Step 8: Gathering Feedback

[1692] Device: Monitors the usage of the provided UI and collects click data, scroll data, and sentiment data.

[1693] Terminal: Sends collected feedback data to the server in real time.

[1694] Step 9: Analysis of Feedback Data

[1695] Server: Stores feedback data sent from terminals in a database and prepares it for re-analysis.

[1696] Generative AI Model: Based on feedback data, the UI design model is retrained and continuously improved.

[1697] Step 10: Regenerate and serve the UI

[1698] Server: Uses the retrained model to generate a newly optimized UI and sends it to the user's device.

[1699] Terminal: Displays and provides the user with the regenerated new UI.

[1700] Through the steps described above, a dynamic and optimal UI is provided based on user behavior and emotional data. This improves the user experience and maximizes the efficiency of service utilization. For example, by detecting when a user frequently views a particular product and the emotions they show interest in it, it is possible to provide product recommendations based on browsing history and adjust the UI based on those emotions.

[1701] (Example 2)

[1702] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1703] Traditional user interfaces (UIs) have been optimized solely based on user behavior data, but this approach fails to take into account user emotional states and positive responses, resulting in insufficient optimization accuracy. Furthermore, feedback collection and analysis for continuously improving the effectiveness of a UI once it has been provided are often inadequate. The objective of this invention is to solve these problems and provide a more intuitive and effective user experience.

[1704] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1705] In this invention,

[1706] The server has means for collecting user behavior data and emotional data,

[1707] A means for analyzing the aforementioned behavioral data and emotional data to identify the user's behavioral patterns and emotional patterns,

[1708] A means for generating a user interface optimized for each user based on the aforementioned behavioral and emotional patterns,

[1709] Means for monitoring the usage of the provided user interface and collecting feedback data,

[1710] A means for analyzing user behavior patterns and emotional patterns using a generative AI model and generating the user interface,

[1711] This includes means for continuously improving the user interface using the collected feedback data.

[1712] This allows for the incorporation of emotional data in addition to behavioral data, and enables continuous improvement of the user interface based on collected feedback data. This, in turn, allows for the delivery of a more personalized, intuitive, and effective user experience.

[1713] "Behavioral data" refers to records of operations and actions performed by users when using a system.

[1714] "Emotional data" refers to information about a user's emotional state, analyzed from their facial expressions and voice.

[1715] "Behavioral patterns" refer to the consistent behavioral patterns of users, based on collected behavioral data.

[1716] An "emotional pattern" is a consistent emotional state of a user, captured based on collected emotional data.

[1717] "User interface (UI)" is a general term for the screens and input methods that users use to operate a system.

[1718] A "generative AI model" is an artificial intelligence model that analyzes user behavior patterns and emotional patterns to generate the optimal user interface.

[1719] "Feedback data" refers to data about the usage and effectiveness of the provided user interface, including click data and scroll data.

[1720] An "optimized user interface" is a user interface that is individually tailored based on the user's behavioral and emotional patterns.

[1721] "Monitoring" refers to the process of monitoring user interface usage in real time and collecting data.

[1722] This invention relates to a system that analyzes user behavior and emotional patterns to provide and improve an optimized user interface (UI). This system collects and analyzes user behavior and emotional data, and improves the quality of the user experience by providing the most suitable UI for each individual user based on the results.

[1723] The system consists of terminals, servers, and generative AI models.

[1724] Data collection

[1725] The device collects data on user actions and behaviors as they use the system. This data includes activities such as browsing web pages, using search functions, and clicking on product details. Furthermore, it uses an emotion engine to acquire real-time emotional data from the user's facial expressions and voice. The collected behavioral and emotional data is transmitted to the server in real time. Encryption technologies such as SSL / TLS are used for data transmission to ensure security.

[1726] Data reception and storage

[1727] The server receives behavioral and emotional data transmitted from the terminal. It performs CRC checks and data integrity checks to verify the accuracy and completeness of the received data. The received data is then stored in a database and labeled according to category (e.g., page views, searches, clicks, emotional state).

[1728] Data cleansing

[1729] The server validates behavioral and sentiment data in the database, removing incomplete and duplicate data. Data integrity tools and scripts are used for this purpose. After a clean dataset is generated, the data is formatted and normalized to improve analysis accuracy.

[1730] Data Analysis

[1731] The server extracts behavioral and emotional data related to a specific user from a cleansed database. This is done using SQL queries and data extraction tools. The extracted data is input into a generative AI model, where the user's behavioral and emotional patterns are analyzed. For example, it might be identified that the user frequently browses products in the "shoes" category and smiles while browsing. Deep learning and machine learning algorithms are used for this analysis.

[1732] Generating an optimized UI

[1733] The generative AI model generates the optimal UI layout for the user based on analyzed behavioral and emotional patterns. Specifically, this includes adjusting the position of navigation menus, customizing product recommendation sections, and making design changes that take emotional states into account. The server prepares the generated UI layout information and necessary resources (images, stylesheets, scripts, etc.) and gets ready to send the optimized UI.

[1734] UI provision

[1735] The server sends an optimized UI to the user's device. This is done using the HTTP protocol. The device displays the received UI in a browser or app, providing the user with a new interface. The display is updated in real time, and user actions are reflected smoothly.

[1736] Feedback gathering and continuous improvement

[1737] The device monitors the usage of the provided UI and collects click data, scroll data, and sentiment data. This data is sent to the server in real time as feedback data. As before, encryption technology is used for transmission to ensure security. The server stores the received feedback data in a database and prepares it for reanalysis. This feedback data is used to retrain the generative AI model, enabling continuous UI improvement.

[1738] Specific example

[1739] Example 1: In the case of an e-commerce site

[1740] User B is using an e-commerce site. The default UI displays a general layout for all users. The system detects that User B repeatedly browses products in the "Fashion" category, and frequently views the "Shoes" page. The emotion engine also detects that User B smiles while browsing. Based on this data, a generative AI model generates an optimal UI for User B, and a special "Shoes" section is displayed at the top of the homepage. Once User B starts using the new UI, their operation logs, click data, and emotion data are collected again, and the system uses this to continuously improve the UI.

[1741] Example of a prompt

[1742] "Analyze the user's behavioral and emotional patterns, and generate a UI that is optimal for that user's preferences. The user has recently been viewing many 'shoe' related products and has shown interest by smiling while doing so."

[1743] Thus, the system of the present invention improves the quality of the user experience by combining and analyzing behavioral data and emotional data, and by providing an optimal and personalized UI for each individual user.

[1744] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1745] Step 1: Data Collection

[1746] Input: User action data, user sentiment data

[1747] Operation and output:

[1748] The device collects behavioral data from users as they use the system. Specifically, this includes web page browsing history, search terms, and click history.

[1749] The device utilizes an emotion engine to collect emotional data in real time from the user's facial expressions and voice. For example, it can detect the moment the user smiles or the tone of their voice.

[1750] The device sends this data to the server in real time. Encryption technologies such as SSL / TLS are used for data transmission to ensure secure transfer of data to the server.

[1751] Step 2: Data reception and storage

[1752] Input: Behavioral data and emotional data sent from the device.

[1753] Operation and output:

[1754] The server receives behavioral and emotional data transmitted from the terminal. To maintain data integrity, CRC checks and data integrity checks are performed.

[1755] The server stores the received data in a database. Each piece of data in the database is assigned a category label (e.g., page views, searches, clicks, sentiment).

[1756] Step 3: Data Cleansing

[1757] Input: Stored behavioral data, emotional data

[1758] Operation and output:

[1759] The server validates behavioral and sentiment data in the database, removing incomplete and duplicate data. This is done using data integrity tools and scripts.

[1760] The server generates a clean dataset and performs data format conversion and normalization to improve analysis accuracy.

[1761] The output consists of cleansed behavioral and emotional data.

[1762] Step 4: Data Analysis

[1763] Input: Cleansed behavioral data, emotional data

[1764] Operation and output:

[1765] The server extracts behavioral and sentiment data related to a specific user from the cleansed database. This is done using SQL queries and data extraction tools.

[1766] The server inputs the extracted data into a generating AI model to analyze the user's behavioral and emotional patterns. For example, it might identify that the user frequently browses products in the "shoes" category and smiles while browsing.

[1767] The output consists of each user's behavioral and emotional patterns.

[1768] Step 5: Generate an optimized UI

[1769] Input: Behavioral patterns, emotional patterns

[1770] Operation and output:

[1771] The generative AI model generates the optimal UI layout for each user based on analyzed behavioral and emotional patterns. Specific examples include adjusting the position of navigation menus and customizing product recommendation sections.

[1772] The server prepares the generated UI layout information and necessary resources (images, stylesheets, scripts, etc.) and gets ready to send the optimized UI.

[1773] The output consists of a user-optimized UI layout design and the necessary resources.

[1774] Step 6: Provide the UI

[1775] Input: Optimized UI layout, resources

[1776] Operation and output:

[1777] The server sends an optimized UI to the user's device. The HTTP protocol is used for this purpose.

[1778] The device displays the received UI in a browser or app, providing the user with a new interface. The display is updated in real time, and user actions are reflected smoothly.

[1779] The output is an optimized UI provided to the user.

[1780] Step 7: Gathering Feedback and Continuous Improvement

[1781] Input: Usage data of the provided UI (click data, scroll data, sentiment data)

[1782] Operation and output:

[1783] The device monitors the usage of the provided UI and collects click data, scroll data, and sentiment data. Specific examples include the frequency of clicks and scroll depth on a particular product page.

[1784] The device sends the collected feedback data to the server in real time.

[1785] The server saves the received feedback data to a database and prepares it for reanalysis.

[1786] The generative AI model retrains its UI design model based on feedback data, enabling continuous improvement.

[1787] The output is an improved user interface for the next version.

[1788] (Application Example 2)

[1789] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1790] Conventional user interface (UI) optimization systems generated UIs based solely on user behavior data, failing to reflect user emotions or real-time states. Consequently, the quality of the user experience was not always high, and user-specific optimization was insufficient. This invention aims to improve the user experience by utilizing both user behavior data and emotional data to provide each user with a highly personalized UI.

[1791] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[1792] In this invention, the server includes means for collecting user behavior data and emotional data, means for analyzing the behavior data and emotional data to identify the user's behavior patterns and emotional patterns, and means for generating a user interface (UI) optimized for each user based on the behavior patterns and emotional patterns. This makes it possible to generate and provide an optimal UI from both the user's behavior and emotions, thereby providing a more personalized user experience suited to individual users.

[1793] "User behavior data" refers to data about a user's actions and behavior when using an online system or application, and specifically includes page viewing history, click count, search history, etc.

[1794] "Emotional data" refers to data related to emotions obtained from the user's facial expressions, voice, gestures, etc., such as data representing the user's reactions like smiling, anger, or surprise.

[1795] "Behavioral patterns" refer to a series of trends and characteristics of user behavior obtained by analyzing user behavior data, such as frequently viewing products in a specific category or accessing the site at a specific time of day.

[1796] An "emotional pattern" refers to a series of tendencies and characteristics of emotions obtained by analyzing emotional data, and describes patterns of how a user exhibits emotions in specific situations.

[1797] An "optimized user interface (UI)" is an interface customized based on user behavior and emotional patterns, resulting in an interface that is most user-friendly and appealing to each individual user, with a well-designed and structured interface.

[1798] "Feedback data" refers to data about the usage of the provided user interface, such as what actions the user performed and what emotions they expressed in which parts of the interface.

[1799] "Methods for continuous improvement" refer to a process of re-evaluating the user interface based on feedback data, repeatedly analyzing it, and optimizing it.

[1800] A "generative AI model" is a software model that includes machine learning and deep learning algorithms to generate the optimal user interface based on user behavior data and emotional data.

[1801] This invention is a system that analyzes user behavior data and emotional data to provide a user interface (UI) optimized for each individual user. The specific method for realizing this system is described below.

[1802] Hardware and software used

[1803] 1. Hardware

[1804] The hardware used will consist of terminals (smartphones, tablets, PCs, etc.) and servers for collecting user behavioral and emotional data. Ideally, the terminals should be equipped with a camera and microphone, which will allow for the collection of user facial and voice data.

[1805] 2. Software

[1806] The following software may be used:

[1807] OpenCV: A library for acquiring and processing camera images.

[1808] SpeechRecognition Library: A library for collecting and analyzing speech.

[1809] Requests library: A library for sending and receiving data using REST APIs.

[1810] Generative AI model: A software model that includes machine learning and deep learning algorithms to generate the optimal UI based on user behavior and emotional data.

[1811] Data collection and analysis

[1812] The device collects user behavior data (e.g., page viewing history, click count, search history, etc.) as the user interacts with it. It also uses the camera and microphone to capture the user's facial expressions and voice in real time, collecting emotional data. This data is transmitted from the device to the server in real time and stored in a database.

[1813] The server analyzes the received behavioral and emotional data to identify the user's behavioral and emotional patterns. Specifically, it analyzes whether the user frequently views products in certain categories and their emotional state while using the products, based on the behavioral data.

[1814] UI optimization and delivery

[1815] The generative AI model generates a user-optimized UI based on analyzed behavioral and emotional patterns. This UI specifically includes adjusting the position of navigation menus, customizing product recommendation sections, and making design changes that take emotional states into account.

[1816] The server sends the generated UI layout information and necessary resources (images, stylesheets, scripts, etc.) to the device, which then displays the new UI based on this data.

[1817] Feedback gathering and continuous improvement

[1818] The device monitors the usage of the provided UI, collecting click data, scroll data, and sentiment data in real time. The feedback data is sent back to the server, and the generative AI model uses this data to retrain the UI design model and make continuous improvements.

[1819] Examples of specific cases and prompt statements

[1820] For example, when a user is browsing "shoes" in the fashion category, if their facial expression is smiling, the emotion data "positive" is recorded. Based on this behavioral and emotion data, a generative AI model displays recommended products related to "shoes" on the home screen.

[1821] Example of a prompt:

[1822] When user A is browsing "shoes," if the user's facial expression is smiling, the emotion data "positive" is recorded and analyzed together with behavioral data. As a result, the user will be shown more recommended products related to "shoes."

[1823] Thus, the system of the present invention can integrate user behavior data and emotional data to provide a highly personalized UI.

[1824] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1825] Step 1:

[1826] Data collection

[1827] The device collects user behavior data (page viewing history, click count, search history, etc.) during user interaction. It also uses a camera and microphone to capture user facial expressions and voice in real time, collecting emotional data. Input here consists of user actions and visual / audio information, while output is behavioral data and emotional data. Specifically, camera images are acquired frame by frame using OpenCV, and audio data is collected using the SpeechRecognition library.

[1828] Step 2:

[1829] Data transmission

[1830] The device sends collected behavioral and emotional data to the server in real time. The input here is the collected behavioral and emotional data, and the output is the transmitted data. Specifically, the Requests library is used to send the behavioral and emotional data to the server.

[1831] Step 3:

[1832] Data reception and storage

[1833] The server receives behavioral and emotional data transmitted from the terminal and stores it in a database. The input here is the transmitted data, and the output is the data stored in the database. Specifically, after receiving the data, it is stored in the database in an appropriate format.

[1834] Step 4:

[1835] Data cleansing

[1836] The server cleanses behavioral and sentiment data in the database, removing duplicates and incomplete data. The input is the raw data stored in the database, and the output is the cleansed data. Specifically, it applies algorithms for data consistency checking and deduplication.

[1837] Step 5:

[1838] Data Analysis

[1839] The server extracts behavioral and emotional data related to a specific user from the cleansed data and inputs it into a generative AI model. The generative AI model analyzes the user's behavioral and emotional patterns. Here, the input is the cleansed data, and the output is the behavioral and emotional patterns. Specifically, it performs data analysis using a machine learning model.

[1840] Step 6:

[1841] UI optimization

[1842] The generative AI model generates a user-optimized UI based on analyzed behavioral and emotional patterns. The input here is behavioral and emotional patterns, and the output is an optimized UI layout. Specific actions include adjusting the position of navigation menus and customizing product recommendation sections.

[1843] Step 7:

[1844] UI provided

[1845] The server sends the generated UI layout information and necessary resources to the device. The device then displays the new UI based on this data. The input here is the UI layout information and resources sent from the server, and the output is the displayed new UI. Specifically, the UI layout information and resources are applied to the user's device.

[1846] Step 8:

[1847] Feedback Collection

[1848] The device monitors the usage of the provided UI and collects click data, scroll data, and sentiment data. The inputs here are user interaction data and visual / audio information, while the output is feedback data. Specifically, it prepares to send user interaction data to the server while continuously logging it.

[1849] Step 9:

[1850] Continuous improvement

[1851] The server stores feedback data sent from the terminal in a database, and the generated AI model retrains the UI design model based on this data, continuously improving it. Here, the input is feedback data, and the output is the updated UI design model. Specifically, it performs a retraining process using feedback data to continuously optimize the UI.

[1852] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1853] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1854] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[1855] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1856] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[1857] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[1858] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[1859] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[1860] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[1861] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[1862] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[1863] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[1864] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[1865] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1866] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[1867] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[1868] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[1869] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[1870] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[1871] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[1872] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[1873] The following is further disclosed regarding the embodiments described above.

[1874] (Claim 1)

[1875] Means of collecting user behavior data,

[1876] A means for analyzing the aforementioned behavioral data to identify user behavior patterns,

[1877] A means for generating an optimized user interface (UI) for each user based on the aforementioned behavioral patterns,

[1878] A system including means for providing the generated user interface to a user.

[1879] (Claim 2)

[1880] The system according to claim 1, further comprising means for collecting the usage status of the provided user interface as feedback data.

[1881] (Claim 3)

[1882] The system according to claim 1, further comprising means for continuously improving the user interface using the aforementioned feedback data.

[1883]

[1884] "Example 1"

[1885] (Claim 1)

[1886] Means of collecting user behavior data,

[1887] Means for transmitting the aforementioned behavioral data to a server in real time,

[1888] Means for storing the aforementioned behavioral data in a database and analyzing it,

[1889] A means of analyzing user behavior patterns using generative AI models,

[1890] A means for generating an optimized user interface (UI) for each user based on the aforementioned behavioral patterns,

[1891] Means for providing the generated user interface to the user,

[1892] A means of monitoring the usage of the provided user interface,

[1893] A means for collecting the aforementioned usage data as feedback data and storing it in a database,

[1894] A means for retraining the user interface using the aforementioned feedback data and continuously optimizing it,

[1895] A system that includes this.

[1896] (Claim 2)

[1897] The system according to claim 1, further comprising means for preparing the operational resources (images, stylesheets, scripts, etc.) of the provided user interface on a server.

[1898] (Claim 3)

[1899] The system according to claim 1, further comprising means for identifying the user's areas of interest based on the analysis results of the generated AI model and dynamically changing the content of the user interface accordingly.

[1900] "Application Example 1"

[1901] (Claim 1)

[1902] Means of collecting user behavior data,

[1903] A means for analyzing the aforementioned behavioral data to identify user behavior patterns,

[1904] A means for generating an optimized user interface (UI) for each user based on the aforementioned behavioral patterns,

[1905] Means for providing the generated user interface to the user,

[1906] A display device for tracking the driver's gaze position and head direction to acquire data,

[1907] A means for changing the optimal information arrangement for the aforementioned display device in real time,

[1908] A system that includes this.

[1909] (Claim 2)

[1910] The system according to claim 1, further comprising means for collecting the usage of a provided user interface as feedback data.

[1911] (Claim 3)

[1912] The system according to claim 1, further comprising means for continuously improving the user interface using the aforementioned feedback data.

[1913] "Example 2 of combining an emotion engine"

[1914] (Claim 1)

[1915] Means for collecting user behavior data and emotional data,

[1916] A means for analyzing the aforementioned behavioral data and emotional data to identify the user's behavioral patterns and emotional patterns,

[1917] A means for generating a user interface optimized for each user based on the aforementioned behavioral and emotional patterns,

[1918] Means for providing the generated user interface to the user,

[1919] A system including means for monitoring the usage of the provided user interface and collecting feedback data.

[1920] (Claim 2)

[1921] The system according to claim 1, further comprising means for continuously improving the user interface using the collected feedback data.

[1922] (Claim 3)

[1923] The system according to claim 1, further comprising means for analyzing user behavior patterns and emotional patterns using a generative AI model to generate the user interface.

[1924] "Application example 2 when combining with an emotional engine"

[1925] (Claim 1)

[1926] Means for collecting user behavior data and emotional data,

[1927] A means for analyzing the aforementioned behavioral data and emotional data to identify the user's behavioral patterns and emotional patterns,

[1928] A means for generating an optimized user interface (UI) for each user based on the aforementioned behavioral and emotional patterns,

[1929] A system including means for providing the generated user interface to a user.

[1930] (Claim 2)

[1931] The system according to claim 1, further comprising means for collecting usage and sentiment data of the provided user interface as feedback data.

[1932] (Claim 3)

[1933] The system according to claim 1, further comprising means for continuously improving the user interface using the aforementioned feedback data. [Explanation of Symbols]

[1934] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. Means of collecting user behavior data, A means for analyzing the aforementioned behavioral data to identify user behavior patterns, A means for generating a user interface optimized for each user based on the aforementioned behavioral patterns, A system including means for providing the generated user interface to a user.

2. The system according to claim 1, further comprising means for collecting the usage status of the provided user interface as feedback data.

3. The system according to claim 1, further comprising means for continuously improving the user interface using the aforementioned feedback data.

Images