System

Abstract

A system is provided.SOLUTION: A system comprising: means for collecting behavioral data of a user; means for analyzing the collected behavioral data and identifying an interest of the user; means for obtaining relevant information and generating summarized content based on the identified interest of the user; and means for delivering and displaying the generated summarized content to a user terminal.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In today's information society, a huge amount of information is generated daily, making it difficult for users to effectively gather the information they need. Furthermore, obtaining information tailored to specific interests can require significant time and effort. This problem is particularly pronounced for users who primarily use mobile devices such as smartphones and tablets. To address these issues, the present invention aims to provide a system that efficiently collects and summarizes information tailored to a user's interests, allowing the user to quickly obtain the information they need. [Means for solving the problem]

[0005] In order to solve the above-mentioned problems, the present invention provides the following means: A system is provided that includes means for collecting user behavioral data, means for analyzing the collected behavioral data and identifying the user's interests, and means for acquiring related information and generating summary content based on the identified user interests. The system also includes means for delivering and displaying the generated summary content to a user terminal. By adding means for transmitting the behavioral data from the user terminal to a server and means for generating attractive thumbnails of the generated summary content and displaying them on the user terminal, a system is realized that allows users to efficiently obtain the information they need in a short amount of time.

[0006] "Behavioral data" is data generated when a user performs actions such as browsing the internet, using applications, and purchasing history.

[0007] "Analysis" is a data processing method for identifying user interests and concerns based on collected behavioral data.

[0008] "Interest identification" is the process of determining what information a user is interested in from analyzed behavioral data.

[0009] "Relevant information" is external information that is obtained based on the identified user's interests.

[0010] "Summary content" is information that is concisely summarized so that long-form related information can be understood in a short amount of time.

[0011] "Delivery" refers to the act of transferring the generated summary content to the user's terminal.

[0012] "Display" refers to the act of making the summarized content visible on the user terminal.

[0013] A "thumbnail" is a small visual element that provides a brief overview or table of contents of a piece of content.

[0014] A "server" is a central processing unit that receives, processes, and stores data sent from user terminals.

[0015] A "user device" is an electronic device such as a smartphone, tablet, or PC that a user actually uses. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0017] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0020] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0021] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0022] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0024] [First embodiment]

[0025] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0026] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0029] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0032] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0036] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0037] The present invention relates to a system and an operating method thereof for enabling a user to efficiently obtain personalized information. The basic components for implementing the present invention are a user terminal, a server, and data exchanged between them.

[0038] User terminal operation

[0039] User devices collect behavioral data when users use the Internet or applications. This behavioral data includes the websites visited, the URLs of pages viewed, the duration of visits, the applications used, and purchase history. This data is periodically sent to the server. Users can also register categories of interest and specific information.

[0040] Example: If a user frequently visits travel sites, their URLs are collected and sent to the server.

[0041] Server Operation

[0042] The server receives the behavioral data sent from the user's device and stores it in a database. The collected data is then analyzed. Machine learning algorithms and natural language processing techniques are used to identify the user's interests. Based on the results of this analysis, related information is collected from the Internet and summary content is generated.

[0043] Example: A server analyzes data from a user's repeated visits to a travel site and identifies that the user is interested in "travel." Based on this information, it gathers the latest travel guide articles and travel deals from the Internet and creates a compact summary.

[0044] Summary content generation and delivery

[0045] The generated summary content is also processed as thumbnails and delivered to the user terminal. These thumbnails contain excerpts of the summary content and visual elements designed to attract the user's attention. The user terminal receives these thumbnails and displays them to the user.

[0046] Example: A summary article of "Top 10 recommended tourist spots in Tokyo" generated by the server is displayed on the user's smartphone screen along with a thumbnail image. By clicking on this, the user can quickly view the summarized information.

[0047] User operations

[0048] Users can click on the thumbnails to view the summary content. This is a very intuitive operation, allowing users to access the information they need in a short amount of time, greatly improving the efficiency of information gathering. Users can also select new categories based on their own interests.

[0049] Example: When a user clicks on the thumbnail of "Top 10 Recommended Tourist Spots in Tokyo," they can immediately view summarized information about the tourist spots, which can be useful when planning their trip.

[0050] This system allows users to efficiently gather the information they need in a short amount of time, even in the midst of an information overload, greatly simplifying everyday information gathering.

[0051] The processing flow will be explained below.

[0052] Step 1:

[0053] Users generate behavioral data when they browse the web or use apps, including the URLs of websites visited, the duration of their visits, the applications used, and their purchase history.

[0054] Step 2:

[0055] The device automatically collects the generated behavioral data, and the collection process runs in the background and is designed to not interfere with user operations.

[0056] Step 3:

[0057] The device sends the collected behavioral data to a server at regular intervals or when a certain amount of data has been accumulated. The transmission is done using a secure communication protocol such as HTTPS.

[0058] Step 4:

[0059] The server stores the behavioral data received from the user device in a database, standardizing the data format and performing data cleansing.

[0060] Step 5:

[0061] The server launches a data analysis module to analyze the received behavioral data, using machine learning algorithms and natural language processing techniques to identify the user's interests.

[0062] Step 6:

[0063] Based on the analysis results, the server collects relevant information from the internet, using web scraping and APIs to retrieve information such as the latest news articles, blog posts, and product reviews.

[0064] Step 7:

[0065] The server generates summary content from the collected relevant information, using a text summarization algorithm to summarize lengthy information in a concise format.

[0066] Step 8:

[0067] The server generates a thumbnail for the summary content, which includes a text and visual image of a portion of the summary content.

[0068] Step 9:

[0069] The server delivers the generated summary content and thumbnails to the user terminal, adjusting the content delivery order to be optimal based on the user's interests.

[0070] Step 10:

[0071] The terminal displays the received summary content and thumbnails to the user. The user interface is intuitive, allowing users to easily find the information they are looking for.

[0072] Step 11:

[0073] Users can click on the displayed thumbnails to view the summary content, which allows users to quickly obtain related information.

[0074] The above steps realize a system that allows users to quickly and efficiently view summarized content based on behavioral data.

[0075] Example 1

[0076] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0077] Conventional information provision systems have difficulty quickly and appropriately providing information that meets users' diverse interests. Furthermore, due to the vast amount of information available, users may take a significant amount of time to obtain the information they need. Furthermore, there is a lack of technology that efficiently utilizes user behavior data to provide personalized content, creating a need for improved user experience.

[0078] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0079] In this invention, the server includes means for analyzing user behavior data and identifying user interests, means for collecting related information from the Internet based on the identified user interests and generating summary content, and means for converting the generated summary content into a visually appealing thumbnail format and delivering and displaying it on the user terminal, thereby enabling users to quickly and efficiently obtain information that matches their interests.

[0080] "User behavioral data" refers to information such as the URLs of websites visited when a user uses the Internet or applications, the URLs of pages viewed, the time of visit, the name and duration of use of the application used, and purchase history.

[0081] The "server" is a central computer system that receives and analyzes behavioral data sent from user terminals, collects related information, and generates summary content.

[0082] "Means for analysis" refers to the process and program for identifying user interests and concerns from collected behavioral data using machine learning algorithms and natural language processing techniques.

[0083] "Means of collecting relevant information" refers to processes such as web scraping techniques and API access to obtain information related to the user's interests from the Internet.

[0084] The "means for generating summary content" is the process of creating content based on collected related information to enable users to understand important information in a short amount of time.

[0085] "Thumbnail processing" is the process by which the generated summary content is converted into a visually appealing and usable format.

[0086] The "means for delivering and displaying" refers to a process for transmitting the generated summary content to a user terminal and displaying it on the screen of the terminal.

[0087] A "secure protocol" is a communication protocol for securely sending and receiving data, an example of which is HTTPS.

[0088] A "machine learning algorithm" is a program that finds patterns and rules from large amounts of data and makes predictions and classifications.

[0089] "Natural language processing technology" is a technology for analyzing and understanding human language.

[0090] The present invention relates to a system and an operating method for enabling users to efficiently obtain personalized information. To implement the system, the following components are required: a user terminal, a server, and data exchanged between them.

[0091] User terminal operation

[0092] User devices collect behavioral data when users use the Internet or applications. This behavioral data includes the websites visited, the URLs of pages viewed, the duration of visits, the applications used, and purchase history. This data is periodically sent to the server. Users can also register categories of interest and specific information.

[0093] Example: If a user frequently visits travel sites, their URLs are collected and sent to the server.

[0094] Server Operation

[0095] The server receives behavioral data sent from the user's device and stores it in a database. The collected data is then analyzed using machine learning algorithms and natural language processing techniques (e.g., Scikit-learn, NLTK) implemented in Python to identify the user's interests. Based on the results of this analysis, related information is collected from the Internet and summary content is generated using Python.

[0096] Example: A server analyzes data on a user's repeated visits to a travel site and identifies that the user is interested in "travel." Based on this information, it collects the latest travel guide articles and travel deals from the Internet and creates summary content using Python libraries (e.g., BeautifulSoup, Requests).

[0097] Summary content generation and delivery

[0098] The generated summary content is also processed as thumbnails and delivered to the user terminal. These thumbnails contain excerpts of the summary content and visual elements designed to attract the user's attention. The user terminal receives these thumbnails and displays them to the user.

[0099] Example: A summary article of "Top 10 recommended tourist spots in Tokyo" generated by the server is displayed on the user's smartphone screen along with a thumbnail image. By clicking on this, the user can quickly view the summarized information.

[0100] User operations

[0101] Users can click on the thumbnails to view the summary content. This is a very intuitive operation, allowing users to access the information they need in a short amount of time, greatly improving the efficiency of information gathering. Users can also select new categories based on their own interests.

[0102] Example: When a user clicks on the thumbnail of "Top 10 Recommended Tourist Spots in Tokyo," they can immediately view summarized information about the tourist spots, which can be useful when planning their trip.

[0103] Prompt Sentence Examples

[0104] Here are some example prompts to input to a generative AI model:

[0105] "We've determined that the user is interested in travel. How can we collect the latest travel guide articles and travel deals and provide them as summary content?"

[0106] This system allows users to efficiently gather the information they need in a short amount of time, even in the midst of an information overload, greatly simplifying everyday information gathering.

[0107] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0108] Step 1:

[0109] The user device collects user behavior data, including the URLs of visited websites, URLs of viewed pages, visit times, names of applications used, and purchase history. This data is recorded in local storage in real time.

[0110] Input: User internet and application behavior

[0111] Output: Behavioral data recorded in local storage

[0112] Specific operation: When a user visits an online shopping site, the URL of the site, the URL of the product page viewed, and the time of the visit are recorded on the device.

[0113] Step 2:

[0114] The collected behavioral data is periodically sent to a server using the secure HTTPS protocol. The data is converted to JSON format before transmission.

[0115] Input: Behavioral data recorded in local storage

[0116] Output: JSON formatted behavior data sent to the server

[0117] Specific operation: The device sends the collected data to the server's API endpoint using HTTPS at regular intervals (e.g., every hour). The data is in JSON format and includes a "URL," "start time," and "end time."

[0118] Step 3:

[0119] The server receives the behavioral data sent from the user terminal and stores it in a database, which is an SQL-based database (e.g., PostgreSQL).

[0120] Input: JSON formatted behavior data sent from the device

[0121] Output: Behavioral data stored in a database (PostgreSQL) table

[0122] Specific operation: The server's API endpoint parses the received JSON data and stores the data in the "UserActions" table in the PostgreSQL database using an INSERT query.

[0123] Step 4:

[0124] The server analyzes the behavioral data stored in the database using machine learning algorithms and natural language processing techniques (e.g., Scikit-learn, NLTK) implemented in Python.

[0125] Input: Behavioral data stored in a database

[0126] Output: Analysis of user interests and concerns

[0127] How it works: The server analyzes behavioral data collected over the past month to identify categories that users repeatedly visit. It then uses Scikit-learn to perform clustering analysis and identify user interests.

[0128] Step 5:

[0129] The server collects relevant information from the Internet based on the analysis results. Web scraping technology (e.g., BeautifulSoup, Scrapy) is used to collect information. The collected information is summarized and summary content is generated using Python libraries (e.g., Gensim, Sumy).

[0130] Input: Parsed user interests

[0131] Output: Summarized content

[0132] What it does: The server collects the latest articles about the specified "travel" from the Internet, parses them with BeautifulSoup, and then creates summary content using the Gensim library.

[0133] Step 6:

[0134] The generated summary content is processed into thumbnail format and delivered to the user's terminal. The thumbnail contains an excerpt of the summary content and visual elements.

[0135] Input: Summarized content

[0136] Output: Summary content in thumbnail format

[0137] Specific operation: The server creates thumbnails by combining a summary article of "Top 10 tourist spots in Tokyo" with related images, and sends them to the user's device.

[0138] Step 7:

[0139] The user terminal displays the received thumbnail to the user, who can then click on the thumbnail to view the detailed summary content.

[0140] Input: Summary content in thumbnail format

[0141] Output: Thumbnail and detailed summary content displayed to the user

[0142] What it does: When a user clicks on the thumbnail of "Top 10 tourist spots in Tokyo," a browser opens and displays a detailed page of the tourist spot with a summary.

[0143] This system's series of processes allows users to efficiently collect information based on their interests and obtain the information they need in a short amount of time.

[0144] (Application example 1)

[0145] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0146] In today's world where Internet usage is widespread, users can access a large amount of information, but it is extremely difficult to efficiently collect the information they need from that vast amount of information. Furthermore, in the case of advertising delivery, advertisements that do not match the user's interests are often displayed, causing user stress and reducing the effectiveness of advertising. The present invention aims to solve these problems by providing a system that effectively utilizes user behavior data to display personalized advertisements.

[0147] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0148] In this invention, the server includes means for collecting user behavioral data, means for analyzing the collected behavioral data and identifying the user's interests, means for acquiring related information and generating summary content based on the identified user's interests, means for processing the generated summary content as thumbnails and delivering them to the user terminal, means for allowing the user to view detailed information by clicking on the thumbnails, and means for allowing the user to select a new category and update their interests. This provides personalized information and advertisements based on the user's behavioral data, allowing users to efficiently access the information they need. It also enables advertisers to deliver advertisements more effectively, which is expected to improve advertising effectiveness.

[0149] "User behavioral data" is information generated when a user uses the Internet or applications, and specifically includes data such as websites visited, URLs of pages viewed, time of visit, applications used, and purchase history.

[0150] "Collection means" refers to software and hardware for collecting user behavior data in real time or periodically.

[0151] "Means of analysis" refers to the algorithms and analytical tools used to analyze collected behavioral data to identify user interests and concerns.

[0152] "Means for acquiring related information based on the identified user's interests" refers to a function for collecting related information from the Internet, etc., based on the user's interests and concerns.

[0153] "Means for generating summary content" refers to a function that compactly summarizes acquired related information and converts it into a format that is easy for the user to understand.

[0154] The "means for processing the summary content as a thumbnail and delivering it to a user terminal" refers to a method for displaying the generated summary content in a visually appealing format and transmitting it to a user terminal.

[0155] "User terminal" refers to a device that is directly operated by a user, such as a smartphone, tablet, or PC.

[0156] "Means for viewing detailed information" refers to the ability to display the full content and related detailed information when a user clicks on a thumbnail.

[0157] "Means for selecting new categories and updating interests" refers to an interface that allows users to select new information categories based on their own interests and have them reflected in the system.

[0158] The "means for delivering in the form of push notification" refers to a function for automatically notifying a user terminal of a thumbnail of the generated summary content.

[0159] The present invention relates to a system that generates personalized summary content by collecting and analyzing user behavior data and efficiently delivers the content to users. This system basically consists of a user terminal, a server, and data exchanged between them.

[0160] User terminal operation

[0161] User devices collect behavioral data when users use the Internet or applications. This behavioral data includes the URLs of websites visited, pages viewed, visit times, applications used, purchase history, etc. This data is periodically sent to a server where it is analyzed.

[0162] Server Operation

[0163] The server receives behavioral data sent from the user's device and stores it in a database. It then analyzes the collected data to identify the user's interests. The analysis method uses machine learning algorithms using Python's Scikit-learn library, as well as natural language processing techniques using Python's NLTK library and generative AI models such as GPT-3 (registered trademark). Based on this, relevant information is collected from the Internet and summary content is generated.

[0164] Summary content generation and delivery

[0165] The generated summary content is then processed into a thumbnail before being delivered to the user's device. This thumbnail includes an excerpt of the summary content and visual elements designed to pique the user's interest. The thumbnail is then delivered to the user's device, such as a smartphone, tablet, or PC, via push notification.

[0166] User operations

[0167] Users can view the summary content by clicking on the displayed thumbnail. This operation allows users to access the information they need quickly and efficiently. Furthermore, users can select new categories based on their interests and have the system reflect that information.

[0168] Specific examples

[0169] For example, if a user frequently visits a travel site, this behavioral data is collected and sent to a server. The server analyzes this data and identifies the user as being interested in "travel." The server then collects the latest travel guide articles and travel deals from the Internet and generates summarized content. This summarized content is then processed into thumbnails and delivered to the user's smartphone as a push notification. When the user clicks on this notification, they can immediately view the summarized information.

[0170] Hardware and software used

[0171] User devices: smartphones (iOS or ANDROID (registered trademark)), tablets, PCs, etc.

[0172] Server: High-performance cloud server (e.g., AWS (registered trademark) EC2)

[0173] Machine learning algorithms: Python's Scikit-learn library

[0174] Natural language processing technology: Python's NLTK library, generative AI models such as GPT-3

[0175] Database: MySQL or a NoSQL database (e.g., MongoDB)

[0176] Examples of prompt statements

[0177] "Generate ads for frequent travel site visitors whose behavioral data includes visits to multiple travel sites. Create a summary ad with a visual element suitable for a thumbnail."

[0178] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0179] Step 1: Collect user behavior data

[0180] The user device collects user behavior data. This behavior data includes the URLs of visited websites, pages viewed, visit times, applications used, purchase history, etc. These data are temporarily stored in local files. The input data is the user behavior data, and the output data is the collected behavior data.

[0181] Step 2: Sending behavioral data to the server

[0182] The terminal periodically transmits the collected behavioral data to the server. This process uses a scheduling algorithm to transmit data at specified time intervals. The input data is the collected behavioral data, and the output data is the behavioral data transmitted to the server.

[0183] Step 3: Analyzing behavioral data

[0184] The server receives the behavioral data and stores it in a database. Next, it uses machine learning algorithms and natural language processing techniques to identify the user's interests. This analysis is performed using Python's Scikit-learn and NLTK libraries. The input data is the behavioral data sent to the server, and the output data is the analysis results.

[0185] Step 4: Obtaining relevant information and generating summary content

[0186] The server collects relevant information from the Internet based on the identified user interests. The retrieved information is summarized using a generative AI model (e.g., GPT-3). Specific prompts are used to generate the summary. The input data are the analysis results and the collected relevant information, and the output data is the summary content.

[0187] Step 5: Generate and edit thumbnails

[0188] The server processes the generated summary content into an attractive thumbnail, which contains an excerpt and visual elements of the summary content. The input data is the summary content, and the output data is the processed thumbnail.

[0189] Step 6: Serving and displaying thumbnails

[0190] The server delivers the processed thumbnail to the user's device in the form of a push notification. The user receives this notification on their device and the thumbnail is displayed. The input data is the processed thumbnail, and the output data is the thumbnail displayed on the user's device.

[0191] Step 7: User interaction

[0192] The user can click on the displayed thumbnails to view the summary content, and can also update their interests by selecting new categories. The input data are the user's clicks and selections, and the output data are the viewed summary content and updated interest data.

[0193] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0194] The present invention relates to a system and its operation method for efficiently obtaining personalized information by utilizing behavioral data and emotional states of users. The basic components for implementing the present invention include a user terminal, a server, and an emotion engine.

[0195] User terminal operation

[0196] User devices collect behavioral data when users use the internet and applications. This behavioral data includes the URLs of websites visited, the duration of visits, the applications used, and purchase history. Biometric sensors (e.g., heart rate, skin temperature, and facial recognition) can also be used to capture user emotions in real time.

[0197] Example: While a user is busy visiting a travel site, an increase in heart rate is detected, which may indicate an emotional state such as excitement or anticipation.

[0198] Emotion Engine Operation

[0199] The emotion engine analyzes behavioral data and biometric sensor data sent from the user's device to identify the user's emotional state. The engine uses machine learning algorithms to determine the user's emotional state and transmits that information to the server.

[0200] Example: The emotion engine combines and analyzes the user's heart rate data, skin temperature, and facial recognition results to determine whether the user is in a state of "expectation."

[0201] Server Operation

[0202] The server receives behavioral and emotional data sent from the user device and the emotion engine. It stores these data in a database and activates a data analysis module to identify the user's interests and emotions. Based on the analysis results, it collects relevant information from the Internet and generates summary content.

[0203] Example: A server identifies travel interests from user behavioral data and detects signs of "expectation" from emotional data. Based on this information, it collects the latest travel guides and discount plans and summarizes them as content with high expectations.

[0204] Summary content generation and delivery

[0205] The server generates summary content from the collected related information, summarizes long-form information in a concise format using a text summarization algorithm, and creates thumbnails of the summary content that match the user's emotional state and delivers them to the user's device.

[0206] Example: The server generates a summary article for "Top 10 recommended tourist spots in Tokyo" and creates thumbnails that express the user's "expectation." For example, thumbnails are created that combine photos of attractive tourist spots with phrases that evoke a sense of anticipation.

[0207] User operations

[0208] Users can view the summary content by clicking on the thumbnails displayed on their devices. This operation is intuitive, and users can quickly obtain the information they need. In addition, information provided is adapted to the user's emotional state, providing a more satisfying user experience.

[0209] Example: When a user clicks on the thumbnail for "Top 10 Recommended Tourist Spots in Tokyo," they can instantly see a summary of the information and use it to help with their planning.

[0210] By combining and analyzing behavioral data and emotional data, this system makes it possible to quickly provide users with the information that is most relevant and matches their emotions, improving the user experience and significantly increasing the efficiency of information gathering.

[0211] The processing flow will be explained below.

[0212] Step 1:

[0213] When users browse the web or use applications, behavioral data is generated, including the URLs of websites visited, the duration of visits, applications used, and purchase history.

[0214] Step 2:

[0215] The device automatically collects user behavior data. This collection process runs in the background and is designed to not interfere with user operations. Data is also collected from biometric sensors (heart rate, skin temperature, facial recognition, etc.) installed on the device.

[0216] Step 3:

[0217] The device sends the collected behavioral data and biometric sensor data to a server at regular intervals or when a certain amount of data has been accumulated. The transmission is carried out using a secure communication protocol such as HTTPS.

[0218] Step 4:

[0219] The server stores the behavioral data and biometric sensor data received from the user device in a database, standardizing the data format and performing data cleansing.

[0220] Step 5:

[0221] The server launches a data analysis module to analyze the received data, using machine learning algorithms and natural language processing techniques to identify the user's interests and emotional state.

[0222] Step 6:

[0223] The emotion engine analyzes the behavioral data and biometric sensor data to identify the user's emotional state (e.g., joy, anticipation, surprise, etc.), and the analysis results are also sent to the server.

[0224] Step 7:

[0225] Based on the analysis results, the server collects relevant information from the internet, using web scraping and APIs to retrieve information such as the latest news articles, blog posts, and product reviews.

[0226] Step 8:

[0227] The server generates summary content from the collected relevant information, uses a text summarization algorithm to summarize long-form information in a concise format, and creates thumbnails of the summary content that are adaptive to the user's emotional state.

[0228] Step 9:

[0229] The server delivers the generated summary content and thumbnails to the user terminal, adjusting the content delivery order to be optimal based on the user's interests and emotions.

[0230] Step 10:

[0231] The terminal displays the received summary content and thumbnails to the user. The user interface is intuitive, allowing users to easily find the information they are looking for.

[0232] Step 11:

[0233] Users can click on the displayed thumbnails to view the summary content, allowing them to quickly obtain relevant information. Furthermore, the displayed information is adapted to the user's emotional state, resulting in a more satisfying information experience.

[0234] The above steps realize a system that allows users to quickly and efficiently view summarized content based on behavioral data and emotion data.

[0235] Example 2

[0236] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0237] Conventional information provision systems provide relevant information based solely on user behavioral data, often without considering the user's emotional state. As a result, personalized information provision that adapts to the user's emotions is lacking, and an improved user experience is desired. In addition, since the amount of information tends to be large, a means of efficiently summarizing and providing information is also required.

[0238] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0239] In this invention, the server includes means for collecting user behavioral data and biometric sensor data, means for analyzing the collected behavioral data and biometric sensor data to identify the user's emotional state, means for acquiring related information and generating summary content based on the identified user's interests and emotional state, and means for creating thumbnails of the generated summary content that match the user's emotional state and delivering them to the user terminal. This makes it possible to provide information that is adapted to the user's emotional state, thereby improving the user experience and significantly increasing the efficiency of information collection.

[0240] "User behavioral data" refers to data generated when a user uses the Internet or applications, such as the URLs of websites visited, the time of visit, applications used, and purchase history.

[0241] "Biometric sensor data" refers to data used to acquire a user's biometric information in real time, specifically including heart rate, skin temperature, and facial expression recognition results.

[0242] "Emotion Engine" refers to the algorithms and systems that analyze collected behavioral and biometric sensor data to identify a user's emotional state.

[0243] "Summary content" refers to information in which related information is reconstructed into a concise format using a summarization algorithm, allowing users to obtain information efficiently.

[0244] "Thumbnail" refers to a visually appealing image and short descriptive text related to the summarized content that matches the user's emotional state.

[0245] A "user terminal" is a device that a user uses to receive and view information, and specifically refers to a smartphone, tablet, PC, etc.

[0246] The present invention relates to a system and its operating method for efficiently providing personalized information by utilizing user behavioral data and biometric sensor data. The main components include a user terminal, a server, and an emotion engine. Each component and its operation will be described in detail below.

[0247] User terminal operation

[0248] User devices collect behavioral data when users use the internet and applications. This behavioral data includes the URLs of websites visited, the duration of visits, applications used, and purchase history. Biosensors (e.g., heart rate, skin temperature, facial expression recognition) can also be used to collect real-time emotional data.

[0249] Example: A user visits a travel site and while viewing a page about a particular tourist destination, the device's biometric sensors detect an increase in heart rate, which indicates emotions such as excitement and anticipation.

[0250] Emotion Engine Operation

[0251] The emotion engine analyzes behavioral data and biometric sensor data sent from the user device to identify the user's emotional state. The engine uses machine learning algorithms (e.g., TENSORFLOW®, PyTorch) to determine the user's emotional state. The identified emotional state information is then sent to the server.

[0252] Example: An emotion engine combines and analyzes heart rate data, skin temperature, and facial expression recognition results to determine when a user is in a state of "expectation."

[0253] Server Operation

[0254] The server receives behavioral and emotional data sent from the user device and the emotion engine. The received data is stored in a database (e.g., MySQL, PostgreSQL). The data analysis module then analyzes the data to identify the user's interests and emotional state. Based on the analysis results, relevant information is collected from the Internet and summary content is generated.

[0255] Example: The server stores the received data in a database, and the data analysis module identifies the user's travel interest from their recent web browsing history and detects the emotion "anticipation." Based on this information, the latest travel guides and discount plan information are collected and reconstructed as summary content.

[0256] Summary content and thumbnail generation

[0257] The server generates summary content based on the collected relevant information. It uses a text summarization algorithm (e.g., BERT summarizer) to condense long-form information into a concise format. It also creates thumbnails of the generated summary content that match the user's emotional state. It uses an image generation algorithm or image editing software (e.g., Photoshop API) to create the thumbnails.

[0258] Example: A server converts a detailed article about "Top 10 tourist spots in Tokyo" into a concise summary and generates an attractive thumbnail to create excitement.

[0259] Summary content and thumbnail delivery

[0260] The server delivers the generated summary content and thumbnails to the user device using push notifications or real-time communication protocols (e.g., WebSocket).

[0261] Example: The server delivers summary content and thumbnails to the user's smartphone app via push notification.

[0262] User operations

[0263] Users can view the summary content by clicking on the thumbnails displayed on their devices. This intuitive operation allows users to quickly obtain the information they need.

[0264] Example: A user clicks on a thumbnail of "Top 10 recommended tourist spots in Tokyo" to view summarized information and use it to help plan their trip.

[0265] Example prompt: "The emotion engine has determined that the user's heart rate increases when viewing a particular web page, indicating that the user is excited."

[0266] As a result, this system combines and analyzes the user's behavioral data and emotional data, and provides information adapted to the user's emotional state, thereby improving the user experience and making information gathering more efficient.

[0267] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0268] Step 1:

[0269] The user device collects user behavioral data and biometric sensor data, including the URLs of visited websites, the time of visit, the applications used, and purchase history. It also collects biometric sensor data (heart rate, skin temperature, and facial expression recognition) in real time.

[0270] Input: The actions and physiological responses of the body as users interact with the internet and applications.

[0271] Output: URL list of visited sites, visit times, app usage history, biometric sensor data.

[0272] Example of how it works: A user visits a travel website and while browsing a page about a particular tourist destination, an increase in heart rate is detected. This data is collected from the device.

[0273] Step 2:

[0274] The user device encrypts the collected behavioral data and biometric sensor data and sends it to the server using HTTPS as the communication protocol.

[0275] Input: Behavioral and biosensor data collected in the previous step.

[0276] Output: Encrypted behavioral and biometric sensor data.

[0277] Specific operation example: The list of URLs visited by the device and heart rate change data are encrypted in JSON format and sent to the server using the HTTPS protocol.

[0278] Step 3:

[0279] The emotion engine analyzes the behavioral data and biometric sensor data provided by the server to identify the user's emotional state. It analyzes the data using machine learning algorithms (e.g., TensorFlow, PyTorch).

[0280] Input: Decrypted behavioral and biometric sensor data.

[0281] Output: Identification of the user's emotional state (e.g., anticipation, excitement, etc.).

[0282] Specific example of operation: The emotion engine combines and analyzes heart rate data, skin temperature, and facial expression recognition results to determine that the user is in an "expectant" state, and sends the result to the server.

[0283] Step 4:

[0284] The server stores the data sent from the user terminal and the emotion engine in a database and activates a data analysis module, which analyzes the data and identifies the user's interests and emotional state.

[0285] Input: Stored behavioral and emotional data.

[0286] Output: Analysis results identifying the user's interests and emotional state.

[0287] Specific example of operation: The server identifies travel interest from user behavioral data and detects signs of "expectation" from emotional data.

[0288] Step 5:

[0289] The server collects relevant information from the Internet based on the identified user's interests and emotional state, using web crawlers and external APIs (e.g., travel information APIs) to obtain the latest information.

[0290] Input: Analysis results identifying the user's interests and emotional state.

[0291] Output: Related information (e.g. travel guide, discount offers, etc.).

[0292] Example of how it works: The server queries the travel information API to get the latest travel guides and discount plan information.

[0293] Step 6:

[0294] The server generates summary content from the collected information, using a text summarization algorithm (e.g., BERT summarizer) to summarize the information in a concise format.

[0295] Input: Related information (e.g. travel guide, discount offers, etc.).

[0296] Output: Summary content.

[0297] Example of how it works: The server converts a detailed article about "Top 10 tourist spots in Tokyo" into a concise summary.

[0298] Step 7:

[0299] The server creates thumbnails that match the emotional state of the generated summary content using image generation algorithms and image editing software (e.g., Photoshop API).

[0300] Input: Summary content, and the user's emotional state.

[0301] Output: Thumbnails adapted to the emotional state.

[0302] Specific example of how it works: The server generates thumbnails that combine photos of attractive tourist spots with phrases that evoke excitement.

[0303] Step 8:

[0304] The server delivers the generated summary content and thumbnails to the user device using push notifications or real-time communication protocols (e.g., WebSocket).

[0305] Input: Abstract content and thumbnail.

[0306] Output: Delivered summary content and thumbnails.

[0307] Specific example of operation: The server delivers summary content and thumbnails to the user's smartphone app via push notification.

[0308] Step 9:

[0309] Users can view the summary content by clicking on the thumbnails displayed on their devices. This intuitive operation allows users to quickly obtain the information they need.

[0310] Input: The thumbnail displayed on the device.

[0311] Output: User viewing summary content.

[0312] Example of specific operation: A user clicks on the thumbnail of "Top 10 recommended tourist spots in Tokyo" and views the summarized information to help plan their trip.

[0313] (Application example 2)

[0314] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0315] Conventional systems provide information based solely on user behavior data, which means that they do not adequately consider the user's emotional state, making it difficult to maximize user satisfaction. Furthermore, personalized advertising based on emotions is not available, making it difficult to maximize advertising effectiveness.

[0316] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0317] In this invention, the server includes means for collecting user behavioral data and biometric data, means for analyzing the collected behavioral data and biometric data to identify the user's emotional state and interests, means for acquiring related information and generating summary content based on the identified user's emotional state and interests, and means for delivering and displaying personalized advertisements based on the generated summary content and emotional state to the user terminal, thereby enabling the provision of information and personalized advertisements optimal for the user's emotional state.

[0318] "User behavior data" is information that records a user's activities on the Internet, such as website browsing history, application usage history, and purchase history.

[0319] "Biometric data" is information that represents a user's biological state or reaction, such as the user's heart rate, skin temperature, or facial recognition results.

[0320] "Emotional state" is information that indicates the user's current psychological state, which is analyzed from the user's biometric data.

[0321] "Relevant information" is information that is determined to be useful or interesting to the user based on the user's behavioral data and emotional state.

[0322] "Summary content" is information that succinctly summarizes related information and is generated so that users can understand it in a short amount of time.

[0323] "Personalized advertising" refers to advertising that is optimized based on a user's behavioral data and emotional state, and is determined to be highly relevant to each individual user.

[0324] A "user terminal" is a communication device such as a smartphone or tablet used by a user, and is a device that collects and displays information.

[0325] The present invention relates to a system for collecting and analyzing user behavioral data and biometric data, and delivering personalized advertisements based on the user's emotional state and interests. Specific embodiments for implementing the present invention will be described below.

[0326] Hardware and software used

[0327] The system of the present invention mainly comprises a user terminal, a server, and an emotion engine.

[0328] User device: A communication device connected to the Internet, such as a smartphone or tablet, that collects user behavioral data and biometric data and sends it to a server.

[0329] Server: Equipped with a database, data analysis module, and ad distribution module, it uses a Python (registered trademark)-based web framework such as Flask or Django.

[0330] Emotion Engine: Uses TensorFlow and PyTorch to implement machine learning models.

[0331] System Operation

[0332] 1. Data Collection:

[0333] The user device collects behavioral data such as the user's website browsing history, app usage history, and purchase history, as well as biometric data such as heart rate, skin temperature, and facial recognition results in real time. For example, it records the user's history of using a shopping app and the timing of when their heart rate increased.

[0334] 2. Data transmission:

[0335] The collected data is sent to the server via secure communication using HTTPS, and the data is encrypted.

[0336] 3. Emotion analysis:

[0337] The server inputs the received data into an emotion engine to analyze the user's emotional state, for example, determining whether the user is relaxed or stressed based on a combination of heart rate and skin temperature.

[0338] 4. Advertisement Selection:

[0339] Based on the emotional state analyzed by the emotion engine, the server searches for relevant information from the database and selects the most suitable personalized advertisement for the user, for example, if the user is relaxed, it will select a refreshing travel advertisement.

[0340] 5. Advertisement Delivery:

[0341] The selected advertisement is then pushed to the user's device and displayed there. At this time, a thumbnail is also generated based on the user's emotional state to make it easier for the user to intuitively understand the advertisement content.

[0342] Specific examples

[0343] While User A is browsing a shopping app on his smartphone, the server detects that his heart rate is higher than normal. The server determines this as stress and displays product advertisements on the smartphone that will help relieve stress.

[0344] Prompt Sentence Examples

[0345] "Collect user behavioral and emotional data and display ads that match your relaxed mood based on your emotional state."

[0346] This system enables optimal information to be provided based on the user's behavior and emotions, enhancing the user experience.

[0347] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0348] Step 1:

[0349] The user device collects user behavioral data and biometric data. Specifically, it records in real time the URLs of websites visited by the user, the time of visit, the applications used, purchase history, heart rate, skin temperature, and facial recognition results. The input is the user's behavior and signals obtained from biometric sensors, and the output is a set of collected data.

[0350] Step 2:

[0351] The user device sends the collected behavioral data and biometric data to the server via secure communication. The communication is encrypted using the HTTPS protocol. The input is the set of collected data, and the output is the data sent to the server.

[0352] Step 3:

[0353] The server inputs the received behavioral data and biometric data into the emotion engine. The emotion engine uses a machine learning model (TensorFlow or PyTorch) to analyze the data and identify the user's emotional state. The analysis process identifies "relaxed" or "stressed" based on heart rate, skin temperature, and facial recognition results. The input is the integrated data sent to the server, and the output is the analyzed user's emotional state.

[0354] Step 4:

[0355] The server searches for relevant information from a database based on the user's emotional state and behavioral data identified by the emotion engine, generates summary content, and selects personalized advertisements that match the user's specific emotional state. The input is the emotional state and behavioral data, and the output is the summary content and personalized advertisements.

[0356] Step 5:

[0357] The server generates attractive thumbnails for the generated summary content and advertisements based on the user's emotional state, providing a display format that is intuitively understandable to users. The input is the summary content and personalized advertisements, and the output is the summary content with thumbnails and advertisements.

[0358] Step 6:

[0359] The server transmits the summary content with thumbnails and the personalized advertisement to the user terminal by push notification, where the input is the summary content with thumbnails and the advertisement, and the output is the notification sent to the user terminal.

[0360] Step 7:

[0361] The user terminal displays the received summary content with thumbnails and advertisements. The user can view the summary content or advertisement details by clicking on the displayed thumbnail. The input is the pushed content and advertisement, and the output is the visual information displayed to the user.

[0362] This system enables optimal information provision and advertisement delivery based on user behavior and emotions, improving the user experience.

[0363] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0364] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0365] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0366] [Second embodiment]

[0367] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0368] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0369] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0370] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0371] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0372] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0373] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0374] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0375] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0376] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0377] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0378] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0379] The present invention relates to a system and an operating method thereof for enabling a user to efficiently obtain personalized information. The basic components for implementing the present invention are a user terminal, a server, and data exchanged between them.

[0380] User terminal operation

[0381] User devices collect behavioral data when users use the Internet or applications. This behavioral data includes the websites visited, the URLs of pages viewed, the duration of visits, the applications used, and purchase history. This data is periodically sent to the server. Users can also register categories of interest and specific information.

[0382] Example: If a user frequently visits travel sites, their URLs are collected and sent to the server.

[0383] Server Operation

[0384] The server receives the behavioral data sent from the user's device and stores it in a database. The collected data is then analyzed. Machine learning algorithms and natural language processing techniques are used to identify the user's interests. Based on the results of this analysis, related information is collected from the Internet and summary content is generated.

[0385] Example: A server analyzes data from a user's repeated visits to a travel site and identifies that the user is interested in "travel." Based on this information, it gathers the latest travel guide articles and travel deals from the Internet and creates a compact summary.

[0386] Summary content generation and delivery

[0387] The generated summary content is also processed as thumbnails and delivered to the user terminal. These thumbnails contain excerpts of the summary content and visual elements designed to attract the user's attention. The user terminal receives these thumbnails and displays them to the user.

[0388] Example: A summary article of "Top 10 recommended tourist spots in Tokyo" generated by the server is displayed on the user's smartphone screen along with a thumbnail image. By clicking on this, the user can quickly view the summarized information.

[0389] User operations

[0390] Users can click on the thumbnails to view the summary content. This is a very intuitive operation, allowing users to access the information they need in a short amount of time, greatly improving the efficiency of information gathering. Users can also select new categories based on their own interests.

[0391] Example: When a user clicks on the thumbnail of "Top 10 Recommended Tourist Spots in Tokyo," they can immediately view summarized information about the tourist spots, which can be useful when planning their trip.

[0392] This system allows users to efficiently gather the information they need in a short amount of time, even in the midst of an information overload, greatly simplifying everyday information gathering.

[0393] The processing flow will be explained below.

[0394] Step 1:

[0395] Users generate behavioral data when they browse the web or use apps, including the URLs of websites visited, the duration of their visits, the applications used, and their purchase history.

[0396] Step 2:

[0397] The device automatically collects the generated behavioral data, and the collection process runs in the background and is designed to not interfere with user operations.

[0398] Step 3:

[0399] The device sends the collected behavioral data to a server at regular intervals or when a certain amount of data has been accumulated. The transmission is done using a secure communication protocol such as HTTPS.

[0400] Step 4:

[0401] The server stores the behavioral data received from the user device in a database, standardizing the data format and performing data cleansing.

[0402] Step 5:

[0403] The server launches a data analysis module to analyze the received behavioral data, using machine learning algorithms and natural language processing techniques to identify the user's interests.

[0404] Step 6:

[0405] Based on the analysis results, the server collects relevant information from the internet, using web scraping and APIs to retrieve information such as the latest news articles, blog posts, and product reviews.

[0406] Step 7:

[0407] The server generates summary content from the collected relevant information, using a text summarization algorithm to summarize lengthy information in a concise format.

[0408] Step 8:

[0409] The server generates a thumbnail for the summary content, which includes a text and visual image of a portion of the summary content.

[0410] Step 9:

[0411] The server delivers the generated summary content and thumbnails to the user terminal, adjusting the content delivery order to be optimal based on the user's interests.

[0412] Step 10:

[0413] The terminal displays the received summary content and thumbnails to the user. The user interface is intuitive, allowing users to easily find the information they are looking for.

[0414] Step 11:

[0415] Users can click on the displayed thumbnails to view the summary content, which allows users to quickly obtain related information.

[0416] The above steps realize a system that allows users to quickly and efficiently view summarized content based on behavioral data.

[0417] Example 1

[0418] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0419] Conventional information provision systems have difficulty quickly and appropriately providing information that meets users' diverse interests. Furthermore, due to the vast amount of information available, users may take a significant amount of time to obtain the information they need. Furthermore, there is a lack of technology that efficiently utilizes user behavior data to provide personalized content, creating a need for improved user experience.

[0420] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0421] In this invention, the server includes means for analyzing user behavior data and identifying user interests, means for collecting related information from the Internet based on the identified user interests and generating summary content, and means for converting the generated summary content into a visually appealing thumbnail format and delivering and displaying it on the user terminal, thereby enabling users to quickly and efficiently obtain information that matches their interests.

[0422] "User behavioral data" refers to information such as the URLs of websites visited when a user uses the Internet or applications, the URLs of pages viewed, the time of visit, the name and duration of use of the application used, and purchase history.

[0423] The "server" is a central computer system that receives and analyzes behavioral data sent from user terminals, collects related information, and generates summary content.

[0424] "Means for analysis" refers to the process and program for identifying user interests and concerns from collected behavioral data using machine learning algorithms and natural language processing techniques.

[0425] "Means of collecting relevant information" refers to processes such as web scraping techniques and API access to obtain information related to the user's interests from the Internet.

[0426] The "means for generating summary content" is the process of creating content based on collected related information to enable users to understand important information in a short amount of time.

[0427] "Thumbnail processing" is the process by which the generated summary content is converted into a visually appealing and usable format.

[0428] The "means for delivering and displaying" refers to a process for transmitting the generated summary content to a user terminal and displaying it on the screen of the terminal.

[0429] A "secure protocol" is a communication protocol for securely sending and receiving data, an example of which is HTTPS.

[0430] A "machine learning algorithm" is a program that finds patterns and rules from large amounts of data and makes predictions and classifications.

[0431] "Natural language processing technology" is a technology for analyzing and understanding human language.

[0432] The present invention relates to a system and an operating method for enabling users to efficiently obtain personalized information. To implement the system, the following components are required: a user terminal, a server, and data exchanged between them.

[0433] User terminal operation

[0434] User devices collect behavioral data when users use the Internet or applications. This behavioral data includes the websites visited, the URLs of pages viewed, the duration of visits, the applications used, and purchase history. This data is periodically sent to the server. Users can also register categories of interest and specific information.

[0435] Example: If a user frequently visits travel sites, their URLs are collected and sent to the server.

[0436] Server Operation

[0437] The server receives behavioral data sent from the user's device and stores it in a database. The collected data is then analyzed using machine learning algorithms and natural language processing techniques (e.g., Scikit-learn, NLTK) implemented in Python to identify the user's interests. Based on the results of this analysis, related information is collected from the Internet and summary content is generated using Python.

[0438] Example: A server analyzes data on a user's repeated visits to a travel site and identifies that the user is interested in "travel." Based on this information, it collects the latest travel guide articles and travel deals from the Internet and creates summary content using Python libraries (e.g., BeautifulSoup, Requests).

[0439] Summary content generation and delivery

[0440] The generated summary content is also processed as thumbnails and delivered to the user terminal. These thumbnails contain excerpts of the summary content and visual elements designed to attract the user's attention. The user terminal receives these thumbnails and displays them to the user.

[0441] Example: A summary article of "Top 10 recommended tourist spots in Tokyo" generated by the server is displayed on the user's smartphone screen along with a thumbnail image. By clicking on this, the user can quickly view the summarized information.

[0442] User operations

[0443] Users can click on the thumbnails to view the summary content. This is a very intuitive operation, allowing users to access the information they need in a short amount of time, greatly improving the efficiency of information gathering. Users can also select new categories based on their own interests.

[0444] Example: When a user clicks on the thumbnail of "Top 10 Recommended Tourist Spots in Tokyo," they can immediately view summarized information about the tourist spots, which can be useful when planning their trip.

[0445] Prompt Sentence Examples

[0446] Here are some example prompts to input to a generative AI model:

[0447] "We've determined that the user is interested in travel. How can we collect the latest travel guide articles and travel deals and provide them as summary content?"

[0448] This system allows users to efficiently gather the information they need in a short amount of time, even in the midst of an information overload, greatly simplifying everyday information gathering.

[0449] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0450] Step 1:

[0451] The user device collects user behavior data, including the URLs of visited websites, URLs of viewed pages, visit times, names of applications used, and purchase history. This data is recorded in local storage in real time.

[0452] Input: User internet and application behavior

[0453] Output: Behavioral data recorded in local storage

[0454] Specific operation: When a user visits an online shopping site, the URL of the site, the URL of the product page viewed, and the time of the visit are recorded on the device.

[0455] Step 2:

[0456] The collected behavioral data is periodically sent to a server using the secure HTTPS protocol. The data is converted to JSON format before transmission.

[0457] Input: Behavioral data recorded in local storage

[0458] Output: JSON formatted behavior data sent to the server

[0459] Specific operation: The device sends the collected data to the server's API endpoint using HTTPS at regular intervals (e.g., every hour). The data is in JSON format and includes a "URL," "start time," and "end time."

[0460] Step 3:

[0461] The server receives the behavioral data sent from the user terminal and stores it in a database, which is an SQL-based database (e.g., PostgreSQL).

[0462] Input: JSON formatted behavior data sent from the device

[0463] Output: Behavioral data stored in a database (PostgreSQL) table

[0464] Specific operation: The server's API endpoint parses the received JSON data and stores the data in the "UserActions" table in the PostgreSQL database using an INSERT query.

[0465] Step 4:

[0466] The server analyzes the behavioral data stored in the database using machine learning algorithms and natural language processing techniques (e.g., Scikit-learn, NLTK) implemented in Python.

[0467] Input: Behavioral data stored in a database

[0468] Output: Analysis of user interests and concerns

[0469] How it works: The server analyzes behavioral data collected over the past month to identify categories that users repeatedly visit. It then uses Scikit-learn to perform clustering analysis and identify user interests.

[0470] Step 5:

[0471] The server collects relevant information from the Internet based on the analysis results. Web scraping technology (e.g., BeautifulSoup, Scrapy) is used to collect information. The collected information is summarized and summary content is generated using Python libraries (e.g., Gensim, Sumy).

[0472] Input: Parsed user interests

[0473] Output: Summarized content

[0474] What it does: The server collects the latest articles about the specified "travel" from the Internet, parses them with BeautifulSoup, and then creates summary content using the Gensim library.

[0475] Step 6:

[0476] The generated summary content is processed into thumbnail format and delivered to the user's terminal. The thumbnail contains an excerpt of the summary content and visual elements.

[0477] Input: Summarized content

[0478] Output: Summary content in thumbnail format

[0479] Specific operation: The server creates thumbnails by combining a summary article of "Top 10 tourist spots in Tokyo" with related images, and sends them to the user's device.

[0480] Step 7:

[0481] The user terminal displays the received thumbnail to the user, who can then click on the thumbnail to view the detailed summary content.

[0482] Input: Summary content in thumbnail format

[0483] Output: Thumbnail and detailed summary content displayed to the user

[0484] What it does: When a user clicks on the thumbnail of "Top 10 tourist spots in Tokyo," a browser opens and displays a detailed page of the tourist spot with a summary.

[0485] This system's series of processes allows users to efficiently collect information based on their interests and obtain the information they need in a short amount of time.

[0486] (Application example 1)

[0487] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0488] In today's world where Internet usage is widespread, users can access a large amount of information, but it is extremely difficult to efficiently collect the information they need from that vast amount of information. Furthermore, in the case of advertising delivery, advertisements that do not match the user's interests are often displayed, causing user stress and reducing the effectiveness of advertising. The present invention aims to solve these problems by providing a system that effectively utilizes user behavior data to display personalized advertisements.

[0489] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0490] In this invention, the server includes means for collecting user behavioral data, means for analyzing the collected behavioral data and identifying the user's interests, means for acquiring related information and generating summary content based on the identified user's interests, means for processing the generated summary content as thumbnails and delivering them to the user terminal, means for allowing the user to view detailed information by clicking on the thumbnails, and means for allowing the user to select a new category and update their interests. This provides personalized information and advertisements based on the user's behavioral data, allowing users to efficiently access the information they need. It also enables advertisers to deliver advertisements more effectively, which is expected to improve advertising effectiveness.

[0491] "User behavioral data" is information generated when a user uses the Internet or applications, and specifically includes data such as websites visited, URLs of pages viewed, time of visit, applications used, and purchase history.

[0492] "Collection means" refers to software and hardware for collecting user behavior data in real time or periodically.

[0493] "Means of analysis" refers to the algorithms and analytical tools used to analyze collected behavioral data to identify user interests and concerns.

[0494] "Means for acquiring related information based on the identified user's interests" refers to a function for collecting related information from the Internet, etc., based on the user's interests and concerns.

[0495] "Means for generating summary content" refers to a function that compactly summarizes acquired related information and converts it into a format that is easy for the user to understand.

[0496] The "means for processing the summary content as a thumbnail and delivering it to a user terminal" refers to a method for displaying the generated summary content in a visually appealing format and transmitting it to a user terminal.

[0497] "User terminal" refers to a device that is directly operated by a user, such as a smartphone, tablet, or PC.

[0498] "Means for viewing detailed information" refers to the ability to display the full content and related detailed information when a user clicks on a thumbnail.

[0499] "Means for selecting new categories and updating interests" refers to an interface that allows users to select new information categories based on their own interests and have them reflected in the system.

[0500] The "means for delivering in the form of push notification" refers to a function for automatically notifying a user terminal of a thumbnail of the generated summary content.

[0501] The present invention relates to a system that generates personalized summary content by collecting and analyzing user behavior data and efficiently delivers the content to users. This system basically consists of a user terminal, a server, and data exchanged between them.

[0502] User terminal operation

[0503] User devices collect behavioral data when users use the Internet or applications. This behavioral data includes the URLs of websites visited, pages viewed, visit times, applications used, purchase history, etc. This data is periodically sent to a server where it is analyzed.

[0504] Server Operation

[0505] The server receives behavioral data sent from the user's device and stores it in a database. The collected data is then analyzed to identify the user's interests. The analysis method used is machine learning algorithms using Python's Scikit-learn library, as well as natural language processing techniques using Python's NLTK library and generative AI models such as GPT-3. Based on this, relevant information is collected from the Internet and summary content is generated.

[0506] Summary content generation and delivery

[0507] The generated summary content is then processed into a thumbnail before being delivered to the user's device. This thumbnail includes an excerpt of the summary content and visual elements designed to pique the user's interest. The thumbnail is then delivered to the user's device, such as a smartphone, tablet, or PC, via push notification.

[0508] User operations

[0509] Users can view the summary content by clicking on the displayed thumbnail. This operation allows users to access the information they need quickly and efficiently. Furthermore, users can select new categories based on their interests and have the system reflect that information.

[0510] Specific examples

[0511] For example, if a user frequently visits a travel site, this behavioral data is collected and sent to a server. The server analyzes this data and identifies the user as being interested in "travel." The server then collects the latest travel guide articles and travel deals from the Internet and generates summarized content. This summarized content is then processed into thumbnails and delivered to the user's smartphone as a push notification. When the user clicks on this notification, they can immediately view the summarized information.

[0512] Hardware and software used

[0513] User devices: smartphones (iOS or Android), tablets, computers, etc.

[0514] Server: High-performance cloud server (e.g. AWS EC2)

[0515] Machine learning algorithms: Python's Scikit-learn library

[0516] Natural language processing technology: Python's NLTK library, generative AI models such as GPT-3

[0517] Database: MySQL or a NoSQL database (e.g. MongoDB)

[0518] Examples of prompt statements

[0519] "Generate ads for frequent travel site visitors whose behavioral data includes visits to multiple travel sites. Create a summary ad with a visual element suitable for a thumbnail."

[0520] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0521] Step 1: Collect user behavior data

[0522] The user device collects user behavior data. This behavior data includes the URLs of visited websites, pages viewed, visit times, applications used, purchase history, etc. These data are temporarily stored in local files. The input data is the user behavior data, and the output data is the collected behavior data.

[0523] Step 2: Sending behavioral data to the server

[0524] The terminal periodically transmits the collected behavioral data to the server. This process uses a scheduling algorithm to transmit data at specified time intervals. The input data is the collected behavioral data, and the output data is the behavioral data transmitted to the server.

[0525] Step 3: Analyzing behavioral data

[0526] The server receives the behavioral data and stores it in a database. Next, it uses machine learning algorithms and natural language processing techniques to identify the user's interests. This analysis is performed using Python's Scikit-learn and NLTK libraries. The input data is the behavioral data sent to the server, and the output data is the analysis results.

[0527] Step 4: Obtaining relevant information and generating summary content

[0528] The server collects relevant information from the Internet based on the identified user interests. The retrieved information is summarized using a generative AI model (e.g., GPT-3). Specific prompts are used to generate the summary. The input data are the analysis results and the collected relevant information, and the output data is the summary content.

[0529] Step 5: Generate and edit thumbnails

[0530] The server processes the generated summary content into an attractive thumbnail, which contains an excerpt and visual elements of the summary content. The input data is the summary content, and the output data is the processed thumbnail.

[0531] Step 6: Serving and displaying thumbnails

[0532] The server delivers the processed thumbnail to the user's device in the form of a push notification. The user receives this notification on their device and the thumbnail is displayed. The input data is the processed thumbnail, and the output data is the thumbnail displayed on the user's device.

[0533] Step 7: User interaction

[0534] The user can click on the displayed thumbnails to view the summary content, and can also update their interests by selecting new categories. The input data are the user's clicks and selections, and the output data are the viewed summary content and updated interest data.

[0535] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0536] The present invention relates to a system and its operation method for efficiently obtaining personalized information by utilizing behavioral data and emotional states of users. The basic components for implementing the present invention include a user terminal, a server, and an emotion engine.

[0537] User terminal operation

[0538] User devices collect behavioral data when users use the internet and applications. This behavioral data includes the URLs of websites visited, the duration of visits, the applications used, and purchase history. Biometric sensors (e.g., heart rate, skin temperature, and facial recognition) can also be used to capture user emotions in real time.

[0539] Example: While a user is busy visiting a travel site, an increase in heart rate is detected, which may indicate an emotional state such as excitement or anticipation.

[0540] Emotion Engine Operation

[0541] The emotion engine analyzes behavioral data and biometric sensor data sent from the user's device to identify the user's emotional state. The engine uses machine learning algorithms to determine the user's emotional state and transmits that information to the server.

[0542] Example: The emotion engine combines and analyzes the user's heart rate data, skin temperature, and facial recognition results to determine whether the user is in a state of "expectation."

[0543] Server Operation

[0544] The server receives behavioral and emotional data sent from the user device and the emotion engine. It stores these data in a database and activates a data analysis module to identify the user's interests and emotions. Based on the analysis results, it collects relevant information from the Internet and generates summary content.

[0545] Example: A server identifies travel interests from user behavioral data and detects signs of "expectation" from emotional data. Based on this information, it collects the latest travel guides and discount plans and summarizes them as content with high expectations.

[0546] Summary content generation and delivery

[0547] The server generates summary content from the collected related information, summarizes long-form information in a concise format using a text summarization algorithm, and creates thumbnails of the summary content that match the user's emotional state and delivers them to the user's device.

[0548] Example: The server generates a summary article for "Top 10 recommended tourist spots in Tokyo" and creates thumbnails that express the user's "expectation." For example, thumbnails are created that combine photos of attractive tourist spots with phrases that evoke a sense of anticipation.

[0549] User operations

[0550] Users can view the summary content by clicking on the thumbnails displayed on their devices. This operation is intuitive, and users can quickly obtain the information they need. In addition, information provided is adapted to the user's emotional state, providing a more satisfying user experience.

[0551] Example: When a user clicks on the thumbnail for "Top 10 Recommended Tourist Spots in Tokyo," they can instantly see a summary of the information and use it to help with their planning.

[0552] By combining and analyzing behavioral data and emotional data, this system makes it possible to quickly provide users with the information that is most relevant and matches their emotions, improving the user experience and significantly increasing the efficiency of information gathering.

[0553] The processing flow will be explained below.

[0554] Step 1:

[0555] When users browse the web or use applications, behavioral data is generated, including the URLs of websites visited, the duration of visits, applications used, and purchase history.

[0556] Step 2:

[0557] The device automatically collects user behavior data. This collection process runs in the background and is designed to not interfere with user operations. Data is also collected from biometric sensors (heart rate, skin temperature, facial recognition, etc.) installed on the device.

[0558] Step 3:

[0559] The device sends the collected behavioral data and biometric sensor data to a server at regular intervals or when a certain amount of data has been accumulated. The transmission is carried out using a secure communication protocol such as HTTPS.

[0560] Step 4:

[0561] The server stores the behavioral data and biometric sensor data received from the user device in a database, standardizing the data format and performing data cleansing.

[0562] Step 5:

[0563] The server launches a data analysis module to analyze the received data, using machine learning algorithms and natural language processing techniques to identify the user's interests and emotional state.

[0564] Step 6:

[0565] The emotion engine analyzes the behavioral data and biometric sensor data to identify the user's emotional state (e.g., joy, anticipation, surprise, etc.), and the analysis results are also sent to the server.

[0566] Step 7:

[0567] Based on the analysis results, the server collects relevant information from the internet, using web scraping and APIs to retrieve information such as the latest news articles, blog posts, and product reviews.

[0568] Step 8:

[0569] The server generates summary content from the collected relevant information, uses a text summarization algorithm to summarize long-form information in a concise format, and creates thumbnails of the summary content that are adaptive to the user's emotional state.

[0570] Step 9:

[0571] The server delivers the generated summary content and thumbnails to the user terminal, adjusting the content delivery order to be optimal based on the user's interests and emotions.

[0572] Step 10:

[0573] The terminal displays the received summary content and thumbnails to the user. The user interface is intuitive, allowing users to easily find the information they are looking for.

[0574] Step 11:

[0575] Users can click on the displayed thumbnails to view the summary content, allowing them to quickly obtain relevant information. Furthermore, the displayed information is adapted to the user's emotional state, resulting in a more satisfying information experience.

[0576] The above steps realize a system that allows users to quickly and efficiently view summarized content based on behavioral data and emotion data.

[0577] Example 2

[0578] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0579] Conventional information provision systems provide relevant information based solely on user behavioral data, often without considering the user's emotional state. As a result, personalized information provision that adapts to the user's emotions is lacking, and an improved user experience is desired. In addition, since the amount of information tends to be large, a means of efficiently summarizing and providing information is also required.

[0580] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0581] In this invention, the server includes means for collecting user behavioral data and biometric sensor data, means for analyzing the collected behavioral data and biometric sensor data to identify the user's emotional state, means for acquiring related information and generating summary content based on the identified user's interests and emotional state, and means for creating thumbnails of the generated summary content that match the user's emotional state and delivering them to the user terminal. This makes it possible to provide information that is adapted to the user's emotional state, thereby improving the user experience and significantly increasing the efficiency of information collection.

[0582] "User behavioral data" refers to data generated when a user uses the Internet or applications, such as the URLs of websites visited, the time of visit, applications used, and purchase history.

[0583] "Biometric sensor data" refers to data used to acquire a user's biometric information in real time, specifically including heart rate, skin temperature, and facial expression recognition results.

[0584] "Emotion Engine" refers to the algorithms and systems that analyze collected behavioral and biometric sensor data to identify a user's emotional state.

[0585] "Summary content" refers to information in which related information is reconstructed into a concise format using a summarization algorithm, allowing users to obtain information efficiently.

[0586] "Thumbnail" refers to a visually appealing image and short descriptive text related to the summarized content that matches the user's emotional state.

[0587] A "user terminal" is a device that a user uses to receive and view information, and specifically refers to a smartphone, tablet, PC, etc.

[0588] The present invention relates to a system and its operating method for efficiently providing personalized information by utilizing user behavioral data and biometric sensor data. The main components include a user terminal, a server, and an emotion engine. Each component and its operation will be described in detail below.

[0589] User terminal operation

[0590] User devices collect behavioral data when users use the internet and applications. This behavioral data includes the URLs of websites visited, the duration of visits, applications used, and purchase history. Biosensors (e.g., heart rate, skin temperature, facial expression recognition) can also be used to collect real-time emotional data.

[0591] Example: A user visits a travel site and while viewing a page about a particular tourist destination, the device's biometric sensors detect an increase in heart rate, which indicates emotions such as excitement and anticipation.

[0592] Emotion Engine Operation

[0593] The emotion engine analyzes behavioral data and biometric sensor data sent from the user's device to identify the user's emotional state. The engine uses machine learning algorithms (e.g., TensorFlow, PyTorch) to determine the user's emotional state. The identified emotional state information is then sent to the server.

[0594] Example: An emotion engine combines and analyzes heart rate data, skin temperature, and facial expression recognition results to determine when a user is in a state of "expectation."

[0595] Server Operation

[0596] The server receives behavioral and emotional data sent from the user device and the emotion engine. The received data is stored in a database (e.g., MySQL, PostgreSQL). The data analysis module then analyzes the data to identify the user's interests and emotional state. Based on the analysis results, relevant information is collected from the Internet and summary content is generated.

[0597] Example: The server stores the received data in a database, and the data analysis module identifies the user's travel interest from their recent web browsing history and detects the emotion "anticipation." Based on this information, the latest travel guides and discount plan information are collected and reconstructed as summary content.

[0598] Summary content and thumbnail generation

[0599] The server generates summary content based on the collected relevant information. It uses a text summarization algorithm (e.g., BERT summarizer) to condense long-form information into a concise format. It also creates thumbnails of the generated summary content that match the user's emotional state. It uses an image generation algorithm or image editing software (e.g., Photoshop API) to create the thumbnails.

[0600] Example: A server converts a detailed article about "Top 10 tourist spots in Tokyo" into a concise summary and generates an attractive thumbnail to create excitement.

[0601] Summary content and thumbnail delivery

[0602] The server delivers the generated summary content and thumbnails to the user device using push notifications or real-time communication protocols (e.g., WebSocket).

[0603] Example: The server delivers summary content and thumbnails to the user's smartphone app via push notification.

[0604] User operations

[0605] Users can view the summary content by clicking on the thumbnails displayed on their devices. This intuitive operation allows users to quickly obtain the information they need.

[0606] Example: A user clicks on a thumbnail of "Top 10 recommended tourist spots in Tokyo" to view summarized information and use it to help plan their trip.

[0607] Example prompt: "The emotion engine has determined that the user's heart rate increases when viewing a particular web page, indicating that the user is excited."

[0608] As a result, this system combines and analyzes the user's behavioral data and emotional data, and provides information adapted to the user's emotional state, thereby improving the user experience and making information gathering more efficient.

[0609] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0610] Step 1:

[0611] The user device collects user behavioral data and biometric sensor data, including the URLs of visited websites, the time of visit, the applications used, and purchase history. It also collects biometric sensor data (heart rate, skin temperature, and facial expression recognition) in real time.

[0612] Input: The actions and physiological responses of the body as users interact with the internet and applications.

[0613] Output: URL list of visited sites, visit times, app usage history, biometric sensor data.

[0614] Example of how it works: A user visits a travel website and while browsing a page about a particular tourist destination, an increase in heart rate is detected. This data is collected from the device.

[0615] Step 2:

[0616] The user device encrypts the collected behavioral data and biometric sensor data and sends it to the server using HTTPS as the communication protocol.

[0617] Input: Behavioral and biosensor data collected in the previous step.

[0618] Output: Encrypted behavioral and biometric sensor data.

[0619] Specific operation example: The list of URLs visited by the device and heart rate change data are encrypted in JSON format and sent to the server using the HTTPS protocol.

[0620] Step 3:

[0621] The emotion engine analyzes the behavioral data and biometric sensor data provided by the server to identify the user's emotional state. It analyzes the data using machine learning algorithms (e.g., TensorFlow, PyTorch).

[0622] Input: Decrypted behavioral and biometric sensor data.

[0623] Output: Identification of the user's emotional state (e.g., anticipation, excitement, etc.).

[0624] Specific example of operation: The emotion engine combines and analyzes heart rate data, skin temperature, and facial expression recognition results to determine that the user is in an "expectant" state, and sends the result to the server.

[0625] Step 4:

[0626] The server stores the data sent from the user terminal and the emotion engine in a database and activates a data analysis module, which analyzes the data and identifies the user's interests and emotional state.

[0627] Input: Stored behavioral and emotional data.

[0628] Output: Analysis results identifying the user's interests and emotional state.

[0629] Specific example of operation: The server identifies travel interest from user behavioral data and detects signs of "expectation" from emotional data.

[0630] Step 5:

[0631] The server collects relevant information from the Internet based on the identified user's interests and emotional state, using web crawlers and external APIs (e.g., travel information APIs) to obtain the latest information.

[0632] Input: Analysis results identifying the user's interests and emotional state.

[0633] Output: Related information (e.g. travel guide, discount offers, etc.).

[0634] Example of how it works: The server queries the travel information API to get the latest travel guides and discount plan information.

[0635] Step 6:

[0636] The server generates summary content from the collected information, using a text summarization algorithm (e.g., BERT summarizer) to summarize the information in a concise format.

[0637] Input: Related information (e.g. travel guide, discount offers, etc.).

[0638] Output: Summary content.

[0639] Example of how it works: The server converts a detailed article about "Top 10 tourist spots in Tokyo" into a concise summary.

[0640] Step 7:

[0641] The server creates thumbnails that match the emotional state of the generated summary content using image generation algorithms and image editing software (e.g., Photoshop API).

[0642] Input: Summary content, and the user's emotional state.

[0643] Output: Thumbnails adapted to the emotional state.

[0644] Specific example of how it works: The server generates thumbnails that combine photos of attractive tourist spots with phrases that evoke excitement.

[0645] Step 8:

[0646] The server delivers the generated summary content and thumbnails to the user device using push notifications or real-time communication protocols (e.g., WebSocket).

[0647] Input: Abstract content and thumbnail.

[0648] Output: Delivered summary content and thumbnails.

[0649] Specific example of operation: The server delivers summary content and thumbnails to the user's smartphone app via push notification.

[0650] Step 9:

[0651] Users can view the summary content by clicking on the thumbnails displayed on their devices. This intuitive operation allows users to quickly obtain the information they need.

[0652] Input: The thumbnail displayed on the device.

[0653] Output: User viewing summary content.

[0654] Example of specific operation: A user clicks on the thumbnail of "Top 10 recommended tourist spots in Tokyo" and views the summarized information to help plan their trip.

[0655] (Application example 2)

[0656] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0657] Conventional systems provide information based solely on user behavior data, which means that they do not adequately consider the user's emotional state, making it difficult to maximize user satisfaction. Furthermore, personalized advertising based on emotions is not available, making it difficult to maximize advertising effectiveness.

[0658] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0659] In this invention, the server includes means for collecting user behavioral data and biometric data, means for analyzing the collected behavioral data and biometric data to identify the user's emotional state and interests, means for acquiring related information and generating summary content based on the identified user's emotional state and interests, and means for delivering and displaying personalized advertisements based on the generated summary content and emotional state to the user terminal, thereby enabling the provision of information and personalized advertisements optimal for the user's emotional state.

[0660] "User behavior data" is information that records a user's activities on the Internet, such as website browsing history, application usage history, and purchase history.

[0661] "Biometric data" is information that represents a user's biological state or reaction, such as the user's heart rate, skin temperature, or facial recognition results.

[0662] "Emotional state" is information that indicates the user's current psychological state, which is analyzed from the user's biometric data.

[0663] "Relevant information" is information that is determined to be useful or interesting to the user based on the user's behavioral data and emotional state.

[0664] "Summary content" is information that succinctly summarizes related information and is generated so that users can understand it in a short amount of time.

[0665] "Personalized advertising" refers to advertising that is optimized based on a user's behavioral data and emotional state, and is determined to be highly relevant to each individual user.

[0666] A "user terminal" is a communication device such as a smartphone or tablet used by a user, and is a device that collects and displays information.

[0667] The present invention relates to a system for collecting and analyzing user behavioral data and biometric data, and delivering personalized advertisements based on the user's emotional state and interests. Specific embodiments for implementing the present invention will be described below.

[0668] Hardware and software used

[0669] The system of the present invention mainly comprises a user terminal, a server, and an emotion engine.

[0670] User device: A communication device connected to the Internet, such as a smartphone or tablet, that collects user behavioral data and biometric data and sends it to a server.

[0671] Server: Equipped with a database, data analysis module, and ad serving module, and uses a Python-based web framework such as Flask or Django.

[0672] Emotion Engine: Uses TensorFlow and PyTorch to implement machine learning models.

[0673] System Operation

[0674] 1. Data Collection:

[0675] The user device collects behavioral data such as the user's website browsing history, app usage history, and purchase history, as well as biometric data such as heart rate, skin temperature, and facial recognition results in real time. For example, it records the user's history of using a shopping app and the timing of when their heart rate increased.

[0676] 2. Data transmission:

[0677] The collected data is sent to the server via secure communication using HTTPS, and the data is encrypted.

[0678] 3. Emotion analysis:

[0679] The server inputs the received data into an emotion engine to analyze the user's emotional state, for example, determining whether the user is relaxed or stressed based on a combination of heart rate and skin temperature.

[0680] 4. Advertisement Selection:

[0681] Based on the emotional state analyzed by the emotion engine, the server searches for relevant information from the database and selects the most suitable personalized advertisement for the user, for example, if the user is relaxed, it will select a refreshing travel advertisement.

[0682] 5. Advertisement Delivery:

[0683] The selected advertisement is then pushed to the user's device and displayed there. At this time, a thumbnail is also generated based on the user's emotional state to make it easier for the user to intuitively understand the advertisement content.

[0684] Specific examples

[0685] While User A is browsing a shopping app on his smartphone, the server detects that his heart rate is higher than normal. The server determines this as stress and displays product advertisements on the smartphone that will help relieve stress.

[0686] Prompt Sentence Examples

[0687] "Collect user behavioral and emotional data and display ads that match your relaxed mood based on your emotional state."

[0688] This system enables optimal information to be provided based on the user's behavior and emotions, enhancing the user experience.

[0689] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0690] Step 1:

[0691] The user device collects user behavioral data and biometric data. Specifically, it records in real time the URLs of websites visited by the user, the time of visit, the applications used, purchase history, heart rate, skin temperature, and facial recognition results. The input is the user's behavior and signals obtained from biometric sensors, and the output is a set of collected data.

[0692] Step 2:

[0693] The user device sends the collected behavioral data and biometric data to the server via secure communication. The communication is encrypted using the HTTPS protocol. The input is the set of collected data, and the output is the data sent to the server.

[0694] Step 3:

[0695] The server inputs the received behavioral data and biometric data into the emotion engine. The emotion engine uses a machine learning model (TensorFlow or PyTorch) to analyze the data and identify the user's emotional state. The analysis process identifies "relaxed" or "stressed" based on heart rate, skin temperature, and facial recognition results. The input is the integrated data sent to the server, and the output is the analyzed user's emotional state.

[0696] Step 4:

[0697] The server searches for relevant information from a database based on the user's emotional state and behavioral data identified by the emotion engine, generates summary content, and selects personalized advertisements that match the user's specific emotional state. The input is the emotional state and behavioral data, and the output is the summary content and personalized advertisements.

[0698] Step 5:

[0699] The server generates attractive thumbnails for the generated summary content and advertisements based on the user's emotional state, providing a display format that is intuitively understandable to users. The input is the summary content and personalized advertisements, and the output is the summary content with thumbnails and advertisements.

[0700] Step 6:

[0701] The server transmits the summary content with thumbnails and the personalized advertisement to the user terminal by push notification, where the input is the summary content with thumbnails and the advertisement, and the output is the notification sent to the user terminal.

[0702] Step 7:

[0703] The user terminal displays the received summary content with thumbnails and advertisements. The user can view the summary content or advertisement details by clicking on the displayed thumbnail. The input is the pushed content and advertisement, and the output is the visual information displayed to the user.

[0704] This system enables optimal information provision and advertisement delivery based on user behavior and emotions, improving the user experience.

[0705] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0706] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0707] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0708] [Third embodiment]

[0709] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0710] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0711] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0712] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0713] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0714] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0715] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0716] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0717] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0718] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0719] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0720] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0721] The present invention relates to a system and an operating method thereof for enabling a user to efficiently obtain personalized information. The basic components for implementing the present invention are a user terminal, a server, and data exchanged between them.

[0722] User terminal operation

[0723] User devices collect behavioral data when users use the Internet or applications. This behavioral data includes the websites visited, the URLs of pages viewed, the duration of visits, the applications used, and purchase history. This data is periodically sent to the server. Users can also register categories of interest and specific information.

[0724] Example: If a user frequently visits travel sites, their URLs are collected and sent to the server.

[0725] Server Operation

[0726] The server receives the behavioral data sent from the user's device and stores it in a database. The collected data is then analyzed. Machine learning algorithms and natural language processing techniques are used to identify the user's interests. Based on the results of this analysis, related information is collected from the Internet and summary content is generated.

[0727] Example: A server analyzes data from a user's repeated visits to a travel site and identifies that the user is interested in "travel." Based on this information, it gathers the latest travel guide articles and travel deals from the Internet and creates a compact summary.

[0728] Summary content generation and delivery

[0729] The generated summary content is also processed as thumbnails and delivered to the user terminal. These thumbnails contain excerpts of the summary content and visual elements designed to attract the user's attention. The user terminal receives these thumbnails and displays them to the user.

[0730] Example: A summary article of "Top 10 recommended tourist spots in Tokyo" generated by the server is displayed on the user's smartphone screen along with a thumbnail image. By clicking on this, the user can quickly view the summarized information.

[0731] User operations

[0732] Users can click on the thumbnails to view the summary content. This is a very intuitive operation, allowing users to access the information they need in a short amount of time, greatly improving the efficiency of information gathering. Users can also select new categories based on their own interests.

[0733] Example: When a user clicks on the thumbnail of "Top 10 Recommended Tourist Spots in Tokyo," they can immediately view summarized information about the tourist spots, which can be useful when planning their trip.

[0734] This system allows users to efficiently gather the information they need in a short amount of time, even in the midst of an information overload, greatly simplifying everyday information gathering.

[0735] The processing flow will be explained below.

[0736] Step 1:

[0737] Users generate behavioral data when they browse the web or use apps, including the URLs of websites visited, the duration of their visits, the applications used, and their purchase history.

[0738] Step 2:

[0739] The device automatically collects the generated behavioral data, and the collection process runs in the background and is designed to not interfere with user operations.

[0740] Step 3:

[0741] The device sends the collected behavioral data to a server at regular intervals or when a certain amount of data has been accumulated. The transmission is done using a secure communication protocol such as HTTPS.

[0742] Step 4:

[0743] The server stores the behavioral data received from the user device in a database, standardizing the data format and performing data cleansing.

[0744] Step 5:

[0745] The server launches a data analysis module to analyze the received behavioral data, using machine learning algorithms and natural language processing techniques to identify the user's interests.

[0746] Step 6:

[0747] Based on the analysis results, the server collects relevant information from the internet, using web scraping and APIs to retrieve information such as the latest news articles, blog posts, and product reviews.

[0748] Step 7:

[0749] The server generates summary content from the collected relevant information, using a text summarization algorithm to summarize lengthy information in a concise format.

[0750] Step 8:

[0751] The server generates a thumbnail for the summary content, which includes a text and visual image of a portion of the summary content.

[0752] Step 9:

[0753] The server delivers the generated summary content and thumbnails to the user terminal, adjusting the content delivery order to be optimal based on the user's interests.

[0754] Step 10:

[0755] The terminal displays the received summary content and thumbnails to the user. The user interface is intuitive, allowing users to easily find the information they are looking for.

[0756] Step 11:

[0757] Users can click on the displayed thumbnails to view the summary content, which allows users to quickly obtain related information.

[0758] The above steps realize a system that allows users to quickly and efficiently view summarized content based on behavioral data.

[0759] Example 1

[0760] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0761] Conventional information provision systems have difficulty quickly and appropriately providing information that meets users' diverse interests. Furthermore, due to the vast amount of information available, users may take a significant amount of time to obtain the information they need. Furthermore, there is a lack of technology that efficiently utilizes user behavior data to provide personalized content, creating a need for improved user experience.

[0762] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0763] In this invention, the server includes means for analyzing user behavior data and identifying user interests, means for collecting related information from the Internet based on the identified user interests and generating summary content, and means for converting the generated summary content into a visually appealing thumbnail format and delivering and displaying it on the user terminal, thereby enabling users to quickly and efficiently obtain information that matches their interests.

[0764] "User behavioral data" refers to information such as the URLs of websites visited when a user uses the Internet or applications, the URLs of pages viewed, the time of visit, the name and duration of use of the application used, and purchase history.

[0765] The "server" is a central computer system that receives and analyzes behavioral data sent from user terminals, collects related information, and generates summary content.

[0766] "Means for analysis" refers to the process and program for identifying user interests and concerns from collected behavioral data using machine learning algorithms and natural language processing techniques.

[0767] "Means of collecting relevant information" refers to processes such as web scraping techniques and API access to obtain information related to the user's interests from the Internet.

[0768] The "means for generating summary content" is the process of creating content based on collected related information to enable users to understand important information in a short amount of time.

[0769] "Thumbnail processing" is the process by which the generated summary content is converted into a visually appealing and usable format.

[0770] The "means for delivering and displaying" refers to a process for transmitting the generated summary content to a user terminal and displaying it on the screen of the terminal.

[0771] A "secure protocol" is a communication protocol for securely sending and receiving data, an example of which is HTTPS.

[0772] A "machine learning algorithm" is a program that finds patterns and rules from large amounts of data and makes predictions and classifications.

[0773] "Natural language processing technology" is a technology for analyzing and understanding human language.

[0774] The present invention relates to a system and an operating method for enabling users to efficiently obtain personalized information. To implement the system, the following components are required: a user terminal, a server, and data exchanged between them.

[0775] User terminal operation

[0776] User devices collect behavioral data when users use the Internet or applications. This behavioral data includes the websites visited, the URLs of pages viewed, the duration of visits, the applications used, and purchase history. This data is periodically sent to the server. Users can also register categories of interest and specific information.

[0777] Example: If a user frequently visits travel sites, their URLs are collected and sent to the server.

[0778] Server Operation

[0779] The server receives behavioral data sent from the user's device and stores it in a database. The collected data is then analyzed using machine learning algorithms and natural language processing techniques (e.g., Scikit-learn, NLTK) implemented in Python to identify the user's interests. Based on the results of this analysis, related information is collected from the Internet and summary content is generated using Python.

[0780] Example: A server analyzes data on a user's repeated visits to a travel site and identifies that the user is interested in "travel." Based on this information, it collects the latest travel guide articles and travel deals from the Internet and creates summary content using Python libraries (e.g., BeautifulSoup, Requests).

[0781] Summary content generation and delivery

[0782] The generated summary content is also processed as thumbnails and delivered to the user terminal. These thumbnails contain excerpts of the summary content and visual elements designed to attract the user's attention. The user terminal receives these thumbnails and displays them to the user.

[0783] Example: A summary article of "Top 10 recommended tourist spots in Tokyo" generated by the server is displayed on the user's smartphone screen along with a thumbnail image. By clicking on this, the user can quickly view the summarized information.

[0784] User operations

[0785] Users can click on the thumbnails to view the summary content. This is a very intuitive operation, allowing users to access the information they need in a short amount of time, greatly improving the efficiency of information gathering. Users can also select new categories based on their own interests.

[0786] Example: When a user clicks on the thumbnail of "Top 10 Recommended Tourist Spots in Tokyo," they can immediately view summarized information about the tourist spots, which can be useful when planning their trip.

[0787] Prompt Sentence Examples

[0788] Here are some example prompts to input to a generative AI model:

[0789] "We've determined that the user is interested in travel. How can we collect the latest travel guide articles and travel deals and provide them as summary content?"

[0790] This system allows users to efficiently gather the information they need in a short amount of time, even in the midst of an information overload, greatly simplifying everyday information gathering.

[0791] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0792] Step 1:

[0793] The user device collects user behavior data, including the URLs of visited websites, URLs of viewed pages, visit times, names of applications used, and purchase history. This data is recorded in local storage in real time.

[0794] Input: User internet and application behavior

[0795] Output: Behavioral data recorded in local storage

[0796] Specific operation: When a user visits an online shopping site, the URL of the site, the URL of the product page viewed, and the time of the visit are recorded on the device.

[0797] Step 2:

[0798] The collected behavioral data is periodically sent to a server using the secure HTTPS protocol. The data is converted to JSON format before transmission.

[0799] Input: Behavioral data recorded in local storage

[0800] Output: JSON formatted behavior data sent to the server

[0801] Specific operation: The device sends the collected data to the server's API endpoint using HTTPS at regular intervals (e.g., every hour). The data is in JSON format and includes a "URL," "start time," and "end time."

[0802] Step 3:

[0803] The server receives the behavioral data sent from the user terminal and stores it in a database, which is an SQL-based database (e.g., PostgreSQL).

[0804] Input: JSON formatted behavior data sent from the device

[0805] Output: Behavioral data stored in a database (PostgreSQL) table

[0806] Specific operation: The server's API endpoint parses the received JSON data and stores the data in the "UserActions" table in the PostgreSQL database using an INSERT query.

[0807] Step 4:

[0808] The server analyzes the behavioral data stored in the database using machine learning algorithms and natural language processing techniques (e.g., Scikit-learn, NLTK) implemented in Python.

[0809] Input: Behavioral data stored in a database

[0810] Output: Analysis of user interests and concerns

[0811] How it works: The server analyzes behavioral data collected over the past month to identify categories that users repeatedly visit. It then uses Scikit-learn to perform clustering analysis and identify user interests.

[0812] Step 5:

[0813] The server collects relevant information from the Internet based on the analysis results. Web scraping technology (e.g., BeautifulSoup, Scrapy) is used to collect information. The collected information is summarized and summary content is generated using Python libraries (e.g., Gensim, Sumy).

[0814] Input: Parsed user interests

[0815] Output: Summarized content

[0816] What it does: The server collects the latest articles about the specified "travel" from the Internet, parses them with BeautifulSoup, and then creates summary content using the Gensim library.

[0817] Step 6:

[0818] The generated summary content is processed into thumbnail format and delivered to the user's terminal. The thumbnail contains an excerpt of the summary content and visual elements.

[0819] Input: Summarized content

[0820] Output: Summary content in thumbnail format

[0821] Specific operation: The server creates thumbnails by combining a summary article of "Top 10 tourist spots in Tokyo" with related images, and sends them to the user's device.

[0822] Step 7:

[0823] The user terminal displays the received thumbnail to the user, who can then click on the thumbnail to view the detailed summary content.

[0824] Input: Summary content in thumbnail format

[0825] Output: Thumbnail and detailed summary content displayed to the user

[0826] What it does: When a user clicks on the thumbnail of "Top 10 tourist spots in Tokyo," a browser opens and displays a detailed page of the tourist spot with a summary.

[0827] This system's series of processes allows users to efficiently collect information based on their interests and obtain the information they need in a short amount of time.

[0828] (Application example 1)

[0829] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0830] In today's world where Internet usage is widespread, users can access a large amount of information, but it is extremely difficult to efficiently collect the information they need from that vast amount of information. Furthermore, in the case of advertising delivery, advertisements that do not match the user's interests are often displayed, causing user stress and reducing the effectiveness of advertising. The present invention aims to solve these problems by providing a system that effectively utilizes user behavior data to display personalized advertisements.

[0831] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0832] In this invention, the server includes means for collecting user behavioral data, means for analyzing the collected behavioral data and identifying the user's interests, means for acquiring related information and generating summary content based on the identified user's interests, means for processing the generated summary content as thumbnails and delivering them to the user terminal, means for allowing the user to view detailed information by clicking on the thumbnails, and means for allowing the user to select a new category and update their interests. This provides personalized information and advertisements based on the user's behavioral data, allowing users to efficiently access the information they need. It also enables advertisers to deliver advertisements more effectively, which is expected to improve advertising effectiveness.

[0833] "User behavioral data" is information generated when a user uses the Internet or applications, and specifically includes data such as websites visited, URLs of pages viewed, time of visit, applications used, and purchase history.

[0834] "Collection means" refers to software and hardware for collecting user behavior data in real time or periodically.

[0835] "Means of analysis" refers to the algorithms and analytical tools used to analyze collected behavioral data to identify user interests and concerns.

[0836] "Means for acquiring related information based on the identified user's interests" refers to a function for collecting related information from the Internet, etc., based on the user's interests and concerns.

[0837] "Means for generating summary content" refers to a function that compactly summarizes acquired related information and converts it into a format that is easy for the user to understand.

[0838] The "means for processing the summary content as a thumbnail and delivering it to a user terminal" refers to a method for displaying the generated summary content in a visually appealing format and transmitting it to a user terminal.

[0839] "User terminal" refers to a device that is directly operated by a user, such as a smartphone, tablet, or PC.

[0840] "Means for viewing detailed information" refers to the ability to display the full content and related detailed information when a user clicks on a thumbnail.

[0841] "Means for selecting new categories and updating interests" refers to an interface that allows users to select new information categories based on their own interests and have them reflected in the system.

[0842] The "means for delivering in the form of push notification" refers to a function for automatically notifying a user terminal of a thumbnail of the generated summary content.

[0843] The present invention relates to a system that generates personalized summary content by collecting and analyzing user behavior data and efficiently delivers the content to users. This system basically consists of a user terminal, a server, and data exchanged between them.

[0844] User terminal operation

[0845] User devices collect behavioral data when users use the Internet or applications. This behavioral data includes the URLs of websites visited, pages viewed, visit times, applications used, purchase history, etc. This data is periodically sent to a server where it is analyzed.

[0846] Server Operation

[0847] The server receives behavioral data sent from the user's device and stores it in a database. The collected data is then analyzed to identify the user's interests. The analysis method used is machine learning algorithms using Python's Scikit-learn library, as well as natural language processing techniques using Python's NLTK library and generative AI models such as GPT-3. Based on this, relevant information is collected from the Internet and summary content is generated.

[0848] Summary content generation and delivery

[0849] The generated summary content is then processed into a thumbnail before being delivered to the user's device. This thumbnail includes an excerpt of the summary content and visual elements designed to pique the user's interest. The thumbnail is then delivered to the user's device, such as a smartphone, tablet, or PC, via push notification.

[0850] User operations

[0851] Users can view the summary content by clicking on the displayed thumbnail. This operation allows users to access the information they need quickly and efficiently. Furthermore, users can select new categories based on their interests and have the system reflect that information.

[0852] Specific examples

[0853] For example, if a user frequently visits a travel site, this behavioral data is collected and sent to a server. The server analyzes this data and identifies the user as being interested in "travel." The server then collects the latest travel guide articles and travel deals from the Internet and generates summarized content. This summarized content is then processed into thumbnails and delivered to the user's smartphone as a push notification. When the user clicks on this notification, they can immediately view the summarized information.

[0854] Hardware and software used

[0855] User devices: smartphones (iOS or Android), tablets, computers, etc.

[0856] Server: High-performance cloud server (e.g. AWS EC2)

[0857] Machine learning algorithms: Python's Scikit-learn library

[0858] Natural language processing technology: Python's NLTK library, generative AI models such as GPT-3

[0859] Database: MySQL or a NoSQL database (e.g. MongoDB)

[0860] Examples of prompt statements

[0861] "Generate ads for frequent travel site visitors whose behavioral data includes visits to multiple travel sites. Create a summary ad with a visual element suitable for a thumbnail."

[0862] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0863] Step 1: Collect user behavior data

[0864] The user device collects user behavior data. This behavior data includes the URLs of visited websites, pages viewed, visit times, applications used, purchase history, etc. These data are temporarily stored in local files. The input data is the user behavior data, and the output data is the collected behavior data.

[0865] Step 2: Sending behavioral data to the server

[0866] The terminal periodically transmits the collected behavioral data to the server. This process uses a scheduling algorithm to transmit data at specified time intervals. The input data is the collected behavioral data, and the output data is the behavioral data transmitted to the server.

[0867] Step 3: Analyzing behavioral data

[0868] The server receives the behavioral data and stores it in a database. Next, it uses machine learning algorithms and natural language processing techniques to identify the user's interests. This analysis is performed using Python's Scikit-learn and NLTK libraries. The input data is the behavioral data sent to the server, and the output data is the analysis results.

[0869] Step 4: Obtaining relevant information and generating summary content

[0870] The server collects relevant information from the Internet based on the identified user interests. The retrieved information is summarized using a generative AI model (e.g., GPT-3). Specific prompts are used to generate the summary. The input data are the analysis results and the collected relevant information, and the output data is the summary content.

[0871] Step 5: Generate and edit thumbnails

[0872] The server processes the generated summary content into an attractive thumbnail, which contains an excerpt and visual elements of the summary content. The input data is the summary content, and the output data is the processed thumbnail.

[0873] Step 6: Serving and displaying thumbnails

[0874] The server delivers the processed thumbnail to the user's device in the form of a push notification. The user receives this notification on their device and the thumbnail is displayed. The input data is the processed thumbnail, and the output data is the thumbnail displayed on the user's device.

[0875] Step 7: User interaction

[0876] The user can click on the displayed thumbnails to view the summary content, and can also update their interests by selecting new categories. The input data are the user's clicks and selections, and the output data are the viewed summary content and updated interest data.

[0877] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0878] The present invention relates to a system and its operation method for efficiently obtaining personalized information by utilizing behavioral data and emotional states of users. The basic components for implementing the present invention include a user terminal, a server, and an emotion engine.

[0879] User terminal operation

[0880] User devices collect behavioral data when users use the internet and applications. This behavioral data includes the URLs of websites visited, the duration of visits, the applications used, and purchase history. Biometric sensors (e.g., heart rate, skin temperature, and facial recognition) can also be used to capture user emotions in real time.

[0881] Example: While a user is busy visiting a travel site, an increase in heart rate is detected, which may indicate an emotional state such as excitement or anticipation.

[0882] Emotion Engine Operation

[0883] The emotion engine analyzes behavioral data and biometric sensor data sent from the user's device to identify the user's emotional state. The engine uses machine learning algorithms to determine the user's emotional state and transmits that information to the server.

[0884] Example: The emotion engine combines and analyzes the user's heart rate data, skin temperature, and facial recognition results to determine whether the user is in a state of "expectation."

[0885] Server Operation

[0886] The server receives behavioral and emotional data sent from the user device and the emotion engine. It stores these data in a database and activates a data analysis module to identify the user's interests and emotions. Based on the analysis results, it collects relevant information from the Internet and generates summary content.

[0887] Example: A server identifies travel interests from user behavioral data and detects signs of "expectation" from emotional data. Based on this information, it collects the latest travel guides and discount plans and summarizes them as content with high expectations.

[0888] Summary content generation and delivery

[0889] The server generates summary content from the collected related information, summarizes long-form information in a concise format using a text summarization algorithm, and creates thumbnails of the summary content that match the user's emotional state and delivers them to the user's device.

[0890] Example: The server generates a summary article for "Top 10 recommended tourist spots in Tokyo" and creates thumbnails that express the user's "expectation." For example, thumbnails are created that combine photos of attractive tourist spots with phrases that evoke a sense of anticipation.

[0891] User operations

[0892] Users can view the summary content by clicking on the thumbnails displayed on their devices. This operation is intuitive, and users can quickly obtain the information they need. In addition, information provided is adapted to the user's emotional state, providing a more satisfying user experience.

[0893] Example: When a user clicks on the thumbnail for "Top 10 Recommended Tourist Spots in Tokyo," they can instantly see a summary of the information and use it to help with their planning.

[0894] By combining and analyzing behavioral data and emotional data, this system makes it possible to quickly provide users with the information that is most relevant and matches their emotions, improving the user experience and significantly increasing the efficiency of information gathering.

[0895] The processing flow will be explained below.

[0896] Step 1:

[0897] When users browse the web or use applications, behavioral data is generated, including the URLs of websites visited, the duration of visits, applications used, and purchase history.

[0898] Step 2:

[0899] The device automatically collects user behavior data. This collection process runs in the background and is designed to not interfere with user operations. Data is also collected from biometric sensors (heart rate, skin temperature, facial recognition, etc.) installed on the device.

[0900] Step 3:

[0901] The device sends the collected behavioral data and biometric sensor data to a server at regular intervals or when a certain amount of data has been accumulated. The transmission is carried out using a secure communication protocol such as HTTPS.

[0902] Step 4:

[0903] The server stores the behavioral data and biometric sensor data received from the user device in a database, standardizing the data format and performing data cleansing.

[0904] Step 5:

[0905] The server launches a data analysis module to analyze the received data, using machine learning algorithms and natural language processing techniques to identify the user's interests and emotional state.

[0906] Step 6:

[0907] The emotion engine analyzes the behavioral data and biometric sensor data to identify the user's emotional state (e.g., joy, anticipation, surprise, etc.), and the analysis results are also sent to the server.

[0908] Step 7:

[0909] Based on the analysis results, the server collects relevant information from the internet, using web scraping and APIs to retrieve information such as the latest news articles, blog posts, and product reviews.

[0910] Step 8:

[0911] The server generates summary content from the collected relevant information, uses a text summarization algorithm to summarize long-form information in a concise format, and creates thumbnails of the summary content that are adaptive to the user's emotional state.

[0912] Step 9:

[0913] The server delivers the generated summary content and thumbnails to the user terminal, adjusting the content delivery order to be optimal based on the user's interests and emotions.

[0914] Step 10:

[0915] The terminal displays the received summary content and thumbnails to the user. The user interface is intuitive, allowing users to easily find the information they are looking for.

[0916] Step 11:

[0917] Users can click on the displayed thumbnails to view the summary content, allowing them to quickly obtain relevant information. Furthermore, the displayed information is adapted to the user's emotional state, resulting in a more satisfying information experience.

[0918] The above steps realize a system that allows users to quickly and efficiently view summarized content based on behavioral data and emotion data.

[0919] Example 2

[0920] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0921] Conventional information provision systems provide relevant information based solely on user behavioral data, often without considering the user's emotional state. As a result, personalized information provision that adapts to the user's emotions is lacking, and an improved user experience is desired. In addition, since the amount of information tends to be large, a means of efficiently summarizing and providing information is also required.

[0922] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0923] In this invention, the server includes means for collecting user behavioral data and biometric sensor data, means for analyzing the collected behavioral data and biometric sensor data to identify the user's emotional state, means for acquiring related information and generating summary content based on the identified user's interests and emotional state, and means for creating thumbnails of the generated summary content that match the user's emotional state and delivering them to the user terminal. This makes it possible to provide information that is adapted to the user's emotional state, thereby improving the user experience and significantly increasing the efficiency of information collection.

[0924] "User behavioral data" refers to data generated when a user uses the Internet or applications, such as the URLs of websites visited, the time of visit, applications used, and purchase history.

[0925] "Biometric sensor data" refers to data used to acquire a user's biometric information in real time, specifically including heart rate, skin temperature, and facial expression recognition results.

[0926] "Emotion Engine" refers to the algorithms and systems that analyze collected behavioral and biometric sensor data to identify a user's emotional state.

[0927] "Summary content" refers to information in which related information is reconstructed into a concise format using a summarization algorithm, allowing users to obtain information efficiently.

[0928] "Thumbnail" refers to a visually appealing image and short descriptive text related to the summarized content that matches the user's emotional state.

[0929] A "user terminal" is a device that a user uses to receive and view information, and specifically refers to a smartphone, tablet, PC, etc.

[0930] The present invention relates to a system and its operating method for efficiently providing personalized information by utilizing user behavioral data and biometric sensor data. The main components include a user terminal, a server, and an emotion engine. Each component and its operation will be described in detail below.

[0931] User terminal operation

[0932] User devices collect behavioral data when users use the internet and applications. This behavioral data includes the URLs of websites visited, the duration of visits, applications used, and purchase history. Biosensors (e.g., heart rate, skin temperature, facial expression recognition) can also be used to collect real-time emotional data.

[0933] Example: A user visits a travel site and while viewing a page about a particular tourist destination, the device's biometric sensors detect an increase in heart rate, which indicates emotions such as excitement and anticipation.

[0934] Emotion Engine Operation

[0935] The emotion engine analyzes behavioral data and biometric sensor data sent from the user's device to identify the user's emotional state. The engine uses machine learning algorithms (e.g., TensorFlow, PyTorch) to determine the user's emotional state. The identified emotional state information is then sent to the server.

[0936] Example: An emotion engine combines and analyzes heart rate data, skin temperature, and facial expression recognition results to determine when a user is in a state of "expectation."

[0937] Server Operation

[0938] The server receives behavioral and emotional data sent from the user device and the emotion engine. The received data is stored in a database (e.g., MySQL, PostgreSQL). The data analysis module then analyzes the data to identify the user's interests and emotional state. Based on the analysis results, relevant information is collected from the Internet and summary content is generated.

[0939] Example: The server stores the received data in a database, and the data analysis module identifies the user's travel interest from their recent web browsing history and detects the emotion "anticipation." Based on this information, the latest travel guides and discount plan information are collected and reconstructed as summary content.

[0940] Summary content and thumbnail generation

[0941] The server generates summary content based on the collected relevant information. It uses a text summarization algorithm (e.g., BERT summarizer) to condense long-form information into a concise format. It also creates thumbnails of the generated summary content that match the user's emotional state. It uses an image generation algorithm or image editing software (e.g., Photoshop API) to create the thumbnails.

[0942] Example: A server converts a detailed article about "Top 10 tourist spots in Tokyo" into a concise summary and generates an attractive thumbnail to create excitement.

[0943] Summary content and thumbnail delivery

[0944] The server delivers the generated summary content and thumbnails to the user device using push notifications or real-time communication protocols (e.g., WebSocket).

[0945] Example: The server delivers summary content and thumbnails to the user's smartphone app via push notification.

[0946] User operations

[0947] Users can view the summary content by clicking on the thumbnails displayed on their devices. This intuitive operation allows users to quickly obtain the information they need.

[0948] Example: A user clicks on a thumbnail of "Top 10 recommended tourist spots in Tokyo" to view summarized information and use it to help plan their trip.

[0949] Example prompt: "The emotion engine has determined that the user's heart rate increases when viewing a particular web page, indicating that the user is excited."

[0950] As a result, this system combines and analyzes the user's behavioral data and emotional data, and provides information adapted to the user's emotional state, thereby improving the user experience and making information gathering more efficient.

[0951] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0952] Step 1:

[0953] The user device collects user behavioral data and biometric sensor data, including the URLs of visited websites, the time of visit, the applications used, and purchase history. It also collects biometric sensor data (heart rate, skin temperature, and facial expression recognition) in real time.

[0954] Input: The actions and physiological responses of the body as users interact with the internet and applications.

[0955] Output: URL list of visited sites, visit times, app usage history, biometric sensor data.

[0956] Example of how it works: A user visits a travel website and while browsing a page about a particular tourist destination, an increase in heart rate is detected. This data is collected from the device.

[0957] Step 2:

[0958] The user device encrypts the collected behavioral data and biometric sensor data and sends it to the server using HTTPS as the communication protocol.

[0959] Input: Behavioral and biosensor data collected in the previous step.

[0960] Output: Encrypted behavioral and biometric sensor data.

[0961] Specific operation example: The list of URLs visited by the device and heart rate change data are encrypted in JSON format and sent to the server using the HTTPS protocol.

[0962] Step 3:

[0963] The emotion engine analyzes the behavioral data and biometric sensor data provided by the server to identify the user's emotional state. It analyzes the data using machine learning algorithms (e.g., TensorFlow, PyTorch).

[0964] Input: Decrypted behavioral and biometric sensor data.

[0965] Output: Identification of the user's emotional state (e.g., anticipation, excitement, etc.).

[0966] Specific example of operation: The emotion engine combines and analyzes heart rate data, skin temperature, and facial expression recognition results to determine that the user is in an "expectant" state, and sends the result to the server.

[0967] Step 4:

[0968] The server stores the data sent from the user terminal and the emotion engine in a database and activates a data analysis module, which analyzes the data and identifies the user's interests and emotional state.

[0969] Input: Stored behavioral and emotional data.

[0970] Output: Analysis results identifying the user's interests and emotional state.

[0971] Specific example of operation: The server identifies travel interest from user behavioral data and detects signs of "expectation" from emotional data.

[0972] Step 5:

[0973] The server collects relevant information from the Internet based on the identified user's interests and emotional state, using web crawlers and external APIs (e.g., travel information APIs) to obtain the latest information.

[0974] Input: Analysis results identifying the user's interests and emotional state.

[0975] Output: Related information (e.g. travel guide, discount offers, etc.).

[0976] Example of how it works: The server queries the travel information API to get the latest travel guides and discount plan information.

[0977] Step 6:

[0978] The server generates summary content from the collected information, using a text summarization algorithm (e.g., BERT summarizer) to summarize the information in a concise format.

[0979] Input: Related information (e.g. travel guide, discount offers, etc.).

[0980] Output: Summary content.

[0981] Example of how it works: The server converts a detailed article about "Top 10 tourist spots in Tokyo" into a concise summary.

[0982] Step 7:

[0983] The server creates thumbnails that match the emotional state of the generated summary content using image generation algorithms and image editing software (e.g., Photoshop API).

[0984] Input: Summary content, and the user's emotional state.

[0985] Output: Thumbnails adapted to the emotional state.

[0986] Specific example of how it works: The server generates thumbnails that combine photos of attractive tourist spots with phrases that evoke excitement.

[0987] Step 8:

[0988] The server delivers the generated summary content and thumbnails to the user device using push notifications or real-time communication protocols (e.g., WebSocket).

[0989] Input: Abstract content and thumbnail.

[0990] Output: Delivered summary content and thumbnails.

[0991] Specific example of operation: The server delivers summary content and thumbnails to the user's smartphone app via push notification.

[0992] Step 9:

[0993] Users can view the summary content by clicking on the thumbnails displayed on their devices. This intuitive operation allows users to quickly obtain the information they need.

[0994] Input: The thumbnail displayed on the device.

[0995] Output: User viewing summary content.

[0996] Example of specific operation: A user clicks on the thumbnail of "Top 10 recommended tourist spots in Tokyo" and views the summarized information to help plan their trip.

[0997] (Application example 2)

[0998] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0999] Conventional systems provide information based solely on user behavior data, which means that they do not adequately consider the user's emotional state, making it difficult to maximize user satisfaction. Furthermore, personalized advertising based on emotions is not available, making it difficult to maximize advertising effectiveness.

[1000] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1001] In this invention, the server includes means for collecting user behavioral data and biometric data, means for analyzing the collected behavioral data and biometric data to identify the user's emotional state and interests, means for acquiring related information and generating summary content based on the identified user's emotional state and interests, and means for delivering and displaying personalized advertisements based on the generated summary content and emotional state to the user terminal, thereby enabling the provision of information and personalized advertisements optimal for the user's emotional state.

[1002] "User behavior data" is information that records a user's activities on the Internet, such as website browsing history, application usage history, and purchase history.

[1003] "Biometric data" is information that represents a user's biological state or reaction, such as the user's heart rate, skin temperature, or facial recognition results.

[1004] "Emotional state" is information that indicates the user's current psychological state, which is analyzed from the user's biometric data.

[1005] "Relevant information" is information that is determined to be useful or interesting to the user based on the user's behavioral data and emotional state.

[1006] "Summary content" is information that succinctly summarizes related information and is generated so that users can understand it in a short amount of time.

[1007] "Personalized advertising" refers to advertising that is optimized based on a user's behavioral data and emotional state, and is determined to be highly relevant to each individual user.

[1008] A "user terminal" is a communication device such as a smartphone or tablet used by a user, and is a device that collects and displays information.

[1009] The present invention relates to a system for collecting and analyzing user behavioral data and biometric data, and delivering personalized advertisements based on the user's emotional state and interests. Specific embodiments for implementing the present invention will be described below.

[1010] Hardware and software used

[1011] The system of the present invention mainly comprises a user terminal, a server, and an emotion engine.

[1012] User device: A communication device connected to the Internet, such as a smartphone or tablet, that collects user behavioral data and biometric data and sends it to a server.

[1013] Server: Equipped with a database, data analysis module, and ad serving module, and uses a Python-based web framework such as Flask or Django.

[1014] Emotion Engine: Uses TensorFlow and PyTorch to implement machine learning models.

[1015] System Operation

[1016] 1. Data Collection:

[1017] The user device collects behavioral data such as the user's website browsing history, app usage history, and purchase history, as well as biometric data such as heart rate, skin temperature, and facial recognition results in real time. For example, it records the user's history of using a shopping app and the timing of when their heart rate increased.

[1018] 2. Data transmission:

[1019] The collected data is sent to the server via secure communication using HTTPS, and the data is encrypted.

[1020] 3. Emotion analysis:

[1021] The server inputs the received data into an emotion engine to analyze the user's emotional state, for example, determining whether the user is relaxed or stressed based on a combination of heart rate and skin temperature.

[1022] 4. Advertisement Selection:

[1023] Based on the emotional state analyzed by the emotion engine, the server searches for relevant information from the database and selects the most suitable personalized advertisement for the user, for example, if the user is relaxed, it will select a refreshing travel advertisement.

[1024] 5. Advertisement Delivery:

[1025] The selected advertisement is then pushed to the user's device and displayed there. At this time, a thumbnail is also generated based on the user's emotional state to make it easier for the user to intuitively understand the advertisement content.

[1026] Specific examples

[1027] While User A is browsing a shopping app on his smartphone, the server detects that his heart rate is higher than normal. The server determines this as stress and displays product advertisements on the smartphone that will help relieve stress.

[1028] Prompt Sentence Examples

[1029] "Collect user behavioral and emotional data and display ads that match your relaxed mood based on your emotional state."

[1030] This system enables optimal information to be provided based on the user's behavior and emotions, enhancing the user experience.

[1031] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1032] Step 1:

[1033] The user device collects user behavioral data and biometric data. Specifically, it records in real time the URLs of websites visited by the user, the time of visit, the applications used, purchase history, heart rate, skin temperature, and facial recognition results. The input is the user's behavior and signals obtained from biometric sensors, and the output is a set of collected data.

[1034] Step 2:

[1035] The user device sends the collected behavioral data and biometric data to the server via secure communication. The communication is encrypted using the HTTPS protocol. The input is the set of collected data, and the output is the data sent to the server.

[1036] Step 3:

[1037] The server inputs the received behavioral data and biometric data into the emotion engine. The emotion engine uses a machine learning model (TensorFlow or PyTorch) to analyze the data and identify the user's emotional state. The analysis process identifies "relaxed" or "stressed" based on heart rate, skin temperature, and facial recognition results. The input is the integrated data sent to the server, and the output is the analyzed user's emotional state.

[1038] Step 4:

[1039] The server searches for relevant information from a database based on the user's emotional state and behavioral data identified by the emotion engine, generates summary content, and selects personalized advertisements that match the user's specific emotional state. The input is the emotional state and behavioral data, and the output is the summary content and personalized advertisements.

[1040] Step 5:

[1041] The server generates attractive thumbnails for the generated summary content and advertisements based on the user's emotional state, providing a display format that is intuitively understandable to users. The input is the summary content and personalized advertisements, and the output is the summary content with thumbnails and advertisements.

[1042] Step 6:

[1043] The server transmits the summary content with thumbnails and the personalized advertisement to the user terminal by push notification, where the input is the summary content with thumbnails and the advertisement, and the output is the notification sent to the user terminal.

[1044] Step 7:

[1045] The user terminal displays the received summary content with thumbnails and advertisements. The user can view the summary content or advertisement details by clicking on the displayed thumbnail. The input is the pushed content and advertisement, and the output is the visual information displayed to the user.

[1046] This system enables optimal information provision and advertisement delivery based on user behavior and emotions, improving the user experience.

[1047] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1048] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1049] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1050] [Fourth embodiment]

[1051] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1052] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1053] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1054] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1055] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1056] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1057] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1058] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1059] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1060] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1061] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1062] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1063] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1064] The present invention relates to a system and an operating method thereof for enabling a user to efficiently obtain personalized information. The basic components for implementing the present invention are a user terminal, a server, and data exchanged between them.

[1065] User terminal operation

[1066] User devices collect behavioral data when users use the Internet or applications. This behavioral data includes the websites visited, the URLs of pages viewed, the duration of visits, the applications used, and purchase history. This data is periodically sent to the server. Users can also register categories of interest and specific information.

[1067] Example: If a user frequently visits travel sites, their URLs are collected and sent to the server.

[1068] Server Operation

[1069] The server receives the behavioral data sent from the user's device and stores it in a database. The collected data is then analyzed. Machine learning algorithms and natural language processing techniques are used to identify the user's interests. Based on the results of this analysis, related information is collected from the Internet and summary content is generated.

[1070] Example: A server analyzes data from a user's repeated visits to a travel site and identifies that the user is interested in "travel." Based on this information, it gathers the latest travel guide articles and travel deals from the Internet and creates a compact summary.

[1071] Summary content generation and delivery

[1072] The generated summary content is also processed as thumbnails and delivered to the user terminal. These thumbnails contain excerpts of the summary content and visual elements designed to attract the user's attention. The user terminal receives these thumbnails and displays them to the user.

[1073] Example: A summary article of "Top 10 recommended tourist spots in Tokyo" generated by the server is displayed on the user's smartphone screen along with a thumbnail image. By clicking on this, the user can quickly view the summarized information.

[1074] User operations

[1075] Users can click on the thumbnails to view the summary content. This is a very intuitive operation, allowing users to access the information they need in a short amount of time, greatly improving the efficiency of information gathering. Users can also select new categories based on their own interests.

[1076] Example: When a user clicks on the thumbnail of "Top 10 Recommended Tourist Spots in Tokyo," they can immediately view summarized information about the tourist spots, which can be useful when planning their trip.

[1077] This system allows users to efficiently gather the information they need in a short amount of time, even in the midst of an information overload, greatly simplifying everyday information gathering.

[1078] The processing flow will be explained below.

[1079] Step 1:

[1080] Users generate behavioral data when they browse the web or use apps, including the URLs of websites visited, the duration of their visits, the applications used, and their purchase history.

[1081] Step 2:

[1082] The device automatically collects the generated behavioral data, and the collection process runs in the background and is designed to not interfere with user operations.

[1083] Step 3:

[1084] The device sends the collected behavioral data to a server at regular intervals or when a certain amount of data has been accumulated. The transmission is done using a secure communication protocol such as HTTPS.

[1085] Step 4:

[1086] The server stores the behavioral data received from the user device in a database, standardizing the data format and performing data cleansing.

[1087] Step 5:

[1088] The server launches a data analysis module to analyze the received behavioral data, using machine learning algorithms and natural language processing techniques to identify the user's interests.

[1089] Step 6:

[1090] Based on the analysis results, the server collects relevant information from the internet, using web scraping and APIs to retrieve information such as the latest news articles, blog posts, and product reviews.

[1091] Step 7:

[1092] The server generates summary content from the collected relevant information, using a text summarization algorithm to summarize lengthy information in a concise format.

[1093] Step 8:

[1094] The server generates a thumbnail for the summary content, which includes a text and visual image of a portion of the summary content.

[1095] Step 9:

[1096] The server delivers the generated summary content and thumbnails to the user terminal, adjusting the content delivery order to be optimal based on the user's interests.

[1097] Step 10:

[1098] The terminal displays the received summary content and thumbnails to the user. The user interface is intuitive, allowing users to easily find the information they are looking for.

[1099] Step 11:

[1100] Users can click on the displayed thumbnails to view the summary content, which allows users to quickly obtain related information.

[1101] The above steps realize a system that allows users to quickly and efficiently view summarized content based on behavioral data.

[1102] Example 1

[1103] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1104] Conventional information provision systems have difficulty quickly and appropriately providing information that meets users' diverse interests. Furthermore, due to the vast amount of information available, users may take a significant amount of time to obtain the information they need. Furthermore, there is a lack of technology that efficiently utilizes user behavior data to provide personalized content, creating a need for improved user experience.

[1105] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1106] In this invention, the server includes means for analyzing user behavior data and identifying user interests, means for collecting related information from the Internet based on the identified user interests and generating summary content, and means for converting the generated summary content into a visually appealing thumbnail format and delivering and displaying it on the user terminal, thereby enabling users to quickly and efficiently obtain information that matches their interests.

[1107] "User behavioral data" refers to information such as the URLs of websites visited when a user uses the Internet or applications, the URLs of pages viewed, the time of visit, the name and duration of use of the application used, and purchase history.

[1108] The "server" is a central computer system that receives and analyzes behavioral data sent from user terminals, collects related information, and generates summary content.

[1109] "Means for analysis" refers to the process and program for identifying user interests and concerns from collected behavioral data using machine learning algorithms and natural language processing techniques.

[1110] "Means of collecting relevant information" refers to processes such as web scraping techniques and API access to obtain information related to the user's interests from the Internet.

[1111] The "means for generating summary content" is the process of creating content based on collected related information to enable users to understand important information in a short amount of time.

[1112] "Thumbnail processing" is the process by which the generated summary content is converted into a visually appealing and usable format.

[1113] The "means for delivering and displaying" refers to a process for transmitting the generated summary content to a user terminal and displaying it on the screen of the terminal.

[1114] A "secure protocol" is a communication protocol for securely sending and receiving data, an example of which is HTTPS.

[1115] A "machine learning algorithm" is a program that finds patterns and rules from large amounts of data and makes predictions and classifications.

[1116] "Natural language processing technology" is a technology for analyzing and understanding human language.

[1117] The present invention relates to a system and an operating method for enabling users to efficiently obtain personalized information. To implement the system, the following components are required: a user terminal, a server, and data exchanged between them.

[1118] User terminal operation

[1119] User devices collect behavioral data when users use the Internet or applications. This behavioral data includes the websites visited, the URLs of pages viewed, the duration of visits, the applications used, and purchase history. This data is periodically sent to the server. Users can also register categories of interest and specific information.

[1120] Example: If a user frequently visits travel sites, their URLs are collected and sent to the server.

[1121] Server Operation

[1122] The server receives behavioral data sent from the user's device and stores it in a database. The collected data is then analyzed using machine learning algorithms and natural language processing techniques (e.g., Scikit-learn, NLTK) implemented in Python to identify the user's interests. Based on the results of this analysis, related information is collected from the Internet and summary content is generated using Python.

[1123] Example: A server analyzes data on a user's repeated visits to a travel site and identifies that the user is interested in "travel." Based on this information, it collects the latest travel guide articles and travel deals from the Internet and creates summary content using Python libraries (e.g., BeautifulSoup, Requests).

[1124] Summary content generation and delivery

[1125] The generated summary content is also processed as thumbnails and delivered to the user terminal. These thumbnails contain excerpts of the summary content and visual elements designed to attract the user's attention. The user terminal receives these thumbnails and displays them to the user.

[1126] Example: A summary article of "Top 10 recommended tourist spots in Tokyo" generated by the server is displayed on the user's smartphone screen along with a thumbnail image. By clicking on this, the user can quickly view the summarized information.

[1127] User operations

[1128] Users can click on the thumbnails to view the summary content. This is a very intuitive operation, allowing users to access the information they need in a short amount of time, greatly improving the efficiency of information gathering. Users can also select new categories based on their own interests.

[1129] Example: When a user clicks on the thumbnail of "Top 10 Recommended Tourist Spots in Tokyo," they can immediately view summarized information about the tourist spots, which can be useful when planning their trip.

[1130] Prompt Sentence Examples

[1131] Here are some example prompts to input to a generative AI model:

[1132] "We've determined that the user is interested in travel. How can we collect the latest travel guide articles and travel deals and provide them as summary content?"

[1133] This system allows users to efficiently gather the information they need in a short amount of time, even in the midst of an information overload, greatly simplifying everyday information gathering.

[1134] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1135] Step 1:

[1136] The user device collects user behavior data, including the URLs of visited websites, URLs of viewed pages, visit times, names of applications used, and purchase history. This data is recorded in local storage in real time.

[1137] Input: User internet and application behavior

[1138] Output: Behavioral data recorded in local storage

[1139] Specific operation: When a user visits an online shopping site, the URL of the site, the URL of the product page viewed, and the time of the visit are recorded on the device.

[1140] Step 2:

[1141] The collected behavioral data is periodically sent to a server using the secure HTTPS protocol. The data is converted to JSON format before transmission.

[1142] Input: Behavioral data recorded in local storage

[1143] Output: JSON formatted behavior data sent to the server

[1144] Specific operation: The device sends the collected data to the server's API endpoint using HTTPS at regular intervals (e.g., every hour). The data is in JSON format and includes a "URL," "start time," and "end time."

[1145] Step 3:

[1146] The server receives the behavioral data sent from the user terminal and stores it in a database, which is an SQL-based database (e.g., PostgreSQL).

[1147] Input: JSON formatted behavior data sent from the device

[1148] Output: Behavioral data stored in a database (PostgreSQL) table

[1149] Specific operation: The server's API endpoint parses the received JSON data and stores the data in the "UserActions" table in the PostgreSQL database using an INSERT query.

[1150] Step 4:

[1151] The server analyzes the behavioral data stored in the database using machine learning algorithms and natural language processing techniques (e.g., Scikit-learn, NLTK) implemented in Python.

[1152] Input: Behavioral data stored in a database

[1153] Output: Analysis of user interests and concerns

[1154] How it works: The server analyzes behavioral data collected over the past month to identify categories that users repeatedly visit. It then uses Scikit-learn to perform clustering analysis and identify user interests.

[1155] Step 5:

[1156] The server collects relevant information from the Internet based on the analysis results. Web scraping technology (e.g., BeautifulSoup, Scrapy) is used to collect information. The collected information is summarized and summary content is generated using Python libraries (e.g., Gensim, Sumy).

[1157] Input: Parsed user interests

[1158] Output: Summarized content

[1159] What it does: The server collects the latest articles about the specified "travel" from the Internet, parses them with BeautifulSoup, and then creates summary content using the Gensim library.

[1160] Step 6:

[1161] The generated summary content is processed into thumbnail format and delivered to the user's terminal. The thumbnail contains an excerpt of the summary content and visual elements.

[1162] Input: Summarized content

[1163] Output: Summary content in thumbnail format

[1164] Specific operation: The server creates thumbnails by combining a summary article of "Top 10 tourist spots in Tokyo" with related images, and sends them to the user's device.

[1165] Step 7:

[1166] The user terminal displays the received thumbnail to the user, who can then click on the thumbnail to view the detailed summary content.

[1167] Input: Summary content in thumbnail format

[1168] Output: Thumbnail and detailed summary content displayed to the user

[1169] What it does: When a user clicks on the thumbnail of "Top 10 tourist spots in Tokyo," a browser opens and displays a detailed page of the tourist spot with a summary.

[1170] This system's series of processes allows users to efficiently collect information based on their interests and obtain the information they need in a short amount of time.

[1171] (Application example 1)

[1172] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1173] In today's world where Internet usage is widespread, users can access a large amount of information, but it is extremely difficult to efficiently collect the information they need from that vast amount of information. Furthermore, in the case of advertising delivery, advertisements that do not match the user's interests are often displayed, causing user stress and reducing the effectiveness of advertising. The present invention aims to solve these problems by providing a system that effectively utilizes user behavior data to display personalized advertisements.

[1174] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1175] In this invention, the server includes means for collecting user behavioral data, means for analyzing the collected behavioral data and identifying the user's interests, means for acquiring related information and generating summary content based on the identified user's interests, means for processing the generated summary content as thumbnails and delivering them to the user terminal, means for allowing the user to view detailed information by clicking on the thumbnails, and means for allowing the user to select a new category and update their interests. This provides personalized information and advertisements based on the user's behavioral data, allowing users to efficiently access the information they need. It also enables advertisers to deliver advertisements more effectively, which is expected to improve advertising effectiveness.

[1176] "User behavioral data" is information generated when a user uses the Internet or applications, and specifically includes data such as websites visited, URLs of pages viewed, time of visit, applications used, and purchase history.

[1177] "Collection means" refers to software and hardware for collecting user behavior data in real time or periodically.

[1178] "Means of analysis" refers to the algorithms and analytical tools used to analyze collected behavioral data to identify user interests and concerns.

[1179] "Means for acquiring related information based on the identified user's interests" refers to a function for collecting related information from the Internet, etc., based on the user's interests and concerns.

[1180] "Means for generating summary content" refers to a function that compactly summarizes acquired related information and converts it into a format that is easy for the user to understand.

[1181] The "means for processing the summary content as a thumbnail and delivering it to a user terminal" refers to a method for displaying the generated summary content in a visually appealing format and transmitting it to a user terminal.

[1182] "User terminal" refers to a device that is directly operated by a user, such as a smartphone, tablet, or PC.

[1183] "Means for viewing detailed information" refers to the ability to display the full content and related detailed information when a user clicks on a thumbnail.

[1184] "Means for selecting new categories and updating interests" refers to an interface that allows users to select new information categories based on their own interests and have them reflected in the system.

[1185] The "means for delivering in the form of push notification" refers to a function for automatically notifying a user terminal of a thumbnail of the generated summary content.

[1186] The present invention relates to a system that generates personalized summary content by collecting and analyzing user behavior data and efficiently delivers the content to users. This system basically consists of a user terminal, a server, and data exchanged between them.

[1187] User terminal operation

[1188] User devices collect behavioral data when users use the Internet or applications. This behavioral data includes the URLs of websites visited, pages viewed, visit times, applications used, purchase history, etc. This data is periodically sent to a server where it is analyzed.

[1189] Server Operation

[1190] The server receives behavioral data sent from the user's device and stores it in a database. The collected data is then analyzed to identify the user's interests. The analysis method used is machine learning algorithms using Python's Scikit-learn library, as well as natural language processing techniques using Python's NLTK library and generative AI models such as GPT-3. Based on this, relevant information is collected from the Internet and summary content is generated.

[1191] Summary content generation and delivery

[1192] The generated summary content is then processed into a thumbnail before being delivered to the user's device. This thumbnail includes an excerpt of the summary content and visual elements designed to pique the user's interest. The thumbnail is then delivered to the user's device, such as a smartphone, tablet, or PC, via push notification.

[1193] User operations

[1194] Users can view the summary content by clicking on the displayed thumbnail. This operation allows users to access the information they need quickly and efficiently. Furthermore, users can select new categories based on their interests and have the system reflect that information.

[1195] Specific examples

[1196] For example, if a user frequently visits a travel site, this behavioral data is collected and sent to a server. The server analyzes this data and identifies the user as being interested in "travel." The server then collects the latest travel guide articles and travel deals from the Internet and generates summarized content. This summarized content is then processed into thumbnails and delivered to the user's smartphone as a push notification. When the user clicks on this notification, they can immediately view the summarized information.

[1197] Hardware and software used

[1198] User devices: smartphones (iOS or Android), tablets, computers, etc.

[1199] Server: High-performance cloud server (e.g. AWS EC2)

[1200] Machine learning algorithms: Python's Scikit-learn library

[1201] Natural language processing technology: Python's NLTK library, generative AI models such as GPT-3

[1202] Database: MySQL or a NoSQL database (e.g. MongoDB)

[1203] Examples of prompt statements

[1204] "Generate ads for frequent travel site visitors whose behavioral data includes visits to multiple travel sites. Create a summary ad with a visual element suitable for a thumbnail."

[1205] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1206] Step 1: Collect user behavior data

[1207] The user device collects user behavior data. This behavior data includes the URLs of visited websites, pages viewed, visit times, applications used, purchase history, etc. These data are temporarily stored in local files. The input data is the user behavior data, and the output data is the collected behavior data.

[1208] Step 2: Sending behavioral data to the server

[1209] The terminal periodically transmits the collected behavioral data to the server. This process uses a scheduling algorithm to transmit data at specified time intervals. The input data is the collected behavioral data, and the output data is the behavioral data transmitted to the server.

[1210] Step 3: Analyzing behavioral data

[1211] The server receives the behavioral data and stores it in a database. Next, it uses machine learning algorithms and natural language processing techniques to identify the user's interests. This analysis is performed using Python's Scikit-learn and NLTK libraries. The input data is the behavioral data sent to the server, and the output data is the analysis results.

[1212] Step 4: Obtaining relevant information and generating summary content

[1213] The server collects relevant information from the Internet based on the identified user interests. The retrieved information is summarized using a generative AI model (e.g., GPT-3). Specific prompts are used to generate the summary. The input data are the analysis results and the collected relevant information, and the output data is the summary content.

[1214] Step 5: Generate and edit thumbnails

[1215] The server processes the generated summary content into an attractive thumbnail, which contains an excerpt and visual elements of the summary content. The input data is the summary content, and the output data is the processed thumbnail.

[1216] Step 6: Serving and displaying thumbnails

[1217] The server delivers the processed thumbnail to the user's device in the form of a push notification. The user receives this notification on their device and the thumbnail is displayed. The input data is the processed thumbnail, and the output data is the thumbnail displayed on the user's device.

[1218] Step 7: User interaction

[1219] The user can click on the displayed thumbnails to view the summary content, and can also update their interests by selecting new categories. The input data are the user's clicks and selections, and the output data are the viewed summary content and updated interest data.

[1220] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1221] The present invention relates to a system and its operation method for efficiently obtaining personalized information by utilizing behavioral data and emotional states of users. The basic components for implementing the present invention include a user terminal, a server, and an emotion engine.

[1222] User terminal operation

[1223] User devices collect behavioral data when users use the internet and applications. This behavioral data includes the URLs of websites visited, the duration of visits, the applications used, and purchase history. Biometric sensors (e.g., heart rate, skin temperature, and facial recognition) can also be used to capture user emotions in real time.

[1224] Example: While a user is busy visiting a travel site, an increase in heart rate is detected, which may indicate an emotional state such as excitement or anticipation.

[1225] Emotion Engine Operation

[1226] The emotion engine analyzes behavioral data and biometric sensor data sent from the user's device to identify the user's emotional state. The engine uses machine learning algorithms to determine the user's emotional state and transmits that information to the server.

[1227] Example: The emotion engine combines and analyzes the user's heart rate data, skin temperature, and facial recognition results to determine whether the user is in a state of "expectation."

[1228] Server Operation

[1229] The server receives behavioral and emotional data sent from the user device and the emotion engine. It stores these data in a database and activates a data analysis module to identify the user's interests and emotions. Based on the analysis results, it collects relevant information from the Internet and generates summary content.

[1230] Example: A server identifies travel interests from user behavioral data and detects signs of "expectation" from emotional data. Based on this information, it collects the latest travel guides and discount plans and summarizes them as content with high expectations.

[1231] Summary content generation and delivery

[1232] The server generates summary content from the collected related information, summarizes long-form information in a concise format using a text summarization algorithm, and creates thumbnails of the summary content that match the user's emotional state and delivers them to the user's device.

[1233] Example: The server generates a summary article for "Top 10 recommended tourist spots in Tokyo" and creates thumbnails that express the user's "expectation." For example, thumbnails are created that combine photos of attractive tourist spots with phrases that evoke a sense of anticipation.

[1234] User operations

[1235] Users can view the summary content by clicking on the thumbnails displayed on their devices. This operation is intuitive, and users can quickly obtain the information they need. In addition, information provided is adapted to the user's emotional state, providing a more satisfying user experience.

[1236] Example: When a user clicks on the thumbnail for "Top 10 Recommended Tourist Spots in Tokyo," they can instantly see a summary of the information and use it to help with their planning.

[1237] By combining and analyzing behavioral data and emotional data, this system makes it possible to quickly provide users with the information that is most relevant and matches their emotions, improving the user experience and significantly increasing the efficiency of information gathering.

[1238] The processing flow will be explained below.

[1239] Step 1:

[1240] When users browse the web or use applications, behavioral data is generated, including the URLs of websites visited, the duration of visits, applications used, and purchase history.

[1241] Step 2:

[1242] The device automatically collects user behavior data. This collection process runs in the background and is designed to not interfere with user operations. Data is also collected from biometric sensors (heart rate, skin temperature, facial recognition, etc.) installed on the device.

[1243] Step 3:

[1244] The device sends the collected behavioral data and biometric sensor data to a server at regular intervals or when a certain amount of data has been accumulated. The transmission is carried out using a secure communication protocol such as HTTPS.

[1245] Step 4:

[1246] The server stores the behavioral data and biometric sensor data received from the user device in a database, standardizing the data format and performing data cleansing.

[1247] Step 5:

[1248] The server launches a data analysis module to analyze the received data, using machine learning algorithms and natural language processing techniques to identify the user's interests and emotional state.

[1249] Step 6:

[1250] The emotion engine analyzes the behavioral data and biometric sensor data to identify the user's emotional state (e.g., joy, anticipation, surprise, etc.), and the analysis results are also sent to the server.

[1251] Step 7:

[1252] Based on the analysis results, the server collects relevant information from the internet, using web scraping and APIs to retrieve information such as the latest news articles, blog posts, and product reviews.

[1253] Step 8:

[1254] The server generates summary content from the collected relevant information, uses a text summarization algorithm to summarize long-form information in a concise format, and creates thumbnails of the summary content that are adaptive to the user's emotional state.

[1255] Step 9:

[1256] The server delivers the generated summary content and thumbnails to the user terminal, adjusting the content delivery order to be optimal based on the user's interests and emotions.

[1257] Step 10:

[1258] The terminal displays the received summary content and thumbnails to the user. The user interface is intuitive, allowing users to easily find the information they are looking for.

[1259] Step 11:

[1260] Users can click on the displayed thumbnails to view the summary content, allowing them to quickly obtain relevant information. Furthermore, the displayed information is adapted to the user's emotional state, resulting in a more satisfying information experience.

[1261] The above steps realize a system that allows users to quickly and efficiently view summarized content based on behavioral data and emotion data.

[1262] Example 2

[1263] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1264] Conventional information provision systems provide relevant information based solely on user behavioral data, often without considering the user's emotional state. As a result, personalized information provision that adapts to the user's emotions is lacking, and an improved user experience is desired. In addition, since the amount of information tends to be large, a means of efficiently summarizing and providing information is also required.

[1265] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1266] In this invention, the server includes means for collecting user behavioral data and biometric sensor data, means for analyzing the collected behavioral data and biometric sensor data to identify the user's emotional state, means for acquiring related information and generating summary content based on the identified user's interests and emotional state, and means for creating thumbnails of the generated summary content that match the user's emotional state and delivering them to the user terminal. This makes it possible to provide information that is adapted to the user's emotional state, thereby improving the user experience and significantly increasing the efficiency of information collection.

[1267] "User behavioral data" refers to data generated when a user uses the Internet or applications, such as the URLs of websites visited, the time of visit, applications used, and purchase history.

[1268] "Biometric sensor data" refers to data used to acquire a user's biometric information in real time, specifically including heart rate, skin temperature, and facial expression recognition results.

[1269] "Emotion Engine" refers to the algorithms and systems that analyze collected behavioral and biometric sensor data to identify a user's emotional state.

[1270] "Summary content" refers to information in which related information is reconstructed into a concise format using a summarization algorithm, allowing users to obtain information efficiently.

[1271] "Thumbnail" refers to a visually appealing image and short descriptive text related to the summarized content that matches the user's emotional state.

[1272] A "user terminal" is a device that a user uses to receive and view information, and specifically refers to a smartphone, tablet, PC, etc.

[1273] The present invention relates to a system and its operating method for efficiently providing personalized information by utilizing user behavioral data and biometric sensor data. The main components include a user terminal, a server, and an emotion engine. Each component and its operation will be described in detail below.

[1274] User terminal operation

[1275] User devices collect behavioral data when users use the internet and applications. This behavioral data includes the URLs of websites visited, the duration of visits, applications used, and purchase history. Biosensors (e.g., heart rate, skin temperature, facial expression recognition) can also be used to collect real-time emotional data.

[1276] Example: A user visits a travel site and while viewing a page about a particular tourist destination, the device's biometric sensors detect an increase in heart rate, which indicates emotions such as excitement and anticipation.

[1277] Emotion Engine Operation

[1278] The emotion engine analyzes behavioral data and biometric sensor data sent from the user's device to identify the user's emotional state. The engine uses machine learning algorithms (e.g., TensorFlow, PyTorch) to determine the user's emotional state. The identified emotional state information is then sent to the server.

[1279] Example: An emotion engine combines and analyzes heart rate data, skin temperature, and facial expression recognition results to determine when a user is in a state of "expectation."

[1280] Server Operation

[1281] The server receives behavioral and emotional data sent from the user device and the emotion engine. The received data is stored in a database (e.g., MySQL, PostgreSQL). The data analysis module then analyzes the data to identify the user's interests and emotional state. Based on the analysis results, relevant information is collected from the Internet and summary content is generated.

[1282] Example: The server stores the received data in a database, and the data analysis module identifies the user's travel interest from their recent web browsing history and detects the emotion "anticipation." Based on this information, the latest travel guides and discount plan information are collected and reconstructed as summary content.

[1283] Summary content and thumbnail generation

[1284] The server generates summary content based on the collected relevant information. It uses a text summarization algorithm (e.g., BERT summarizer) to condense long-form information into a concise format. It also creates thumbnails of the generated summary content that match the user's emotional state. It uses an image generation algorithm or image editing software (e.g., Photoshop API) to create the thumbnails.

[1285] Example: A server converts a detailed article about "Top 10 tourist spots in Tokyo" into a concise summary and generates an attractive thumbnail to create excitement.

[1286] Summary content and thumbnail delivery

[1287] The server delivers the generated summary content and thumbnails to the user device using push notifications or real-time communication protocols (e.g., WebSocket).

[1288] Example: The server delivers summary content and thumbnails to the user's smartphone app via push notification.

[1289] User operations

[1290] Users can view the summary content by clicking on the thumbnails displayed on their devices. This intuitive operation allows users to quickly obtain the information they need.

[1291] Example: A user clicks on a thumbnail of "Top 10 recommended tourist spots in Tokyo" to view summarized information and use it to help plan their trip.

[1292] Example prompt: "The emotion engine has determined that the user's heart rate increases when viewing a particular web page, indicating that the user is excited."

[1293] As a result, this system combines and analyzes the user's behavioral data and emotional data, and provides information adapted to the user's emotional state, thereby improving the user experience and making information gathering more efficient.

[1294] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1295] Step 1:

[1296] The user device collects user behavioral data and biometric sensor data, including the URLs of visited websites, the time of visit, the applications used, and purchase history. It also collects biometric sensor data (heart rate, skin temperature, and facial expression recognition) in real time.

[1297] Input: The actions and physiological responses of the body as users interact with the internet and applications.

[1298] Output: URL list of visited sites, visit times, app usage history, biometric sensor data.

[1299] Example of how it works: A user visits a travel website and while browsing a page about a particular tourist destination, an increase in heart rate is detected. This data is collected from the device.

[1300] Step 2:

[1301] The user device encrypts the collected behavioral data and biometric sensor data and sends it to the server using HTTPS as the communication protocol.

[1302] Input: Behavioral and biosensor data collected in the previous step.

[1303] Output: Encrypted behavioral and biometric sensor data.

[1304] Specific operation example: The list of URLs visited by the device and heart rate change data are encrypted in JSON format and sent to the server using the HTTPS protocol.

[1305] Step 3:

[1306] The emotion engine analyzes the behavioral data and biometric sensor data provided by the server to identify the user's emotional state. It analyzes the data using machine learning algorithms (e.g., TensorFlow, PyTorch).

[1307] Input: Decrypted behavioral and biometric sensor data.

[1308] Output: Identification of the user's emotional state (e.g., anticipation, excitement, etc.).

[1309] Specific example of operation: The emotion engine combines and analyzes heart rate data, skin temperature, and facial expression recognition results to determine that the user is in an "expectant" state, and sends the result to the server.

[1310] Step 4:

[1311] The server stores the data sent from the user terminal and the emotion engine in a database and activates a data analysis module, which analyzes the data and identifies the user's interests and emotional state.

[1312] Input: Stored behavioral and emotional data.

[1313] Output: Analysis results identifying the user's interests and emotional state.

[1314] Specific example of operation: The server identifies travel interest from user behavioral data and detects signs of "expectation" from emotional data.

[1315] Step 5:

[1316] The server collects relevant information from the Internet based on the identified user's interests and emotional state, using web crawlers and external APIs (e.g., travel information APIs) to obtain the latest information.

[1317] Input: Analysis results identifying the user's interests and emotional state.

[1318] Output: Related information (e.g. travel guide, discount offers, etc.).

[1319] Example of how it works: The server queries the travel information API to get the latest travel guides and discount plan information.

[1320] Step 6:

[1321] The server generates summary content from the collected information, using a text summarization algorithm (e.g., BERT summarizer) to summarize the information in a concise format.

[1322] Input: Related information (e.g. travel guide, discount offers, etc.).

[1323] Output: Summary content.

[1324] Example of how it works: The server converts a detailed article about "Top 10 tourist spots in Tokyo" into a concise summary.

[1325] Step 7:

[1326] The server creates thumbnails that match the emotional state of the generated summary content using image generation algorithms and image editing software (e.g., Photoshop API).

[1327] Input: Summary content, and the user's emotional state.

[1328] Output: Thumbnails adapted to the emotional state.

[1329] Specific example of how it works: The server generates thumbnails that combine photos of attractive tourist spots with phrases that evoke excitement.

[1330] Step 8:

[1331] The server delivers the generated summary content and thumbnails to the user device using push notifications or real-time communication protocols (e.g., WebSocket).

[1332] Input: Abstract content and thumbnail.

[1333] Output: Delivered summary content and thumbnails.

[1334] Specific example of operation: The server delivers summary content and thumbnails to the user's smartphone app via push notification.

[1335] Step 9:

[1336] Users can view the summary content by clicking on the thumbnails displayed on their devices. This intuitive operation allows users to quickly obtain the information they need.

[1337] Input: The thumbnail displayed on the device.

[1338] Output: User viewing summary content.

[1339] Example of specific operation: A user clicks on the thumbnail of "Top 10 recommended tourist spots in Tokyo" and views the summarized information to help plan their trip.

[1340] (Application example 2)

[1341] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1342] Conventional systems provide information based solely on user behavior data, which means that they do not adequately consider the user's emotional state, making it difficult to maximize user satisfaction. Furthermore, personalized advertising based on emotions is not available, making it difficult to maximize advertising effectiveness.

[1343] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1344] In this invention, the server includes means for collecting user behavioral data and biometric data, means for analyzing the collected behavioral data and biometric data to identify the user's emotional state and interests, means for acquiring related information and generating summary content based on the identified user's emotional state and interests, and means for delivering and displaying personalized advertisements based on the generated summary content and emotional state to the user terminal, thereby enabling the provision of information and personalized advertisements optimal for the user's emotional state.

[1345] "User behavior data" is information that records a user's activities on the Internet, such as website browsing history, application usage history, and purchase history.

[1346] "Biometric data" is information that represents a user's biological state or reaction, such as the user's heart rate, skin temperature, or facial recognition results.

[1347] "Emotional state" is information that indicates the user's current psychological state, which is analyzed from the user's biometric data.

[1348] "Relevant information" is information that is determined to be useful or interesting to the user based on the user's behavioral data and emotional state.

[1349] "Summary content" is information that succinctly summarizes related information and is generated so that users can understand it in a short amount of time.

[1350] "Personalized advertising" refers to advertising that is optimized based on a user's behavioral data and emotional state, and is determined to be highly relevant to each individual user.

[1351] A "user terminal" is a communication device such as a smartphone or tablet used by a user, and is a device that collects and displays information.

[1352] The present invention relates to a system for collecting and analyzing user behavioral data and biometric data, and delivering personalized advertisements based on the user's emotional state and interests. Specific embodiments for implementing the present invention will be described below.

[1353] Hardware and software used

[1354] The system of the present invention mainly comprises a user terminal, a server, and an emotion engine.

[1355] User device: A communication device connected to the Internet, such as a smartphone or tablet, that collects user behavioral data and biometric data and sends it to a server.

[1356] Server: Equipped with a database, data analysis module, and ad serving module, and uses a Python-based web framework such as Flask or Django.

[1357] Emotion Engine: Uses TensorFlow and PyTorch to implement machine learning models.

[1358] System Operation

[1359] 1. Data Collection:

[1360] The user device collects behavioral data such as the user's website browsing history, app usage history, and purchase history, as well as biometric data such as heart rate, skin temperature, and facial recognition results in real time. For example, it records the user's history of using a shopping app and the timing of when their heart rate increased.

[1361] 2. Data transmission:

[1362] The collected data is sent to the server via secure communication using HTTPS, and the data is encrypted.

[1363] 3. Emotion analysis:

[1364] The server inputs the received data into an emotion engine to analyze the user's emotional state, for example, determining whether the user is relaxed or stressed based on a combination of heart rate and skin temperature.

[1365] 4. Advertisement Selection:

[1366] Based on the emotional state analyzed by the emotion engine, the server searches for relevant information from the database and selects the most suitable personalized advertisement for the user, for example, if the user is relaxed, it will select a refreshing travel advertisement.

[1367] 5. Advertisement Delivery:

[1368] The selected advertisement is then pushed to the user's device and displayed there. At this time, a thumbnail is also generated based on the user's emotional state to make it easier for the user to intuitively understand the advertisement content.

[1369] Specific examples

[1370] While User A is browsing a shopping app on his smartphone, the server detects that his heart rate is higher than normal. The server determines this as stress and displays product advertisements on the smartphone that will help relieve stress.

[1371] Prompt Sentence Examples

[1372] "Collect user behavioral and emotional data and display ads that match your relaxed mood based on your emotional state."

[1373] This system enables optimal information to be provided based on the user's behavior and emotions, enhancing the user experience.

[1374] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1375] Step 1:

[1376] The user device collects user behavioral data and biometric data. Specifically, it records in real time the URLs of websites visited by the user, the time of visit, the applications used, purchase history, heart rate, skin temperature, and facial recognition results. The input is the user's behavior and signals obtained from biometric sensors, and the output is a set of collected data.

[1377] Step 2:

[1378] The user device sends the collected behavioral data and biometric data to the server via secure communication. The communication is encrypted using the HTTPS protocol. The input is the set of collected data, and the output is the data sent to the server.

[1379] Step 3:

[1380] The server inputs the received behavioral data and biometric data into the emotion engine. The emotion engine uses a machine learning model (TensorFlow or PyTorch) to analyze the data and identify the user's emotional state. The analysis process identifies "relaxed" or "stressed" based on heart rate, skin temperature, and facial recognition results. The input is the integrated data sent to the server, and the output is the analyzed user's emotional state.

[1381] Step 4:

[1382] The server searches for relevant information from a database based on the user's emotional state and behavioral data identified by the emotion engine, generates summary content, and selects personalized advertisements that match the user's specific emotional state. The input is the emotional state and behavioral data, and the output is the summary content and personalized advertisements.

[1383] Step 5:

[1384] The server generates attractive thumbnails for the generated summary content and advertisements based on the user's emotional state, providing a display format that is intuitively understandable to users. The input is the summary content and personalized advertisements, and the output is the summary content with thumbnails and advertisements.

[1385] Step 6:

[1386] The server transmits the summary content with thumbnails and the personalized advertisement to the user terminal by push notification, where the input is the summary content with thumbnails and the advertisement, and the output is the notification sent to the user terminal.

[1387] Step 7:

[1388] The user terminal displays the received summary content with thumbnails and advertisements. The user can view the summary content or advertisement details by clicking on the displayed thumbnail. The input is the pushed content and advertisement, and the output is the visual information displayed to the user.

[1389] This system enables optimal information provision and advertisement delivery based on user behavior and emotions, improving the user experience.

[1390] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1391] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1392] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1393] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1394] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1395] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1396] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1397] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1398] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1399] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1400] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1401] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1402] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1403] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1404] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1405] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1406] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1407] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1408] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1409] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1410] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1411] The following is further disclosed regarding the above embodiment.

[1412] (Claim 1)

[1413] a means for collecting user behavior data;

[1414] means for analyzing collected behavioral data to identify user interests;

[1415] means for obtaining relevant information and generating summary content based on the identified user interests;

[1416] means for delivering and displaying the generated summary content to a user terminal;

[1417] A system including:

[1418] (Claim 2)

[1419] 10. The system of claim 1, further comprising means for transmitting the behavioral data from the user terminal to the server.

[1420] (Claim 3)

[1421] 10. The system of claim 1, further comprising means for generating and displaying an attractive thumbnail of the generated summary content on the user terminal.

[1422] "Example 1"

[1423] (Claim 1)

[1424] a means for collecting user behavior data;

[1425] means for transmitting the collected behavioral data to a server;

[1426] means for analyzing the collected behavioral data on the server to identify user interests;

[1427] means for collecting relevant information from the Internet and generating summary content based on the identified user's interests;

[1428] a means for processing the generated summary content into a visually appealing thumbnail format and delivering and displaying the thumbnail to a user terminal;

[1429] A system including:

[1430] (Claim 2)

[1431] 10. The system of claim 1, further comprising means for transmitting the behavioral data from the user terminal to the server using a secure protocol.

[1432] (Claim 3)

[1433] 10. The system of claim 1, further comprising means for analyzing the behavioral data to identify user interests using machine learning algorithms and natural language processing techniques.

[1434] "Application Example 1"

[1435] (Claim 1)

[1436] a means for collecting user behavior data;

[1437] means for analyzing collected behavioral data to identify user interests;

[1438] means for obtaining relevant information and generating summary content based on the identified user interests;

[1439] a means for processing the generated summary content into a thumbnail and delivering it to a user terminal;

[1440] A means for users to click on a thumbnail to view more information,

[1441] a means for users to select new categories and update their interests;

[1442] A system including:

[1443] (Claim 2)

[1444] 10. The system of claim 1, further comprising means for transmitting the behavioral data from the user terminal to the server.

[1445] (Claim 3)

[1446] 10. The system of claim 1, further comprising means for generating an attractive thumbnail for the generated summary content and delivering it to the user terminal in the form of a push notification.

[1447] "Example 2: Combining Emotion Engines"

[1448] (Claim 1)

[1449] means for collecting user behavioral data and biometric sensor data;

[1450] means for analyzing the collected behavioral data and biometric sensor data to identify the user's emotional state;

[1451] means for obtaining relevant information and generating summary content based on the identified user's interests and emotional state;

[1452] a means for generating a thumbnail of the generated summary content that matches the emotional state of the user and delivering the thumbnail to the user terminal;

[1453] A system including:

[1454] (Claim 2)

[1455] 10. The system of claim 1, further comprising means for transmitting the behavioral data and biometric sensor data from the user terminal to the server.

[1456] (Claim 3)

[1457] 2. The system according to claim 1, further comprising means for delivering the generated summary content to a user terminal and enabling intuitive browsing of information based on a user operation.

[1458] "Application example 2 when combining emotion engines"

[1459] (Claim 1)

[1460] means for collecting user behavioral data and biometric data;

[1461] means for analyzing the collected behavioral and biometric data to identify the user's emotional state and interests;

[1462] means for obtaining relevant information and generating summary content based on the identified user's emotional state and interests;

[1463] a means for delivering and displaying a personalized advertisement based on the generated summary content and the emotional state to a user terminal;

[1464] A system including:

[1465] (Claim 2)

[1466] 10. The system of claim 1, further comprising means for transmitting the behavioral data and the biometric data from the user terminal to the server.

[1467] (Claim 3)

[1468] 10. The system of claim 1, further comprising means for generating and displaying, on the user terminal, an attractive thumbnail for the advertisement based on the generated summary content and the emotional state. [Explanation of symbols]

[1469] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. a means for collecting user behavior data; means for analyzing collected behavioral data to identify user interests; means for obtaining relevant information and generating summary content based on the identified user interests; means for delivering and displaying the generated summary content to a user terminal; A system including:

2. The system of claim 1 , further comprising means for transmitting the behavioral data from the user terminal to the server.

3. The system of claim 1 , further comprising means for generating an attractive thumbnail for the generated summary content and displaying it on the user terminal.

Images