System

Abstract

To provide a system.SOLUTION: A system contains: means of estimating an emotion state of a user on the basis of the voice, text input, facial expression, and biological information on the user; means of collecting unread or unviewed information from a plurality of digital contents; and means of recommending an optimal content among the collected digital contents on the basis of the emotion state.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In today's society, where digital content is increasing, there is a problem that users are unable to digest the information they have saved to "read later" or "watch later," and end up leaving it unread or unwatched. [Means for solving the problem]

[0005] To solve this problem, we propose a system that collects unread or unwatched information and summarizes it using artificial intelligence. The system then converts the summarized information into audio and provides it to users in their free time, such as while doing housework or commuting. Furthermore, the system automatically collects recommended information based on the user's usage history and generates summary information that includes that information, thereby providing personalized information. [Brief explanation of the drawings]

[0006] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 2 is a sequence diagram showing a flow of processing in the data processing system according to the first embodiment of the first form example. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1 of Embodiment 1. [Figure 13] FIG. 10 is a sequence diagram showing a processing flow of a data processing system in a second embodiment of the second form example. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 of Embodiment Example 2. [Figure 15] FIG. 10 is a sequence diagram showing the flow of processing in a data processing system according to a third embodiment of the third embodiment. [Figure 16] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 3 of Embodiment 3. [Figure 17] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the first embodiment of the first form example when an emotion engine is combined. [Figure 18] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1 of Form Example 1 when an emotion engine is combined. [Figure 19] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the second embodiment of the second form example when an emotion engine is combined. [Figure 20] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 of Form Example 2 when an emotion engine is combined. [Figure 21] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the third embodiment of the third form example when an emotion engine is combined. [Figure 22] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 3 of Form Example 3 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0007] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0008] First, the terms used in the following description will be explained.

[0009] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, the processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (TENSOR PROCESSING UNIT (registered trademark)).

[0010] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0011] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0012] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0013] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0014] [First embodiment]

[0015] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0016] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0017] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0018] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0019] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0020] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0021] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0022] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0023] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0024] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0025] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0026] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0027] "Example 1"

[0028] In one embodiment of the present invention, unread or unwatched information is collected by using APIs that access various digital content platforms used by users, such as YouTube's "Watch Later" list, unread messages in messenger apps, and unread lists in bookmark services.

[0029] "Example 2"

[0030] Natural language processing technology is used as an artificial intelligence method to summarize collected information. Specifically, collected text information is analyzed and a summary is generated. This summary expresses the main content of the original information in a shortened form.

[0031] "Example 3"

[0032] Text-to-speech (TTS) technology is used to convert the summarized information into audio and output it. Specifically, the summarized text information is converted into audio data and provided in a form that the user can listen to.

[0033] "Example 4"

[0034] Machine learning technology is used to automatically collect recommended information based on a user's usage history and generate summary information that includes that information. Specifically, the system learns the user's past browsing history and preferences, and then collects and summarizes new recommended information based on that information. This makes it possible to efficiently provide useful information to users.

[0035] The processing flow of each embodiment will be described below.

[0036] "Example 1"

[0037] Step 1: We access the various digital content platforms you use to collect information, such as your YouTube Watch Later list, unread messages in your messaging app, or unread lists in your bookmarking services.

[0038] Step 2: Centralize unread or unwatched information obtained from each platform, allowing users to view information across multiple platforms in one place.

[0039] "Example 2"

[0040] Step 1: Analyze the collected information using natural language processing technology. Through this analysis, the main content of the information is extracted.

[0041] Step 2: Based on the extracted key content, a summary of the information is generated. This summary represents the main content of the original information in a shortened form.

[0042] "Example 3"

[0043] Step 1: Convert the summarized information into audio data using text-to-speech (TTS) technology.

[0044] Step 2: The converted audio data is provided to the user in a form that they can listen to, allowing them to digest the information during their free time, such as during housework or on their way to work.

[0045] "Example 4"

[0046] Step 1: Learn about the user's past browsing history and preferences using machine learning techniques.

[0047] Step 2: Based on the learned results, new recommendations are collected.

[0048] Step 3: The collected recommendation information is summarized and a summary containing it is generated, which makes it possible to efficiently provide useful information to users.

[0049] Example 1

[0050] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0051] Today's users use multiple digital content platforms, and unread or unwatched information often accumulates on each platform. However, there is a lack of a way to centrally manage this information and efficiently extract and provide important information. As a result, users are more likely to miss important information, and information management becomes cumbersome.

[0052] The identification process by the identification processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means. In this invention, the server includes a means for accessing multiple digital content platforms, a means for collecting unread or unviewed information, a means for converting the collected information into a unified format, a means for analyzing the converted information and extracting important information, and a means for providing the extracted information to the user. This allows the user to centrally manage unread or unviewed information from multiple platforms and efficiently check important information.

[0053] "Digital Content Platform" means an online service that enables users to view, listen to or store content in digital form.

[0054] "Means of access" refers to the methods and technologies by which a server connects to a particular digital content platform and obtains the required data.

[0055] "Unread or unwatched information" refers to digital content such as messages, videos, or articles that a user has not yet viewed.

[0056] "Means of collection" refers to the methods or technologies by which the server obtains unread or unwatched information from the digital content platform.

[0057] "Means of converting into a unified format" refers to methods and technologies for converting data collected from different platforms into a consistent format.

[0058] "Means of analysis" refers to the methods and techniques used to analyze the collected and transformed data and extract information that is important to users.

[0059] "Means of extracting important information" refers to methods and techniques for selecting information that is valuable to users from the analyzed data.

[0060] "Means of providing" refers to the methods and technologies used to notify or display extracted important information to the user.

[0061] MODE FOR CARRYING OUT THE INVENTION

[0062] This invention provides a system that efficiently collects unread or unviewed information from multiple digital content platforms used by users, extracts important information, and provides it to users. Specific embodiments of this system are described below.

[0063] System configuration

[0064] Hardware and software configuration:

[0065] The server uses the following hardware and software to implement the system.

[0066] Hardware: A server with a powerful processor, sufficient memory, and ample storage.

[0067] Software: Libraries for API access (e.g., HTTP client libraries), libraries for data analysis (e.g., Python's Pandas, NumPy), database management systems (e.g., MySQL (registered trademark), PostgreSQL)

[0068] Specific APIs:

[0069] Digital content platform APIs: YouTube Data API, Facebook Messenger API, Pocket API, etc.

[0070] Program processing

[0071] The server collects and processes data in the following manner:

[0072] 1. Prepare for API access:

[0073] The server obtains API keys and authentication information for each platform and prepares API access, such as setting an API key for the YouTube Data API, an access token for the Facebook Messenger API, and a consumer key for the Pocket API.

[0074] 2. Data Collection:

[0075] The server calls each platform's API to collect unread or unwatched content, such as YouTube's "Watch Later" list, Facebook Messenger's unread messages, and Pocket's unread bookmarks.

[0076] 3. Data Processing:

[0077] The server converts the collected data into a unified format and assigns unread or unwatched statuses. For example, it converts data obtained from each platform into JSON format.

[0078] 4. Data Analysis:

[0079] The server analyzes the collected data and extracts information that is important to the user, such as unread messages containing specific keywords.

[0080] 5. Providing results:

[0081] The server processes and analyzes the data and provides it to users, who can then check their unread or unwatched information through a dedicated dashboard and notifications.

[0082] Specific examples

[0083] If a user uses YouTube, Facebook Messenger, or Pocket, the server:

[0084] 1. The server sets the API key for the YouTube Data API and retrieves the "Watch Later" list.

[0085] 2. The server sets the access token for the Facebook Messenger API and retrieves the unread messages.

[0086] 3. The server sets the Pocket API consumer key and retrieves unread bookmarks.

[0087] 4. Convert the collected data into JSON format and mark it as unread or unwatched.

[0088] 5. Extract important information and notify the user.

[0089] Example prompt sentence:

[0090] "Generate a program that gathers your YouTube Watch Later list, unread messages in Facebook Messenger, and unread bookmarks in Pocket, extracts the important information, and notifies you."

[0091] In this way, the server realizes a system that efficiently collects unread or unviewed information from various digital content platforms used by users and provides it to users.

[0092] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0093] Step 1: Prepare for API access

[0094] The server obtains the API key and authentication information for each platform and prepares for API access. Specifically, it reads the API key and authentication information from the configuration file and sets the API endpoint for each platform. The input requires the API key and authentication information, and the output is a state where API access is possible.

[0095] Step 2: Data collection

[0096] The server calls the API of each platform to collect unread or unwatched information. Specifically, it uses the YouTube Data API to obtain the "Watch Later" list, the Facebook Messenger API to obtain unread messages, and the Pocket API to obtain unread bookmarks. The input requires a state where API access is ready, and the output is the unread or unwatched information obtained from each platform.

[0097] Step 3: Data processing

[0098] The server converts the collected data into a unified format and assigns an unread or unviewed status. Specifically, it converts the data obtained from each platform into JSON format and assigns an unread or unviewed status. The input requires the collected unread or unviewed information, and the output is the data converted into a unified format.

[0099] Step 4: Data analysis

[0100] The server analyzes the collected data and extracts information that is important to the user. Specifically, it extracts messages containing specific keywords from unread messages, extracts highly rated videos from unwatched YouTube videos, and extracts articles with specific tags from unread bookmarks. The input requires data converted into a unified format, and the output is data from which important information has been extracted.

[0101] Step 5: Delivering results

[0102] The server provides the processed and analyzed data to the user. Specifically, it displays the analysis results on a dashboard and notifies the user of important unread messages and unwatched videos. The user can check the unread or unwatched information through the dashboard and take necessary actions. The input is data from which important information has been extracted, and the output is the information provided to the user.

[0103] (Application example 1)

[0104] Next, a description will be given of Application Example 1 of Embodiment Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0105] Today's users consume a wide variety of digital content, and the amount of unread or unviewed information tends to increase. However, there is a lack of ways to efficiently manage this information and notify users at the appropriate time, which can lead to users missing important information. To solve this problem, a system is needed that centrally collects unread or unviewed information from multiple digital content and notifies users at the appropriate time.

[0106] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0107] In this invention, the server includes means for collecting unread or unviewed information from multiple digital contents, artificial intelligence means for summarizing the collected information, means for converting the summarized information into audio and outputting it, and means for centrally managing the collected information and notifying the user at an appropriate time. This allows the user to efficiently manage unread or unviewed information from multiple digital contents and receive notifications at an appropriate time without missing important information.

[0108] "Digital content" is a general term for information and media provided via the Internet or electronic devices.

[0109] "Unread or unviewed information" refers to digital content that the user has not yet read or viewed.

[0110] "Means of collection" refers to methods or techniques for obtaining unread or unviewed information from multiple digital content.

[0111] "Artificial intelligence means for summarizing" refers to artificial intelligence techniques used to summarize collected information succinctly.

[0112] "Means for converting and outputting speech" refers to methods or technologies for converting summarized information into speech and providing it to the user.

[0113] "Means for centralized management" refers to methods or technologies for managing collected unread or unviewed information in an integrated manner within a single system.

[0114] "Means of notifying at the appropriate time" refers to methods and technologies for notifying users of information at the most effective time.

[0115] The system for implementing this invention collects unread or unviewed information from multiple digital contents and notifies the user at an appropriate time. The system consists of three main elements: a server, a terminal, and a user.

[0116] server

[0117] The server includes the following means:

[0118] 1. Collection method: The server accesses multiple digital content platforms (e.g., video sharing services, messenger apps, bookmark services) and collects information on unread or unwatched content. Specifically, it obtains the information using the API of each platform.

[0119] 2. Artificial intelligence means of summarization: Using artificial intelligence techniques to summarize collected information, for example, using natural language processing (NLP) techniques to summarize long messages or articles in a concise manner.

[0120] 3. Voice output: The summarized information is converted into voice and provided to the user. Text-to-speech technology (TTS) is used to convert text information into voice.

[0121] 4. Centralized management: Collected unread or unwatched information is consolidated and managed in a single database, allowing users to view all information in a single interface.

[0122] 5. Notifications at the right time: Notifications are sent at the optimal time based on the user's behavioral patterns and schedule. For example, users can receive notifications during their free time, such as during their commute or while doing housework.

[0123] Terminal

[0124] A terminal is a device used by a user, such as a smartphone or tablet. An application is installed on the terminal to receive and display notifications sent from the server. This application has the following functions:

[0125] 1. Receiving and displaying notifications: Receives notifications sent from the server and displays them to the user. Notifications are displayed in the form of popups or banners.

[0126] 2. Audio playback: The audio summary information is played back so that users can listen to it through earphones or speakers.

[0127] user

[0128] Users can use the system to efficiently manage unread or unwatched information and receive timely notifications. Users can check notifications through the application and view detailed information as needed.

[0129] Specific examples

[0130] For example, if a user has five videos added to their "watch later" list on a video sharing service, three unread messages on a messaging app, and two unread articles on a bookmarking service, the server will collect, summarize, and convert this information into audio. It will then send notifications to users during their commute, providing them with the information.

[0131] Prompt Sentence Examples

[0132] A user has five unwatched videos in their "Watch Later" list on a video sharing service. The titles are "Introduction to Programming," "The Future of AI," "Data Science Fundamentals," "Applications of Machine Learning," and "Deep Learning Fundamentals."

[0133] The user has three unread messages in their messenger app: "Hi, how are you?", "Regarding tomorrow's meeting," and "What are your plans for the weekend?"

[0134] The user has two unread articles in their bookmarks service, titled "Current Technology Trends" and "Programming Best Practices."

[0135] Use this information to generate a reminder for your users.

[0136] In this way, users can efficiently manage their information without missing any important information.

[0137] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0138] Step 1:

[0139] The server accesses multiple digital content platforms and collects unread or unviewed information. Specifically, it uses each platform's API to obtain information such as the user's "Watch Later" list, unread messages, and unread bookmarks. The input is each platform's API key or token, and the output is a list of unread or unviewed information.

[0140] Step 2:

[0141] The server uses artificial intelligence techniques to summarize the collected information. Specifically, it uses natural language processing (NLP) techniques to concisely summarize long messages and articles. The input is a list of unread or unwatched information, and the output is the summarized information.

[0142] Step 3:

[0143] The server converts the summarized information into speech. Text-to-speech (TTS) technology is used to convert text information into speech. The input is the summarized information and the output is audio data.

[0144] Step 4:

[0145] The server consolidates and manages the collected unread or unwatched information into a single database, allowing users to view all information in a single interface. The input is a list of unread or unwatched information, and the output is the consolidated database.

[0146] Step 5:

[0147] The server sends notifications at optimal times based on the user's behavioral patterns and schedule. For example, users can receive notifications during their free time, such as while commuting or doing housework. The input is the user's behavioral patterns and schedule information, and the output is notification data.

[0148] Step 6:

[0149] The device receives notifications sent from the server and displays them to the user. Notifications can be displayed in the form of popups or banners. The input is the notification data and the output is the notification that is displayed to the user.

[0150] Step 7:

[0151] The terminal plays back the audio summary information, and the user can listen to the information through earphones or speakers. The input is audio data, and the output is the audio being played back.

[0152] Step 8:

[0153] The user can view the notification through the application and view more information if necessary. The input is the displayed notification, and the output is the user's action (such as viewing more information).

[0154] Example 2

[0155] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0156] In modern society, the amount of information is increasing, making it difficult for users to efficiently obtain the information they need. In addition, when obtaining information, users may not have time to read text information or may prefer to obtain information by voice. Furthermore, there is a demand for a system that automatically collects and summarizes recommended information based on the user's usage history.

[0157] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0158] In this invention, the server includes a means for collecting unread or unviewed information, a natural language processing means for analyzing the collected text information and generating a summary, a means for inputting a prompt sentence into a generation AI model to generate a summary, and a means for converting the summarized information into audio and outputting it. This allows users to efficiently obtain the information they need and acquire it by voice. It is also possible to automatically collect recommended information based on the user's usage history and generate summary information including that information.

[0159] "Unread or unviewed information" refers to digital content that a user has not yet read or viewed.

[0160] "Means for collecting" refers to a function or device for automatically collecting unread or unviewed information.

[0161] "Text information" refers to information written in text, including news articles, reports, blog posts, etc.

[0162] "Analyzing" means analyzing text information using techniques such as tokenization, part-of-speech tagging, and dependency analysis.

[0163] "Natural language processing means" refers to technologies and algorithms for analyzing text information and generating summaries.

[0164] A "generative AI model" is a model that uses artificial intelligence to summarize and generate text information, and examples include GPT-3 (registered trademark) and BERT.

[0165] A "prompt" is an instruction given to a generative AI model, containing specific instructions for generating a summary.

[0166] "Means for generating a summary" refers to the functionality or device for inputting a prompt sentence into a generative AI model and generating a summary.

[0167] "Means for converting text information into audio and outputting it" refers to a function or device for converting text information into audio and providing it to the user.

[0168] "Usage history" refers to a record of the information and actions a user has previously accessed.

[0169] "Recommended information" refers to information that is automatically selected based on the user's usage history and is presumed to be useful to the user.

[0170] The present invention provides a system that enables a user to efficiently acquire information and acquire information by voice. A specific embodiment of this system will be described below.

[0171] First, the user enters the text information they want to summarize into their device. For example, they paste the text information, such as a news article or report, into an input form. The device then sends the entered text information to the server as an HTTP request. At this time, the text information is packaged in JSON format.

[0172] The server uses the Python programming language and natural language processing libraries (e.g., spaCy or NLTK) to analyze the received text information. The server tokenizes the text and performs part-of-speech tagging and dependency analysis, which allows it to understand the structure of the text information and extract key content.

[0173] Next, the server inputs a prompt to a generative AI model (such as GPT-3 or BERT) based on the analysis results. The prompt includes instructions for generating a summary. For example, the prompt might be, "Please summarize the following text." The generative AI model generates a summary based on this prompt and the analysis results.

[0174] The generated summary is sent from the server to the device. At this time, the summary is packaged in JSON format. The device displays the summary received from the server to the user. The user can check the summary result on the device screen.

[0175] The server then uses text-to-speech synthesis technology (e.g., a text-to-speech engine) to convert the summarized information into audio and output it. The audio summary information is provided in a format that the user can listen to in their spare time, such as while doing housework or on their way to work.

[0176] As a specific example, suppose a user wants to summarize the news article "A major traffic accident occurred in Tokyo, injuring five people. The cause of the accident is under investigation." In this case, the user enters the news article into an input form on their device, and the device sends the news article to the server. The server tokenizes the news article and performs part-of-speech tagging and dependency analysis. The server then inputs a prompt statement, "Please summarize the following text," into the generative AI model. The generative AI model generates the summary, "A major traffic accident occurred in Tokyo, injuring five people. The cause of the accident is under investigation." The server sends the generated summary to the device, which then displays the summary result to the user.

[0177] In this way, users can easily obtain a summary of text information, and by using audio-translated summary information, users can efficiently acquire information.

[0178] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0179] Step 1:

[0180] The user inputs text information to be summarized into the terminal.

[0181] Specifically, a user copies and pastes text information such as a news article or report into an input form on a terminal, and the text information entered by the user is obtained as input.

[0182] Step 2:

[0183] The terminal transmits the input text information to the server.

[0184] Specifically, the device sends the entered text information to the server as an HTTP request. At this time, the text information is packaged in JSON format. The text information entered by the user is obtained as input, and the JSON format data to be sent to the server is generated as output.

[0185] Step 3:

[0186] The server parses the received text information.

[0187] Specifically, the server uses the Python programming language and natural language processing libraries (e.g., spaCy and NLTK) to tokenize text information, tag parts of speech, and perform dependency analysis. As input, it receives JSON-formatted text information sent from the device, and as output, it generates the analysis results.

[0188] Step 4:

[0189] The server inputs a prompt sentence into the generative AI model based on the analysis results.

[0190] Specifically, the server generates a prompt saying, "Please summarize the following text," and inputs it into the generative AI model along with the analysis results. The analysis results and the prompt are obtained as input, and the data to be input into the generative AI model is generated as output.

[0191] Step 5:

[0192] A generative AI model generates the summary.

[0193] Specifically, a generative AI model (e.g., GPT-3 or BERT) generates a summary based on a prompt and the analysis results. The input is the prompt and the analysis results, and the output is the generated summary.

[0194] Step 6:

[0195] The server transmits the generated summary to the terminal.

[0196] Specifically, the server packages the summary obtained from the generative AI model in JSON format and sends it to the device. The generated summary is obtained as input, and the JSON-formatted data sent to the device is generated as output.

[0197] Step 7:

[0198] The terminal displays the summary results to the user.

[0199] Specifically, the terminal displays the summary received from the server to the user. As input, it receives the JSON-formatted summary sent from the server, and as output, it generates the summary result to be displayed to the user.

[0200] Step 8:

[0201] The server outputs the summarized information as a voice.

[0202] Specifically, the server converts the summary into audio using text-to-speech synthesis technology (e.g., a text-to-speech engine), taking the generated summary as input and generating audio data as output.

[0203] Step 9:

[0204] The terminal provides the user with a voiced summary of the information.

[0205] Specifically, the terminal plays back the audio data and provides it in a form that the user can listen to in their free time, such as while doing housework or on the way to work. As input, the audio data sent from the server is obtained, and as output, audio information to be provided to the user is generated.

[0206] (Application example 2)

[0207] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0208] In modern society, a huge amount of digital content is generated every day, and users are required to efficiently obtain the information they need from it. However, due to the information overload, users are likely to miss important information, and they spend a lot of time obtaining information, which is a problem. In particular, there is the issue of it being difficult to grasp important information such as news articles in a short time.

[0209] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting unread or unviewed information from multiple digital contents, artificial intelligence means for summarizing the collected information, means for converting the summarized information into audio and outputting it, and means for summarizing the collected news articles and providing them to the user. This enables the user to efficiently obtain important information from a vast amount of information and grasp it in a short amount of time.

[0210] "Digital content" means information or media that is electronically generated, stored, or distributed.

[0211] "Unread or unviewed information" means information that the user has not yet read or viewed.

[0212] "Collection methods" refer to the methods and techniques used to obtain the required data from a particular source.

[0213] "Artificial intelligence means for summarizing" refers to a technology that uses natural language processing technology to analyze collected text information and express its main content in a shortened form.

[0214] "Means for converting text information into audio and outputting it" refers to technology for converting text information into audio and providing it to the user.

[0215] A "news article" is a piece of text that describes current events or information provided by the press or other sources.

[0216] "Means of delivery to users" refers to the methods and technologies used to deliver the summarized information in a form that is easy for users to access.

[0217] The system for implementing the present invention includes a server, a user terminal, and an artificial intelligence means. The details of each component and its operation will be described below.

[0218] server

[0219] The server has a means for collecting unread or unwatched information from multiple digital content. Specifically, it sends HTTP requests to obtain the latest news articles from external sources such as news APIs and saves the obtained data in JSON format. The server also has an artificial intelligence means for summarizing the collected information. Here, it uses a pre-trained summarization model using the Hugging Face transformers library. It also has a means for converting the summarized information into audio and outputting it. For audio conversion, it uses speech synthesis technology such as the Google® Text-to-Speech API.

[0220] User Device

[0221] The user terminal is a device such as a smartphone or tablet that receives the summary information provided by the server and displays it to the user. The user can view and listen to the summary of the news article through the application. The user terminal also records the user's usage history and sends it to the server, providing data for providing personalized recommendations.

[0222] Artificial Intelligence Tools

[0223] The AI ​​tool uses natural language processing techniques to analyze collected text information and generate summaries. Specifically, it uses the Hugging Face transformers library to represent the key content of news articles in a shortened form. The summarization model has been trained on a large amount of news article data in advance, enabling it to generate summaries with high accuracy.

[0224] Specific examples

[0225] For example, consider the case where the following news articles are collected:

[0226] Original news article:

[0227] Title: "COVID-19 vaccination progresses"

[0228] Content: "COVID-19 vaccinations are progressing rapidly in Japan. The government plans to offer the vaccine to the entire population in the coming months..."

[0229] For this news article, the following prompt sentence is input to the generative AI model:

[0230] Example prompt sentence:

[0231] "COVID-19 vaccination campaigns are progressing. COVID-19 vaccinations are progressing rapidly in Japan. The government plans to offer the vaccine to the entire population in the coming months..."

[0232] The generated summary results are as follows:

[0233] Summary results:

[0234] "COVID-19 vaccinations are progressing rapidly in Japan, and the government plans to offer the vaccine to all citizens."

[0235] This summary is converted into audio and sent to the user's device, allowing the user to grasp the main news in a short amount of time.

[0236] In this way, the server, user terminal, and artificial intelligence means work together, enabling users to efficiently obtain important information from a vast amount of information and grasp it in a short amount of time.

[0237] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0238] Step 1:

[0239] The server collects the latest news articles from external sources such as a news API by sending HTTP requests and saving the retrieved data in JSON format. The input is the news API endpoint, and the output is the news article data in JSON format.

[0240] Step 2:

[0241] The server analyzes the collected news article data and extracts the content of each article. Specifically, it extracts information such as the article title, text, and publication date and time from the JSON data. The input is news article data in JSON format, and the output is the text information of each news article.

[0242] Step 3:

[0243] The server uses a generative AI model to summarize the extracted text information of news articles. Specifically, it uses the Hugging Face transformers library to input a prompt sentence into the summarization model and obtains the summary result. The input is the text information of the news article, and the output is the summarized text information.

[0244] Step 4:

[0245] The server converts the summarized text information into audio. Specifically, it uses speech synthesis technology such as the Google Text-to-Speech API to convert the text information into audio data. The input is the summarized text information, and the output is audio data.

[0246] Step 5:

[0247] The server sends the voice data and summary text information to the user device. Specifically, it sends data to the user device via push notifications or API requests. The input is the voice data and summary text information, and the output is data delivery to the user device.

[0248] Step 6:

[0249] The user terminal displays and plays the received summary text information and audio data. Specifically, it displays the summary text and plays the audio data through an application. The input is the audio data and summary text information sent from the server, and the output is information provided to the user.

[0250] Step 7:

[0251] The user can view the provided summary information and listen to it aloud. Specifically, by operating the application, the user can check the summary of the news article and play it aloud. The input is the summary information displayed on the user's device, and the output is the user's information acquisition.

[0252] Step 8:

[0253] The user device records the user's usage history and sends it to the server. Specifically, it saves the history of articles the user has viewed and audio they have played, and periodically uploads it to the server. The input is the user's usage history data, and the output is the transmission of the history data to the server.

[0254] Step 9:

[0255] The server analyzes the received usage history data and generates personalized recommendations for the user. Specifically, it selects relevant news articles based on the usage history and generates summary information. The input is the user's usage history data, and the output is personalized summary information.

[0256] Step 10:

[0257] The server then sends the personalized summary information back to the user's device and provides it to the user. Specifically, it sends data via push notifications or API requests. The input is the personalized summary information, and the output is the delivery of data to the user's device.

[0258] Example 3

[0259] Next, a description will be given of a third embodiment of the third embodiment. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0260] In modern society, users have more and more opportunities to access a large amount of digital content, but it is difficult to efficiently obtain useful information from that content. It is also difficult for users to find time to obtain information in their busy daily lives. In particular, there is a demand for a means for users to obtain information by effectively utilizing their free time, such as while doing housework or on their way to work.

[0261] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[0262] In this invention, the server includes means for collecting user usage history, means for automatically collecting recommended information based on the collected usage history, artificial intelligence means for summarizing the collected information, means for converting the summarized information into voice data, and means for providing the voice data to the user's terminal. This allows users to efficiently obtain useful information even in their busy daily lives and effectively utilize free time such as during housework or on the way to work.

[0263] "Means for collecting user usage history" refers to a function for collecting behavioral data when a user browses a website or application.

[0264] The "means for automatically collecting recommended information" is a function for automatically collecting highly relevant information from the Internet based on the user's past usage history and preferences.

[0265] "Artificial intelligence means" are systems that include machine learning models and natural language processing techniques used to analyze and summarize collected information.

[0266] The "means for converting the summarized information into audio data" is a function that uses text-to-speech (TTS) technology to convert the summarized information in text form into audio form.

[0267] The "means for providing voice data to the user's terminal" is a function for transmitting the generated voice data to the user's terminal and providing it in a reproducible form.

[0268] This invention is a system that allows users to efficiently obtain useful information and make effective use of free time in their daily lives. This system performs a series of processes: collects the user's usage history, automatically collects recommended information based on that history, summarizes the collected information, converts the summarized information into voice data, and provides the voice data to the user's terminal.

[0269] Hardware and software used

[0270] Hardware: Servers, terminals (PCs, smartphones)

[0271] Software: Text-to-Speech (TTS) engine, machine learning models, database management system (DBMS)

[0272] Specific operation of the system

[0273] 1. Collection of user usage history

[0274] User: Browse websites and applications.

[0275] Device: Collects information about the user's browsing history and preferences, such as the URLs of pages visited, the duration of browsing, and the links clicked.

[0276] Server: Stores browsing history and preference information sent from your device in a database.

[0277] 2. Automatic collection of recommendations

[0278] Server: Uses machine learning models to learn users' browsing history and preferences. Specifically, it uses Python libraries such as Scikit-learn and TENSORFLOW (registered trademark) to analyze user behavior patterns.

[0279] Server: Based on the learning results, the server automatically collects new recommended information from the Internet, for example, retrieving related articles and product information from news sites and shopping sites.

[0280] 3. Summary information generation

[0281] Server: Uses natural language processing (NLP) techniques to summarize collected recommendations, specifically using Python libraries such as NLTK and spaCy to create short summaries of long articles and product descriptions.

[0282] Server: Generates summarized text information and stores it in a database, for example, summarizing news articles or product features.

[0283] 4. Audio translation of summary information

[0284] Server: Inputs the stored summary text information into a text-to-speech (TTS) engine, specifically using a TTS service such as Google Text-to-Speech API or Amazon Polly.

[0285] TTS engine: Converts summary text into audio data, for example, converting a news article summary into an audio file (e.g., MP3 format).

[0286] 5. Provision of audio data

[0287] User: Uses the device to request voice data, for example, opens a smartphone app and types the voice command "play me the latest news."

[0288] Server: Sends the generated audio data to the user's device by streaming the audio file via an HTTP request.

[0289] Device: Plays the received audio data and provides it in a form that the user can listen to. For example, the audio can be played through a smartphone speaker or earphones.

[0290] Specific examples

[0291] Example 1: News summarization and audio transcription

[0292] User: Frequent news site visitor.

[0293] Device: Collects user browsing history (URLs of visited news articles and viewing times).

[0294] Server: The collected browsing history is stored in a database and analyzed using machine learning models.

[0295] Server: Collects and summarizes the latest news articles based on user interests.

[0296] Server: The summarized news article is input into the TTS engine and converted into voice data.

[0297] User: Requests "Listen to the latest news" on their smartphone during their commute.

[0298] Server: Sends voice data to the user's smartphone.

[0299] Terminal: Plays audio data and allows the user to listen to the news.

[0300] Example 2: Product Recommendations

[0301] User: Visits an online shopping site.

[0302] Device: Collects user purchase and browsing history.

[0303] Server: Stores the collected history in a database and analyzes it using machine learning models.

[0304] Server: Collects and summarizes new products based on user interests.

[0305] Server: The summarized product description is input into the TTS engine and converted into voice data.

[0306] User: While doing housework, the user requests on their smartphone, "Tell me what products you recommend."

[0307] Server: Sends voice data to the user's smartphone.

[0308] Terminal: The audio data is played and the user listens to the product information.

[0309] Prompt Sentence Examples

[0310] "Collect recommended news articles based on the user's past browsing history, summarize them, and convert them into audio data."

[0311] "Based on your online shopping site usage history, please summarize new product recommendations and provide them to you via voice."

[0312] This system allows users to efficiently acquire information and easily obtain useful information in their daily lives. The flow of the identification process in the third embodiment will be described with reference to FIG.

[0313] Step 1:

[0314] Collection of user usage history

[0315] User: Browse websites and applications.

[0316] Device: Collects information about the user's browsing history and preferences, such as the URLs of pages visited, the duration of browsing, and the links clicked.

[0317] Input: User browsing behavior data (URL, browsing time, clicked links, etc.)

[0318] Output: Collected browsing history data

[0319] Specific operation: The device monitors user behavior data in real time and sends the collected data to the server.

[0320] Step 2:

[0321] Automatic collection of recommendations

[0322] Server: Uses machine learning models to learn users' browsing history and preferences, specifically analyzing user behavior patterns using Python libraries such as Scikit-learn and TensorFlow.

[0323] Server: Based on the learning results, the server automatically collects new recommended information from the Internet, for example, retrieving related articles and product information from news sites and shopping sites.

[0324] Input: Collected browsing history data

[0325] Output: Recommendations (news articles, product information, etc.)

[0326] What it does: The server runs machine learning models to gather relevant information from the internet based on the user's interests and preferences.

[0327] Step 3:

[0328] Generate summary information

[0329] Server: Uses natural language processing (NLP) techniques to summarize collected recommendations, specifically using Python libraries such as NLTK and spaCy to create short summaries of long articles and product descriptions.

[0330] Server: Generates summarized text information and stores it in a database, for example, summarizing news articles or product features.

[0331] Input: Recommended information (news articles, product information, etc.)

[0332] Output: Summarized text information

[0333] How it works: The server uses NLP technology to analyze the collected information, extract key points, and generate a summary.

[0334] Step 4:

[0335] Audio translation of summary information

[0336] Server: Inputs the stored summary text information into a text-to-speech (TTS) engine, specifically using a TTS service such as Google Text-to-Speech API or Amazon Polly.

[0337] TTS engine: Converts summary text into audio data, for example, converting a news article summary into an audio file (e.g., MP3 format).

[0338] Input: Summarized text information

[0339] Output: Audio data (MP3 format, etc.)

[0340] Specific operation: The server sends the summary text to the TTS engine, which generates the audio data.

[0341] Step 5:

[0342] Providing audio data

[0343] User: Uses the device to request voice data, for example, opens a smartphone app and types the voice command "play me the latest news."

[0344] Server: Sends the generated audio data to the user's device by streaming the audio file via an HTTP request.

[0345] Device: Plays the received audio data and provides it in a form that the user can listen to. For example, the audio can be played through a smartphone speaker or earphones.

[0346] Input: User request (e.g. voice command)

[0347] Output: Playback of audio data

[0348] Specific operation: Upon receiving a user request, the server sends the corresponding audio data to the terminal, and the terminal plays the audio data.

[0349] (Application example 3)

[0350] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0351] In modern society, users have access to a vast amount of digital content, but it is difficult to efficiently collect and consume information that is useful to them. It is also difficult for users to find time to obtain information in their busy daily lives. In particular, there is a demand for methods to obtain information that allow users to effectively utilize their free time, such as during their commute or while doing housework.

[0352] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[0353] In this invention, the server includes means for collecting unread or unviewed information from a plurality of digital contents, artificial intelligence means for summarizing the collected information, means for converting the summarized information into audio and outputting it, means for automatically collecting recommended information based on the user's usage history, means for summarizing the collected recommended information, means for converting the summarized recommended information into audio and outputting it, and means for providing the summarized audio information in a form that the user can listen to in their spare time. This allows the user to efficiently collect information that is useful to them and effectively utilize their spare time to obtain information.

[0354] "Digital content" is a general term for information and media provided in electronic form.

[0355] "Unread or unviewed information" refers to information that the user has not yet read or viewed.

[0356] "Collection methods" refers to the methods and techniques used to collect specific information.

[0357] "Artificial intelligence means for summarizing" refers to artificial intelligence techniques used to summarize collected information in a concise manner.

[0358] "Means for converting text information into audio and outputting it" refers to a method for converting text information into audio data and providing it to the user.

[0359] "Usage history" refers to a record of the information and actions a user has previously accessed.

[0360] "Recommended information" refers to information selected based on the user's interests and concerns.

[0361] "Automatic collection means" refers to a method of automatically collecting information based on specific conditions.

[0362] "Summarization methods" refer to methods and techniques for summarizing information succinctly.

[0363] "Means for converting text information into audio and outputting it" refers to a method for converting text information into audio data and providing it to the user.

[0364] "Free time" refers to free time when a user is not engaged in other activities.

[0365] "A form that can be listened to without interruption" refers to a form in which the user can receive audio information without having to perform any specific operation.

[0366] The system for implementing this invention is composed of a server, a user terminal, and related software components. The details of each component and its role will be explained below.

[0367] server

[0368] The server has the following main functions:

[0369] 1. Information collection methods:

[0370] The server collects unread or unwatched information from multiple digital content sources, using external sources such as news APIs and RSS feeds.

[0371] 2. Artificial Intelligence Means:

[0372] The server uses natural language processing techniques to summarize the collected information, specifically using Hugging Face's Transformers library to summarize the collected text data.

[0373] 3. Vocalization methods:

[0374] The server uses the Google Text-to-Speech (gTTS) library to convert the summarized information into audio data, which generates an audio file from the text information.

[0375] 4. How we collect recommendations based on your usage history:

[0376] The server automatically collects recommendations based on the user's browsing history and preferences, using machine learning algorithms to analyze the user's behavioral patterns.

[0377] 5. Recommended information summary methods:

[0378] The server again uses natural language processing techniques to summarize the collected recommendations.

[0379] 6. Audio recommendation methods:

[0380] The server again uses the Google Text-to-Speech (gTTS) library to convert the summarized recommendations into audio data.

[0381] 7. Audio information delivery methods:

[0382] The server provides the summarized audio information in a format that users can listen to at their leisure by sending the audio file to the user's terminal and allowing the user to play it on their smartphone or other device.

[0383] User terminal

[0384] The user terminal is a device such as a smartphone or tablet, and has the following functions:

[0385] 1. Receiving and playing audio files:

[0386] The user terminal is equipped with an application for receiving and playing audio files sent from the server, allowing the user to listen to audio information during their free time, such as while commuting or doing housework.

[0387] Specific examples

[0388] For example, if a user is interested in "technology," the server collects the latest technology news, summarizes it, and provides it in audio format, allowing the user to listen to the latest technology news through earphones during their commute.

[0389] Prompt Sentence Examples

[0390] Build an application that gathers the latest news articles, summarizes them and delivers them to you via audio, based on the user's preference: Technology.

[0391] In this way, users can efficiently collect information that is useful to them and can obtain information by effectively utilizing their free time.

[0392] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[0393] Step 1:

[0394] The server collects multiple digital content from external sources such as news APIs and RSS feeds. As input, it receives category information based on the user's interests and retrieves digital content including unread or unwatched information as output. Specifically, it sends an API request and parses the retrieved JSON data to extract the required information.

[0395] Step 2:

[0396] The server uses natural language processing techniques to summarize the collected digital content. It receives the collected text data as input and generates summarized text information as output. Specifically, it uses Hugging Face's Transformers library to summarize the text data.

[0397] Step 3:

[0398] The server converts the summarized text information into audio data. It receives the summarized text information as input and generates an audio file as output. Specifically, it uses the Google Text-to-Speech (gTTS) library to convert the text information into an audio file (e.g., MP3 format).

[0399] Step 4:

[0400] The server automatically collects recommended information based on the user's usage history. It receives the user's past browsing history and preference data as input, and obtains recommended digital content as output. Specifically, it uses machine learning algorithms to analyze the user's behavioral patterns and collect highly relevant information.

[0401] Step 5:

[0402] The server again uses natural language processing techniques to summarize the collected recommendations. It receives the recommendation text data as input and generates summarized recommendations as output. Specifically, it again uses Hugging Face's Transformers library to summarize the text data.

[0403] Step 6:

[0404] The server converts the summarized recommendation information into audio data. It receives the summarized recommendation information as input and generates an audio file as output. Specifically, it again uses the Google Text-to-Speech (gTTS) library to convert the text information into an audio file.

[0405] Step 7:

[0406] The server sends the summarized audio information to the user terminal. As input, it receives the generated audio file and as output, it sends the audio file to the user terminal using an HTTP request so that the user can play it on their smartphone or other device.

[0407] Step 8:

[0408] The user terminal receives and plays the audio file sent from the server. It receives the audio file sent from the server as input and plays the audio information as output. Specifically, it plays the audio file using a dedicated application, allowing the user to listen to it during their free time, such as during their commute or while doing housework.

[0409] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0410] "Example 1"

[0411] In one embodiment of the present invention, the emotion engine collects and analyzes information such as the user's voice, text input, facial expression, and biometric information (e.g., heart rate and electrodermal activity) to recognize the user's emotions. Specifically, the emotion engine infers the user's emotional state from the tone and pitch of the user's voice when speaking to the system, the choice of words in text input, changes in facial expression, and fluctuations in biometric information.

[0412] "Example 2"

[0413] The emotion engine automatically collects recommended information based on the user's emotional state. For example, if a user is expressing joy, it will collect fun news and entertainment information that may enhance that emotion. Conversely, if a user is expressing sadness, it will collect soothing content and positive information that may soothe that emotion.

[0414] "Example 3" The emotion engine also optimizes the way the audio summary information is output depending on the user's emotional state. For example, if the user is in a relaxed emotional state, the tone and speed of the audio will be adjusted to a slower tone. Conversely, if the user is in an excited emotional state, the tone and speed of the audio will be adjusted to a more active tone.

[0415] "Example 4"

[0416] Furthermore, the emotion engine learns changes in the user's emotional state and uses the results to collect recommended information and optimize voice conversion. For example, if a user tends to express joy after hearing certain information, the engine will prioritize collecting information similar to that information. Also, if a user tends to express favorable emotions toward a certain voice tone or speed, the engine will prioritize using that tone or speed.

[0417] The processing flow of each embodiment will be described below.

[0418] "Example 1"

[0419] Step 1: Collect information such as the user's voice, text input, facial expressions, and biometric information (e.g., heart rate and electrodermal activity).

[0420] Step 2: Analyze the collected information and estimate the user's emotional state.

[0421] "Example 2"

[0422] Step 1: Recognize the user's emotional state with the emotion engine.

[0423] Step 2: Automatically collect recommended information based on your emotional state.

[0424] "Example 3"

[0425] Step 1: Recognize the user's emotional state with the emotion engine.

[0426] Step 2: Optimize the output method of the spoken summary information according to the user's emotional state.

[0427] "Example 4"

[0428] Step 1: The emotion engine learns the user's emotional state and its changes.

[0429] Step 2: Based on the learning results, collect recommended information and optimize the voice conversion.

[0430] Example 1

[0431] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0432] In today's digital society, users have access to a large amount of digital content, and the amount of unread or unviewed information is increasing, making it difficult to efficiently manage and digest it. Furthermore, providing information without considering the user's emotional state can reduce user satisfaction. Therefore, there is a need for a system that efficiently collects and summarizes unread or unviewed information and provides it appropriately according to the user's emotional state.

[0433] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0434] In this invention, the server includes means for collecting unread or unviewed information from multiple digital contents, artificial intelligence means for summarizing the collected information, means for converting the summarized information into voice and outputting it, means for collecting voice, text input, facial expression, and biometric information to recognize the user's emotions, and means for analyzing the collected information and estimating the user's emotional state. This makes it possible to efficiently manage unread or unviewed information and provide appropriate information according to the user's emotional state.

[0435] "Digital content" is a general term for information and media provided in electronic form, including video, audio, text, and images.

[0436] "Unread or unviewed information" refers to digital content information that a user has not yet read or viewed.

[0437] "Collection methods" refers to the methods and technologies used to obtain unread or unwatched information from a particular digital content platform.

[0438] "Artificial intelligence means" refers to techniques such as machine learning and natural language processing used to analyze and summarize collected information.

[0439] "Means for converting and outputting speech" refers to the technology or device for converting summarized information into speech and providing it to the user.

[0440] "Means of collecting voice, text input, facial expressions, and biometric information for emotion recognition" refers to methods and technologies for obtaining the data necessary to understand a user's emotional state.

[0441] "Means for analyzing and inferring a user's emotional state" refers to methods and technologies for assessing and inferring a user's emotions based on collected data.

[0442] MODE FOR CARRYING OUT THE INVENTION

[0443] This invention is a system that collects unread or unviewed information from multiple digital content platforms used by a user and provides the information appropriately according to the user's emotional state. Specific embodiments of this system are described below.

[0444] Information gathering process

[0445] The server prepares API keys and authentication information for accessing various digital content platforms. This allows authenticated requests to be sent to each platform. Specifically, the server obtains and stores the YouTube API key, the access token for the messenger app, the consumer key for the bookmark service, and so on.

[0446] The server then sends a request to each platform's API to obtain information about unread or unwatched videos. This allows the server to collect information about videos that the user has not yet viewed or read. Specifically, the server uses the YouTube API to obtain information about videos in the "Watch Later" list, the Messenger app's API to obtain unread messages, and the bookmark service's API to obtain unread bookmarks.

[0447] The collected information is stored in a database by the server, which allows the data to be used efficiently for subsequent processing.

[0448] Emotion Engine Process

[0449] The device collects information necessary to recognize the user's emotions, including voice, text input, facial expressions, biometric information, etc. Specifically, it uses a microphone to collect the user's voice, monitors keyboard input to collect text input, uses a camera to capture facial expressions, and biometric sensors to collect heart rate and electrodermal activity.

[0450] The device then analyzes the collected information to infer the user's emotional state, including through analysis of voice recognition software (e.g., voice recognition APIs), natural language processing tools (e.g., generative AI models), facial recognition software (e.g., facial recognition APIs), and biometric sensor data. Specifically, the device analyzes the tone and pitch of the voice, word choice in the text, changes in facial expression, and biometric fluctuations.

[0451] Based on the analysis results, the device estimates the user's emotional state, allowing it to understand the user's current emotional state and respond appropriately.

[0452] Examples of specific examples and prompts

[0453] As a concrete example, if you want to use the YouTube API to retrieve unwatched videos from your "Watch Later" list, you would use the following prompt:

[0454] Example prompt sentence:

[0455] "Get unwatched videos from your YouTube Watch Later list."

[0456] Also, when analyzing the user's voice tone and pitch to infer their emotional state, use prompts like the following:

[0457] Example prompt sentence:

[0458] "Analyze the user's voice tone and pitch to estimate their emotional state."

[0459] In this way, the server and the device work together to efficiently collect and manage the user's unread or unviewed information, and provide appropriate information according to the user's emotional state.

[0460] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0461] Step 1:

[0462] The server prepares API keys and authentication information for accessing various digital content platforms. Specifically, it obtains and stores on the server the YouTube API key, the access token for the messenger app, the consumer key for the bookmark service, etc.

[0463] Input: API key or authentication information

[0464] Output: Ready to send authenticated requests

[0465] Specific operation: Obtain an API key and access token and save them in the server configuration file.

[0466] Step 2:

[0467] The server sends requests to each platform's API to get information about unread or unwatched videos. Specifically, it uses the YouTube API to get information about videos in the "Watch Later" list, the Messenger app's API to get unread messages, and the bookmark service's API to get unread bookmarks.

[0468] Input: Credentials, API request

[0469] Output: Unread or unheard information

[0470] Specific operation: Sends an API request to retrieve unread or unwatched information from each platform.

[0471] Step 3:

[0472] The server stores the collected unread or unwatched information in a database, which allows the data to be used efficiently in subsequent processing.

[0473] Input: Unread or unwatched information

[0474] Output: Information stored in the database

[0475] Specific operation: Execute a query to insert the retrieved information into a database.

[0476] Step 4:

[0477] The device collects the information needed to recognize the user's emotions by using a microphone to capture the user's voice, monitoring keyboard input to collect text input, a camera to capture facial expressions, and biometric sensors to collect heart rate and electrodermal activity.

[0478] Input: User voice, text input, facial expressions, biometric information

[0479] Output: Collected emotion recognition data

[0480] What it does: Collects data using microphones, keyboards, cameras, and biometric sensors.

[0481] Step 5:

[0482] The device analyzes the collected information to infer the user's emotional state, specifically by analyzing data from voice recognition software, natural language processing tools, facial recognition software, and biometric sensors.

[0483] Input: Collected emotion recognition data

[0484] Output: Estimation of the user's emotional state

[0485] Specific actions: Analyzes tone and pitch of voice, analyzes word choice in text, analyzes changes in facial expressions, and evaluates fluctuations in biometric information.

[0486] Step 6:

[0487] The device estimates the user's emotional state based on the analysis results, allowing it to understand the user's current emotional state and respond appropriately.

[0488] Input: Analysis results

[0489] Output: Estimation of the user's emotional state

[0490] Specific operation: Integrates the analysis results and runs an algorithm to estimate the user's emotional state.

[0491] (Application example 1)

[0492] Next, a description will be given of Application Example 1 of Embodiment Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0493] Today's digital content users are surrounded by a lot of unread or unwatched information, making it difficult to efficiently manage it and digest it in a timely manner. Furthermore, there is a lack of ways to provide optimal content according to the user's emotional state, resulting in a lack of an improved user experience.

[0494] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0495] In this invention, the server includes means for collecting unread or unviewed information from a plurality of digital contents, artificial intelligence means for summarizing the collected information, means for converting the summarized information into voice and outputting it, emotion recognition means for collecting and analyzing the user's voice, text input, facial expression, and biometric information, and means for recommending optimal content based on the emotion recognition means, thereby making it possible to provide optimal content according to the user's emotional state.

[0496] "Digital content" is a general term for information and media provided via the Internet or electronic devices.

[0497] "Unread or unviewed information" refers to digital content that a user has not yet read or viewed.

[0498] "Collection methods" means the methods or technologies used to obtain unread or unwatched information from a digital content platform.

[0499] "Artificial intelligence means" refers to algorithms or systems that analyze and summarize collected information.

[0500] "Means for converting and outputting speech" refers to the technology or device that converts the summarized information into speech and provides it to the user.

[0501] "Emotion recognition means" refers to technology or systems that collect and analyze a user's voice, text input, facial expressions, and biometric information to estimate the user's emotional state.

[0502] "Recommendation means" refers to methods and technologies for recommending optimal content to users based on emotion recognition means.

[0503] A system for carrying out this invention is configured as follows: A server includes means for collecting unread or unviewed information from a plurality of digital contents, artificial intelligence means for summarizing the collected information, means for converting the summarized information into voice and outputting it, emotion recognition means for collecting and analyzing a user's voice, text input, facial expression, and biometric information, and means for recommending optimal content based on the emotion recognition means.

[0504] Hardware and software used

[0505] Hardware: Smartphone (microphone, camera, heart rate sensor, etc.)

[0506] Software: Python, EmotionRecognizer library, ContentRetriever library, various APIs (APIs for digital content platforms)

[0507] Data processing and calculation

[0508] 1. Data Collection:

[0509] Voice Data: We collect your voice using your smartphone's microphone.

[0510] Text data: Collects user-entered text.

[0511] Facial Expression Data: Collecting the user's facial expressions using the smartphone camera.

[0512] Biometric data: We collect your biometric information using the heart rate sensor on your smartphone.

[0513] 2. Emotion recognition:

[0514] The server analyzes the collected voice data, text data, facial expression data, and biometric data using the EmotionRecognizer library to estimate the user's emotional state.

[0515] 3. Content Collection:

[0516] The server collects unread or unviewed content using the API of the digital content platform.

[0517] 4. Content Recommendation:

[0518] The server recommends the most suitable content in the ContentRetriever library based on the estimated emotional state.

[0519] Specific examples

[0520] For example, if a user is feeling stressed, the server will recommend relaxing music or videos, and if the user is feeling excited, it will recommend entertaining content.

[0521] Prompt Sentence Examples

[0522] "If the user's emotional state is estimated to be 'stressed,' recommend relaxing music or videos. For example, select relaxing content from the 'Watch Later' list of a digital content platform."

[0523] In this way, it becomes possible to provide optimal content according to the user's emotional state.

[0524] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0525] Step 1:

[0526] The server collects the user's voice data using the smartphone's microphone. The voice data includes the content, tone, and pitch of what the user said. The input is the user's voice, and the output is the collected voice data.

[0527] Step 2:

[0528] The server collects text data from the user using the smartphone's text input function. The text data includes strings of characters and word choices entered by the user. The input is the user's text input, and the output is the collected text data.

[0529] Step 3:

[0530] The server collects facial expression data of the user using the smartphone camera. The facial expression data includes changes in the user's facial expressions and facial features. The input is the user's facial expression, and the output is the collected facial expression data.

[0531] Step 4:

[0532] The server collects the user's biometric data using the smartphone's heart rate sensor. The biometric data includes the user's heart rate and electrodermal activity. The input is the user's biometric information, and the output is the collected biometric data.

[0533] Step 5:

[0534] The server analyzes the collected voice data, text data, facial expression data, and biometric data using the EmotionRecognizer library to estimate the user's emotional state. The input is the collected data, and the output is the estimated emotional state.

[0535] Step 6:

[0536] The server collects unread or unviewed content using the API of the digital content platform, where the input is the API key or endpoint of the digital content platform and the output is the collected unread or unviewed content.

[0537] Step 7:

[0538] The server recommends the most suitable content based on the estimated emotional state using the ContentRetriever library. The input is the estimated emotional state and collected unread or unwatched content, and the output is the recommended content.

[0539] Step 8:

[0540] The server provides the recommended content to the user. Specifically, it plays a voiced summary of the content to the user through the smartphone. The input is the recommended content, and the output is the voiced summary provided to the user. The server may also make recommendations at optimal times based on the user's behavioral patterns and schedule.

[0541] Example 2

[0542] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0543] Conventional information gathering systems do not provide information according to the user's emotional state, making it difficult to provide information appropriate to the user's emotions. Furthermore, collected information is not adequately summarized or converted into audio, making it difficult for users to acquire information efficiently. This leads to issues such as reduced user satisfaction and a worsening efficiency of information utilization.

[0544] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0545] In this invention, the server includes means for collecting unread or unviewed information from multiple digital contents, artificial intelligence means using natural language processing technology to summarize the collected information, means for detecting the emotional state of the user and automatically collecting recommended information according to that emotion, and means for converting the summarized information into audio and outputting it. This makes it possible to provide information according to the emotional state of the user, allowing for efficient information acquisition.

[0546] "Digital content" means information or media stored or distributed in electronic form.

[0547] "Unread or unviewed information" means information that the user has not yet read or viewed.

[0548] "Natural language processing technology" refers to technology that allows computers to understand, interpret, and generate human language.

[0549] "Artificial intelligence means" refers to systems that automatically process information using machine learning and data analysis.

[0550] An "emotional state" is a state that indicates a user's current feelings or mood.

[0551] "Recommended information" refers to information that is useful or interesting to the user, selected based on the user's emotional state and usage history.

[0552] "Speech" refers to the process of converting text information into speech.

[0553] "Output" refers to providing processed information to a user.

[0554] MODE FOR CARRYING OUT THE INVENTION

[0555] The present invention is a system for providing information according to a user's emotional state, and is implemented as follows.

[0556] Information gathering and summary generation

[0557] The server collects text information from online news sites, social media, and other sources. Specifically, it obtains the latest news articles using RSS feeds and APIs. The collected text information is then analyzed using natural language processing technology. The software used includes "natural language processing APIs" and "generative AI models."

[0558] Examples:

[0559] Original information: "The spread of COVID-19 has led to lockdowns in many countries. Economic activity has stagnated, affecting many businesses."

[0560] Summary: "The spread of COVID-19 has led to lockdowns in many countries and economic activity has stagnated."

[0561] Example prompt sentence:

[0562] "Summarize the following text: The coronavirus outbreak is spreading, leading to lockdowns in many countries. Economic activity is slowing down, affecting many businesses."

[0563] Information gathering using emotion engines

[0564] The device detects the user's emotional state and automatically collects recommended information based on that emotion. The emotion engine uses an "emotion analysis API" and "text analysis tools."

[0565] Examples:

[0566] User's emotional state: joy

[0567] Information collected: Fun news and entertainment information

[0568] User's emotional state: Sadness

[0569] Information collected: Soothing content and positive information

[0570] Example prompt sentence:

[0571] "If a user is expressing an emotion of joy, gather fun news and entertainment information that may enhance that emotion."

[0572] "If users are expressing feelings of sadness, seek out soothing content and positive information that may alleviate those feelings."

[0573] Vocalization and output

[0574] The server converts the summarized information into audio and provides it to the user. The audio is generated using a speech synthesis API. The audio information is provided in a format that users can listen to during their free time, such as while doing housework or on their way to work.

[0575] This system allows users to acquire information efficiently and receive appropriate information according to their emotional state.

[0576] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0577] Program processing flow

[0578] Step 1: Gather information

[0579] The server collects text information from news sites and social networking sites on the Internet. It uses RSS feeds and API URLs as input and obtains the collected text information as output.

[0580] Specific behavior:

[0581] The server periodically checks the RSS feed of the news site to see if there are any new articles.

[0582] The server retrieves social media posts through the API.

[0583] Step 2: Text analysis

[0584] The server analyzes the collected text information using natural language processing technology. The collected text information is used as input, and the analysis results (tokenization, part-of-speech tagging, sentence structure analysis, etc.) are obtained as output.

[0585] Specific behavior:

[0586] The server uses a natural language processing API to tokenize the text and tag each token with a part of speech.

[0587] The server analyzes the structure of the sentence and extracts relationships such as subject, predicate, and object.

[0588] Step 3: Summary generation

[0589] The server generates a summary based on the analysis results. It uses the analysis results as input and obtains the summarized text information as output.

[0590] Specific behavior:

[0591] The server inputs the analysis results into a generative AI model and creates a prompt sentence that generates a summary.

[0592] The server inputs a prompt sentence into the generative AI model and obtains a summary.

[0593] Example prompt sentence:

[0594] "Summarize the following text: The coronavirus outbreak is spreading, leading to lockdowns in many countries. Economic activity is slowing down, affecting many businesses."

[0595] Step 4: Detecting Emotional State

[0596] The device analyzes the user's input (text message or voice) to detect the emotional state. As input, it uses the user's text message or voice data. As output, it obtains the detected emotional state.

[0597] Specific behavior:

[0598] The device sends the text message entered by the user to the sentiment analysis API for sentiment analysis.

[0599] The device converts the user's voice input into text and performs sentiment analysis using text analysis tools.

[0600] Step 5: Gather recommendations

[0601] The device collects recommendation information according to the detected emotional state. It uses the detected emotional state as input and obtains the recommendation information as output.

[0602] Specific behavior:

[0603] If the user expresses joy, the device will use the news API to retrieve entertainment-related articles.

[0604] The device collects information from websites that offer soothing content if the user expresses sadness.

[0605] Example prompt sentence:

[0606] "If a user is expressing an emotion of joy, gather fun news and entertainment information that may enhance that emotion."

[0607] "If users are expressing feelings of sadness, seek out soothing content and positive information that may alleviate those feelings."

[0608] Step 6: Audio and Output

[0609] The server converts the summarized information into audio and provides it to the user. It uses the summarized text information as input and obtains the audio information as output.

[0610] Specific behavior:

[0611] The server uses a speech synthesis API to convert the summarized text information into speech.

[0612] The server outputs the vocalized information through an interface for presentation to the user.

[0613] In this way, the user can acquire information efficiently and receive appropriate information according to their emotional state.

[0614] (Application example 2)

[0615] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0616] Conventional digital content delivery systems have difficulty providing personalized information according to the user's emotional state, and have been unable to increase user satisfaction. Furthermore, there is a lack of a way for users to efficiently obtain the information they need without being overwhelmed by a large amount of information. Furthermore, there is a need to keep users' emotions positive by providing appropriate content according to their emotions.

[0617] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0618] In this invention, the server includes means for collecting unread or unviewed information from multiple digital content items, artificial intelligence means for summarizing the collected information, means for converting the summarized information into audio and outputting it, means for analyzing the user's emotional state, and means for automatically collecting recommended information according to the user's emotional state. This enables personalized information to be provided according to the user's emotional state, thereby increasing user satisfaction. Furthermore, the user can efficiently obtain the information they need without being overwhelmed by a large amount of information. Furthermore, by providing appropriate content according to the user's emotions, it is possible to maintain a positive mood for the user.

[0619] "Digital content" is a general term for information and media that are electronically generated, stored, and distributed.

[0620] "Unread or unviewed information" refers to digital content that a user has not yet read or viewed.

[0621] "Gathering means" refers to the technologies and methods used to obtain digital content from the Internet or databases.

[0622] "Artificial intelligence means" refers to systems or algorithms that use techniques such as machine learning and natural language processing to analyze data and perform specific tasks.

[0623] To "summarize" means to express the main content of the original information in a shortened form.

[0624] "Means for converting text into audio and outputting it" refers to the technology or method for converting text information into audio and making it audible to the user.

[0625] A "user's emotional state" refers to the emotions or mood a user is feeling at a particular point in time.

[0626] "Analytical means" refers to techniques and methods for analyzing data and extracting specific information or patterns.

[0627] "Recommended information" is information that is useful or interesting to the user, selected based on the user's interests and emotional state.

[0628] "Automatic collection methods" are technologies or methods that allow a system to collect information without user intervention.

[0629] A system for carrying out this invention has the following configuration: a server includes means for collecting unread or unviewed information from a plurality of digital contents, artificial intelligence means for summarizing the collected information, means for converting the summarized information into audio and outputting it, means for analyzing the emotional state of a user, and means for automatically collecting recommended information according to the emotional state of the user.

[0630] System Program

[0631] The server collects digital content from external data sources such as news APIs. The collected digital content is summarized using natural language processing technology. This summarization uses the Hugging Face transformers library. The summarized information is then converted into audio using speech synthesis technology and provided to the user.

[0632] The user's emotional state is analyzed in real time using an emotion analysis model. This emotion analysis is performed using the Hugging Face pipeline. Based on the analyzed emotional state, information suitable for the user is automatically collected and recommended. This information is then summarized and converted into audio and provided to the user.

[0633] Hardware and software used

[0634] Hardware: Servers, user devices (smartphones, tablets, PCs, etc.)

[0635] Software: News API, Hugging Face transformers library, emotion analysis model, speech synthesis software

[0636] Specific examples

[0637] For example, if a user is using a smartphone, the server collects the latest news articles from a news API. The collected news articles are summarized using natural language processing technology. The summarized news articles are then converted into audio using speech synthesis technology, which the user can listen to on their smartphone.

[0638] If the user expresses the emotion of "joy," the emotion analysis model will analyze this emotion and automatically collect fun news and entertainment information, which will be summarized, converted into audio, and provided to the user.

[0639] Prompt Sentence Examples

[0640] If the user's emotion is "joy," show them some fun news or entertainment information.

[0641] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0642] Step 1:

[0643] The server collects digital content from external data sources such as news APIs.

[0644] Input: News API endpoint

[0645] Output: A list of the latest news articles

[0646] What it does: The server uses an HTTP request to access the news API to retrieve the latest news articles, which are then returned to the server in JSON format.

[0647] Step 2:

[0648] The server summarizes the collected digital content using natural language processing techniques.

[0649] Input: A list of recent news articles

[0650] Output: A list of summarized news articles

[0651] What it does: The server uses Hugging Face's transformers library to summarize the content of each news article, which is a shortened version of the main points of the original news article.

[0652] Step 3:

[0653] The server converts the summarized information into audio using speech synthesis technology and provides it to the user.

[0654] Input: A list of summarized news articles

[0655] Output: Audio file

[0656] What it does: The server uses speech synthesis software to convert the summarized news article into audio, which is then sent to the user's device for listening.

[0657] Step 4:

[0658] The server analyzes the user's emotional state.

[0659] Input: Real-time user data (e.g. text, voice, facial expressions, etc.)

[0660] Output: User's emotional state (e.g., happy, sad, angry, etc.)

[0661] How it works: The server uses the emotion analysis model to analyze the user's real-time data and identify the user's emotional state. This analysis uses the Hugging Face pipeline.

[0662] Step 5:

[0663] The server automatically collects recommended information based on the user's emotional state.

[0664] Input: User's emotional state

[0665] Output: A list of recommended news articles

[0666] Specific operation: The server recollects appropriate news articles from the news API based on the user's emotional state. For example, if the user expresses "joy," it will collect fun news and entertainment information.

[0667] Step 6:

[0668] The server then summarizes the recollected news articles, converts them into audio, and provides them to the user.

[0669] Input: A list of recommended news articles

[0670] Output: Audio file

[0671] Specific operation: As in steps 2 and 3, the server summarizes the collected news articles and converts them into audio using speech synthesis technology. The generated audio file is sent to the user's device, where the user can listen to it.

[0672] Example 3

[0673] Next, a description will be given of a third embodiment of the third embodiment. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0674] Conventional information gathering systems have problems in that they do not provide sufficient recommended information based on the user's usage history and do not provide optimal voice output according to the user's emotional state. Furthermore, they lack the ability to learn changes in the user's emotional state and optimize information gathering and voice output based on that information, making it difficult to provide optimal information to the user. This reduces user satisfaction and prevents effective use of information.

[0675] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[0676] In this invention, the server includes means for collecting unread or unviewed information from multiple digital content items, artificial intelligence means for summarizing the collected information, means for converting the summarized information into audio and outputting it, means for detecting the user's emotional state and optimizing the output method of the converted audio summary information, and means for learning changes in the user's emotional state and optimizing the collection of recommended information and audio output based on the learning results. This makes it possible to provide recommended information based on the user's usage history, and also realizes optimal audio output according to the user's emotional state. Furthermore, by learning changes in the user's emotional state, the collection of information and audio output are optimized, making it possible to provide optimal information to the user.

[0677] "Digital content" means information or media stored or distributed in electronic form.

[0678] "Unread or unviewed information" means information that has not yet been viewed or listened to by the user.

[0679] "Means of collection" refers to the methods and technologies used to obtain the necessary information from digital content.

[0680] "Artificial intelligence means for summarizing" refers to artificial intelligence techniques used to summarize collected information succinctly.

[0681] "Means for converting text information into audio and outputting it" refers to technology for converting text information into audio data and providing it to the user.

[0682] "Means for detecting emotional state" refers to technology that analyzes the user's emotions and understands their state.

[0683] "Means for optimizing the output method" refers to techniques for adjusting the tone and speed of audio output depending on the user's emotional state.

[0684] "Means for learning changes in emotional state" refers to technology that records changes in the user's emotions and uses them to improve the system's behavior.

[0685] "Means for optimizing the collection and vocalization of recommended information" refers to technology for collecting optimal information and vocalizing it based on the user's emotional state and usage history.

[0686] The present invention is a system that provides optimal information based on a user's usage history and emotional state, and converts it into audio. Specific embodiments of this system will be described below.

[0687] 1. Generating the system program

[0688] The server automatically collects recommended information based on the user's usage history and generates a program that generates summary information containing that information. This program uses machine learning technology to learn the user's past browsing history and preferences, and has the ability to collect and summarize new recommended information.

[0689] 2. Explanation of program processing

[0690] The server processes the data in the following steps:

[0691] Data collection

[0692] The server collects data about users' browsing history and preferences, including the web pages they visit, the links they click, and the duration of their visits. Specifically, the server collects data using analytics tools such as Google Analytics and Mixpanel.

[0693] Training a machine learning model

[0694] The server uses the collected data to train machine learning models that learn user preferences and interests and predict new recommendations using libraries such as TensorFlow and PyTorch.

[0695] Recommendation collection and summarization

[0696] The server uses the trained model to collect new recommendations and summarize them using natural language processing techniques, specifically generative AI models such as BERT and GPT-3.

[0697] Text-to-Speech (TTS)

[0698] The server uses TTS technology, such as Google Text-to-Speech API or Amazon Polly, to convert the summarized text information into audio data.

[0699] Optimization by Emotion Engine

[0700] The server detects the user's emotional state and optimizes the output of the spoken summary information using emotion analysis software (e.g., Affectiva or IBM Watson® Tone Analyzer).

[0701] Feedback Loop

[0702] The server learns changes in the user's emotional state and, based on the results, collects recommended information and optimizes voice conversion.

[0703] 3. Examples and prompts

[0704] For example, if a user has viewed many articles about "travel" in the past, the server will collect new articles related to "travel," summarize them, and translate them into audio. If the user is relaxed, the audio will be provided in a slower tone.

[0705] Example prompt sentence:

[0706] "Based on the user's past browsing history, collect, summarize, and voice new travel recommendations. If the user is relaxed, adjust the voiceover tone to a slower tone."

[0707] In this way, the server can efficiently provide useful information to the user and perform voice output in an optimal form according to the emotional state of the user. The flow of the identification process in the third embodiment will be described with reference to FIG.

[0708] Step 1:

[0709] Data collection

[0710] The server collects data on users' browsing history and preferences. As input, it obtains data such as the web pages they visited, the links they clicked, and the duration of their visit. Specifically, it uses analytics tools such as Google Analytics and Mixpanel to collect the data. As output, the collected data is stored in a database.

[0711] Specific behavior:

[0712] The server records the URLs of the web pages that users visit.

[0713] The server stores information about the links that users click.

[0714] The server measures the viewing time of each page and stores the data in a database.

[0715] Step 2:

[0716] Training a machine learning model

[0717] The server uses the collected data to train a machine learning model. As input, it preprocesses the collected user data and converts it into a format suitable for the machine learning model. Specifically, it uses TensorFlow or PyTorch to train the model. As output, a trained model is generated.

[0718] Specific behavior:

[0719] The server preprocesses the collected data, imputing missing values ​​and normalizing the data.

[0720] The server trains a machine learning model using the preprocessed data.

[0721] The server stores the trained model for use in subsequent processing.

[0722] Step 3:

[0723] Recommendation collection and summarization

[0724] The server uses the trained model to collect new recommendations and summarize them. It uses the trained model and new information collected from the internet as input. Specifically, it uses a generative AI model such as BERT or GPT-3 to perform summarization. The output is a summarized text.

[0725] Specific behavior:

[0726] The server uses the trained model to collect recommendations for users from the Internet.

[0727] The server inputs prompts into the generative AI model to summarize the collected information.

[0728] The server stores the generated summary.

[0729] Step 4:

[0730] Text-to-Speech (TTS)

[0731] The server uses TTS technology to convert the summarized text information into speech data. As input, the summarized text information is input to the TTS software. Specific software examples include Google Text-to-Speech API and Amazon Polly. As output, the generated speech data is obtained.

[0732] Specific behavior:

[0733] The server inputs the summarized text information into the TTS software.

[0734] The server receives and stores the generated voice data.

[0735] The server transmits the audio data to the user's terminal.

[0736] Step 5:

[0737] Optimization by Emotion Engine

[0738] The server detects the user's emotional state and optimizes the output method of the voice summary information. As input, it takes the user's emotional state data. To detect the emotional state, it uses emotion analysis software (e.g., Affectiva or IBM Watson Tone Analyzer). As output, it obtains optimized voice data.

[0739] Specific behavior:

[0740] The server acquires emotional state data from the user's device.

[0741] The server uses emotion analysis software to analyze the user's emotional state.

[0742] The server adjusts the tone and speed of the voice based on the analysis results.

[0743] Step 6:

[0744] Feedback Loop

[0745] The server learns changes in the user's emotional state and uses the results to collect recommended information and optimize voice conversion. Data on changes in the user's emotional state is taken as input. Optimized recommended information and voice data are obtained as output.

[0746] Specific behavior:

[0747] The server records the user's emotional state after hearing specific information.

[0748] The server uses the recorded emotion data to retrain the machine learning model.

[0749] The server uses the retrained model to gather new recommendations and optimize the speech generation.

[0750] (Application example 3)

[0751] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0752] In modern society, users have access to a vast amount of digital content, but it is difficult to efficiently collect and consume information that is useful to them. In addition, there is a lack of ways to optimize the way information is provided to users according to their emotional state, which can lead to a decrease in satisfaction with the way information is received. Furthermore, there is a need for information delivery methods that allow users to make effective use of their free time.

[0753] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[0754] In this invention, the server includes means for collecting unread or unviewed information from multiple digital content items, artificial intelligence means for summarizing the collected information, means for converting the summarized information into audio and outputting it, emotion engine means for adjusting the tone and speed of the audio according to the user's emotional state, and machine learning means for automatically collecting recommended information based on the user's past browsing history and preferences and generating summary information including the information. This allows users to efficiently collect information that is useful to them and receive it in an optimal way according to their emotional state. It also allows users to effectively use their free time to consume information.

[0755] "Digital content" is a general term for information and media provided in electronic form.

[0756] "Unread or unviewed information" refers to information that the user has not yet read or viewed.

[0757] "Collection means" refers to the methods and techniques used to obtain the required information from a particular source.

[0758] "Artificial intelligence means for summarizing" refers to artificial intelligence techniques used to summarize collected information succinctly.

[0759] "Means for converting text information into audio and outputting it" refers to technology for converting text information into audio data and providing it to the user.

[0760] "Emotional state" refers to a user's current psychological state or mood.

[0761] "Emotional engine means" refers to technology for adjusting the tone and speed of speech based on the emotional state of the user.

[0762] "Browsing history" refers to a record of web pages and content that a user has previously accessed.

[0763] "Preferences" refer to the types of information or content in which a user is particularly interested.

[0764] "Machine learning methods" refer to technologies that learn patterns based on data and make predictions and classifications.

[0765] "Recommended information" refers to information that is likely to be useful to a user, selected based on the user's past behavior and preferences.

[0766] A system for carrying out this invention collects unread or unviewed information from multiple digital contents, summarizes the collected information, and outputs the summarized information as audio. It also includes an emotion engine means for adjusting the tone and speed of the audio according to the user's emotional state, and machine learning means for automatically collecting recommended information based on the user's past browsing history and preferences, and generating summary information including the information.

[0767] Hardware and software used

[0768] Hardware: Smartphones, smart glasses, servers

[0769] Software: Python, pyttsx3 (TTS library), requests (HTTP request library), textblob (natural language processing library), scikit-learn (machine learning library)

[0770] Processing flow

[0771] 1. Collecting news articles

[0772] The server uses HTTP requests to retrieve the latest news articles from the news API, thereby collecting information that the user has not yet read or watched.

[0773] 2. News article summaries

[0774] The server uses a natural language processing library (textblob) and a machine learning library (scikit-learn) to summarize the collected news articles. Specifically, it vectorizes the articles using TfidfVectorizer and generates summaries using KMeans clustering.

[0775] 3. Audio

[0776] The server converts the summarized text into audio data using a TTS library (pyttsx3), allowing the user to receive the information by voice.

[0777] 4. Emotion Engine

[0778] The server adjusts the tone and speed of the voice depending on the user's emotional state, for example, slowing down the voice if the user is relaxed and increasing the voice speed if the user is excited.

[0779] 5. Collecting recommendations

[0780] The server automatically collects recommended information based on the user's browsing history and preferences using machine learning techniques, and generates summary information that includes the information, thereby efficiently providing useful information to the user.

[0781] Specific examples

[0782] User A: I've read a lot of technology-related articles in the past, so I'd like to hear a summary of the latest technology news.

[0783] User B: Prefers sports news, so they receive a summary of the latest sports news in audio format.

[0784] Prompt Sentence Examples

[0785] Build an application that automatically collects, summarizes, and delivers recommended news articles to a user based on their browsing history. Include the ability to use an emotion engine to adjust the tone and speed of the speech depending on the user's emotional state.

[0786] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[0787] Step 1:

[0788] The server retrieves the latest news articles from the news API using the HTTP request library (requests). It uses the news API endpoint URL as input and gets a list of retrieved news articles as output. Specifically, it sends an API request and parses the JSON data returned as a response to extract the article title and description.

[0789] Step 2:

[0790] The server uses a natural language processing library (textblob) and a machine learning library (scikit-learn) to summarize the collected news articles. It uses a list of news articles as input and obtains summarized text as output. Specifically, it vectorizes the articles using TfidfVectorizer, extracts important parts using KMeans clustering, and generates summaries.

[0791] Step 3:

[0792] The server converts the summarized text into speech data using a TTS library (pyttsx3). It uses the summarized text as input and obtains speech data as output. Specifically, it initializes the pyttsx3 engine, converts the text into speech, and saves it as an audio file.

[0793] Step 4:

[0794] The server adjusts the tone and speed of the voice depending on the user's emotional state. It uses the user's emotional state (relaxed, excited, etc.) as input and obtains adjusted voice data as output. Specifically, it sets pyttsx3 properties (rate and volume) based on the emotional state and adjusts the tone and speed of the voice.

[0795] Step 5:

[0796] The server uses machine learning techniques to automatically collect recommended information based on the user's past browsing history and preferences, and generates summary information including that information. The server uses the user's browsing history and preference data as input, and obtains summary information including the recommended information as output. Specifically, the server uses a machine learning algorithm to learn the user's preferences, collects new information based on that information, and generates a summary.

[0797] Step 6:

[0798] The user receives the audio summary information using a terminal and can listen to it during their free time, such as while doing housework or commuting. The adjusted audio data is used as input, and the user listens to the audio information as output. Specifically, the audio data is played back and provided to the user through a device such as a smartphone or smart glasses.

[0799] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0800] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0801] Another example of generative AI is Gemini (registered trademark) (Internet search engine). <url: https: gemini.google.com ?hl="ja">) are mentioned.

[0802] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0803] [Second embodiment]

[0804] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0805] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0806] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0807] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0808] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0809] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0810] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0811] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0812] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0813] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0814] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0815] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0816] "Example 1"

[0817] In one embodiment of the present invention, APIs that access various digital content platforms used by users are used to collect unread or unwatched information, such as from YouTube's "Watch Later" list, unread messages in messenger apps, and unread lists in bookmark services.

[0818] "Example 2"

[0819] Natural language processing technology is used as an artificial intelligence method to summarize collected information. Specifically, collected text information is analyzed and a summary is generated. This summary expresses the main content of the original information in a shortened form.

[0820] "Example 3"

[0821] Text-to-speech (TTS) technology is used to convert the summarized information into audio and output it. Specifically, the summarized text information is converted into audio data and provided in a form that the user can listen to.

[0822] "Example 4"

[0823] Machine learning technology is used to automatically collect recommended information based on a user's usage history and generate summary information that includes that information. Specifically, the system learns the user's past browsing history and preferences, and then collects and summarizes new recommended information based on that information. This makes it possible to efficiently provide useful information to users.

[0824] The processing flow of each embodiment will be described below.

[0825] "Example 1"

[0826] Step 1: We access the various digital content platforms you use to collect information, such as your YouTube Watch Later list, unread messages in your messaging app, or unread lists in your bookmarking services.

[0827] Step 2: Centralize unread or unwatched information obtained from each platform, allowing users to view information across multiple platforms in one place.

[0828] "Example 2"

[0829] Step 1: Analyze the collected information using natural language processing technology. Through this analysis, the main content of the information is extracted.

[0830] Step 2: Based on the extracted key content, a summary of the information is generated. This summary represents the main content of the original information in a shortened form.

[0831] "Example 3"

[0832] Step 1: Convert the summarized information into audio data using text-to-speech (TTS) technology.

[0833] Step 2: The converted audio data is provided to the user in a form that they can listen to, allowing them to digest the information during their free time, such as during housework or on their way to work.

[0834] "Example 4"

[0835] Step 1: Learn about the user's past browsing history and preferences using machine learning techniques.

[0836] Step 2: Based on the learned results, new recommendations are collected.

[0837] Step 3: The collected recommendation information is summarized and a summary containing it is generated, which makes it possible to efficiently provide useful information to users.

[0838] Example 1

[0839] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0840] Today's users use multiple digital content platforms, and unread or unwatched information often accumulates on each platform. However, there is a lack of a way to centrally manage this information and efficiently extract and provide important information. As a result, users are more likely to miss important information, and information management becomes cumbersome.

[0841] The identification process by the identification processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means. In this invention, the server includes a means for accessing multiple digital content platforms, a means for collecting unread or unviewed information, a means for converting the collected information into a unified format, a means for analyzing the converted information and extracting important information, and a means for providing the extracted information to the user. This allows the user to centrally manage unread or unviewed information from multiple platforms and efficiently check important information.

[0842] "Digital Content Platform" means an online service that enables users to view, listen to or store content in digital form.

[0843] "Means of access" refers to the methods and technologies by which a server connects to a particular digital content platform and obtains the required data.

[0844] "Unread or unwatched information" refers to digital content such as messages, videos, or articles that a user has not yet viewed.

[0845] "Means of collection" refers to the methods or technologies by which the server obtains unread or unwatched information from the digital content platform.

[0846] "Means of converting into a unified format" refers to methods and technologies for converting data collected from different platforms into a consistent format.

[0847] "Means of analysis" refers to the methods and techniques used to analyze the collected and transformed data and extract information that is important to users.

[0848] "Means of extracting important information" refers to methods and techniques for selecting information that is valuable to users from the analyzed data.

[0849] "Means of providing" refers to the methods and technologies used to notify or display extracted important information to the user.

[0850] MODE FOR CARRYING OUT THE INVENTION

[0851] This invention provides a system that efficiently collects unread or unviewed information from multiple digital content platforms used by users, extracts important information, and provides it to users. Specific embodiments of this system are described below.

[0852] System configuration

[0853] Hardware and software configuration:

[0854] The server uses the following hardware and software to implement the system.

[0855] Hardware: A server with a powerful processor, sufficient memory, and ample storage.

[0856] Software: Libraries for API access (e.g., HTTP client libraries), libraries for data analysis (e.g., Python's Pandas, NumPy), database management systems (e.g., MySQL, PostgreSQL)

[0857] Specific APIs:

[0858] Digital content platform APIs: YouTube Data API, Facebook Messenger API, Pocket API, etc.

[0859] Program processing

[0860] The server collects and processes data in the following manner:

[0861] 1. Prepare for API access:

[0862] The server obtains API keys and authentication information for each platform and prepares API access, such as setting an API key for the YouTube Data API, an access token for the Facebook Messenger API, and a consumer key for the Pocket API.

[0863] 2. Data Collection:

[0864] The server calls each platform's API to collect unread or unwatched content, such as YouTube's "Watch Later" list, Facebook Messenger's unread messages, and Pocket's unread bookmarks.

[0865] 3. Data Processing:

[0866] The server converts the collected data into a unified format and assigns unread or unwatched statuses. For example, it converts data obtained from each platform into JSON format.

[0867] 4. Data Analysis:

[0868] The server analyzes the collected data and extracts information that is important to the user, such as unread messages containing specific keywords.

[0869] 5. Providing results:

[0870] The server processes and analyzes the data and provides it to users, who can then check their unread or unwatched information through a dedicated dashboard and notifications.

[0871] Specific examples

[0872] If a user uses YouTube, Facebook Messenger, or Pocket, the server:

[0873] 1. The server sets the API key for the YouTube Data API and retrieves the "Watch Later" list.

[0874] 2. The server sets the access token for the Facebook Messenger API and retrieves the unread messages.

[0875] 3. The server sets the Pocket API consumer key and retrieves unread bookmarks.

[0876] 4. Convert the collected data into JSON format and mark it as unread or unwatched.

[0877] 5. Extract important information and notify the user.

[0878] Example prompt sentence:

[0879] "Generate a program that gathers your YouTube Watch Later list, unread messages in Facebook Messenger, and unread bookmarks in Pocket, extracts the important information, and notifies you."

[0880] In this way, the server realizes a system that efficiently collects unread or unviewed information from various digital content platforms used by users and provides it to users.

[0881] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0882] Step 1: Prepare for API access

[0883] The server obtains the API key and authentication information for each platform and prepares for API access. Specifically, it reads the API key and authentication information from the configuration file and sets the API endpoint for each platform. The input requires the API key and authentication information, and the output is a state where API access is possible.

[0884] Step 2: Data collection

[0885] The server calls the API of each platform to collect unread or unwatched information. Specifically, it uses the YouTube Data API to obtain the "Watch Later" list, the Facebook Messenger API to obtain unread messages, and the Pocket API to obtain unread bookmarks. The input requires a state where API access is ready, and the output is the unread or unwatched information obtained from each platform.

[0886] Step 3: Data processing

[0887] The server converts the collected data into a unified format and assigns an unread or unviewed status. Specifically, it converts the data obtained from each platform into JSON format and assigns an unread or unviewed status. The input requires the collected unread or unviewed information, and the output is the data converted into a unified format.

[0888] Step 4: Data analysis

[0889] The server analyzes the collected data and extracts information that is important to the user. Specifically, it extracts messages containing specific keywords from unread messages, extracts highly rated videos from unwatched YouTube videos, and extracts articles with specific tags from unread bookmarks. The input requires data converted into a unified format, and the output is data from which important information has been extracted.

[0890] Step 5: Delivering results

[0891] The server provides the processed and analyzed data to the user. Specifically, it displays the analysis results on a dashboard and notifies the user of important unread messages and unwatched videos. The user can check the unread or unwatched information through the dashboard and take necessary actions. The input is data from which important information has been extracted, and the output is the information provided to the user.

[0892] (Application example 1)

[0893] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0894] Today's users consume a wide variety of digital content, and the amount of unread or unviewed information tends to increase. However, there is a lack of ways to efficiently manage this information and notify users at the appropriate time, which can lead to users missing important information. To solve this problem, a system is needed that centrally collects unread or unviewed information from multiple digital content and notifies users at the appropriate time.

[0895] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0896] In this invention, the server includes means for collecting unread or unviewed information from multiple digital contents, artificial intelligence means for summarizing the collected information, means for converting the summarized information into audio and outputting it, and means for centrally managing the collected information and notifying the user at an appropriate time. This allows the user to efficiently manage unread or unviewed information from multiple digital contents and receive notifications at an appropriate time without missing important information.

[0897] "Digital content" is a general term for information and media provided via the Internet or electronic devices.

[0898] "Unread or unviewed information" refers to digital content that the user has not yet read or viewed.

[0899] "Means of collection" refers to methods or techniques for obtaining unread or unviewed information from multiple digital content.

[0900] "Artificial intelligence means for summarizing" refers to artificial intelligence techniques used to summarize collected information succinctly.

[0901] "Means for converting and outputting speech" refers to methods or technologies for converting summarized information into speech and providing it to the user.

[0902] "Means for centralized management" refers to methods or technologies for managing collected unread or unviewed information in an integrated manner within a single system.

[0903] "Means of notifying at the appropriate time" refers to methods and technologies for notifying users of information at the most effective time.

[0904] The system for implementing this invention collects unread or unviewed information from multiple digital contents and notifies the user at an appropriate time. The system consists of three main elements: a server, a terminal, and a user.

[0905] server

[0906] The server includes the following means:

[0907] 1. Collection method: The server accesses multiple digital content platforms (e.g., video sharing services, messenger apps, bookmark services) and collects information on unread or unwatched content. Specifically, it obtains the information using the API of each platform.

[0908] 2. Artificial intelligence means of summarization: Using artificial intelligence techniques to summarize collected information, for example, using natural language processing (NLP) techniques to summarize long messages or articles in a concise manner.

[0909] 3. Voice output: The summarized information is converted into voice and provided to the user. Text-to-speech technology (TTS) is used to convert text information into voice.

[0910] 4. Centralized management: Collected unread or unwatched information is consolidated and managed in a single database, allowing users to view all information in a single interface.

[0911] 5. Notifications at the right time: Notifications are sent at the optimal time based on the user's behavioral patterns and schedule. For example, users can receive notifications during their free time, such as during their commute or while doing housework.

[0912] Terminal

[0913] A terminal is a device used by a user, such as a smartphone or tablet. An application is installed on the terminal to receive and display notifications sent from the server. This application has the following functions:

[0914] 1. Receiving and displaying notifications: Receives notifications sent from the server and displays them to the user. Notifications are displayed in the form of popups or banners.

[0915] 2. Audio playback: The audio summary information is played back so that users can listen to it through earphones or speakers.

[0916] user

[0917] Users can use the system to efficiently manage unread or unwatched information and receive timely notifications. Users can check notifications through the application and view detailed information as needed.

[0918] Specific examples

[0919] For example, if a user has five videos added to their "watch later" list on a video sharing service, three unread messages on a messaging app, and two unread articles on a bookmarking service, the server will collect, summarize, and convert this information into audio. It will then send notifications to users during their commute, providing them with the information.

[0920] Prompt Sentence Examples

[0921] A user has five unwatched videos in their "Watch Later" list on a video sharing service. The titles are "Introduction to Programming," "The Future of AI," "Data Science Fundamentals," "Applications of Machine Learning," and "Deep Learning Fundamentals."

[0922] The user has three unread messages in their messenger app: "Hi, how are you?", "Regarding tomorrow's meeting," and "What are your plans for the weekend?"

[0923] The user has two unread articles in their bookmarks service, titled "Current Technology Trends" and "Programming Best Practices."

[0924] Use this information to generate a reminder for your users.

[0925] In this way, users can efficiently manage their information without missing any important information.

[0926] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0927] Step 1:

[0928] The server accesses multiple digital content platforms and collects unread or unviewed information. Specifically, it uses each platform's API to obtain information such as the user's "Watch Later" list, unread messages, and unread bookmarks. The input is each platform's API key or token, and the output is a list of unread or unviewed information.

[0929] Step 2:

[0930] The server uses artificial intelligence techniques to summarize the collected information. Specifically, it uses natural language processing (NLP) techniques to concisely summarize long messages and articles. The input is a list of unread or unwatched information, and the output is the summarized information.

[0931] Step 3:

[0932] The server converts the summarized information into speech. Text-to-speech (TTS) technology is used to convert text information into speech. The input is the summarized information and the output is audio data.

[0933] Step 4:

[0934] The server consolidates and manages the collected unread or unwatched information into a single database, allowing users to view all information in a single interface. The input is a list of unread or unwatched information, and the output is the consolidated database.

[0935] Step 5:

[0936] The server sends notifications at optimal times based on the user's behavioral patterns and schedule. For example, users can receive notifications during their free time, such as while commuting or doing housework. The input is the user's behavioral patterns and schedule information, and the output is notification data.

[0937] Step 6:

[0938] The device receives notifications sent from the server and displays them to the user. Notifications can be displayed in the form of popups or banners. The input is the notification data and the output is the notification that is displayed to the user.

[0939] Step 7:

[0940] The terminal plays back the audio summary information, and the user can listen to the information through earphones or speakers. The input is audio data, and the output is the audio being played back.

[0941] Step 8:

[0942] The user can view the notification through the application and view more information if necessary. The input is the displayed notification, and the output is the user's action (such as viewing more information).

[0943] Example 2

[0944] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0945] In modern society, the amount of information is increasing, making it difficult for users to efficiently obtain the information they need. In addition, when obtaining information, users may not have time to read text information or may prefer to obtain information by voice. Furthermore, there is a demand for a system that automatically collects and summarizes recommended information based on the user's usage history.

[0946] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0947] In this invention, the server includes a means for collecting unread or unviewed information, a natural language processing means for analyzing the collected text information and generating a summary, a means for inputting a prompt sentence into a generation AI model to generate a summary, and a means for converting the summarized information into audio and outputting it. This allows users to efficiently obtain the information they need and acquire it by voice. It is also possible to automatically collect recommended information based on the user's usage history and generate summary information including that information.

[0948] "Unread or unviewed information" refers to digital content that a user has not yet read or viewed.

[0949] "Means for collecting" refers to a function or device for automatically collecting unread or unviewed information.

[0950] "Text information" refers to information written in text, including news articles, reports, blog posts, etc.

[0951] "Analyzing" means analyzing text information using techniques such as tokenization, part-of-speech tagging, and dependency analysis.

[0952] "Natural language processing means" refers to technologies and algorithms for analyzing text information and generating summaries.

[0953] A "generative AI model" is a model that uses artificial intelligence to summarize and generate text information, and examples include GPT-3 and BERT.

[0954] A "prompt" is an instruction given to a generative AI model, containing specific instructions for generating a summary.

[0955] "Means for generating a summary" refers to the functionality or device for inputting a prompt sentence into a generative AI model and generating a summary.

[0956] "Means for converting text information into audio and outputting it" refers to a function or device for converting text information into audio and providing it to the user.

[0957] "Usage history" refers to a record of the information and actions a user has previously accessed.

[0958] "Recommended information" refers to information that is automatically selected based on the user's usage history and is presumed to be useful to the user.

[0959] The present invention provides a system that enables a user to efficiently acquire information and acquire information by voice. A specific embodiment of this system will be described below.

[0960] First, the user enters the text information they want to summarize into their device. For example, they paste the text information, such as a news article or report, into an input form. The device then sends the entered text information to the server as an HTTP request. At this time, the text information is packaged in JSON format.

[0961] The server uses the Python programming language and natural language processing libraries (e.g., spaCy or NLTK) to analyze the received text information. The server tokenizes the text and performs part-of-speech tagging and dependency analysis, which allows it to understand the structure of the text information and extract key content.

[0962] Next, the server inputs a prompt to a generative AI model (such as GPT-3 or BERT) based on the analysis results. The prompt includes instructions for generating a summary. For example, the prompt might be, "Please summarize the following text." The generative AI model generates a summary based on this prompt and the analysis results.

[0963] The generated summary is sent from the server to the device. At this time, the summary is packaged in JSON format. The device displays the summary received from the server to the user. The user can check the summary result on the device screen.

[0964] The server then uses text-to-speech synthesis technology (e.g., a text-to-speech engine) to convert the summarized information into audio and output it. The audio summary information is provided in a format that the user can listen to in their spare time, such as while doing housework or on their way to work.

[0965] As a specific example, suppose a user wants to summarize the news article "A major traffic accident occurred in Tokyo, injuring five people. The cause of the accident is under investigation." In this case, the user enters the news article into an input form on their device, and the device sends the news article to the server. The server tokenizes the news article and performs part-of-speech tagging and dependency analysis. The server then inputs a prompt statement, "Please summarize the following text," into the generative AI model. The generative AI model generates the summary, "A major traffic accident occurred in Tokyo, injuring five people. The cause of the accident is under investigation." The server sends the generated summary to the device, which then displays the summary result to the user.

[0966] In this way, users can easily obtain a summary of text information, and by using audio-translated summary information, users can efficiently acquire information.

[0967] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0968] Step 1:

[0969] The user inputs text information to be summarized into the terminal.

[0970] Specifically, a user copies and pastes text information such as a news article or report into an input form on a terminal, and the text information entered by the user is obtained as input.

[0971] Step 2:

[0972] The terminal transmits the input text information to the server.

[0973] Specifically, the device sends the entered text information to the server as an HTTP request. At this time, the text information is packaged in JSON format. The text information entered by the user is obtained as input, and the JSON format data to be sent to the server is generated as output.

[0974] Step 3:

[0975] The server parses the received text information.

[0976] Specifically, the server uses the Python programming language and natural language processing libraries (e.g., spaCy and NLTK) to tokenize text information, tag parts of speech, and perform dependency analysis. As input, it receives JSON-formatted text information sent from the device, and as output, it generates the analysis results.

[0977] Step 4:

[0978] The server inputs a prompt sentence into the generative AI model based on the analysis results.

[0979] Specifically, the server generates a prompt saying, "Please summarize the following text," and inputs it into the generative AI model along with the analysis results. The analysis results and the prompt are obtained as input, and the data to be input into the generative AI model is generated as output.

[0980] Step 5:

[0981] A generative AI model generates the summary.

[0982] Specifically, a generative AI model (e.g., GPT-3 or BERT) generates a summary based on a prompt and the analysis results. The input is the prompt and the analysis results, and the output is the generated summary.

[0983] Step 6:

[0984] The server transmits the generated summary to the terminal.

[0985] Specifically, the server packages the summary obtained from the generative AI model in JSON format and sends it to the device. The generated summary is obtained as input, and the JSON-formatted data sent to the device is generated as output.

[0986] Step 7:

[0987] The terminal displays the summary results to the user.

[0988] Specifically, the terminal displays the summary received from the server to the user. As input, it receives the JSON-formatted summary sent from the server, and as output, it generates the summary result to be displayed to the user.

[0989] Step 8:

[0990] The server outputs the summarized information as a voice.

[0991] Specifically, the server converts the summary into audio using text-to-speech synthesis technology (e.g., a text-to-speech engine), taking the generated summary as input and generating audio data as output.

[0992] Step 9:

[0993] The terminal provides the user with a voiced summary of the information.

[0994] Specifically, the terminal plays back the audio data and provides it in a form that the user can listen to in their free time, such as while doing housework or on the way to work. As input, the audio data sent from the server is obtained, and as output, audio information to be provided to the user is generated.

[0995] (Application example 2)

[0996] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0997] In modern society, a huge amount of digital content is generated every day, and users are required to efficiently obtain the information they need from it. However, due to the information overload, users are likely to miss important information, and they spend a lot of time obtaining information, which is a problem. In particular, there is the issue of it being difficult to grasp important information such as news articles in a short time.

[0998] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting unread or unviewed information from multiple digital contents, artificial intelligence means for summarizing the collected information, means for converting the summarized information into audio and outputting it, and means for summarizing the collected news articles and providing them to the user. This enables the user to efficiently obtain important information from a vast amount of information and grasp it in a short amount of time.

[0999] "Digital content" means information or media that is electronically generated, stored, or distributed.

[1000] "Unread or unviewed information" means information that the user has not yet read or viewed.

[1001] "Collection methods" refer to the methods and techniques used to obtain the required data from a particular source.

[1002] "Artificial intelligence means for summarizing" refers to a technology that uses natural language processing technology to analyze collected text information and express its main content in a shortened form.

[1003] "Means for converting text information into audio and outputting it" refers to technology for converting text information into audio and providing it to the user.

[1004] A "news article" is a piece of text that describes current events or information provided by the press or other sources.

[1005] "Means of delivery to users" refers to the methods and technologies used to deliver the summarized information in a form that is easy for users to access.

[1006] The system for implementing the present invention includes a server, a user terminal, and an artificial intelligence means. The details of each component and its operation will be described below.

[1007] server

[1008] The server has a means of collecting unread or unwatched information from multiple digital content sources. Specifically, it sends HTTP requests to obtain the latest news articles from external sources such as news APIs, and saves the obtained data in JSON format. The server also has an artificial intelligence means of summarizing the collected information. Here, it uses a pre-trained summarization model using the Hugging Face transformers library. It also has a means of converting the summarized information into audio and outputting it. For audio conversion, it uses speech synthesis technology such as the Google Text-to-Speech API.

[1009] User Device

[1010] The user terminal is a device such as a smartphone or tablet that receives the summary information provided by the server and displays it to the user. The user can view and listen to the summary of the news article through the application. The user terminal also records the user's usage history and sends it to the server, providing data for providing personalized recommendations.

[1011] Artificial Intelligence Tools

[1012] The AI ​​tool uses natural language processing techniques to analyze collected text information and generate summaries. Specifically, it uses the Hugging Face transformers library to represent the key content of news articles in a shortened form. The summarization model has been trained on a large amount of news article data in advance, enabling it to generate summaries with high accuracy.

[1013] Specific examples

[1014] For example, consider the case where the following news articles are collected:

[1015] Original news article:

[1016] Title: "COVID-19 vaccination progresses"

[1017] Content: "COVID-19 vaccinations are progressing rapidly in Japan. The government plans to offer the vaccine to the entire population in the coming months..."

[1018] For this news article, the following prompt sentence is input to the generative AI model:

[1019] Example prompt sentence:

[1020] "COVID-19 vaccination campaigns are progressing. COVID-19 vaccinations are progressing rapidly in Japan. The government plans to offer the vaccine to the entire population in the coming months..."

[1021] The generated summary results are as follows:

[1022] Summary results:

[1023] "COVID-19 vaccinations are progressing rapidly in Japan, and the government plans to offer the vaccine to all citizens."

[1024] This summary is converted into audio and sent to the user's device, allowing the user to grasp the main news in a short amount of time.

[1025] In this way, the server, user terminal, and artificial intelligence means work together, enabling users to efficiently obtain important information from a vast amount of information and grasp it in a short amount of time.

[1026] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1027] Step 1:

[1028] The server collects the latest news articles from external sources such as a news API by sending HTTP requests and saving the retrieved data in JSON format. The input is the news API endpoint, and the output is the news article data in JSON format.

[1029] Step 2:

[1030] The server analyzes the collected news article data and extracts the content of each article. Specifically, it extracts information such as the article title, text, and publication date and time from the JSON data. The input is news article data in JSON format, and the output is the text information of each news article.

[1031] Step 3:

[1032] The server uses a generative AI model to summarize the extracted text information of news articles. Specifically, it uses the Hugging Face transformers library to input a prompt sentence into the summarization model and obtains the summary result. The input is the text information of the news article, and the output is the summarized text information.

[1033] Step 4:

[1034] The server converts the summarized text information into audio. Specifically, it uses speech synthesis technology such as the Google Text-to-Speech API to convert the text information into audio data. The input is the summarized text information, and the output is audio data.

[1035] Step 5:

[1036] The server sends the voice data and summary text information to the user device. Specifically, it sends data to the user device via push notifications or API requests. The input is the voice data and summary text information, and the output is data delivery to the user device.

[1037] Step 6:

[1038] The user terminal displays and plays the received summary text information and audio data. Specifically, it displays the summary text and plays the audio data through an application. The input is the audio data and summary text information sent from the server, and the output is information provided to the user.

[1039] Step 7:

[1040] The user can view the provided summary information and listen to it aloud. Specifically, by operating the application, the user can check the summary of the news article and play it aloud. The input is the summary information displayed on the user's device, and the output is the user's information acquisition.

[1041] Step 8:

[1042] The user device records the user's usage history and sends it to the server. Specifically, it saves the history of articles the user has viewed and audio they have played, and periodically uploads it to the server. The input is the user's usage history data, and the output is the transmission of the history data to the server.

[1043] Step 9:

[1044] The server analyzes the received usage history data and generates personalized recommendations for the user. Specifically, it selects relevant news articles based on the usage history and generates summary information. The input is the user's usage history data, and the output is personalized summary information.

[1045] Step 10:

[1046] The server then sends the personalized summary information back to the user's device and provides it to the user. Specifically, it sends data via push notifications or API requests. The input is the personalized summary information, and the output is the delivery of data to the user's device.

[1047] Example 3

[1048] Next, a description will be given of Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1049] In modern society, users have more and more opportunities to access a large amount of digital content, but it is difficult to efficiently obtain useful information from that content. It is also difficult for users to find time to obtain information in their busy daily lives. In particular, there is a demand for a means for users to obtain information by effectively utilizing their free time, such as while doing housework or on their way to work.

[1050] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[1051] In this invention, the server includes means for collecting user usage history, means for automatically collecting recommended information based on the collected usage history, artificial intelligence means for summarizing the collected information, means for converting the summarized information into voice data, and means for providing the voice data to the user's terminal. This allows users to efficiently obtain useful information even in their busy daily lives and effectively utilize free time such as during housework or on the way to work.

[1052] "Means for collecting user usage history" refers to a function for collecting behavioral data when a user browses a website or application.

[1053] The "means for automatically collecting recommended information" is a function for automatically collecting highly relevant information from the Internet based on the user's past usage history and preferences.

[1054] "Artificial intelligence means" are systems that include machine learning models and natural language processing techniques used to analyze and summarize collected information.

[1055] The "means for converting the summarized information into audio data" is a function that uses text-to-speech (TTS) technology to convert the summarized information in text form into audio form.

[1056] The "means for providing voice data to the user's terminal" is a function for transmitting the generated voice data to the user's terminal and providing it in a reproducible form.

[1057] This invention is a system that allows users to efficiently obtain useful information and make effective use of free time in their daily lives. This system performs a series of processes: collects the user's usage history, automatically collects recommended information based on that history, summarizes the collected information, converts the summarized information into voice data, and provides the voice data to the user's terminal.

[1058] Hardware and software used

[1059] Hardware: Servers, terminals (PCs, smartphones)

[1060] Software: Text-to-Speech (TTS) engine, machine learning models, database management system (DBMS)

[1061] Specific operation of the system

[1062] 1. Collection of user usage history

[1063] User: Browse websites and applications.

[1064] Device: Collects information about the user's browsing history and preferences, such as the URLs of pages visited, the duration of browsing, and the links clicked.

[1065] Server: Stores browsing history and preference information sent from your device in a database.

[1066] 2. Automatic collection of recommendations

[1067] Server: Uses machine learning models to learn users' browsing history and preferences, specifically analyzing user behavior patterns using Python libraries such as Scikit-learn and TensorFlow.

[1068] Server: Based on the learning results, the server automatically collects new recommended information from the Internet, for example, retrieving related articles and product information from news sites and shopping sites.

[1069] 3. Summary information generation

[1070] Server: Uses natural language processing (NLP) techniques to summarize collected recommendations, specifically using Python libraries such as NLTK and spaCy to create short summaries of long articles and product descriptions.

[1071] Server: Generates summarized text information and stores it in a database, for example, summarizing news articles or product features.

[1072] 4. Audio translation of summary information

[1073] Server: Inputs the stored summary text information into a text-to-speech (TTS) engine, specifically using a TTS service such as Google Text-to-Speech API or Amazon Polly.

[1074] TTS engine: Converts summary text into audio data, for example, converting a news article summary into an audio file (e.g., MP3 format).

[1075] 5. Provision of audio data

[1076] User: Uses the device to request voice data, for example, opens a smartphone app and types the voice command "play me the latest news."

[1077] Server: Sends the generated audio data to the user's device by streaming the audio file via an HTTP request.

[1078] Device: Plays the received audio data and provides it in a form that the user can listen to. For example, the audio can be played through a smartphone speaker or earphones.

[1079] Specific examples

[1080] Example 1: News summarization and audio transcription

[1081] User: Frequent news site visitor.

[1082] Device: Collects user browsing history (URLs of visited news articles and viewing times).

[1083] Server: The collected browsing history is stored in a database and analyzed using machine learning models.

[1084] Server: Collects and summarizes the latest news articles based on user interests.

[1085] Server: The summarized news article is input into the TTS engine and converted into voice data.

[1086] User: Requests "Listen to the latest news" on their smartphone during their commute.

[1087] Server: Sends voice data to the user's smartphone.

[1088] Terminal: Plays audio data and allows the user to listen to the news.

[1089] Example 2: Product Recommendations

[1090] User: Visits an online shopping site.

[1091] Device: Collects user purchase and browsing history.

[1092] Server: Stores the collected history in a database and analyzes it using machine learning models.

[1093] Server: Collects and summarizes new products based on user interests.

[1094] Server: The summarized product description is input into the TTS engine and converted into voice data.

[1095] User: While doing housework, the user requests on their smartphone, "Tell me what products you recommend."

[1096] Server: Sends voice data to the user's smartphone.

[1097] Terminal: The audio data is played and the user listens to the product information.

[1098] Prompt Sentence Examples

[1099] "Collect recommended news articles based on the user's past browsing history, summarize them, and convert them into audio data."

[1100] "Based on your online shopping site usage history, please summarize new product recommendations and provide them to you via voice."

[1101] This system allows users to efficiently acquire information and easily obtain useful information in their daily lives. The flow of the identification process in the third embodiment will be described with reference to FIG.

[1102] Step 1:

[1103] Collection of user usage history

[1104] User: Browse websites and applications.

[1105] Device: Collects information about the user's browsing history and preferences, such as the URLs of pages visited, the duration of browsing, and the links clicked.

[1106] Input: User browsing behavior data (URL, browsing time, clicked links, etc.)

[1107] Output: Collected browsing history data

[1108] Specific operation: The device monitors user behavior data in real time and sends the collected data to the server.

[1109] Step 2:

[1110] Automatic collection of recommendations

[1111] Server: Uses machine learning models to learn users' browsing history and preferences, specifically analyzing user behavior patterns using Python libraries such as Scikit-learn and TensorFlow.

[1112] Server: Based on the learning results, the server automatically collects new recommended information from the Internet, for example, retrieving related articles and product information from news sites and shopping sites.

[1113] Input: Collected browsing history data

[1114] Output: Recommendations (news articles, product information, etc.)

[1115] What it does: The server runs machine learning models to gather relevant information from the internet based on the user's interests and preferences.

[1116] Step 3:

[1117] Generate summary information

[1118] Server: Uses natural language processing (NLP) techniques to summarize collected recommendations, specifically using Python libraries such as NLTK and spaCy to create short summaries of long articles and product descriptions.

[1119] Server: Generates summarized text information and stores it in a database, for example, summarizing news articles or product features.

[1120] Input: Recommended information (news articles, product information, etc.)

[1121] Output: Summarized text information

[1122] How it works: The server uses NLP technology to analyze the collected information, extract key points, and generate a summary.

[1123] Step 4:

[1124] Audio translation of summary information

[1125] Server: Inputs the stored summary text information into a text-to-speech (TTS) engine, specifically using a TTS service such as Google Text-to-Speech API or Amazon Polly.

[1126] TTS engine: Converts summary text into audio data, for example, converting a news article summary into an audio file (e.g., MP3 format).

[1127] Input: Summarized text information

[1128] Output: Audio data (MP3 format, etc.)

[1129] Specific operation: The server sends the summary text to the TTS engine, which generates the audio data.

[1130] Step 5:

[1131] Providing audio data

[1132] User: Uses the device to request voice data, for example, opens a smartphone app and types the voice command "play me the latest news."

[1133] Server: Sends the generated audio data to the user's device by streaming the audio file via an HTTP request.

[1134] Device: Plays the received audio data and provides it in a form that the user can listen to. For example, the audio can be played through a smartphone speaker or earphones.

[1135] Input: User request (e.g. voice command)

[1136] Output: Playback of audio data

[1137] Specific operation: Upon receiving a user request, the server sends the corresponding audio data to the terminal, and the terminal plays the audio data.

[1138] (Application example 3)

[1139] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1140] In modern society, users have access to a vast amount of digital content, but it is difficult to efficiently collect and consume information that is useful to them. It is also difficult for users to find time to obtain information in their busy daily lives. In particular, there is a demand for methods to obtain information that allow users to effectively utilize their free time, such as during their commute or while doing housework.

[1141] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[1142] In this invention, the server includes means for collecting unread or unviewed information from a plurality of digital contents, artificial intelligence means for summarizing the collected information, means for converting the summarized information into audio and outputting it, means for automatically collecting recommended information based on the user's usage history, means for summarizing the collected recommended information, means for converting the summarized recommended information into audio and outputting it, and means for providing the summarized audio information in a form that the user can listen to in their spare time. This allows the user to efficiently collect information that is useful to them and effectively utilize their spare time to obtain information.

[1143] "Digital content" is a general term for information and media provided in electronic form.

[1144] "Unread or unviewed information" refers to information that the user has not yet read or viewed.

[1145] "Collection methods" refers to the methods and techniques used to collect specific information.

[1146] "Artificial intelligence means for summarizing" refers to artificial intelligence techniques used to summarize collected information in a concise manner.

[1147] "Means for converting text information into audio and outputting it" refers to a method for converting text information into audio data and providing it to the user.

[1148] "Usage history" refers to a record of the information and actions a user has previously accessed.

[1149] "Recommended information" refers to information selected based on the user's interests and concerns.

[1150] "Automatic collection means" refers to a method of automatically collecting information based on specific conditions.

[1151] "Summarization methods" refer to methods and techniques for summarizing information succinctly.

[1152] "Means for converting text information into audio and outputting it" refers to a method for converting text information into audio data and providing it to the user.

[1153] "Free time" refers to free time when a user is not engaged in other activities.

[1154] "A form that can be listened to without interruption" refers to a form in which the user can receive audio information without having to perform any specific operation.

[1155] The system for implementing this invention is composed of a server, a user terminal, and related software components. The details of each component and its role will be explained below.

[1156] server

[1157] The server has the following main functions:

[1158] 1. Information collection methods:

[1159] The server collects unread or unwatched information from multiple digital content sources, using external sources such as news APIs and RSS feeds.

[1160] 2. Artificial Intelligence Means:

[1161] The server uses natural language processing techniques to summarize the collected information, specifically using Hugging Face's Transformers library to summarize the collected text data.

[1162] 3. Vocalization methods:

[1163] The server uses the Google Text-to-Speech (gTTS) library to convert the summarized information into audio data, which generates an audio file from the text information.

[1164] 4. How we collect recommendations based on your usage history:

[1165] The server automatically collects recommendations based on the user's browsing history and preferences, using machine learning algorithms to analyze the user's behavioral patterns.

[1166] 5. Recommended information summary methods:

[1167] The server again uses natural language processing techniques to summarize the collected recommendations.

[1168] 6. Audio recommendation methods:

[1169] The server again uses the Google Text-to-Speech (gTTS) library to convert the summarized recommendations into audio data.

[1170] 7. Audio information delivery methods:

[1171] The server provides the summarized audio information in a format that users can listen to at their leisure by sending the audio file to the user's terminal and allowing the user to play it on their smartphone or other device.

[1172] User terminal

[1173] The user terminal is a device such as a smartphone or tablet, and has the following functions:

[1174] 1. Receiving and playing audio files:

[1175] The user terminal is equipped with an application for receiving and playing audio files sent from the server, allowing the user to listen to audio information during their free time, such as while commuting or doing housework.

[1176] Specific examples

[1177] For example, if a user is interested in "technology," the server collects the latest technology news, summarizes it, and provides it in audio format, allowing the user to listen to the latest technology news through earphones during their commute.

[1178] Prompt Sentence Examples

[1179] Build an application that gathers the latest news articles, summarizes them and delivers them to you via audio, based on the user's preference: Technology.

[1180] In this way, users can efficiently collect information that is useful to them and can obtain information by effectively utilizing their free time.

[1181] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[1182] Step 1:

[1183] The server collects multiple digital content from external sources such as news APIs and RSS feeds. As input, it receives category information based on the user's interests and retrieves digital content including unread or unwatched information as output. Specifically, it sends an API request and parses the retrieved JSON data to extract the required information.

[1184] Step 2:

[1185] The server uses natural language processing techniques to summarize the collected digital content. It receives the collected text data as input and generates summarized text information as output. Specifically, it uses Hugging Face's Transformers library to summarize the text data.

[1186] Step 3:

[1187] The server converts the summarized text information into audio data. It receives the summarized text information as input and generates an audio file as output. Specifically, it uses the Google Text-to-Speech (gTTS) library to convert the text information into an audio file (e.g., MP3 format).

[1188] Step 4:

[1189] The server automatically collects recommended information based on the user's usage history. It receives the user's past browsing history and preference data as input, and obtains recommended digital content as output. Specifically, it uses machine learning algorithms to analyze the user's behavioral patterns and collect highly relevant information.

[1190] Step 5:

[1191] The server again uses natural language processing techniques to summarize the collected recommendations. It receives the recommendation text data as input and generates summarized recommendations as output. Specifically, it again uses Hugging Face's Transformers library to summarize the text data.

[1192] Step 6:

[1193] The server converts the summarized recommendation information into audio data. It receives the summarized recommendation information as input and generates an audio file as output. Specifically, it again uses the Google Text-to-Speech (gTTS) library to convert the text information into an audio file.

[1194] Step 7:

[1195] The server sends the summarized audio information to the user terminal. As input, it receives the generated audio file and as output, it sends the audio file to the user terminal using an HTTP request so that the user can play it on their smartphone or other device.

[1196] Step 8:

[1197] The user terminal receives and plays the audio file sent from the server. It receives the audio file sent from the server as input and plays the audio information as output. Specifically, it plays the audio file using a dedicated application, allowing the user to listen to it during their free time, such as during their commute or while doing housework.

[1198] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1199] "Example 1"

[1200] In one embodiment of the present invention, the emotion engine collects and analyzes information such as the user's voice, text input, facial expression, and biometric information (e.g., heart rate and electrodermal activity) to recognize the user's emotions. Specifically, the emotion engine infers the user's emotional state from the tone and pitch of the user's voice when speaking to the system, the choice of words in text input, changes in facial expression, and fluctuations in biometric information.

[1201] "Example 2"

[1202] The emotion engine automatically collects recommended information based on the user's emotional state. For example, if a user is expressing joy, it will collect fun news and entertainment information that may enhance that emotion. Conversely, if a user is expressing sadness, it will collect soothing content and positive information that may soothe that emotion.

[1203] "Example 3" The emotion engine also optimizes the way the audio summary information is output depending on the user's emotional state. For example, if the user is in a relaxed emotional state, the tone and speed of the audio will be adjusted to a slower tone. Conversely, if the user is in an excited emotional state, the tone and speed of the audio will be adjusted to a more active tone.

[1204] "Example 4"

[1205] Furthermore, the emotion engine learns changes in the user's emotional state and uses the results to collect recommended information and optimize voice conversion. For example, if a user tends to express joy after hearing certain information, the engine will prioritize collecting information similar to that information. Also, if a user tends to express favorable emotions toward a certain voice tone or speed, the engine will prioritize using that tone or speed.

[1206] The processing flow of each embodiment will be described below.

[1207] "Example 1"

[1208] Step 1: Collect information such as the user's voice, text input, facial expressions, and biometric information (e.g., heart rate and electrodermal activity).

[1209] Step 2: Analyze the collected information and estimate the user's emotional state.

[1210] "Example 2"

[1211] Step 1: Recognize the user's emotional state with the emotion engine.

[1212] Step 2: Automatically collect recommended information based on your emotional state.

[1213] "Example 3"

[1214] Step 1: Recognize the user's emotional state with the emotion engine.

[1215] Step 2: Optimize the output method of the spoken summary information according to the user's emotional state.

[1216] "Example 4"

[1217] Step 1: The emotion engine learns the user's emotional state and its changes.

[1218] Step 2: Based on the learning results, collect recommended information and optimize the voice conversion.

[1219] Example 1

[1220] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1221] In today's digital society, users have access to a large amount of digital content, and the amount of unread or unviewed information is increasing, making it difficult to efficiently manage and digest it. Furthermore, providing information without considering the user's emotional state can reduce user satisfaction. Therefore, there is a need for a system that efficiently collects and summarizes unread or unviewed information and provides it appropriately according to the user's emotional state.

[1222] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1223] In this invention, the server includes means for collecting unread or unviewed information from multiple digital contents, artificial intelligence means for summarizing the collected information, means for converting the summarized information into voice and outputting it, means for collecting voice, text input, facial expression, and biometric information to recognize the user's emotions, and means for analyzing the collected information and estimating the user's emotional state. This makes it possible to efficiently manage unread or unviewed information and provide appropriate information according to the user's emotional state.

[1224] "Digital content" is a general term for information and media provided in electronic form, including video, audio, text, and images.

[1225] "Unread or unviewed information" refers to digital content information that a user has not yet read or viewed.

[1226] "Collection methods" refers to the methods and technologies used to obtain unread or unwatched information from a particular digital content platform.

[1227] "Artificial intelligence means" refers to techniques such as machine learning and natural language processing used to analyze and summarize collected information.

[1228] "Means for converting and outputting speech" refers to the technology or device for converting summarized information into speech and providing it to the user.

[1229] "Means of collecting voice, text input, facial expressions, and biometric information for emotion recognition" refers to methods and technologies for obtaining the data necessary to understand a user's emotional state.

[1230] "Means for analyzing and inferring a user's emotional state" refers to methods and technologies for assessing and inferring a user's emotions based on collected data.

[1231] MODE FOR CARRYING OUT THE INVENTION

[1232] This invention is a system that collects unread or unviewed information from multiple digital content platforms used by a user and provides the information appropriately according to the user's emotional state. Specific embodiments of this system are described below.

[1233] Information gathering process

[1234] The server prepares API keys and authentication information for accessing various digital content platforms. This allows authenticated requests to be sent to each platform. Specifically, the server obtains and stores the YouTube API key, the access token for the messenger app, the consumer key for the bookmark service, and so on.

[1235] The server then sends a request to each platform's API to obtain information about unread or unwatched videos. This allows the server to collect information about videos that the user has not yet viewed or read. Specifically, the server uses the YouTube API to obtain information about videos in the "Watch Later" list, the Messenger app's API to obtain unread messages, and the bookmark service's API to obtain unread bookmarks.

[1236] The collected information is stored in a database by the server, which allows the data to be used efficiently for subsequent processing.

[1237] Emotion Engine Process

[1238] The device collects information necessary to recognize the user's emotions, including voice, text input, facial expressions, biometric information, etc. Specifically, it uses a microphone to collect the user's voice, monitors keyboard input to collect text input, uses a camera to capture facial expressions, and biometric sensors to collect heart rate and electrodermal activity.

[1239] The device then analyzes the collected information to infer the user's emotional state, including through analysis of voice recognition software (e.g., voice recognition APIs), natural language processing tools (e.g., generative AI models), facial recognition software (e.g., facial recognition APIs), and biometric sensor data. Specifically, the device analyzes the tone and pitch of the voice, word choice in the text, changes in facial expression, and biometric fluctuations.

[1240] Based on the analysis results, the device estimates the user's emotional state, allowing it to understand the user's current emotional state and respond appropriately.

[1241] Examples of specific examples and prompts

[1242] As a concrete example, if you want to use the YouTube API to retrieve unwatched videos from your "Watch Later" list, you would use the following prompt:

[1243] Example prompt sentence:

[1244] "Get unwatched videos from your YouTube Watch Later list."

[1245] Also, when analyzing the user's voice tone and pitch to infer their emotional state, use prompts like the following:

[1246] Example prompt sentence:

[1247] "Analyze the user's voice tone and pitch to estimate their emotional state."

[1248] In this way, the server and the device work together to efficiently collect and manage the user's unread or unviewed information, and provide appropriate information according to the user's emotional state.

[1249] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1250] Step 1:

[1251] The server prepares API keys and authentication information for accessing various digital content platforms. Specifically, it obtains and stores on the server the YouTube API key, the access token for the messenger app, the consumer key for the bookmark service, etc.

[1252] Input: API key or authentication information

[1253] Output: Ready to send authenticated requests

[1254] Specific operation: Obtain an API key and access token and save them in the server configuration file.

[1255] Step 2:

[1256] The server sends requests to each platform's API to get information about unread or unwatched videos. Specifically, it uses the YouTube API to get information about videos in the "Watch Later" list, the Messenger app's API to get unread messages, and the bookmark service's API to get unread bookmarks.

[1257] Input: Credentials, API request

[1258] Output: Unread or unheard information

[1259] Specific operation: Sends an API request to retrieve unread or unwatched information from each platform.

[1260] Step 3:

[1261] The server stores the collected unread or unwatched information in a database, which allows the data to be used efficiently in subsequent processing.

[1262] Input: Unread or unwatched information

[1263] Output: Information stored in the database

[1264] Specific operation: Execute a query to insert the retrieved information into a database.

[1265] Step 4:

[1266] The device collects the information needed to recognize the user's emotions by using a microphone to capture the user's voice, monitoring keyboard input to collect text input, a camera to capture facial expressions, and biometric sensors to collect heart rate and electrodermal activity.

[1267] Input: User voice, text input, facial expressions, biometric information

[1268] Output: Collected emotion recognition data

[1269] What it does: Collects data using microphones, keyboards, cameras, and biometric sensors.

[1270] Step 5:

[1271] The device analyzes the collected information to infer the user's emotional state, specifically by analyzing data from voice recognition software, natural language processing tools, facial recognition software, and biometric sensors.

[1272] Input: Collected emotion recognition data

[1273] Output: Estimation of the user's emotional state

[1274] Specific actions: Analyzes tone and pitch of voice, analyzes word choice in text, analyzes changes in facial expressions, and evaluates fluctuations in biometric information.

[1275] Step 6:

[1276] The device estimates the user's emotional state based on the analysis results, allowing it to understand the user's current emotional state and respond appropriately.

[1277] Input: Analysis results

[1278] Output: Estimation of the user's emotional state

[1279] Specific operation: Integrates the analysis results and runs an algorithm to estimate the user's emotional state.

[1280] (Application example 1)

[1281] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1282] Today's digital content users are surrounded by a lot of unread or unwatched information, making it difficult to efficiently manage it and digest it in a timely manner. Furthermore, there is a lack of ways to provide optimal content according to the user's emotional state, resulting in a lack of an improved user experience.

[1283] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1284] In this invention, the server includes means for collecting unread or unviewed information from a plurality of digital contents, artificial intelligence means for summarizing the collected information, means for converting the summarized information into voice and outputting it, emotion recognition means for collecting and analyzing the user's voice, text input, facial expression, and biometric information, and means for recommending optimal content based on the emotion recognition means, thereby making it possible to provide optimal content according to the user's emotional state.

[1285] "Digital content" is a general term for information and media provided via the Internet or electronic devices.

[1286] "Unread or unviewed information" refers to digital content that a user has not yet read or viewed.

[1287] "Collection methods" means the methods or technologies used to obtain unread or unwatched information from a digital content platform.

[1288] "Artificial intelligence means" refers to algorithms or systems that analyze and summarize collected information.

[1289] "Means for converting and outputting speech" refers to the technology or device that converts the summarized information into speech and provides it to the user.

[1290] "Emotion recognition means" refers to technology or systems that collect and analyze a user's voice, text input, facial expressions, and biometric information to estimate the user's emotional state.

[1291] "Recommendation means" refers to methods and technologies for recommending optimal content to users based on emotion recognition means.

[1292] A system for carrying out this invention is configured as follows: A server includes means for collecting unread or unviewed information from a plurality of digital contents, artificial intelligence means for summarizing the collected information, means for converting the summarized information into voice and outputting it, emotion recognition means for collecting and analyzing a user's voice, text input, facial expression, and biometric information, and means for recommending optimal content based on the emotion recognition means.

[1293] Hardware and software used

[1294] Hardware: Smartphone (microphone, camera, heart rate sensor, etc.)

[1295] Software: Python, EmotionRecognizer library, ContentRetriever library, various APIs (APIs for digital content platforms)

[1296] Data processing and calculation

[1297] 1. Data Collection:

[1298] Voice Data: We collect your voice using your smartphone's microphone.

[1299] Text data: Collects user-entered text.

[1300] Facial Expression Data: Collecting the user's facial expressions using the smartphone camera.

[1301] Biometric data: We collect your biometric information using the heart rate sensor on your smartphone.

[1302] 2. Emotion recognition:

[1303] The server analyzes the collected voice data, text data, facial expression data, and biometric data using the EmotionRecognizer library to estimate the user's emotional state.

[1304] 3. Content Collection:

[1305] The server collects unread or unviewed content using the API of the digital content platform.

[1306] 4. Content Recommendation:

[1307] The server recommends the most suitable content in the ContentRetriever library based on the estimated emotional state.

[1308] Specific examples

[1309] For example, if a user is feeling stressed, the server will recommend relaxing music or videos, and if the user is feeling excited, it will recommend entertaining content.

[1310] Prompt Sentence Examples

[1311] "If the user's emotional state is estimated to be 'stressed,' recommend relaxing music or videos. For example, select relaxing content from the 'Watch Later' list of a digital content platform."

[1312] In this way, it becomes possible to provide optimal content according to the user's emotional state.

[1313] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1314] Step 1:

[1315] The server collects the user's voice data using the smartphone's microphone. The voice data includes the content, tone, and pitch of what the user said. The input is the user's voice, and the output is the collected voice data.

[1316] Step 2:

[1317] The server collects text data from the user using the smartphone's text input function. The text data includes strings of characters and word choices entered by the user. The input is the user's text input, and the output is the collected text data.

[1318] Step 3:

[1319] The server collects facial expression data of the user using the smartphone camera. The facial expression data includes changes in the user's facial expressions and facial features. The input is the user's facial expression, and the output is the collected facial expression data.

[1320] Step 4:

[1321] The server collects the user's biometric data using the smartphone's heart rate sensor. The biometric data includes the user's heart rate and electrodermal activity. The input is the user's biometric information, and the output is the collected biometric data.

[1322] Step 5:

[1323] The server analyzes the collected voice data, text data, facial expression data, and biometric data using the EmotionRecognizer library to estimate the user's emotional state. The input is the collected data, and the output is the estimated emotional state.

[1324] Step 6:

[1325] The server collects unread or unviewed content using the API of the digital content platform, where the input is the API key or endpoint of the digital content platform and the output is the collected unread or unviewed content.

[1326] Step 7:

[1327] The server recommends the most suitable content based on the estimated emotional state using the ContentRetriever library. The input is the estimated emotional state and collected unread or unwatched content, and the output is the recommended content.

[1328] Step 8:

[1329] The server provides the recommended content to the user. Specifically, it plays a voiced summary of the content to the user through the smartphone. The input is the recommended content, and the output is the voiced summary provided to the user. The server may also make recommendations at optimal times based on the user's behavioral patterns and schedule.

[1330] Example 2

[1331] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1332] Conventional information gathering systems do not provide information according to the user's emotional state, making it difficult to provide information appropriate to the user's emotions. Furthermore, collected information is not adequately summarized or converted into audio, making it difficult for users to acquire information efficiently. This leads to issues such as reduced user satisfaction and a worsening efficiency of information utilization.

[1333] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1334] In this invention, the server includes means for collecting unread or unviewed information from multiple digital contents, artificial intelligence means using natural language processing technology to summarize the collected information, means for detecting the emotional state of the user and automatically collecting recommended information according to that emotion, and means for converting the summarized information into audio and outputting it. This makes it possible to provide information according to the emotional state of the user, allowing for efficient information acquisition.

[1335] "Digital content" means information or media stored or distributed in electronic form.

[1336] "Unread or unviewed information" means information that the user has not yet read or viewed.

[1337] "Natural language processing technology" refers to technology that allows computers to understand, interpret, and generate human language.

[1338] "Artificial intelligence means" refers to systems that automatically process information using machine learning and data analysis.

[1339] An "emotional state" is a state that indicates a user's current feelings or mood.

[1340] "Recommended information" refers to information that is useful or interesting to the user, selected based on the user's emotional state and usage history.

[1341] "Speech" refers to the process of converting text information into speech.

[1342] "Output" refers to providing processed information to a user.

[1343] MODE FOR CARRYING OUT THE INVENTION

[1344] The present invention is a system for providing information according to a user's emotional state, and is implemented as follows.

[1345] Information gathering and summary generation

[1346] The server collects text information from online news sites, social media, and other sources. Specifically, it obtains the latest news articles using RSS feeds and APIs. The collected text information is then analyzed using natural language processing technology. The software used includes "natural language processing APIs" and "generative AI models."

[1347] Examples:

[1348] Original information: "The spread of COVID-19 has led to lockdowns in many countries. Economic activity has stagnated, affecting many businesses."

[1349] Summary: "The spread of COVID-19 has led to lockdowns in many countries and economic activity has stagnated."

[1350] Example prompt sentence:

[1351] "Summarize the following text: The coronavirus outbreak is spreading, leading to lockdowns in many countries. Economic activity is slowing down, affecting many businesses."

[1352] Information gathering using emotion engines

[1353] The device detects the user's emotional state and automatically collects recommended information based on that emotion. The emotion engine uses an "emotion analysis API" and "text analysis tools."

[1354] Examples:

[1355] User's emotional state: joy

[1356] Information collected: Fun news and entertainment information

[1357] User's emotional state: Sadness

[1358] Information collected: Soothing content and positive information

[1359] Example prompt sentence:

[1360] "If a user is expressing an emotion of joy, gather fun news and entertainment information that may enhance that emotion."

[1361] "If users are expressing feelings of sadness, seek out soothing content and positive information that may alleviate those feelings."

[1362] Vocalization and output

[1363] The server converts the summarized information into audio and provides it to the user. The audio is generated using a speech synthesis API. The audio information is provided in a format that users can listen to during their free time, such as while doing housework or on their way to work.

[1364] This system allows users to acquire information efficiently and receive appropriate information according to their emotional state.

[1365] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1366] Program processing flow

[1367] Step 1: Gather information

[1368] The server collects text information from news sites and social networking sites on the Internet. It uses RSS feeds and API URLs as input and obtains the collected text information as output.

[1369] Specific behavior:

[1370] The server periodically checks the RSS feed of the news site to see if there are any new articles.

[1371] The server retrieves social media posts through the API.

[1372] Step 2: Text analysis

[1373] The server analyzes the collected text information using natural language processing technology. The collected text information is used as input, and the analysis results (tokenization, part-of-speech tagging, sentence structure analysis, etc.) are obtained as output.

[1374] Specific behavior:

[1375] The server uses a natural language processing API to tokenize the text and tag each token with a part of speech.

[1376] The server analyzes the structure of the sentence and extracts relationships such as subject, predicate, and object.

[1377] Step 3: Summary generation

[1378] The server generates a summary based on the analysis results. It uses the analysis results as input and obtains the summarized text information as output.

[1379] Specific behavior:

[1380] The server inputs the analysis results into a generative AI model and creates a prompt sentence that generates a summary.

[1381] The server inputs a prompt sentence into the generative AI model and obtains a summary.

[1382] Example prompt sentence:

[1383] "Summarize the following text: The coronavirus outbreak is spreading, leading to lockdowns in many countries. Economic activity is slowing down, affecting many businesses."

[1384] Step 4: Detecting Emotional State

[1385] The device analyzes the user's input (text message or voice) to detect the emotional state. As input, it uses the user's text message or voice data. As output, it obtains the detected emotional state.

[1386] Specific behavior:

[1387] The device sends the text message entered by the user to the sentiment analysis API for sentiment analysis.

[1388] The device converts the user's voice input into text and performs sentiment analysis using text analysis tools.

[1389] Step 5: Gather recommendations

[1390] The device collects recommendation information according to the detected emotional state. It uses the detected emotional state as input and obtains the recommendation information as output.

[1391] Specific behavior:

[1392] If the user expresses joy, the device will use the news API to retrieve entertainment-related articles.

[1393] The device collects information from websites that offer soothing content if the user expresses sadness.

[1394] Example prompt sentence:

[1395] "If a user is expressing an emotion of joy, gather fun news and entertainment information that may enhance that emotion."

[1396] "If users are expressing feelings of sadness, seek out soothing content and positive information that may alleviate those feelings."

[1397] Step 6: Audio and Output

[1398] The server converts the summarized information into audio and provides it to the user. It uses the summarized text information as input and obtains the audio information as output.

[1399] Specific behavior:

[1400] The server uses a speech synthesis API to convert the summarized text information into speech.

[1401] The server outputs the vocalized information through an interface for presentation to the user.

[1402] In this way, the user can acquire information efficiently and receive appropriate information according to their emotional state.

[1403] (Application example 2)

[1404] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1405] Conventional digital content delivery systems have difficulty providing personalized information according to the user's emotional state, and have been unable to increase user satisfaction. Furthermore, there is a lack of a way for users to efficiently obtain the information they need without being overwhelmed by a large amount of information. Furthermore, there is a need to keep users' emotions positive by providing appropriate content according to their emotions.

[1406] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1407] In this invention, the server includes means for collecting unread or unviewed information from multiple digital content items, artificial intelligence means for summarizing the collected information, means for converting the summarized information into audio and outputting it, means for analyzing the user's emotional state, and means for automatically collecting recommended information according to the user's emotional state. This enables personalized information to be provided according to the user's emotional state, thereby increasing user satisfaction. Furthermore, the user can efficiently obtain the information they need without being overwhelmed by a large amount of information. Furthermore, by providing appropriate content according to the user's emotions, it is possible to maintain a positive mood for the user.

[1408] "Digital content" is a general term for information and media that are electronically generated, stored, and distributed.

[1409] "Unread or unviewed information" refers to digital content that a user has not yet read or viewed.

[1410] "Gathering means" refers to the technologies and methods used to obtain digital content from the Internet or databases.

[1411] "Artificial intelligence means" refers to systems or algorithms that use techniques such as machine learning and natural language processing to analyze data and perform specific tasks.

[1412] To "summarize" means to express the main content of the original information in a shortened form.

[1413] "Means for converting text into audio and outputting it" refers to the technology or method for converting text information into audio and making it audible to the user.

[1414] A "user's emotional state" refers to the emotions or mood a user is feeling at a particular point in time.

[1415] "Analytical means" refers to techniques and methods for analyzing data and extracting specific information or patterns.

[1416] "Recommended information" is information that is useful or interesting to the user, selected based on the user's interests and emotional state.

[1417] "Automatic collection methods" are technologies or methods that allow a system to collect information without user intervention.

[1418] A system for carrying out this invention has the following configuration: a server includes means for collecting unread or unviewed information from a plurality of digital contents, artificial intelligence means for summarizing the collected information, means for converting the summarized information into audio and outputting it, means for analyzing the emotional state of a user, and means for automatically collecting recommended information according to the emotional state of the user.

[1419] System Program

[1420] The server collects digital content from external data sources such as news APIs. The collected digital content is summarized using natural language processing technology. This summarization uses the Hugging Face transformers library. The summarized information is then converted into audio using speech synthesis technology and provided to the user.

[1421] The user's emotional state is analyzed in real time using an emotion analysis model. This emotion analysis is performed using the Hugging Face pipeline. Based on the analyzed emotional state, information suitable for the user is automatically collected and recommended. This information is then summarized and converted into audio and provided to the user.

[1422] Hardware and software used

[1423] Hardware: Servers, user devices (smartphones, tablets, PCs, etc.)

[1424] Software: News API, Hugging Face transformers library, emotion analysis model, speech synthesis software

[1425] Specific examples

[1426] For example, if a user is using a smartphone, the server collects the latest news articles from a news API. The collected news articles are summarized using natural language processing technology. The summarized news articles are then converted into audio using speech synthesis technology, which the user can listen to on their smartphone.

[1427] If the user expresses the emotion of "joy," the emotion analysis model will analyze this emotion and automatically collect fun news and entertainment information, which will be summarized, converted into audio, and provided to the user.

[1428] Prompt Sentence Examples

[1429] If the user's emotion is "joy," show them some fun news or entertainment information.

[1430] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1431] Step 1:

[1432] The server collects digital content from external data sources such as news APIs.

[1433] Input: News API endpoint

[1434] Output: A list of the latest news articles

[1435] What it does: The server uses an HTTP request to access the news API to retrieve the latest news articles, which are then returned to the server in JSON format.

[1436] Step 2:

[1437] The server summarizes the collected digital content using natural language processing techniques.

[1438] Input: A list of recent news articles

[1439] Output: A list of summarized news articles

[1440] What it does: The server uses Hugging Face's transformers library to summarize the content of each news article, which is a shortened version of the main points of the original news article.

[1441] Step 3:

[1442] The server converts the summarized information into audio using speech synthesis technology and provides it to the user.

[1443] Input: A list of summarized news articles

[1444] Output: Audio file

[1445] What it does: The server uses speech synthesis software to convert the summarized news article into audio, which is then sent to the user's device for listening.

[1446] Step 4:

[1447] The server analyzes the user's emotional state.

[1448] Input: Real-time user data (e.g. text, voice, facial expressions, etc.)

[1449] Output: User's emotional state (e.g., happy, sad, angry, etc.)

[1450] How it works: The server uses the emotion analysis model to analyze the user's real-time data and identify the user's emotional state. This analysis uses the Hugging Face pipeline.

[1451] Step 5:

[1452] The server automatically collects recommended information based on the user's emotional state.

[1453] Input: User's emotional state

[1454] Output: A list of recommended news articles

[1455] Specific operation: The server recollects appropriate news articles from the news API based on the user's emotional state. For example, if the user expresses "joy," it will collect fun news and entertainment information.

[1456] Step 6:

[1457] The server then summarizes the recollected news articles, converts them into audio, and provides them to the user.

[1458] Input: A list of recommended news articles

[1459] Output: Audio file

[1460] Specific operation: As in steps 2 and 3, the server summarizes the collected news articles and converts them into audio using speech synthesis technology. The generated audio file is sent to the user's device, where the user can listen to it.

[1461] Example 3

[1462] Next, a description will be given of Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1463] Conventional information gathering systems have problems in that they do not provide sufficient recommended information based on the user's usage history and do not provide optimal voice output according to the user's emotional state. Furthermore, they lack the ability to learn changes in the user's emotional state and optimize information gathering and voice output based on that information, making it difficult to provide optimal information to the user. This reduces user satisfaction and prevents effective use of information.

[1464] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[1465] In this invention, the server includes means for collecting unread or unviewed information from multiple digital content items, artificial intelligence means for summarizing the collected information, means for converting the summarized information into audio and outputting it, means for detecting the user's emotional state and optimizing the output method of the converted audio summary information, and means for learning changes in the user's emotional state and optimizing the collection of recommended information and audio output based on the learning results. This makes it possible to provide recommended information based on the user's usage history, and also realizes optimal audio output according to the user's emotional state. Furthermore, by learning changes in the user's emotional state, the collection of information and audio output are optimized, making it possible to provide optimal information to the user.

[1466] "Digital content" means information or media stored or distributed in electronic form.

[1467] "Unread or unviewed information" means information that has not yet been viewed or listened to by the user.

[1468] "Means of collection" refers to the methods and technologies used to obtain the necessary information from digital content.

[1469] "Artificial intelligence means for summarizing" refers to artificial intelligence techniques used to summarize collected information succinctly.

[1470] "Means for converting text information into audio and outputting it" refers to technology for converting text information into audio data and providing it to the user.

[1471] "Means for detecting emotional state" refers to technology that analyzes the user's emotions and understands their state.

[1472] "Means for optimizing the output method" refers to techniques for adjusting the tone and speed of audio output depending on the user's emotional state.

[1473] "Means for learning changes in emotional state" refers to technology that records changes in the user's emotions and uses them to improve the system's behavior.

[1474] "Means for optimizing the collection and vocalization of recommended information" refers to technology for collecting optimal information and vocalizing it based on the user's emotional state and usage history.

[1475] The present invention is a system that provides optimal information based on a user's usage history and emotional state, and converts it into audio. Specific embodiments of this system will be described below.

[1476] 1. Generating the system program

[1477] The server automatically collects recommended information based on the user's usage history and generates a program that generates summary information containing that information. This program uses machine learning technology to learn the user's past browsing history and preferences, and has the ability to collect and summarize new recommended information.

[1478] 2. Explanation of program processing

[1479] The server processes the data in the following steps:

[1480] Data collection

[1481] The server collects data about users' browsing history and preferences, including the web pages they visit, the links they click, and the duration of their visits. Specifically, the server collects data using analytics tools such as Google Analytics and Mixpanel.

[1482] Training a machine learning model

[1483] The server uses the collected data to train machine learning models that learn user preferences and interests and predict new recommendations using libraries such as TensorFlow and PyTorch.

[1484] Recommendation collection and summarization

[1485] The server uses the trained model to collect new recommendations and summarize them using natural language processing techniques, specifically generative AI models such as BERT and GPT-3.

[1486] Text-to-Speech (TTS)

[1487] The server uses TTS technology, such as Google Text-to-Speech API or Amazon Polly, to convert the summarized text information into audio data.

[1488] Optimization by Emotion Engine

[1489] The server detects the user's emotional state and optimizes the output of the spoken summary information using emotion analysis software (e.g., Affectiva or IBM Watson Tone Analyzer).

[1490] Feedback Loop

[1491] The server learns changes in the user's emotional state and, based on the results, collects recommended information and optimizes voice conversion.

[1492] 3. Examples and prompts

[1493] For example, if a user has viewed many articles about "travel" in the past, the server will collect new articles related to "travel," summarize them, and translate them into audio. If the user is relaxed, the audio will be provided in a slower tone.

[1494] Example prompt sentence:

[1495] "Based on the user's past browsing history, collect, summarize, and voice new travel recommendations. If the user is relaxed, adjust the voiceover tone to a slower tone."

[1496] In this way, the server can efficiently provide useful information to the user and perform voice output in an optimal form according to the emotional state of the user. The flow of the identification process in the third embodiment will be described with reference to FIG.

[1497] Step 1:

[1498] Data collection

[1499] The server collects data on users' browsing history and preferences. As input, it obtains data such as the web pages they visited, the links they clicked, and the duration of their visit. Specifically, it uses analytics tools such as Google Analytics and Mixpanel to collect the data. As output, the collected data is stored in a database.

[1500] Specific behavior:

[1501] The server records the URLs of the web pages that users visit.

[1502] The server stores information about the links that users click.

[1503] The server measures the viewing time of each page and stores the data in a database.

[1504] Step 2:

[1505] Training a machine learning model

[1506] The server uses the collected data to train a machine learning model. As input, it preprocesses the collected user data and converts it into a format suitable for the machine learning model. Specifically, it uses TensorFlow or PyTorch to train the model. As output, a trained model is generated.

[1507] Specific behavior:

[1508] The server preprocesses the collected data, imputing missing values ​​and normalizing the data.

[1509] The server trains a machine learning model using the preprocessed data.

[1510] The server stores the trained model for use in subsequent processing.

[1511] Step 3:

[1512] Recommendation collection and summarization

[1513] The server uses the trained model to collect new recommendations and summarize them. It uses the trained model and new information collected from the internet as input. Specifically, it uses a generative AI model such as BERT or GPT-3 to perform summarization. The output is a summarized text.

[1514] Specific behavior:

[1515] The server uses the trained model to collect recommendations for users from the Internet.

[1516] The server inputs prompts into the generative AI model to summarize the collected information.

[1517] The server stores the generated summary.

[1518] Step 4:

[1519] Text-to-Speech (TTS)

[1520] The server uses TTS technology to convert the summarized text information into speech data. As input, the summarized text information is input to the TTS software. Specific software examples include Google Text-to-Speech API and Amazon Polly. As output, the generated speech data is obtained.

[1521] Specific behavior:

[1522] The server inputs the summarized text information into the TTS software.

[1523] The server receives and stores the generated voice data.

[1524] The server transmits the audio data to the user's terminal.

[1525] Step 5:

[1526] Optimization by Emotion Engine

[1527] The server detects the user's emotional state and optimizes the output method of the voice summary information. As input, it takes the user's emotional state data. To detect the emotional state, it uses emotion analysis software (e.g., Affectiva or IBM Watson Tone Analyzer). As output, it obtains optimized voice data.

[1528] Specific behavior:

[1529] The server acquires emotional state data from the user's device.

[1530] The server uses emotion analysis software to analyze the user's emotional state.

[1531] The server adjusts the tone and speed of the voice based on the analysis results.

[1532] Step 6:

[1533] Feedback Loop

[1534] The server learns changes in the user's emotional state and uses the results to collect recommended information and optimize voice conversion. Data on changes in the user's emotional state is taken as input. Optimized recommended information and voice data are obtained as output.

[1535] Specific behavior:

[1536] The server records the user's emotional state after hearing specific information.

[1537] The server uses the recorded emotion data to retrain the machine learning model.

[1538] The server uses the retrained model to gather new recommendations and optimize the speech generation.

[1539] (Application example 3)

[1540] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1541] In modern society, users have access to a vast amount of digital content, but it is difficult to efficiently collect and consume information that is useful to them. In addition, there is a lack of ways to optimize the way information is provided to users according to their emotional state, which can lead to a decrease in satisfaction with the way information is received. Furthermore, there is a need for information delivery methods that allow users to make effective use of their free time.

[1542] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[1543] In this invention, the server includes means for collecting unread or unviewed information from multiple digital content items, artificial intelligence means for summarizing the collected information, means for converting the summarized information into audio and outputting it, emotion engine means for adjusting the tone and speed of the audio according to the user's emotional state, and machine learning means for automatically collecting recommended information based on the user's past browsing history and preferences and generating summary information including the information. This allows users to efficiently collect information that is useful to them and receive it in an optimal way according to their emotional state. It also allows users to effectively use their free time to consume information.

[1544] "Digital content" is a general term for information and media provided in electronic form.

[1545] "Unread or unviewed information" refers to information that the user has not yet read or viewed.

[1546] "Collection means" refers to the methods and techniques used to obtain the required information from a particular source.

[1547] "Artificial intelligence means for summarizing" refers to artificial intelligence techniques used to summarize collected information succinctly.

[1548] "Means for converting text information into audio and outputting it" refers to technology for converting text information into audio data and providing it to the user.

[1549] "Emotional state" refers to a user's current psychological state or mood.

[1550] "Emotional engine means" refers to technology for adjusting the tone and speed of speech based on the emotional state of the user.

[1551] "Browsing history" refers to a record of web pages and content that a user has previously accessed.

[1552] "Preferences" refer to the types of information or content in which a user is particularly interested.

[1553] "Machine learning methods" refer to technologies that learn patterns based on data and make predictions and classifications.

[1554] "Recommended information" refers to information that is likely to be useful to a user, selected based on the user's past behavior and preferences.

[1555] A system for carrying out this invention collects unread or unviewed information from multiple digital contents, summarizes the collected information, and outputs the summarized information as audio. It also includes an emotion engine means for adjusting the tone and speed of the audio according to the user's emotional state, and machine learning means for automatically collecting recommended information based on the user's past browsing history and preferences, and generating summary information including the information.

[1556] Hardware and software used

[1557] Hardware: Smartphones, smart glasses, servers

[1558] Software: Python, pyttsx3 (TTS library), requests (HTTP request library), textblob (natural language processing library), scikit-learn (machine learning library)

[1559] Processing flow

[1560] 1. Collecting news articles

[1561] The server uses HTTP requests to retrieve the latest news articles from the news API, thereby collecting information that the user has not yet read or watched.

[1562] 2. News article summaries

[1563] The server uses a natural language processing library (textblob) and a machine learning library (scikit-learn) to summarize the collected news articles. Specifically, it vectorizes the articles using TfidfVectorizer and generates summaries using KMeans clustering.

[1564] 3. Audio

[1565] The server converts the summarized text into audio data using a TTS library (pyttsx3), allowing the user to receive the information by voice.

[1566] 4. Emotion Engine

[1567] The server adjusts the tone and speed of the voice depending on the user's emotional state, for example, slowing down the voice if the user is relaxed and increasing the voice speed if the user is excited.

[1568] 5. Collecting recommendations

[1569] The server automatically collects recommended information based on the user's browsing history and preferences using machine learning techniques, and generates summary information that includes the information, thereby efficiently providing useful information to the user.

[1570] Specific examples

[1571] User A: I've read a lot of technology-related articles in the past, so I'd like to hear a summary of the latest technology news.

[1572] User B: Prefers sports news, so they receive a summary of the latest sports news in audio format.

[1573] Prompt Sentence Examples

[1574] Build an application that automatically collects, summarizes, and delivers recommended news articles to a user based on their browsing history. Include the ability to use an emotion engine to adjust the tone and speed of the speech depending on the user's emotional state.

[1575] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[1576] Step 1:

[1577] The server retrieves the latest news articles from the news API using the HTTP request library (requests). It uses the news API endpoint URL as input and gets a list of retrieved news articles as output. Specifically, it sends an API request and parses the JSON data returned as a response to extract the article title and description.

[1578] Step 2:

[1579] The server uses a natural language processing library (textblob) and a machine learning library (scikit-learn) to summarize the collected news articles. It uses a list of news articles as input and obtains summarized text as output. Specifically, it vectorizes the articles using TfidfVectorizer, extracts important parts using KMeans clustering, and generates summaries.

[1580] Step 3:

[1581] The server converts the summarized text into speech data using a TTS library (pyttsx3). It uses the summarized text as input and obtains speech data as output. Specifically, it initializes the pyttsx3 engine, converts the text into speech, and saves it as an audio file.

[1582] Step 4:

[1583] The server adjusts the tone and speed of the voice depending on the user's emotional state. It uses the user's emotional state (relaxed, excited, etc.) as input and obtains adjusted voice data as output. Specifically, it sets pyttsx3 properties (rate and volume) based on the emotional state and adjusts the tone and speed of the voice.

[1584] Step 5:

[1585] The server uses machine learning techniques to automatically collect recommended information based on the user's past browsing history and preferences, and generates summary information including that information. The server uses the user's browsing history and preference data as input, and obtains summary information including the recommended information as output. Specifically, the server uses a machine learning algorithm to learn the user's preferences, collects new information based on that information, and generates a summary.

[1586] Step 6:

[1587] The user receives the audio summary information using a terminal and can listen to it during their free time, such as while doing housework or commuting. The adjusted audio data is used as input, and the user listens to the audio information as output. Specifically, the audio data is played back and provided to the user through a device such as a smartphone or smart glasses.

[1588] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1589] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1590] Another example of generative AI is Gemini (internet search engine). <url: https: gemini.google.com ?hl="ja">) are mentioned.

[1591] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1592] [Third embodiment]

[1593] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1594] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1595] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1596] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1597] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1598] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1599] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1600] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1601] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1602] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1603] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1604] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[1605] "Example 1"

[1606] In one embodiment of the present invention, APIs that access various digital content platforms used by users are used to collect unread or unwatched information, such as from YouTube's "Watch Later" list, unread messages in messenger apps, and unread lists in bookmark services.

[1607] "Example 2"

[1608] Natural language processing technology is used as an artificial intelligence method to summarize collected information. Specifically, collected text information is analyzed and a summary is generated. This summary expresses the main content of the original information in a shortened form.

[1609] "Example 3"

[1610] Text-to-speech (TTS) technology is used to convert the summarized information into audio and output it. Specifically, the summarized text information is converted into audio data and provided in a form that the user can listen to.

[1611] "Example 4"

[1612] Machine learning technology is used to automatically collect recommended information based on a user's usage history and generate summary information that includes that information. Specifically, the system learns the user's past browsing history and preferences, and then collects and summarizes new recommended information based on that information. This makes it possible to efficiently provide useful information to users.

[1613] The processing flow of each embodiment will be described below.

[1614] "Example 1"

[1615] Step 1: We access the various digital content platforms you use to collect information, such as your YouTube Watch Later list, unread messages in your messaging app, or unread lists in your bookmarking services.

[1616] Step 2: Centralize unread or unwatched information obtained from each platform, allowing users to view information across multiple platforms in one place.

[1617] "Example 2"

[1618] Step 1: Analyze the collected information using natural language processing technology. Through this analysis, the main content of the information is extracted.

[1619] Step 2: Based on the extracted key content, a summary of the information is generated. This summary represents the main content of the original information in a shortened form.

[1620] "Example 3"

[1621] Step 1: Convert the summarized information into audio data using text-to-speech (TTS) technology.

[1622] Step 2: The converted audio data is provided to the user in a form that they can listen to, allowing them to digest the information during their free time, such as during housework or on their way to work.

[1623] "Example 4"

[1624] Step 1: Learn about the user's past browsing history and preferences using machine learning techniques.

[1625] Step 2: Based on the learned results, new recommendations are collected.

[1626] Step 3: The collected recommendation information is summarized and a summary containing it is generated, which makes it possible to efficiently provide useful information to users.

[1627] Example 1

[1628] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1629] Today's users use multiple digital content platforms, and unread or unwatched information often accumulates on each platform. However, there is a lack of a way to centrally manage this information and efficiently extract and provide important information. As a result, users are more likely to miss important information, and information management becomes cumbersome.

[1630] The identification process by the identification processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means. In this invention, the server includes a means for accessing multiple digital content platforms, a means for collecting unread or unviewed information, a means for converting the collected information into a unified format, a means for analyzing the converted information and extracting important information, and a means for providing the extracted information to the user. This allows the user to centrally manage unread or unviewed information from multiple platforms and efficiently check important information.

[1631] "Digital Content Platform" means an online service that enables users to view, listen to or store content in digital form.

[1632] "Means of access" refers to the methods and technologies by which a server connects to a particular digital content platform and obtains the required data.

[1633] "Unread or unwatched information" refers to digital content such as messages, videos, or articles that a user has not yet viewed.

[1634] "Means of collection" refers to the methods or technologies by which the server obtains unread or unwatched information from the digital content platform.

[1635] "Means of converting into a unified format" refers to methods and technologies for converting data collected from different platforms into a consistent format.

[1636] "Means of analysis" refers to the methods and techniques used to analyze the collected and transformed data and extract information that is important to users.

[1637] "Means of extracting important information" refers to methods and techniques for selecting information that is valuable to users from the analyzed data.

[1638] "Means of providing" refers to the methods and technologies used to notify or display extracted important information to the user.

[1639] MODE FOR CARRYING OUT THE INVENTION

[1640] This invention provides a system that efficiently collects unread or unviewed information from multiple digital content platforms used by users, extracts important information, and provides it to users. Specific embodiments of this system are described below.

[1641] System configuration

[1642] Hardware and software configuration:

[1643] The server uses the following hardware and software to implement the system.

[1644] Hardware: A server with a powerful processor, sufficient memory, and ample storage.

[1645] Software: Libraries for API access (e.g., HTTP client libraries), libraries for data analysis (e.g., Python's Pandas, NumPy), database management systems (e.g., MySQL, PostgreSQL)

[1646] Specific APIs:

[1647] Digital content platform APIs: YouTube Data API, Facebook Messenger API, Pocket API, etc.

[1648] Program processing

[1649] The server collects and processes data in the following manner:

[1650] 1. Prepare for API access:

[1651] The server obtains API keys and authentication information for each platform and prepares API access, such as setting an API key for the YouTube Data API, an access token for the Facebook Messenger API, and a consumer key for the Pocket API.

[1652] 2. Data Collection:

[1653] The server calls each platform's API to collect unread or unwatched content, such as YouTube's "Watch Later" list, Facebook Messenger's unread messages, and Pocket's unread bookmarks.

[1654] 3. Data Processing:

[1655] The server converts the collected data into a unified format and assigns unread or unwatched statuses. For example, it converts data obtained from each platform into JSON format.

[1656] 4. Data Analysis:

[1657] The server analyzes the collected data and extracts information that is important to the user, such as unread messages containing specific keywords.

[1658] 5. Providing results:

[1659] The server processes and analyzes the data and provides it to users, who can then check their unread or unwatched information through a dedicated dashboard and notifications.

[1660] Specific examples

[1661] If a user uses YouTube, Facebook Messenger, or Pocket, the server:

[1662] 1. The server sets the API key for the YouTube Data API and retrieves the "Watch Later" list.

[1663] 2. The server sets the access token for the Facebook Messenger API and retrieves the unread messages.

[1664] 3. The server sets the Pocket API consumer key and retrieves unread bookmarks.

[1665] 4. Convert the collected data into JSON format and mark it as unread or unwatched.

[1666] 5. Extract important information and notify the user.

[1667] Example prompt sentence:

[1668] "Generate a program that gathers your YouTube Watch Later list, unread messages in Facebook Messenger, and unread bookmarks in Pocket, extracts the important information, and notifies you."

[1669] In this way, the server realizes a system that efficiently collects unread or unviewed information from various digital content platforms used by users and provides it to users.

[1670] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1671] Step 1: Prepare for API access

[1672] The server obtains the API key and authentication information for each platform and prepares for API access. Specifically, it reads the API key and authentication information from the configuration file and sets the API endpoint for each platform. The input requires the API key and authentication information, and the output is a state where API access is possible.

[1673] Step 2: Data collection

[1674] The server calls the API of each platform to collect unread or unwatched information. Specifically, it uses the YouTube Data API to obtain the "Watch Later" list, the Facebook Messenger API to obtain unread messages, and the Pocket API to obtain unread bookmarks. The input requires a state where API access is ready, and the output is the unread or unwatched information obtained from each platform.

[1675] Step 3: Data processing

[1676] The server converts the collected data into a unified format and assigns an unread or unviewed status. Specifically, it converts the data obtained from each platform into JSON format and assigns an unread or unviewed status. The input requires the collected unread or unviewed information, and the output is the data converted into a unified format.

[1677] Step 4: Data analysis

[1678] The server analyzes the collected data and extracts information that is important to the user. Specifically, it extracts messages containing specific keywords from unread messages, extracts highly rated videos from unwatched YouTube videos, and extracts articles with specific tags from unread bookmarks. The input requires data converted into a unified format, and the output is data from which important information has been extracted.

[1679] Step 5: Delivering results

[1680] The server provides the processed and analyzed data to the user. Specifically, it displays the analysis results on a dashboard and notifies the user of important unread messages and unwatched videos. The user can check the unread or unwatched information through the dashboard and take necessary actions. The input is data from which important information has been extracted, and the output is the information provided to the user.

[1681] (Application example 1)

[1682] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1683] Today's users consume a wide variety of digital content, and the amount of unread or unviewed information tends to increase. However, there is a lack of ways to efficiently manage this information and notify users at the appropriate time, which can lead to users missing important information. To solve this problem, a system is needed that centrally collects unread or unviewed information from multiple digital content and notifies users at the appropriate time.

[1684] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1685] In this invention, the server includes means for collecting unread or unviewed information from multiple digital contents, artificial intelligence means for summarizing the collected information, means for converting the summarized information into audio and outputting it, and means for centrally managing the collected information and notifying the user at an appropriate time. This allows the user to efficiently manage unread or unviewed information from multiple digital contents and receive notifications at an appropriate time without missing important information.

[1686] "Digital content" is a general term for information and media provided via the Internet or electronic devices.

[1687] "Unread or unviewed information" refers to digital content that the user has not yet read or viewed.

[1688] "Means of collection" refers to methods or techniques for obtaining unread or unviewed information from multiple digital content.

[1689] "Artificial intelligence means for summarizing" refers to artificial intelligence techniques used to summarize collected information succinctly.

[1690] "Means for converting and outputting speech" refers to methods or technologies for converting summarized information into speech and providing it to the user.

[1691] "Means for centralized management" refers to methods or technologies for managing collected unread or unviewed information in an integrated manner within a single system.

[1692] "Means of notifying at the appropriate time" refers to methods and technologies for notifying users of information at the most effective time.

[1693] The system for implementing this invention collects unread or unviewed information from multiple digital contents and notifies the user at an appropriate time. The system consists of three main elements: a server, a terminal, and a user.

[1694] server

[1695] The server includes the following means:

[1696] 1. Collection method: The server accesses multiple digital content platforms (e.g., video sharing services, messenger apps, bookmark services) and collects information on unread or unwatched content. Specifically, it obtains the information using the API of each platform.

[1697] 2. Artificial intelligence means of summarization: Using artificial intelligence techniques to summarize collected information, for example, using natural language processing (NLP) techniques to summarize long messages or articles in a concise manner.

[1698] 3. Voice output: The summarized information is converted into voice and provided to the user. Text-to-speech technology (TTS) is used to convert text information into voice.

[1699] 4. Centralized management: Collected unread or unwatched information is consolidated and managed in a single database, allowing users to view all information in a single interface.

[1700] 5. Notifications at the right time: Notifications are sent at the optimal time based on the user's behavioral patterns and schedule. For example, users can receive notifications during their free time, such as during their commute or while doing housework.

[1701] Terminal

[1702] A terminal is a device used by a user, such as a smartphone or tablet. An application is installed on the terminal to receive and display notifications sent from the server. This application has the following functions:

[1703] 1. Receiving and displaying notifications: Receives notifications sent from the server and displays them to the user. Notifications are displayed in the form of popups or banners.

[1704] 2. Audio playback: The audio summary information is played back so that users can listen to it through earphones or speakers.

[1705] user

[1706] Users can use the system to efficiently manage unread or unwatched information and receive timely notifications. Users can check notifications through the application and view detailed information as needed.

[1707] Specific examples

[1708] For example, if a user has five videos added to their "watch later" list on a video sharing service, three unread messages on a messaging app, and two unread articles on a bookmarking service, the server will collect, summarize, and convert this information into audio. It will then send notifications to users during their commute, providing them with the information.

[1709] Prompt Sentence Examples

[1710] A user has five unwatched videos in their "Watch Later" list on a video sharing service. The titles are "Introduction to Programming," "The Future of AI," "Data Science Fundamentals," "Applications of Machine Learning," and "Deep Learning Fundamentals."

[1711] The user has three unread messages in their messenger app: "Hi, how are you?", "Regarding tomorrow's meeting," and "What are your plans for the weekend?"

[1712] The user has two unread articles in their bookmarks service, titled "Current Technology Trends" and "Programming Best Practices."

[1713] Use this information to generate a reminder for your users.

[1714] In this way, users can efficiently manage their information without missing any important information.

[1715] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1716] Step 1:

[1717] The server accesses multiple digital content platforms and collects unread or unviewed information. Specifically, it uses each platform's API to obtain information such as the user's "Watch Later" list, unread messages, and unread bookmarks. The input is each platform's API key or token, and the output is a list of unread or unviewed information.

[1718] Step 2:

[1719] The server uses artificial intelligence techniques to summarize the collected information. Specifically, it uses natural language processing (NLP) techniques to concisely summarize long messages and articles. The input is a list of unread or unwatched information, and the output is the summarized information.

[1720] Step 3:

[1721] The server converts the summarized information into speech. Text-to-speech (TTS) technology is used to convert text information into speech. The input is the summarized information and the output is audio data.

[1722] Step 4:

[1723] The server consolidates and manages the collected unread or unwatched information into a single database, allowing users to view all information in a single interface. The input is a list of unread or unwatched information, and the output is the consolidated database.

[1724] Step 5:

[1725] The server sends notifications at optimal times based on the user's behavioral patterns and schedule. For example, users can receive notifications during their free time, such as while commuting or doing housework. The input is the user's behavioral patterns and schedule information, and the output is notification data.

[1726] Step 6:

[1727] The device receives notifications sent from the server and displays them to the user. Notifications can be displayed in the form of popups or banners. The input is the notification data and the output is the notification that is displayed to the user.

[1728] Step 7:

[1729] The terminal plays back the audio summary information, and the user can listen to the information through earphones or speakers. The input is audio data, and the output is the audio being played back.

[1730] Step 8:

[1731] The user can view the notification through the application and view more information if necessary. The input is the displayed notification, and the output is the user's action (such as viewing more information).

[1732] Example 2

[1733] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1734] In modern society, the amount of information is increasing, making it difficult for users to efficiently obtain the information they need. In addition, when obtaining information, users may not have time to read text information or may prefer to obtain information by voice. Furthermore, there is a demand for a system that automatically collects and summarizes recommended information based on the user's usage history.

[1735] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1736] In this invention, the server includes a means for collecting unread or unviewed information, a natural language processing means for analyzing the collected text information and generating a summary, a means for inputting a prompt sentence into a generation AI model to generate a summary, and a means for converting the summarized information into audio and outputting it. This allows users to efficiently obtain the information they need and acquire it by voice. It is also possible to automatically collect recommended information based on the user's usage history and generate summary information including that information.

[1737] "Unread or unviewed information" refers to digital content that a user has not yet read or viewed.

[1738] "Means for collecting" refers to a function or device for automatically collecting unread or unviewed information.

[1739] "Text information" refers to information written in text, including news articles, reports, blog posts, etc.

[1740] "Analyzing" means analyzing text information using techniques such as tokenization, part-of-speech tagging, and dependency analysis.

[1741] "Natural language processing means" refers to technologies and algorithms for analyzing text information and generating summaries.

[1742] A "generative AI model" is a model that uses artificial intelligence to summarize and generate text information, and examples include GPT-3 and BERT.

[1743] A "prompt" is an instruction given to a generative AI model, containing specific instructions for generating a summary.

[1744] "Means for generating a summary" refers to the functionality or device for inputting a prompt sentence into a generative AI model and generating a summary.

[1745] "Means for converting text information into audio and outputting it" refers to a function or device for converting text information into audio and providing it to the user.

[1746] "Usage history" refers to a record of the information and actions a user has previously accessed.

[1747] "Recommended information" refers to information that is automatically selected based on the user's usage history and is presumed to be useful to the user.

[1748] The present invention provides a system that enables a user to efficiently acquire information and acquire information by voice. A specific embodiment of this system will be described below.

[1749] First, the user enters the text information they want to summarize into their device. For example, they paste the text information, such as a news article or report, into an input form. The device then sends the entered text information to the server as an HTTP request. At this time, the text information is packaged in JSON format.

[1750] The server uses the Python programming language and natural language processing libraries (e.g., spaCy or NLTK) to analyze the received text information. The server tokenizes the text and performs part-of-speech tagging and dependency analysis, which allows it to understand the structure of the text information and extract key content.

[1751] Next, the server inputs a prompt to a generative AI model (such as GPT-3 or BERT) based on the analysis results. The prompt includes instructions for generating a summary. For example, the prompt might be, "Please summarize the following text." The generative AI model generates a summary based on this prompt and the analysis results.

[1752] The generated summary is sent from the server to the device. At this time, the summary is packaged in JSON format. The device displays the summary received from the server to the user. The user can check the summary result on the device screen.

[1753] The server then uses text-to-speech synthesis technology (e.g., a text-to-speech engine) to convert the summarized information into audio and output it. The audio summary information is provided in a format that the user can listen to in their spare time, such as while doing housework or on their way to work.

[1754] As a specific example, suppose a user wants to summarize the news article "A major traffic accident occurred in Tokyo, injuring five people. The cause of the accident is under investigation." In this case, the user enters the news article into an input form on their device, and the device sends the news article to the server. The server tokenizes the news article and performs part-of-speech tagging and dependency analysis. The server then inputs a prompt statement, "Please summarize the following text," into the generative AI model. The generative AI model generates the summary, "A major traffic accident occurred in Tokyo, injuring five people. The cause of the accident is under investigation." The server sends the generated summary to the device, which then displays the summary result to the user.

[1755] In this way, users can easily obtain a summary of text information, and by using audio-translated summary information, users can efficiently acquire information.

[1756] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1757] Step 1:

[1758] The user inputs text information to be summarized into the terminal.

[1759] Specifically, a user copies and pastes text information such as a news article or report into an input form on a terminal, and the text information entered by the user is obtained as input.

[1760] Step 2:

[1761] The terminal transmits the input text information to the server.

[1762] Specifically, the device sends the entered text information to the server as an HTTP request. At this time, the text information is packaged in JSON format. The text information entered by the user is obtained as input, and the JSON format data to be sent to the server is generated as output.

[1763] Step 3:

[1764] The server parses the received text information.

[1765] Specifically, the server uses the Python programming language and natural language processing libraries (e.g., spaCy and NLTK) to tokenize text information, tag parts of speech, and perform dependency analysis. As input, it receives JSON-formatted text information sent from the device, and as output, it generates the analysis results.

[1766] Step 4:

[1767] The server inputs a prompt sentence into the generative AI model based on the analysis results.

[1768] Specifically, the server generates a prompt saying, "Please summarize the following text," and inputs it into the generative AI model along with the analysis results. The analysis results and the prompt are obtained as input, and the data to be input into the generative AI model is generated as output.

[1769] Step 5:

[1770] A generative AI model generates the summary.

[1771] Specifically, a generative AI model (e.g., GPT-3 or BERT) generates a summary based on a prompt and the analysis results. The input is the prompt and the analysis results, and the output is the generated summary.

[1772] Step 6:

[1773] The server transmits the generated summary to the terminal.

[1774] Specifically, the server packages the summary obtained from the generative AI model in JSON format and sends it to the device. The generated summary is obtained as input, and the JSON-formatted data sent to the device is generated as output.

[1775] Step 7:

[1776] The terminal displays the summary results to the user.

[1777] Specifically, the terminal displays the summary received from the server to the user. As input, it receives the JSON-formatted summary sent from the server, and as output, it generates the summary result to be displayed to the user.

[1778] Step 8:

[1779] The server outputs the summarized information as a voice.

[1780] Specifically, the server converts the summary into audio using text-to-speech synthesis technology (e.g., a text-to-speech engine), taking the generated summary as input and generating audio data as output.

[1781] Step 9:

[1782] The terminal provides the user with a voiced summary of the information.

[1783] Specifically, the terminal plays back the audio data and provides it in a form that the user can listen to in their free time, such as while doing housework or on the way to work. As input, the audio data sent from the server is obtained, and as output, audio information to be provided to the user is generated.

[1784] (Application example 2)

[1785] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1786] In modern society, a huge amount of digital content is generated every day, and users are required to efficiently obtain the information they need from it. However, due to the information overload, users are likely to miss important information, and they spend a lot of time obtaining information, which is a problem. In particular, there is the issue of it being difficult to grasp important information such as news articles in a short time.

[1787] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting unread or unviewed information from multiple digital contents, artificial intelligence means for summarizing the collected information, means for converting the summarized information into audio and outputting it, and means for summarizing the collected news articles and providing them to the user. This enables the user to efficiently obtain important information from a vast amount of information and grasp it in a short amount of time.

[1788] "Digital content" means information or media that is electronically generated, stored, or distributed.

[1789] "Unread or unviewed information" means information that the user has not yet read or viewed.

[1790] "Collection methods" refer to the methods and techniques used to obtain the required data from a particular source.

[1791] "Artificial intelligence means for summarizing" refers to a technology that uses natural language processing technology to analyze collected text information and express its main content in a shortened form.

[1792] "Means for converting text information into audio and outputting it" refers to technology for converting text information into audio and providing it to the user.

[1793] A "news article" is a piece of text that describes current events or information provided by the press or other sources.

[1794] "Means of delivery to users" refers to the methods and technologies used to deliver the summarized information in a form that is easy for users to access.

[1795] The system for implementing the present invention includes a server, a user terminal, and an artificial intelligence means. The details of each component and its operation will be described below.

[1796] server

[1797] The server has a means of collecting unread or unwatched information from multiple digital content sources. Specifically, it sends HTTP requests to obtain the latest news articles from external sources such as news APIs, and saves the obtained data in JSON format. The server also has an artificial intelligence means of summarizing the collected information. Here, it uses a pre-trained summarization model using the Hugging Face transformers library. It also has a means of converting the summarized information into audio and outputting it. For audio conversion, it uses speech synthesis technology such as the Google Text-to-Speech API.

[1798] User Device

[1799] The user terminal is a device such as a smartphone or tablet that receives the summary information provided by the server and displays it to the user. The user can view and listen to the summary of the news article through the application. The user terminal also records the user's usage history and sends it to the server, providing data for providing personalized recommendations.

[1800] Artificial Intelligence Tools

[1801] The AI ​​tool uses natural language processing techniques to analyze collected text information and generate summaries. Specifically, it uses the Hugging Face transformers library to represent the key content of news articles in a shortened form. The summarization model has been trained on a large amount of news article data in advance, enabling it to generate summaries with high accuracy.

[1802] Specific examples

[1803] For example, consider the case where the following news articles are collected:

[1804] Original news article:

[1805] Title: "COVID-19 vaccination progresses"

[1806] Content: "COVID-19 vaccinations are progressing rapidly in Japan. The government plans to offer the vaccine to the entire population in the coming months..."

[1807] For this news article, the following prompt sentence is input to the generative AI model:

[1808] Example prompt sentence:

[1809] "COVID-19 vaccination campaigns are progressing. COVID-19 vaccinations are progressing rapidly in Japan. The government plans to offer the vaccine to the entire population in the coming months..."

[1810] The generated summary results are as follows:

[1811] Summary results:

[1812] "COVID-19 vaccinations are progressing rapidly in Japan, and the government plans to offer the vaccine to all citizens."

[1813] This summary is converted into audio and sent to the user's device, allowing the user to grasp the main news in a short amount of time.

[1814] In this way, the server, user terminal, and artificial intelligence means work together, enabling users to efficiently obtain important information from a vast amount of information and grasp it in a short amount of time.

[1815] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1816] Step 1:

[1817] The server collects the latest news articles from external sources such as a news API by sending HTTP requests and saving the retrieved data in JSON format. The input is the news API endpoint, and the output is the news article data in JSON format.

[1818] Step 2:

[1819] The server analyzes the collected news article data and extracts the content of each article. Specifically, it extracts information such as the article title, text, and publication date and time from the JSON data. The input is news article data in JSON format, and the output is the text information of each news article.

[1820] Step 3:

[1821] The server uses a generative AI model to summarize the extracted text information of news articles. Specifically, it uses the Hugging Face transformers library to input a prompt sentence into the summarization model and obtains the summary result. The input is the text information of the news article, and the output is the summarized text information.

[1822] Step 4:

[1823] The server converts the summarized text information into audio. Specifically, it uses speech synthesis technology such as the Google Text-to-Speech API to convert the text information into audio data. The input is the summarized text information, and the output is audio data.

[1824] Step 5:

[1825] The server sends the voice data and summary text information to the user device. Specifically, it sends data to the user device via push notifications or API requests. The input is the voice data and summary text information, and the output is data delivery to the user device.

[1826] Step 6:

[1827] The user terminal displays and plays the received summary text information and audio data. Specifically, it displays the summary text and plays the audio data through an application. The input is the audio data and summary text information sent from the server, and the output is information provided to the user.

[1828] Step 7:

[1829] The user can view the provided summary information and listen to it aloud. Specifically, by operating the application, the user can check the summary of the news article and play it aloud. The input is the summary information displayed on the user's device, and the output is the user's information acquisition.

[1830] Step 8:

[1831] The user device records the user's usage history and sends it to the server. Specifically, it saves the history of articles the user has viewed and audio they have played, and periodically uploads it to the server. The input is the user's usage history data, and the output is the transmission of the history data to the server.

[1832] Step 9:

[1833] The server analyzes the received usage history data and generates personalized recommendations for the user. Specifically, it selects relevant news articles based on the usage history and generates summary information. The input is the user's usage history data, and the output is personalized summary information.

[1834] Step 10:

[1835] The server then sends the personalized summary information back to the user's device and provides it to the user. Specifically, it sends data via push notifications or API requests. The input is the personalized summary information, and the output is the delivery of data to the user's device.

[1836] Example 3

[1837] Next, a third embodiment of the third embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1838] In modern society, users have more and more opportunities to access a large amount of digital content, but it is difficult to efficiently obtain useful information from that content. It is also difficult for users to find time to obtain information in their busy daily lives. In particular, there is a demand for a means for users to obtain information by effectively utilizing their free time, such as while doing housework or on their way to work.

[1839] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[1840] In this invention, the server includes means for collecting user usage history, means for automatically collecting recommended information based on the collected usage history, artificial intelligence means for summarizing the collected information, means for converting the summarized information into voice data, and means for providing the voice data to the user's terminal. This allows users to efficiently obtain useful information even in their busy daily lives and effectively utilize free time such as during housework or on the way to work.

[1841] "Means for collecting user usage history" refers to a function for collecting behavioral data when a user browses a website or application.

[1842] The "means for automatically collecting recommended information" is a function for automatically collecting highly relevant information from the Internet based on the user's past usage history and preferences.

[1843] "Artificial intelligence means" are systems that include machine learning models and natural language processing techniques used to analyze and summarize collected information.

[1844] The "means for converting the summarized information into audio data" is a function that uses text-to-speech (TTS) technology to convert the summarized information in text form into audio form.

[1845] The "means for providing voice data to the user's terminal" is a function for transmitting the generated voice data to the user's terminal and providing it in a reproducible form.

[1846] This invention is a system that allows users to efficiently obtain useful information and make effective use of free time in their daily lives. This system performs a series of processes: collects the user's usage history, automatically collects recommended information based on that history, summarizes the collected information, converts the summarized information into voice data, and provides the voice data to the user's terminal.

[1847] Hardware and software used

[1848] Hardware: Servers, terminals (PCs, smartphones)

[1849] Software: Text-to-Speech (TTS) engine, machine learning models, database management system (DBMS)

[1850] Specific operation of the system

[1851] 1. Collection of user usage history

[1852] User: Browse websites and applications.

[1853] Device: Collects information about the user's browsing history and preferences, such as the URLs of pages visited, the duration of browsing, and the links clicked.

[1854] Server: Stores browsing history and preference information sent from your device in a database.

[1855] 2. Automatic collection of recommendations

[1856] Server: Uses machine learning models to learn users' browsing history and preferences, specifically analyzing user behavior patterns using Python libraries such as Scikit-learn and TensorFlow.

[1857] Server: Based on the learning results, the server automatically collects new recommended information from the Internet, for example, retrieving related articles and product information from news sites and shopping sites.

[1858] 3. Summary information generation

[1859] Server: Uses natural language processing (NLP) techniques to summarize collected recommendations, specifically using Python libraries such as NLTK and spaCy to create short summaries of long articles and product descriptions.

[1860] Server: Generates summarized text information and stores it in a database, for example, summarizing news articles or product features.

[1861] 4. Audio translation of summary information

[1862] Server: Inputs the stored summary text information into a text-to-speech (TTS) engine, specifically using a TTS service such as Google Text-to-Speech API or Amazon Polly.

[1863] TTS engine: Converts summary text into audio data, for example, converting a news article summary into an audio file (e.g., MP3 format).

[1864] 5. Provision of audio data

[1865] User: Uses the device to request voice data, for example, opens a smartphone app and types the voice command "play me the latest news."

[1866] Server: Sends the generated audio data to the user's device by streaming the audio file via an HTTP request.

[1867] Device: Plays the received audio data and provides it in a form that the user can listen to. For example, the audio can be played through a smartphone speaker or earphones.

[1868] Specific examples

[1869] Example 1: News summarization and audio transcription

[1870] User: Frequent news site visitor.

[1871] Device: Collects user browsing history (URLs of visited news articles and viewing times).

[1872] Server: The collected browsing history is stored in a database and analyzed using machine learning models.

[1873] Server: Collects and summarizes the latest news articles based on user interests.

[1874] Server: The summarized news article is input into the TTS engine and converted into voice data.

[1875] User: Requests "Listen to the latest news" on their smartphone during their commute.

[1876] Server: Sends voice data to the user's smartphone.

[1877] Terminal: Plays audio data and allows the user to listen to the news.

[1878] Example 2: Product Recommendations

[1879] User: Visits an online shopping site.

[1880] Device: Collects user purchase and browsing history.

[1881] Server: Stores the collected history in a database and analyzes it using machine learning models.

[1882] Server: Collects and summarizes new products based on user interests.

[1883] Server: The summarized product description is input into the TTS engine and converted into voice data.

[1884] User: While doing housework, the user requests on their smartphone, "Tell me what products you recommend."

[1885] Server: Sends voice data to the user's smartphone.

[1886] Terminal: The audio data is played and the user listens to the product information.

[1887] Prompt Sentence Examples

[1888] "Collect recommended news articles based on the user's past browsing history, summarize them, and convert them into audio data."

[1889] "Based on your online shopping site usage history, please summarize new product recommendations and provide them to you via voice."

[1890] This system allows users to efficiently acquire information and easily obtain useful information in their daily lives. The flow of the identification process in the third embodiment will be described with reference to FIG.

[1891] Step 1:

[1892] Collection of user usage history

[1893] User: Browse websites and applications.

[1894] Device: Collects information about the user's browsing history and preferences, such as the URLs of pages visited, the duration of browsing, and the links clicked.

[1895] Input: User browsing behavior data (URL, browsing time, clicked links, etc.)

[1896] Output: Collected browsing history data

[1897] Specific operation: The device monitors user behavior data in real time and sends the collected data to the server.

[1898] Step 2:

[1899] Automatic collection of recommendations

[1900] Server: Uses machine learning models to learn users' browsing history and preferences, specifically analyzing user behavior patterns using Python libraries such as Scikit-learn and TensorFlow.

[1901] Server: Based on the learning results, the server automatically collects new recommended information from the Internet, for example, retrieving related articles and product information from news sites and shopping sites.

[1902] Input: Collected browsing history data

[1903] Output: Recommendations (news articles, product information, etc.)

[1904] What it does: The server runs machine learning models to ga...

Claims

1. means for estimating an emotional state of a user based on the user's voice, text input, facial expression, and biometric information; A means for collecting unread or unviewed information from a plurality of digital contents; A means for recommending optimal content from among collected digital content based on the emotional state; A system including:

2. 2. The system of claim 1, further comprising means for summarizing the collected information and generating audio summary information of the summarized information.

3. The system according to claim 1 , wherein the means for recommending the optimal content recommends the optimal content at an optimal timing based on the user's behavioral patterns and schedule.

Images