system

Abstract

Provide a system. A method for collecting news articles, A means for analyzing the text of collected news articles and extracting key points; A means of summarizing news articles based on key points; means for transmitting the summarized news to a user terminal; means for displaying a news summary on a user terminal; A system including:

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In modern society, people receive a huge amount of information from many sources, and they need to efficiently manage it and quickly obtain only the information they need. However, individually checking information from various sources, such as news, email, and video, takes time and effort, and there is also a risk of missing important information. Therefore, there is a need for an effective means to solve the problem of information overload and efficiently obtain and understand the information they need. [Means for solving the problem]

[0005] The present invention provides a system that efficiently summarizes information collected from multiple information sources and provides it to users. Specifically, the system includes means for collecting news articles, analyzing them to extract key points, and sending the summarized news to a user terminal, means for receiving emails, analyzing their contents to extract important information, and sending the summarized email to a user terminal, and means for receiving video URLs, analyzing the audio and video of the video to extract important parts, and sending the summarized video to a user terminal. This allows users to quickly and efficiently obtain and understand the information they need.

[0006] A "news article" is a piece of text published on the Internet that provides information on politics, economics, sports, etc.

[0007] "Collecting" is the act of obtaining specific information from multiple sources on the Internet.

[0008] "Analyzing" refers to the process of structurally analyzing the acquired information and identifying its contents and key points.

[0009] The "gist" is a part of the information that is particularly important and expresses the content concisely.

[0010] "Summarizing" means to summarize information in a concise manner, extracting only the main points and shortening the information.

[0011] A "user terminal" is an electronic device (e.g., a smartphone, a personal computer, a tablet, etc.) that a user operates and uses to receive information.

[0012] "Email" is a digital message sent and received over the Internet, a means of communication that can include text, images, files, etc.

[0013] "Audio" refers to the language and sound information reproduced in the video.

[0014] "Video" refers to visual information displayed in a moving image.

[0015] "Viewing history" is a record of videos that a user has viewed in the past.

[0016] The "important part" refers to the main information in a video, document, etc., or the part of the content that is valuable to the user. [Brief explanation of the drawings]

[0017] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0018] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0019] First, the terms used in the following description will be explained.

[0020] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0021] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0022] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0023] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0024] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0025] [First embodiment]

[0026] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0027] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0028] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0029] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0030] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0031] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0032] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0033] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0034] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0035] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0036] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0037] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0038] This invention is a system that efficiently summarizes information from three types of information sources: news, email, and video, and provides it to users. To implement this system, a server that runs a program with the following functions, a terminal, and user operations are required.

[0039] News article summary function

[0040] News article collection

[0041] The server has a mechanism to access various news sites on the Internet and automatically collect the latest news articles. For example, it filters by category, such as politics, economics, and sports, and extracts the necessary news articles.

[0042] News article analysis

[0043] The server analyzes the collected news articles using natural language processing (NLP) algorithms. Using the latest NLP techniques, such as BERT and GPT models, it extracts important keywords and key points from the articles. This analysis determines the context and importance of the text and identifies the information needed for summarization.

[0044] News article summaries

[0045] Based on the results of the analysis, the server summarizes the news article in a few lines, leaving out the main points and important details of the news and eliminating redundant parts.

[0046] Sending and displaying news summaries

[0047] The server generates a document summarizing the summarized news and sends it to the user's terminal.

[0048] Users can easily check the summarized news on their devices.

[0049] Email Summary Feature

[0050] Get email

[0051] The device accesses the user's email account (e.g., Gmail or Outlook) and retrieves all of the emails received for the day.

[0052] Email Analysis

[0053] The terminal sends the acquired email data (body, subject, sender information, etc.) to the server.

[0054] Based on the data sent, the server analyzes the content of each email using natural language processing (NLP) algorithms to extract important information and keywords.

[0055] Email Summary

[0056] The server extracts important keywords and key points from each email and then summarises them into a few lines, concisely summarising important meeting schedules or urgent requests.

[0057] Sending and viewing summary emails

[0058] The server generates a list of summarized emails and sends them to the user's terminal.

[0059] The user can easily check the summarized email on the terminal.

[0060] Video summary function

[0061] Get the video URL

[0062] The user inputs their viewing history and a specified URL into the device.

[0063] Video data analysis

[0064] The terminal transmits the specified URL to the server, and the server accesses the URL to obtain the video data.

[0065] The server converts the audio of the video into text using automatic speech recognition (ASR) technology, and then analyzes the video data to extract important scenes, for example, by identifying important parts based on audio containing specific keywords or scene changes.

[0066] Video Summary

[0067] The server generates a video digest based on the extracted key points, summarizing important announcements and interesting moments into short clips.

[0068] Sending and displaying summary videos

[0069] The server transmits the summarized video file to the user's terminal.

[0070] The user can view the digest video generated on the terminal and quickly understand the main content.

[0071] Specific examples

[0072] News article summary examples

[0073] For example, if the server retrieves a "political news article" from a nationally known news site, it will analyze the text for the title "Cabinet reshuffle" using the BERT model, extracting "new cabinet members," "policy changes," "key statements," etc., and generate a summary based on this.

[0074] Example of an email summary

[0075] For example, a device retrieves an email with the subject "Meeting Schedule" from a user's email account and sends the contents to the server. The server extracts information such as the meeting date and time, names of participants, and main agenda items from the body of the email and generates a summary.

[0076] Video summary examples

[0077] For example, if a user selects the URL of a "sports highlight video" from their viewing history, the server retrieves the corresponding video and performs voice recognition and video analysis. As a result, it extracts "important goal scenes" and "interview highlights," and generates a summary clip based on these.

[0078] In this way, the present invention realizes a system that efficiently summarizes news, email, and video information and provides it to users, allowing them to quickly and concisely grasp the information they need.

[0079] The processing flow will be explained below.

[0080] News article summary function

[0081] Step 1: Gather news articles

[0082] The server accesses the specified news site and collects the URLs of the latest news articles, sometimes using the news site's API.

[0083] Step 2: Get news articles

[0084] The server retrieves HTML data from the collected URLs and converts it into a format that is easy to analyze. It extracts the news text, title, date and time, etc.

[0085] Step 3: Analyzing the news article

[0086] The server uses natural language processing (NLP) algorithms to analyze the text of retrieved news articles, using models such as BERT and GPT to extract key keywords and key points from each article.

[0087] Step 4: Summarize the news article

[0088] The server then summarises the article based on the analysis results, combining multiple algorithms to generate an optimal summary that always includes important information, such as important dates, people's names, and events.

[0089] Step 5: Submit your news summary

[0090] The server generates a document summarizing the summarized news and sends it to the user's terminal.

[0091] Step 6: Viewing News Summary

[0092] Users can easily view summaries of news on their devices, and can filter news by category and importance.

[0093] Email Summary Feature

[0094] Step 1: Get email

[0095] The device accesses the user's email account and retrieves all of the emails received in one day. It uses RestAPI or similar to retrieve data from the email server.

[0096] Step 2: Send email data

[0097] The terminal sends the acquired email data (subject, body, sender information, etc.) to the server.

[0098] Step 3: Parse the email

[0099] The server uses NLP algorithms to extract obviously important information, such as meeting schedules, project progress, and requests that require urgent attention.

[0100] Step 4: Email Summary

[0101] The server then summarises each email into a few lines based on the extracted keywords and key points, concisely summarising the key points so that the content can be understood at a glance.

[0102] Step 5: Send a summary email

[0103] The server generates a list of summarized mails and sends it to the user's terminal.

[0104] Step 6: View the summary email

[0105] Users can view summaries of emails compiled on their device, and it also includes the ability to filter out unnecessary emails and display only the important ones.

[0106] Video summary function

[0107] Step 1: Get the video URL

[0108] The user inputs the viewing history or the URL of the specified video into the device.

[0109] Step 2: Sending video data

[0110] The terminal sends the specified URL to the server.

[0111] Step 3: Getting the video

[0112] The server accesses the URL and retrieves the video data. YouTube (registered trademark) API or Tver API is often used.

[0113] Step 4: Audio-visual analysis of the video

[0114] The server transcribes the audio from the captured video using automatic speech recognition (ASR) technology and performs scene analysis of the video to identify important parts, such as audio containing specific keywords or scene changes.

[0115] Step 5: Summarize your video

[0116] Based on the extracted key points, the server generates a digest video that summarizes the important scenes of the video in less than five minutes.

[0117] Step 6: Submit your summary video

[0118] The server transmits the digested video to the user terminal.

[0119] Step 7: Displaying the summary video

[0120] Users can play the digest video generated on their device and quickly check the main content.

[0121] The above processing steps realize a system that efficiently summarizes news, email, and video information, allowing users to quickly obtain the information they need.

[0122] Example 1

[0123] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0124] In modern society, as the amount of information increases, it is becoming increasingly difficult to efficiently grasp important information. In particular, there is a lack of methods to quickly retrieve, summarize, and present necessary information from various sources, such as news articles, emails, and videos. This forces users to manually find important parts from a vast amount of information, which is a time-consuming and labor-intensive process.

[0125] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0126] In this invention, the server includes a means for collecting news information, a means for analyzing the collected news information using a natural language processing algorithm to extract key points, and a means for summarizing the news information in a few lines based on the key points. This enables summarization of news articles. The server also includes a means for acquiring electronic communications, a means for analyzing the content of the acquired electronic communications to extract important information, and a means for summarizing the electronic communications in a few lines based on the important information. This enables summarization of emails. The server also includes a means for acquiring URLs of video information, a means for analyzing the audio and video of the acquired video information to extract important parts, and a means for summarizing the video information into short clips based on the extracted important parts. This enables summarization of videos. These functions allow users to efficiently acquire important information from different information sources and grasp it in a short amount of time.

[0127] "News information" refers to the content and articles of news sites published on the Internet.

[0128] "Natural language processing algorithms" refers to technologies that analyze text data and evaluate context and importance. Specifically, this includes models such as BERT and GPT.

[0129] "Key points" refer to important information or keywords found in news information, electronic communications, and video information.

[0130] "Summarizing" means compressing long pieces of information into a concise summary of only the main points and important information.

[0131] "Electronic communications" refers to communications such as emails exchanged over the Internet.

[0132] "Speech recognition technology" refers to technology that converts voice data into text data. Specifically, it includes ASR (automatic speech recognition) technology.

[0133] "Video information" refers to the content and images of videos that can be viewed on the Internet.

[0134] A "URL" is an address used to specify a specific resource on the Internet.

[0135] "Analyze" refers to analyzing data or information using algorithms to find specific patterns or important elements.

[0136] "User terminal" refers to a device used by a user to receive and display information, including, but not limited to, a smartphone, tablet, or computer.

[0137] "Document" refers to a digital document, such as a text file or PDF, that compiles summarized information.

[0138] "Generate" refers to creating data or information from scratch using a computer.

[0139] "Transmitting" refers to sending data or information from one location to another over a network.

[0140] "Display" refers to visually presenting data or information so that it can be read by a user.

[0141] A "short clip" refers to a short piece of video created by cutting out important parts from a longer video.

[0142] This invention is a system that collects information from three types of information sources: news information, electronic communications, and video information, and efficiently summarizes each of them and provides them to users. To implement this system, a server that runs a program with the following functions, a terminal, and user operations are required.

[0143] News article summary function

[0144] Gathering news information

[0145] The server sets a Cron job to access a news site, for example, every day at 9:00 a.m., to collect the latest news information. The server uses the news site's API to extract the news title and text data of the news body.

[0146] News information analysis

[0147] The server analyzes the collected news information using natural language processing algorithms. Specifically, it uses generative AI models such as BERT and GPT to tokenize the text data, evaluate its context and importance, extract important keywords and key points, and store the results.

[0148] News summary

[0149] The server summarizes the news information in a few lines based on the extracted keywords and key points, concisely retaining only the main points and eliminating redundant parts.

[0150] Sending and displaying summary news information

[0151] The server generates a document containing the summarized news information and sends it to the user's terminal, for example, in PDF or HTML format.

[0152] To enable a user to check summarized news information on a terminal and efficiently grasp the latest news.

[0153] Summary function of electronic communications

[0154] Obtaining Electronic Communications

[0155] The device accesses the user's email account using the IMAP protocol and retrieves electronic communications for a specified period of time. The connection is made using a secure authentication protocol (such as OAuth).

[0156] Analysis of electronic communications

[0157] The device sends captured electronic communications to a server, which then uses generative AI models such as BERT and GPT to analyze the email data and extract key information and keywords.

[0158] Electronic Communications Summary

[0159] The server summarizes electronic communications in a few lines based on extracted keywords and key points, concisely summarizing important meeting schedules and information requiring urgent action.

[0160] Summary of electronic communication transmission and display

[0161] The server generates a document from the summarized electronic communication and transmits it to the user's terminal.

[0162] The user can view summarized electronic communications on a terminal and quickly grasp important information.

[0163] Video summary function

[0164] Get the video URL

[0165] The user enters their viewing history and a specified URL into the device, and the device then sends the URL to the server.

[0166] Video information analysis

[0167] The server accesses the specified URL and retrieves the video data. The server then downloads the correct data using the YouTube API or similar.

[0168] The server uses automatic speech recognition (ASR) technology to convert the audio in the video into text and simultaneously analyzes the video data, for example, to detect lines containing important keywords or scene changes.

[0169] Video information summary

[0170] The server generates a video digest based on the analysis results, compiling important announcements and interesting moments into short clips.

[0171] Sending and displaying summary video information

[0172] The server sends the summarized video file to the user's device, where it can be streamed using a dedicated app.

[0173] Users can watch the summary video on their device and quickly grasp the important content.

[0174] Specific examples

[0175] Examples of news summaries

[0176] For example, the server retrieves "political news information" from a domestic news site and analyzes it using the BERT model. The analysis extracts "new cabinet members," "policy changes," "major statements," etc., and generates a summary based on this.

[0177] Examples of Electronic Communications Summaries

[0178] For example, a terminal retrieves an electronic message with the subject "Meeting Schedule" from a user's email account and sends it to a server. The server extracts information such as the meeting date and time, participant names, and main agenda items from the message body and generates a summary.

[0179] Example of video summary

[0180] For example, if a user selects the URL of a "sports highlight video" from their viewing history, the server retrieves and analyzes the corresponding video. As a result of voice recognition and video analysis, it extracts "goal scenes" and "interview highlights," and generates a summary clip based on these.

[0181] Prompt Sentence Examples

[0182] "Please summarize the latest political news article."

[0183] "Please tell me the highlights of the email I received today."

[0184] "Summarize the key scenes in this sports video."

[0185] In this way, the present invention realizes a system that efficiently summarizes news information, electronic communications, and video information and provides them to users, allowing them to quickly and concisely grasp the information they need.

[0186] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0187] News article summary function

[0188] Step 1: Gather news information

[0189] The server sets up a Cron job to access a news site on the Internet, for example, every day at 9:00 AM.

[0190] Input: List of news site URLs

[0191] The server accesses each URL and uses the news site's API to obtain the latest news information.

[0192] Data processing: Parse HTML to extract news titles and text data.

[0193] Output: A list of news articles in text format

[0194] Step 2: Analyzing the news information

[0195] The server analyzes the news information collected in the previous step using a natural language processing algorithm.

[0196] Input: News article list

[0197] The server uses generative AI models such as BERT or GPT to tokenize the text data and evaluate its context and importance.

[0198] Data calculation: Extract important keywords and key points.

[0199] Output: A dataset showing key keywords and key points

[0200] Step 3: Summarize the news information

[0201] The server summarizes the news information in a few lines based on the extracted keywords and key points.

[0202] Input: A dataset showing important keywords and key points

[0203] Data processing: Keep only the main points and remove redundant parts.

[0204] Output: Summary

[0205] Step 4: Send and display summary news information

[0206] The server generates a document containing summarized news information and transmits it to the user's terminal.

[0207] Input: Abstract

[0208] Data processing: Generate documents in PDF or HTML format.

[0209] Output: A summary of the news document

[0210] The user checks the summarized news information on the terminal.

[0211] Summary function of electronic communications

[0212] Step 1: Obtaining Electronic Communications

[0213] The device accesses the user's email account using the IMAP protocol.

[0214] Enter your email account information

[0215] The device captures electronic communications for a specified period of time.

[0216] Data processing: Read data from received emails.

[0217] Output: Electronic communication data

[0218] Step 2: Analyzing Electronic Communications

[0219] The terminal transmits the captured electronic communication to a server.

[0220] Input: Electronic communication data

[0221] The server analyzes the data using a generative AI model such as BERT or GPT.

[0222] Data calculations: Extracting important information and keywords.

[0223] Output: Data showing important information and keywords

[0224] Step 3: Summarizing Electronic Communications

[0225] The server summarizes the electronic communication in a few lines based on the extracted keywords and key points.

[0226] Input: Data that indicates important information or keywords

[0227] Data processing: Concisely summarize important meeting schedules and information that requires urgent action.

[0228] Output: Summary

[0229] Step 4: Sending and displaying summary electronic communications

[0230] The server generates a document from the summarized electronic communication and transmits it to the user's terminal.

[0231] Input: Abstract

[0232] Data processing: Generate documents in PDF or HTML format.

[0233] Output: A summary of the electronic communication

[0234] The user reviews the summarized electronic communication at the terminal.

[0235] Video summary function

[0236] Step 1: Get the video URL

[0237] The user enters their viewing history and a specified URL into the device, and the device then sends the URL to the server.

[0238] Input: Video URL

[0239] Output: URL data

[0240] Step 2: Analyze video information

[0241] The server accesses the specified URL and acquires the video data.

[0242] Input: URL data

[0243] The server downloads the video data using the YouTube API or similar.

[0244] Data processing: Uses automatic speech recognition (ASR) technology to convert voice into text and also analyzes video data.

[0245] Output: Analyzed text data and video data

[0246] Step 3: Summary of video information

[0247] The server generates a video digest based on the analysis results.

[0248] Input: Analyzed text data and video data

[0249] Data processing: Summarize important scenes and quotes into short clips.

[0250] Output: Summary clip

[0251] Step 4: Send and display summary video information

[0252] The server sends the summarized video file to the user's terminal.

[0253] Input: Summary clip

[0254] Data processing: Streaming playback becomes possible using a dedicated app.

[0255] Output: Summarized video file

[0256] Users can watch the summary video on their device and quickly grasp the important content.

[0257] (Application example 1)

[0258] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0259] In today's information-saturated world, users need to quickly and efficiently understand the vast amount of news articles, emails, and video information. It is particularly difficult to extract and provide users with only the most important information from each source. Furthermore, there is a lack of a way to easily summarize this information and provide it to users in an easily accessible format.

[0260] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0261] In this invention, the server includes means for collecting news articles, means for analyzing the text of the collected news articles and extracting key points, means for summarizing the news articles based on the key points, means for sending the summarized news to a user terminal, means for displaying the summarized news on the user terminal, and means for optimizing the summaries using a generative AI model and prompt sentences, thereby enabling users to quickly grasp only the important key points from a large amount of information.

[0262] A "news article" is a report or information about a current event or topic provided through online or offline media.

[0263] "Email" is a digital message sent or received over the Internet or other network.

[0264] A "video" is a multimedia file that combines audio and video and is played continuously.

[0265] A summary is a short summary of the important elements or key points of the original information.

[0266] A "user terminal" is a device used by a user, including a smartphone, tablet, or PC.

[0267] A "generative AI model" is an artificial intelligence model that uses machine learning algorithms to generate text and data.

[0268] A "prompt" is text input to a generative AI model that takes the form of an instruction or question to the model.

[0269] A "collection method" is a method or system for obtaining data or information from a particular source.

[0270] An "analysis tool" is a method or system for processing collected data or information and extracting meaningful elements.

[0271] A "transmission means" is a method or system for sending data or information from one system to another.

[0272] A "display means" is a method or system for visually presenting information to a user, and includes devices such as a screen.

[0273] This invention is an information summarization system for efficiently providing various information to users. This system summarizes information from three types of information sources: news, email, and video, and provides it to users concisely. To implement this, a server, a terminal, and user operations are required.

[0274] News article summary function

[0275] News article collection

[0276] The server accesses various news sites to collect the latest news articles, and then filters the articles by category, such as politics, economics, or sports, to extract the necessary news articles.

[0277] News article analysis

[0278] The server analyzes the collected news articles using natural language processing (NLP) algorithms, such as the BERT model, to extract the article's context and important keywords.

[0279] News article summaries

[0280] The server then summarises the news article based on the analysis, keeping the key points and keywords and removing redundant parts.

[0281] Sending and displaying news summaries

[0282] The server sends the summarized news to the user's terminal, where the user can check the concisely summarized news.

[0283] Email Summary Feature

[0284] Get email

[0285] The terminal accesses the user's email account and retrieves received emails.

[0286] Email Analysis

[0287] The device sends the email data to the server, which then uses NLP algorithms to analyze the email body and extract important information and keywords.

[0288] Email Summary

[0289] The server then uses this extracted data to summarize the email, concisely summarizing important schedules and information requiring urgent action.

[0290] Sending and viewing summary emails

[0291] The server sends the summarized email to the user's terminal, where the user can check the summarized email.

[0292] Video summary function

[0293] Get the video URL

[0294] The user inputs their viewing history and the specified URL into the device.

[0295] Video data analysis

[0296] The device sends the URL to the server, which retrieves the corresponding video. The audio data is converted into text using automatic speech recognition (ASR), and important scenes are extracted from the video data.

[0297] Video Summary

[0298] Based on the analysis data, the server generates a summary clip of the video containing important scenes and keywords.

[0299] Sending and displaying summary videos

[0300] The server sends the summarized video to the user's device, where the user can watch the summarized video in a short time.

[0301] Hardware and Software Used

[0302] The system's main hardware consists of a server and user devices (smartphones, tablets, and PCs). The software used includes Python, the BERT model, the Transformers library, the requests library, and speech recognition technology.

[0303] Examples and prompts

[0304] As a concrete example, consider a case where a user checks a news article summary on their smartphone. For example, the following prompt sentence is input to the generative AI model:

[0305] "Summarize the article: A new cabinet has been announced, with key changes."

[0306] An example output is:

[0307] "A new cabinet has been announced, with key members being changed."

[0308] Users can view this output on their smartphones.

[0309] This system allows users to quickly grasp only the important points from a large amount of information.

[0310] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0311] Step 1:

[0312] Collect news articles (server)

[0313] The server accesses various news sites and automatically collects the latest news articles. The input is the URL of the news site, and the output is the collected news article data. The server filters these articles by category and saves the collected data.

[0314] Step 2:

[0315] News article analysis (server)

[0316] The server analyzes the collected news article text using a natural language processing (NLP) algorithm. Specifically, it uses the BERT model to extract the article's context and important keywords. The input is the collected news article data, and the output is the analysis results: important keywords and key points.

[0317] Step 3:

[0318] News article summaries (server)

[0319] The server summarizes the news article based on the keywords and key points extracted in the previous step, eliminating redundant parts and picking out only the important information. The input is the analysis result, and the output is the summarized news article text.

[0320] Step 4:

[0321] Sending summary news (server)

[0322] The server sends the summarized news article to the user's terminal, where the input is the summarized news article text and the output is the summarized news sent to the user's terminal.

[0323] Step 5:

[0324] Displaying news summaries (terminal)

[0325] The user terminal displays the received summary news. The input is the summary news sent from the server, and the output is the news content displayed on the user interface.

[0326] Step 6:

[0327] Retrieving email (terminal)

[0328] The terminal accesses the user's email account and retrieves all received emails for one day. The input is the email account authentication information, and the output is the retrieved email data.

[0329] Step 7:

[0330] Email analysis (terminal)

[0331] The device sends the acquired email data to the server, which then uses NLP algorithms to analyze the content. The input is the email data, and the output is the extraction of important information and keywords.

[0332] Step 8:

[0333] Email Abstract (Server)

[0334] The server summarizes each email in a few lines based on the extracted important keywords and information. The input is the analysis result, and the output is the summarized email text.

[0335] Step 9:

[0336] Sending summary emails (server)

[0337] The server sends a list of summarized emails to the user terminal, where the input is the summarized email text and the output is the summarized email sent to the user terminal.

[0338] Step 10:

[0339] Display summary email (terminal)

[0340] The user terminal displays the received summary email. The input is the summary email sent from the server, and the output is the email content displayed on the user interface.

[0341] Step 11:

[0342] Get the video URL (device)

[0343] The user inputs their viewing history and the URL of the video they want to view into the device. The input is the URL of the video they specified, and the output is that URL information.

[0344] Step 12:

[0345] Video data analysis (server)

[0346] The device sends the URL of the specified video to the server. The server accesses the URL and retrieves the video data. The audio data is converted into text using speech recognition technology, and important scenes are extracted from the video data. The input is the video URL, and the output is the speech recognition results and the extraction of important scenes.

[0347] Step 13:

[0348] Video summary (server)

[0349] The server generates a video digest clip based on the extracted keypoints. The input is the speech recognition result and the extraction of important scenes, and the output is a summarized video clip.

[0350] Step 14:

[0351] Sending summary video (server)

[0352] The server sends the summarized video file to the user's terminal, where the input is the summarized video clip and the output is the summarized video file sent to the user's terminal.

[0353] Step 15:

[0354] Display summary video (device)

[0355] The user terminal plays the received summary video. The input is the summary video file sent from the server, and the output is the summary video that is played.

[0356] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0357] This invention combines a system that collects information from three types of information sources (news, email, and video), efficiently summarizes it, and provides it to users with an emotion engine that recognizes user emotions. Implementing this system requires a server that runs a program with the following functions, a terminal, and user operation.

[0358] News article summarization and sentiment engine

[0359] News article collection

[0360] The server has a mechanism to access various news sites on the Internet and automatically collect the latest news articles. For example, it filters by category, such as politics, economics, and sports, and extracts the necessary news articles.

[0361] News article analysis

[0362] The server analyzes the collected news article text using natural language processing (NLP) algorithms, using the latest NLP techniques such as BERT and GPT models to extract important keywords and key points from the articles.

[0363] News article summaries

[0364] Based on the results of the analysis, the server summarizes the news article in a few lines, leaving out the main points and important details of the news and eliminating redundant parts.

[0365] Emotion Recognition and Filtering

[0366] The server uses an emotion engine to recognize the user's current emotional state, for example, by analyzing the user's facial expressions and voice through a camera.

[0367] The server filters news articles appropriate for the user based on the perceived emotional state: for example, if the user is feeling stressed, it prioritizes positive news articles.

[0368] Sending and displaying news summaries

[0369] The server generates a document summarizing the summarized news and sends it to the user's terminal.

[0370] Users can easily check filtered news summaries on their devices.

[0371] Email summary and sentiment engine

[0372] Get email

[0373] The device accesses the user's email account (e.g., Gmail or Outlook) and retrieves all of the emails received for the day.

[0374] Email Analysis

[0375] The terminal sends the acquired email data (body, subject, sender information, etc.) to the server.

[0376] Based on the data sent, the server analyzes the content of each email using natural language processing (NLP) algorithms to extract important information and keywords.

[0377] Email Summary

[0378] The server extracts important keywords and key points from each email and then summarises them into a few lines, concisely summarising important meeting schedules or urgent requests.

[0379] Emotion recognition and priority display

[0380] The server uses an emotion engine to recognize the user's current emotional state.

[0381] The server prioritizes which emails to display based on the user's emotional state, for example, displaying less urgent emails to a stressed user first.

[0382] Sending and viewing summary emails

[0383] The server generates a list of summarized emails and sends them to the user's terminal.

[0384] Users can view emails on their devices summarized in order of importance according to their emotions.

[0385] Video summary function and emotion engine

[0386] Get the video URL

[0387] The user inputs their viewing history or a specified URL into the device.

[0388] Video data analysis

[0389] The terminal transmits the specified URL to the server, and the server accesses the URL to obtain the video data.

[0390] The server transcribes the audio from the captured video using automatic speech recognition (ASR) technology and analyzes the video data to identify important parts, for example, based on audio containing specific keywords or scene changes.

[0391] Video Summary

[0392] Based on the extracted key points, the server generates a digest video that summarizes the important scenes of the video in less than five minutes.

[0393] Emotion recognition and content regulation

[0394] The server uses an emotion engine to recognize the user's emotional state.

[0395] The server automatically selects and tailors the summary video to suit the user based on the recognized emotional state, for example prioritizing relaxing content if the user is tired.

[0396] Sending and displaying summary videos

[0397] The server transmits the digested video to the user's terminal.

[0398] Users can play the digest video generated on their device and quickly check the main content.

[0399] Specific examples

[0400] Specific news article examples

[0401] For example, if a user is in an emotional state where they want to relax after work, the server will recognize the user's current emotions and provide a summary of positive articles preferentially, allowing the user to obtain important information while reducing stress.

[0402] Specific examples of email

[0403] For example, if a user is checking email during busy morning hours, the server will recognize through its emotion engine that the user is feeling anxious or stressed. Based on this, the server will prioritize and summarize and display emails with high urgency.

[0404] Video examples

[0405] For example, if a user is tired and wants to relax, the server will recognize the user's current emotions and prioritize and summarize relaxing videos based on that emotion. The user can watch relaxing videos in a short amount of time.

[0406] In this way, by combining emotion engines, it becomes possible to provide information that is more in line with the user's needs, improving the user experience.

[0407] The processing flow will be explained below.

[0408] News article summarization and sentiment engine

[0409] Step 1: Gather news articles

[0410] The server accesses various news sites on the Internet and collects the URLs of the latest news articles, for example, by obtaining the necessary data from RSS feeds or APIs.

[0411] Step 2: Get news articles

[0412] The server retrieves the HTML data from the collected URLs and extracts the body of the news article, the title, and the date and time.

[0413] Step 3: Analyzing the news article

[0414] The server uses natural language processing (NLP) algorithms to analyze the text of the news article, extracting key keywords and gist information and identifying important information.

[0415] Step 4: Summarize the news article

[0416] The server then summarises the news article in a few lines based on the extracted key points and keywords, concisely summarising the main points.

[0417] Step 5: Emotion Recognition

[0418] The device captures the user's facial expressions and voice through a camera and microphone, and analyzes them with an emotion engine. For example, emotions are identified from the user's facial expressions and tone of voice.

[0419] Step 6: Filtering

[0420] The server then filters appropriate news articles based on the results of the emotion engine according to the user's emotional state. For example, if the user feels like relaxing, it prioritizes positive news.

[0421] Step 7: Submit your news summary

[0422] The server generates a document summarizing the filtered news articles and sends it to the user's terminal.

[0423] Step 8: Viewing Summary News

[0424] Users can easily check filtered news summaries on their devices.

[0425] Email summary and sentiment engine

[0426] Step 1: Get email

[0427] The device accesses the user's email account and retrieves all of the emails received in one day, using an email protocol (e.g., IMAP or POP3).

[0428] Step 2: Send email data

[0429] The terminal sends the acquired email data (body, subject, sender information, etc.) to the server.

[0430] Step 3: Parse the email

[0431] The server analyzes the email content using natural language processing (NLP) algorithms to extract important information and keywords, such as meeting schedules or urgent requests.

[0432] Step 4: Email Summary

[0433] The server then summarises each email in a few lines based on the extracted key keywords and information, concisely summarising the key points.

[0434] Step 5: Emotion Recognition

[0435] The device captures the user's facial expressions and voice through a camera and microphone, and analyzes them with an emotion engine. For example, emotions are identified from the user's facial expressions and tone of voice.

[0436] Step 6: Prioritize Display

[0437] The server then prioritizes which emails to display based on the results of the emotion engine and the user's emotional state. For example, if the user is feeling stressed, emails with a low level of urgency will be displayed first.

[0438] Step 7: Send a summary email

[0439] The server generates a prioritized summary mail list based on the emotional state and transmits it to the user's terminal.

[0440] Step 8: View the summary email

[0441] The user can check the emails summarized according to priority on the terminal.

[0442] Video summary function and emotion engine

[0443] Step 1: Get the video URL

[0444] The user inputs the viewing history or the URL of the specified video into the device.

[0445] Step 2: Sending video data

[0446] The terminal sends the specified URL to the server.

[0447] Step 3: Getting the video

[0448] The server accesses the transmitted URL and acquires the video data.

[0449] Step 4: Audio-visual analysis of the video

[0450] The server transcribes the audio from the captured video using automatic speech recognition (ASR) technology and analyzes the video data to identify important parts, such as audio containing key keywords and scene changes.

[0451] Step 5: Summarize your video

[0452] The server generates a video digest based on the extracted key points, summarizing important scenes into short clips.

[0453] Step 6: Emotion Recognition

[0454] The device captures the user's facial expressions and voice through a camera and microphone, and analyzes them with an emotion engine. For example, emotions are identified from the user's facial expressions and tone of voice.

[0455] Step 7: Adjust the content

[0456] Based on the results of the emotion engine, the server selects a summary video that best suits the user's emotional state and adjusts the content accordingly. For example, if the user wants to relax, it will prioritize videos with relaxing content.

[0457] Step 8: Submit your summary video

[0458] The server transmits the summarized digest video to the user's terminal.

[0459] Step 9: Displaying the Summary Video

[0460] Users can play the digest video generated on their device and quickly check the main content.

[0461] The above processing steps realize a system that efficiently summarizes news, email, and video information, and further adjusts the content provided based on the user's emotional state.

[0462] Example 2

[0463] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0464] Conventional summarization systems for news articles, emails, and videos provide information uniformly without considering the user's emotional state, making it difficult to provide content that adapts to the user's psychological state. Furthermore, efficient summarization of large amounts of information and appropriate provision to the user requires advanced natural language processing and emotion recognition technologies. Consequently, to improve the user experience, it is necessary to adjust the priority and content of information according to the user's emotional state.

[0465] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0466] In this invention, the server includes means for collecting news articles, means for analyzing the text of the collected news articles using a natural language processing algorithm and extracting key points, means for summarizing the news articles based on the key points, means for filtering the news articles based on the emotional state using an emotion engine that recognizes the emotional state of the user, means for transmitting the summarized news to a user terminal, and means for displaying the summarized news on the user terminal, thereby making it possible to provide news articles that are adapted to the emotional state of the user.

[0467] The server also includes means for acquiring emails, means for analyzing the contents of the acquired emails using a natural language processing algorithm and extracting important information, means for summarizing the emails based on the important information, means for determining the priority of the emails based on the emotional state having an emotion engine that recognizes the emotional state of the user, means for sending the summarized emails to the user terminal, and means for displaying the summarized emails on the user terminal, thereby enabling the importance of emails to be determined and provided in accordance with the emotional state of the user.

[0468] The server further includes a means for acquiring a URL of the video, a means for analyzing the audio and video of the acquired video using voice recognition technology and video analysis technology and extracting important parts, a means for summarizing the video based on the extracted important parts, a means having an emotion engine that recognizes the emotional state of the user and adjusting the content of the video based on the emotional state, a means for transmitting the summarized video to the user terminal, and a means for displaying the summarized video on the user terminal, thereby making it possible to provide video content adapted to the emotional state of the user.

[0469] These measures enable the provision of information according to the user's emotional state, improving the user experience.

[0470] The "means for collecting news articles" is a function that allows the server to access news sites on the Internet and automatically obtain the latest news articles.

[0471] "Natural language processing algorithms" are technologies for analyzing text data and understanding the meaning and structure of language, and include BERT and GPT models.

[0472] "Key point extraction" is a function that identifies and extracts important keywords and key information from the body of a news article or email.

[0473] The "means for summarizing news articles" is a function that summarizes the text of a news article in a concise format based on the extracted important information.

[0474] "Means for retrieving email" refers to a function that accesses a user's email account (e.g., Gmail or Outlook) and retrieves all received emails.

[0475] The "means for determining the priority of e-mails" is a function for determining the display order of received e-mails based on the emotional state of the user.

[0476] The "means for transmitting summarized e-mail to a user terminal" is a function for transmitting the contents of the summarized e-mail to a user terminal.

[0477] A "user terminal" is an electronic device such as a computer, smartphone, or tablet that a user uses as an interface.

[0478] The "emotion engine" is a technology for recognizing and analyzing the user's emotional state from their facial expressions and voice.

[0479] The "means for filtering based on emotional state" is a function that selects the content of news articles or emails to be displayed based on the recognized emotional state of the user.

[0480] "Means for obtaining a video URL" refers to a function for obtaining viewing history or a URL specified by the user.

[0481] "Speech recognition technology" is a technology for analyzing voice data and converting it into text.

[0482] "Video analysis technology" is a technology that analyzes video data to identify important scenes and keywords.

[0483] The "means for summarizing a video based on extracted important parts" is a function that extracts important parts of a video and generates a digest video that can be viewed in a short amount of time.

[0484] This system collects information from three sources: news articles, emails, and videos, and efficiently summarizes and provides it to users.The system incorporates an emotion engine that recognizes the user's emotions, and can provide content that adapts to the user's psychological state.

[0485] Hardware and software used

[0486] server

[0487] The server has the following features:

[0488] Gathering news articles: Accessing news sites on the Internet, specifically using Python's BeautifulSoup library and Scrapy framework.

[0489] Natural Language Processing: Using natural language processing algorithms like BERT and GPT models to parse the body of news articles and emails.

[0490] Speech Recognition: Uses the Google® Cloud Speech-to-Text API and PyDub library to analyze the audio in the video.

[0491] Video Analysis: Use FFmpeg and video analysis techniques to identify important scenes in the video.

[0492] Emotion Recognition: Uses technologies such as OpenCV and Dlib, Google Cloud Speech-to-Text API to analyze the user's facial expressions and voice.

[0493] Terminal

[0494] Accessing the user's email account (e.g., Gmail or Outlook) to retrieve email.

[0495] The acquired data is sent to the server.

[0496] It has the function of displaying summary news, summary emails, and summary videos sent from the server.

[0497] User

[0498] Use your device to check news, emails, and video summaries.

[0499] You can request a video summary by entering your viewing history or a specified URL into your device.

[0500] Data processing and calculation methods

[0501] News article collection

[0502] The server periodically accesses news sites to retrieve the latest news articles, which are then filtered by category and stored in a database.

[0503] News article analysis

[0504] The server analyzes the collected article text using natural language processing algorithms, such as BERT and GPT models, to extract key points and keywords from the article.

[0505] News article summaries

[0506] Based on the extracted key points, the server summarizes the news article, consolidating the important information into a few lines and eliminating redundant parts.

[0507] Emotion Recognition and Filtering

[0508] The emotion engine recognizes the user's current emotional state. It analyzes input data from the camera and microphone through facial expression recognition APIs and voice emotion analysis APIs. Based on the recognized emotional state, it selects appropriate news articles and emails for the user.

[0509] Email capture and analysis

[0510] The device accesses the user's email account and sends a day's worth of received emails to a server, which then analyzes the email content using natural language processing algorithms to extract important information.

[0511] Email Summary

[0512] Based on the extracted important information, the server summarizes the email, concisely summarizing urgent matters and important events.

[0513] Video analysis and summarization

[0514] The device sends the specified URL to the server, which then transcribes the audio from the video using speech recognition technology and analyzes the video data to identify important parts, generating a digest video based on key scenes.

[0515] Examples and prompts

[0516] Specific news article examples

[0517] For example, if a user is in an emotional state where they want to relax after work, the server can recognize the user's current emotions and provide a summary of positive articles preferentially, thus allowing the user to obtain important information while reducing stress.

[0518] Example prompt: "I'm in a relaxed mood today, so please summarize a positive news article for me."

[0519] Specific examples of email

[0520] For example, if a user is checking email during busy morning hours, the server will recognize through its emotion engine that the user is feeling anxious or stressed. Based on this, the server will prioritize and summarize and display emails with high urgency.

[0521] Example prompt: "It's a busy morning, so please prioritize and summarize the most urgent emails."

[0522] Video examples

[0523] For example, if a user is tired and wants to relax, the server will recognize the user's current emotions and prioritize and summarize relaxing videos based on that emotion. The user can watch relaxing videos in a short amount of time.

[0524] Example prompt: "I want to relax, so please summarize the video at the specified URL and create a digest video."

[0525] The above is a specific embodiment for carrying out the present invention, which allows for the provision of information according to the emotional state of the user, thereby improving the user experience.

[0526] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0527] News article summarization and sentiment engine

[0528] Step 1: Gather news articles

[0529] Input: List of news site URLs

[0530] Processing: The server uses Python's BeautifulSoup library and Scrapy framework to access news sites and retrieve the latest news articles.

[0531] Output: News article data (title, text, URL, publication date)

[0532] Specific operation: The server accesses the news site, extracts article data from the HTML document, and stores it in a database.

[0533] Step 2: Analyzing the news article

[0534] Input: The text of a news article stored in a database

[0535] Processing: The server uses BERT or GPT models to analyze the text of the news article and extract important keywords and key points.

[0536] Output: Extracted keywords and key points

[0537] Specific operation: Apply NLP model to analyze the text and temporarily save the extracted keywords and main points.

[0538] Step 3: Summarize the news article

[0539] Input: Extracted keywords and key points

[0540] Processing: The server uses the extracted information to summarize the news article in a few lines.

[0541] Output: A summarized news article

[0542] What it does: It summarizes news articles, keeping only the key points and removing redundant parts, and stores the summaries in a database.

[0543] Step 4: Emotion Recognition and Filtering

[0544] Input: User facial expression and voice data from camera and microphone

[0545] Processing: The server analyzes the user's emotions using OpenCV, Dlib, and the Google Cloud Speech-to-Text API.

[0546] Output: Perceived emotional state

[0547] Specific operation: Analyzes data collected through the camera and microphone to recognize the user's emotional state in real time.

[0548] Step 5: Filtering news articles

[0549] Input: Recognized emotional state, summarized news article

[0550] Processing: The server selects suitable news articles based on the recognized emotional state.

[0551] Output: Filtered summary news article

[0552] What it does: Select news articles that users find most appealing based on their emotional state.

[0553] Step 6: Send and display news summaries

[0554] Input: Filtered summary news articles

[0555] Processing: The server sends the summary news to the user's terminal, and the terminal displays the summary news.

[0556] Output: A summary news article displayed on the user's terminal

[0557] Specific operation: The device that receives the news article from the server displays it on the screen.

[0558] Email summary and sentiment engine

[0559] Step 1: Get email

[0560] Input: User's email account information

[0561] Processing: The device accesses the mail server (e.g., Gmail, Outlook) and retrieves one day's worth of received emails.

[0562] Output: Received email data

[0563] Specific operation: The device logs into the email account and downloads all received emails.

[0564] Step 2: Parse the email

[0565] Input: Received email data

[0566] Processing: The device sends the received email data to the server, which uses NLP algorithms to analyze the email content and extract important information.

[0567] Output: Extracted keywords and important information

[0568] What it does: The server analyzes the body of the email and extracts urgent matters and important schedules.

[0569] Step 3: Email Summary

[0570] Input: Extracted keywords and important information

[0571] Processing: The server uses this information to summarize each email.

[0572] Output: Abridged email

[0573] What it does: Summarizes emails by condensing the most important information into a few lines and removing redundant content.

[0574] Step 4: Emotion recognition and priority display

[0575] Input: User facial expression and voice data from camera and microphone

[0576] Processing: The server recognizes the user's emotional state in real time and prioritizes emails based on the emotional state.

[0577] Output: Prioritized emails

[0578] Specific operation: The server sets priorities based on the content of the email and the results of emotion recognition.

[0579] Step 5: Send and view summary emails

[0580] Input: Prioritized summary email

[0581] Processing: The server sends the summary email to the user's terminal, and the terminal displays the summary email.

[0582] Output: Summary email displayed on user terminal

[0583] Specific operation: The device that receives the email from the server displays the contents of the email on the screen.

[0584] Video summary function and emotion engine

[0585] Step 1: Get the video URL

[0586] Input: Viewing history or specified URL

[0587] Processing: The user inputs their viewing history or a specified URL into their device.

[0588] Output: The URL entered

[0589] Specific behavior: The user enters a URL into an input form on the device.

[0590] Step 2: Analyze the video data

[0591] Input: Entered URL

[0592] Processing: The device sends the specified URL to the server. The server accesses the URL and retrieves the video data. The server transcribes the audio using the Google Cloud Speech-to-Text API and analyzes the video data using FFmpeg.

[0593] Output: Transcribed audio data, analyzed video data

[0594] What it does: The server downloads the video from the URL and processes the audio and video separately.

[0595] Step 3: Summarize the video

[0596] Input: Analyzed audio and video data

[0597] Processing: The server extracts important scenes and creates a digest video of less than 5 minutes.

[0598] Output: Digest video

[0599] Specific actions: Edit important scenes into a short video.

[0600] Step 4: Emotion recognition and content adjustment

[0601] Input: User facial expression and voice data from camera and microphone

[0602] Processing: The server uses an emotion engine to recognize the user's emotional state and adjust the content of the summarized digest video.

[0603] Output: Adjusted digest video

[0604] Specific operation: The server adjusts and optimizes the content of the video according to the user's emotional state.

[0605] Step 5: Send and display summary video

[0606] Input: Adjusted digest video

[0607] Processing: The server sends the summary video to the user's terminal, and the terminal plays the summary video.

[0608] Output: Summary video displayed on the user's device

[0609] Specific operation: The device that receives the video from the server plays it.

[0610] Through these steps, the user can efficiently obtain appropriate information according to their emotional state.

[0611] (Application example 2)

[0612] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0613] In today's world, a vast amount of information is available via the Internet, making it difficult for users to efficiently obtain the information they need. Furthermore, few systems exist that provide optimal information based on the user's emotional state, and there is a need for systems that can quickly provide useful information to users in specific situations. Particularly during busy times or high-stress situations, there is a need for systems that can prioritize information that helps users relax or that is of high urgency.

[0614] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0615] In this invention, the server includes means for collecting news articles, means for analyzing the text of the collected news articles and extracting key points, means for summarizing the news articles based on the key points, means for transmitting the summarized news to a user terminal, means for displaying the summarized news on the user terminal, means for recognizing the emotional state of the user, and means for filtering the news articles based on the recognized emotional state and providing the user with the most appropriate summarized news. This makes it possible to provide the user with the most appropriate news article information according to their emotional state and improve the user's information acquisition efficiency.

[0616] A "news article" is written information about current events that is published on the Internet.

[0617] "Email" means an electronic message sent or received over the Internet.

[0618] "Video" is a digital medium that includes video and audio to convey information visually and aurally.

[0619] "Gist" refers to the main content or important parts of a news article, email, or video.

[0620] "Emotional state" refers to the user's current psychological response or mood, as recognized using the emotion engine.

[0621] "Filtering" is the process of selecting and providing relevant information based on a user's emotional state.

[0622] "Summarizing" is the act of shortening the content of a news article, email, or video, leaving out only the main points and expressing them concisely.

[0623] "Terminal" refers to a device that allows a user to receive and view information, such as a smartphone, smart glasses, or a head-mounted display.

[0624] An "emotion engine" is a technology that analyzes emotions from a user's facial expressions, voice, etc., and recognizes their emotional state.

[0625] A "user" is a person who uses an information service to efficiently retrieve and consume news articles, emails, and videos.

[0626] The present invention combines a system that collects information from three types of information sources - news articles, emails, and videos - and efficiently summarizes and provides it to users with an emotion engine that recognizes the user's emotions.

[0627] The main components of this system will be described.

[0628] News article summarization and sentiment engine

[0629] News article collection

[0630] The server accesses various news sites on the Internet and automatically collects the latest news articles. The articles are classified into categories for filtering purposes.

[0631] News article analysis

[0632] The server analyzes the text of collected news articles using natural language processing (NLP) algorithms, such as BERT or GPT models (using Hugging Face's Transformers library), to extract important keywords and key points from the articles.

[0633] News article summaries

[0634] Based on the extracted keywords and key points, the server summarizes the news article in a few lines, eliminating redundant parts and leaving only the main points.

[0635] Emotion Recognition and Filtering

[0636] The emotion engine analyzes the user's facial expressions and voice to recognize their current emotional state, using OpenFace, DeepFace, or the Microsoft® Azure® Emotion API.

[0637] Based on the user's emotional state, the server filters out appropriate news articles, for example, if the user is feeling stressed, it prioritizes positive news.

[0638] Sending and displaying news summaries

[0639] The summarized news is sent from the server to the user's device, where the user can easily check the news articles according to their emotions.

[0640] Email summary and sentiment engine

[0641] Get email

[0642] The device accesses the user's email account (e.g., Gmail or Outlook) and retrieves all of the emails received for the day.

[0643] Email Analysis

[0644] The captured email data (body, subject, sender information, etc.) is sent to a server and analyzed using NLP algorithms.

[0645] Email Summary

[0646] Extract key information and keywords and summarize each email in a few lines.

[0647] Emotion recognition and priority display

[0648] The emotion engine recognizes the user's emotional state and prioritizes which emails to display based on that state. For example, if the user is feeling stressed, emails with a low level of urgency will be prioritized.

[0649] Sending and viewing summary emails

[0650] The server generates a list of summarized emails and sends it to the user's device, where the user can view emails sorted by importance.

[0651] Video summary function and emotion engine

[0652] Get the video URL

[0653] The user inputs their viewing history and a specified URL into the device.

[0654] Video data analysis

[0655] The terminal transmits the specified URL to the server, and the server accesses the URL to obtain the video data.

[0656] The audio from the captured video is transcribed using ASR technology, and the video data is analyzed to identify important parts.

[0657] Video Summary

[0658] Based on the important parts, a digest video of the video is generated that is less than 5 minutes long.

[0659] Emotion recognition and content regulation

[0660] The emotion engine recognizes the user's emotional state and adjusts the content of the summary video based on that state: if the user is tired, relaxing content will be prioritized.

[0661] Sending and displaying summary videos

[0662] The summarized video is sent from the server to the user's device, allowing the user to quickly check the main content on the device.

[0663] Specific examples

[0664] Specific news article examples

[0665] For example, if a user is in an emotional state where they want to relax after work, the server will recognize the user's current emotions and prioritize providing positive articles.

[0666] Examples of prompt statements

[0667] "What news articles should be suggested if the user's emotional state is perceived as tired?"

[0668] The present invention allows users to acquire information more efficiently and provides appropriate information according to their emotional state, thereby improving the user experience and reducing stress and promoting relaxation.

[0669] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0670] Step 1:

[0671] News article collection

[0672] The server accesses various news sites on the Internet, automatically collects the latest news articles, filters them by category, and extracts the necessary news articles.

[0673] Input: A list of URLs for news sites on the Internet

[0674] Data processing: Extracting news article text using scraping technology

[0675] Output: A list of collected news articles

[0676] Step 2:

[0677] News article analysis

[0678] The server analyzes the collected news articles using natural language processing (NLP) algorithms, such as BERT and GPT models, to extract important keywords and key points from the articles.

[0679] Input: A list of collected news articles

[0680] Data processing: Analysis using NLP algorithms

[0681] Output: A list of extracted keywords and key points

[0682] Step 3:

[0683] News article summaries

[0684] The server then summarises the news article into a few lines based on the extracted keywords and key points, eliminating redundant parts and leaving only the main points.

[0685] Input: A list of extracted keywords and key points

[0686] Data processing: Compression by summarization algorithms

[0687] Output: A summarized news article

[0688] Step 4:

[0689] Recognition of emotional states

[0690] The server uses an emotion engine to recognize the user's emotional state, analyzing facial expression data and voice data sent from the user's device to identify the user's emotional state.

[0691] Input: facial expression data, voice data

[0692] Data processing: Analysis using emotion recognition algorithms

[0693] Output: User's emotional state

[0694] Step 5:

[0695] News article filtering

[0696] The server filters news articles appropriate for the user based on the user's perceived emotional state, prioritizing positive news for users who are feeling stressed.

[0697] Input: Summarized news article, user's emotional state

[0698] Data processing: Selection by filtering algorithm

[0699] Output: A list of news articles relevant to the user

[0700] Step 6:

[0701] Sending news summaries

[0702] The server transmits the summarized news to the user's terminal.

[0703] Input: A list of news articles relevant to the user

[0704] Data processing: Data transmission protocol (HTTP / Sockets)

[0705] Output: News received on the user's device

[0706] Step 7:

[0707] Viewing news summaries

[0708] The user's device displays the received news summary, allowing the user to easily check news articles according to their emotions.

[0709] Input: A summarized news article

[0710] Data processing: Display processing on the user interface

[0711] Output: User views of news articles

[0712] Next, similar steps are taken for email and video summaries and the emotion engine, but the specific steps are similar to the process for news articles: they are analyzed and summarized specifically for each source, filtered according to the user's emotional state, and then displayed on the device.

[0713] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0714] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0715] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0716] [Second embodiment]

[0717] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0718] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0719] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0720] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0721] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0722] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0723] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0724] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0725] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0726] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0727] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0728] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0729] This invention is a system that efficiently summarizes information from three types of information sources: news, email, and video, and provides it to users. To implement this system, a server that runs a program with the following functions, a terminal, and user operations are required.

[0730] News article summary function

[0731] News article collection

[0732] The server has a mechanism to access various news sites on the Internet and automatically collect the latest news articles. For example, it filters by category, such as politics, economics, and sports, and extracts the necessary news articles.

[0733] News article analysis

[0734] The server analyzes the collected news articles using natural language processing (NLP) algorithms. Using the latest NLP techniques, such as BERT and GPT models, it extracts important keywords and key points from the articles. This analysis determines the context and importance of the text and identifies the information needed for summarization.

[0735] News article summaries

[0736] Based on the results of the analysis, the server summarizes the news article in a few lines, leaving out the main points and important details of the news and eliminating redundant parts.

[0737] Sending and displaying news summaries

[0738] The server generates a document summarizing the summarized news and sends it to the user's terminal.

[0739] Users can easily check the summarized news on their devices.

[0740] Email Summary Feature

[0741] Get email

[0742] The device accesses the user's email account (e.g., Gmail or Outlook) and retrieves all of the emails received for the day.

[0743] Email Analysis

[0744] The terminal sends the acquired email data (body, subject, sender information, etc.) to the server.

[0745] Based on the data sent, the server analyzes the content of each email using natural language processing (NLP) algorithms to extract important information and keywords.

[0746] Email Summary

[0747] The server extracts important keywords and key points from each email and then summarises them into a few lines, concisely summarising important meeting schedules or urgent requests.

[0748] Sending and viewing summary emails

[0749] The server generates a list of summarized emails and sends them to the user's terminal.

[0750] The user can easily check the summarized email on the terminal.

[0751] Video summary function

[0752] Get the video URL

[0753] The user inputs their viewing history and a specified URL into the device.

[0754] Video data analysis

[0755] The terminal transmits the specified URL to the server, and the server accesses the URL to obtain the video data.

[0756] The server converts the audio of the video into text using automatic speech recognition (ASR) technology, and then analyzes the video data to extract important scenes, for example, by identifying important parts based on audio containing specific keywords or scene changes.

[0757] Video Summary

[0758] The server generates a video digest based on the extracted key points, summarizing important announcements and interesting moments into short clips.

[0759] Sending and displaying summary videos

[0760] The server transmits the summarized video file to the user's terminal.

[0761] The user can view the digest video generated on the terminal and quickly understand the main content.

[0762] Specific examples

[0763] News article summary examples

[0764] For example, if the server retrieves a "political news article" from a nationally known news site, it will analyze the text for the title "Cabinet reshuffle" using the BERT model, extracting "new cabinet members," "policy changes," "key statements," etc., and generate a summary based on this.

[0765] Example of an email summary

[0766] For example, a device retrieves an email with the subject "Meeting Schedule" from a user's email account and sends the contents to the server. The server extracts information such as the meeting date and time, names of participants, and main agenda items from the body of the email and generates a summary.

[0767] Video summary examples

[0768] For example, if a user selects the URL of a "sports highlight video" from their viewing history, the server retrieves the corresponding video and performs voice recognition and video analysis. As a result, it extracts "important goal scenes" and "interview highlights," and generates a summary clip based on these.

[0769] In this way, the present invention realizes a system that efficiently summarizes news, email, and video information and provides it to users, allowing them to quickly and concisely grasp the information they need.

[0770] The processing flow will be explained below.

[0771] News article summary function

[0772] Step 1: Gather news articles

[0773] The server accesses the specified news site and collects the URLs of the latest news articles, sometimes using the news site's API.

[0774] Step 2: Get news articles

[0775] The server retrieves HTML data from the collected URLs and converts it into a format that is easy to analyze. It extracts the news text, title, date and time, etc.

[0776] Step 3: Analyzing the news article

[0777] The server uses natural language processing (NLP) algorithms to analyze the text of retrieved news articles, using models such as BERT and GPT to extract key keywords and key points from each article.

[0778] Step 4: Summarize the news article

[0779] The server then summarises the article based on the analysis results, combining multiple algorithms to generate an optimal summary that always includes important information, such as important dates, people's names, and events.

[0780] Step 5: Submit your news summary

[0781] The server generates a document summarizing the summarized news and sends it to the user's terminal.

[0782] Step 6: Viewing News Summary

[0783] Users can easily view summaries of news on their devices, and can filter news by category and importance.

[0784] Email Summary Feature

[0785] Step 1: Get email

[0786] The device accesses the user's email account and retrieves all of the emails received in one day. It uses RestAPI or similar to retrieve data from the email server.

[0787] Step 2: Send email data

[0788] The terminal sends the acquired email data (subject, body, sender information, etc.) to the server.

[0789] Step 3: Parse the email

[0790] The server uses NLP algorithms to extract obviously important information, such as meeting schedules, project progress, and requests that require urgent attention.

[0791] Step 4: Email Summary

[0792] The server then summarises each email into a few lines based on the extracted keywords and key points, concisely summarising the key points so that the content can be understood at a glance.

[0793] Step 5: Send a summary email

[0794] The server generates a list of summarized mails and sends it to the user's terminal.

[0795] Step 6: View the summary email

[0796] Users can view summaries of emails compiled on their device, and it also includes the ability to filter out unnecessary emails and display only the important ones.

[0797] Video summary function

[0798] Step 1: Get the video URL

[0799] The user inputs the viewing history or the URL of the specified video into the device.

[0800] Step 2: Sending video data

[0801] The terminal sends the specified URL to the server.

[0802] Step 3: Getting the video

[0803] The server accesses the URL and retrieves the video data. YouTube API or Tver API is often used.

[0804] Step 4: Audio-visual analysis of the video

[0805] The server transcribes the audio from the captured video using automatic speech recognition (ASR) technology and performs scene analysis of the video to identify important parts, such as audio containing specific keywords or scene changes.

[0806] Step 5: Summarize your video

[0807] Based on the extracted key points, the server generates a digest video that summarizes the important scenes of the video in less than five minutes.

[0808] Step 6: Submit your summary video

[0809] The server transmits the digested video to the user terminal.

[0810] Step 7: Displaying the summary video

[0811] Users can play the digest video generated on their device and quickly check the main content.

[0812] The above processing steps realize a system that efficiently summarizes news, email, and video information, allowing users to quickly obtain the information they need.

[0813] Example 1

[0814] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0815] In modern society, as the amount of information increases, it is becoming increasingly difficult to efficiently grasp important information. In particular, there is a lack of methods to quickly retrieve, summarize, and present necessary information from various sources, such as news articles, emails, and videos. This forces users to manually find important parts from a vast amount of information, which is a time-consuming and labor-intensive process.

[0816] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0817] In this invention, the server includes a means for collecting news information, a means for analyzing the collected news information using a natural language processing algorithm to extract key points, and a means for summarizing the news information in a few lines based on the key points. This enables summarization of news articles. The server also includes a means for acquiring electronic communications, a means for analyzing the content of the acquired electronic communications to extract important information, and a means for summarizing the electronic communications in a few lines based on the important information. This enables summarization of emails. The server also includes a means for acquiring URLs of video information, a means for analyzing the audio and video of the acquired video information to extract important parts, and a means for summarizing the video information into short clips based on the extracted important parts. This enables summarization of videos. These functions allow users to efficiently acquire important information from different information sources and grasp it in a short amount of time.

[0818] "News information" refers to the content and articles of news sites published on the Internet.

[0819] "Natural language processing algorithms" refers to technologies that analyze text data and evaluate context and importance. Specifically, this includes models such as BERT and GPT.

[0820] "Key points" refer to important information or keywords found in news information, electronic communications, and video information.

[0821] "Summarizing" means compressing long pieces of information into a concise summary of only the main points and important information.

[0822] "Electronic communications" refers to communications such as emails exchanged over the Internet.

[0823] "Speech recognition technology" refers to technology that converts voice data into text data. Specifically, it includes ASR (automatic speech recognition) technology.

[0824] "Video information" refers to the content and images of videos that can be viewed on the Internet.

[0825] A "URL" is an address used to specify a specific resource on the Internet.

[0826] "Analyze" refers to analyzing data or information using algorithms to find specific patterns or important elements.

[0827] "User terminal" refers to a device used by a user to receive and display information, including, but not limited to, a smartphone, tablet, or computer.

[0828] "Document" refers to a digital document, such as a text file or PDF, that compiles summarized information.

[0829] "Generate" refers to creating data or information from scratch using a computer.

[0830] "Transmitting" refers to sending data or information from one location to another over a network.

[0831] "Display" refers to visually presenting data or information so that it can be read by a user.

[0832] A "short clip" refers to a short piece of video created by cutting out important parts from a longer video.

[0833] This invention is a system that collects information from three types of information sources: news information, electronic communications, and video information, and efficiently summarizes each of them and provides them to users. To implement this system, a server that runs a program with the following functions, a terminal, and user operations are required.

[0834] News article summary function

[0835] Gathering news information

[0836] The server sets a Cron job to access a news site, for example, every day at 9:00 a.m., to collect the latest news information. The server uses the news site's API to extract the news title and text data of the news body.

[0837] News information analysis

[0838] The server analyzes the collected news information using natural language processing algorithms. Specifically, it uses generative AI models such as BERT and GPT to tokenize the text data, evaluate its context and importance, extract important keywords and key points, and store the results.

[0839] News summary

[0840] The server summarizes the news information in a few lines based on the extracted keywords and key points, concisely retaining only the main points and eliminating redundant parts.

[0841] Sending and displaying summary news information

[0842] The server generates a document containing the summarized news information and sends it to the user's terminal, for example, in PDF or HTML format.

[0843] To enable a user to check summarized news information on a terminal and efficiently grasp the latest news.

[0844] Summary function of electronic communications

[0845] Obtaining Electronic Communications

[0846] The device accesses the user's email account using the IMAP protocol and retrieves electronic communications for a specified period of time. The connection is made using a secure authentication protocol (such as OAuth).

[0847] Analysis of electronic communications

[0848] The device sends captured electronic communications to a server, which then uses generative AI models such as BERT and GPT to analyze the email data and extract key information and keywords.

[0849] Electronic Communications Summary

[0850] The server summarizes electronic communications in a few lines based on extracted keywords and key points, concisely summarizing important meeting schedules and information requiring urgent action.

[0851] Summary of electronic communication transmission and display

[0852] The server generates a document from the summarized electronic communication and transmits it to the user's terminal.

[0853] The user can view summarized electronic communications on a terminal and quickly grasp important information.

[0854] Video summary function

[0855] Get the video URL

[0856] The user enters their viewing history and a specified URL into the device, and the device then sends the URL to the server.

[0857] Video information analysis

[0858] The server accesses the specified URL and retrieves the video data. The server then downloads the correct data using the YouTube API or similar.

[0859] The server uses automatic speech recognition (ASR) technology to convert the audio in the video into text and simultaneously analyzes the video data, for example, to detect lines containing important keywords or scene changes.

[0860] Video information summary

[0861] The server generates a video digest based on the analysis results, compiling important announcements and interesting moments into short clips.

[0862] Sending and displaying summary video information

[0863] The server sends the summarized video file to the user's device, where it can be streamed using a dedicated app.

[0864] Users can watch the summary video on their device and quickly grasp the important content.

[0865] Specific examples

[0866] Examples of news summaries

[0867] For example, the server retrieves "political news information" from a domestic news site and analyzes it using the BERT model. The analysis extracts "new cabinet members," "policy changes," "major statements," etc., and generates a summary based on this.

[0868] Examples of Electronic Communications Summaries

[0869] For example, a terminal retrieves an electronic message with the subject "Meeting Schedule" from a user's email account and sends it to a server. The server extracts information such as the meeting date and time, participant names, and main agenda items from the message body and generates a summary.

[0870] Example of video summary

[0871] For example, if a user selects the URL of a "sports highlight video" from their viewing history, the server retrieves and analyzes the corresponding video. As a result of voice recognition and video analysis, it extracts "goal scenes" and "interview highlights," and generates a summary clip based on these.

[0872] Prompt Sentence Examples

[0873] "Please summarize the latest political news article."

[0874] "Please tell me the highlights of the email I received today."

[0875] "Summarize the key scenes in this sports video."

[0876] In this way, the present invention realizes a system that efficiently summarizes news information, electronic communications, and video information and provides them to users, allowing them to quickly and concisely grasp the information they need.

[0877] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0878] News article summary function

[0879] Step 1: Gather news information

[0880] The server sets up a Cron job to access a news site on the Internet, for example, every day at 9:00 AM.

[0881] Input: List of news site URLs

[0882] The server accesses each URL and uses the news site's API to obtain the latest news information.

[0883] Data processing: Parse HTML to extract news titles and text data.

[0884] Output: A list of news articles in text format

[0885] Step 2: Analyzing the news information

[0886] The server analyzes the news information collected in the previous step using a natural language processing algorithm.

[0887] Input: News article list

[0888] The server uses generative AI models such as BERT or GPT to tokenize the text data and evaluate its context and importance.

[0889] Data calculation: Extract important keywords and key points.

[0890] Output: A dataset showing key keywords and key points

[0891] Step 3: Summarize the news information

[0892] The server summarizes the news information in a few lines based on the extracted keywords and key points.

[0893] Input: A dataset showing important keywords and key points

[0894] Data processing: Keep only the main points and remove redundant parts.

[0895] Output: Summary

[0896] Step 4: Send and display summary news information

[0897] The server generates a document containing summarized news information and transmits it to the user's terminal.

[0898] Input: Abstract

[0899] Data processing: Generate documents in PDF or HTML format.

[0900] Output: A summary of the news document

[0901] The user checks the summarized news information on the terminal.

[0902] Summary function of electronic communications

[0903] Step 1: Obtaining Electronic Communications

[0904] The device accesses the user's email account using the IMAP protocol.

[0905] Enter your email account information

[0906] The device captures electronic communications for a specified period of time.

[0907] Data processing: Read data from received emails.

[0908] Output: Electronic communication data

[0909] Step 2: Analyzing Electronic Communications

[0910] The terminal transmits the captured electronic communication to a server.

[0911] Input: Electronic communication data

[0912] The server analyzes the data using a generative AI model such as BERT or GPT.

[0913] Data calculations: Extracting important information and keywords.

[0914] Output: Data showing important information and keywords

[0915] Step 3: Summarizing Electronic Communications

[0916] The server summarizes the electronic communication in a few lines based on the extracted keywords and key points.

[0917] Input: Data that indicates important information or keywords

[0918] Data processing: Concisely summarize important meeting schedules and information that requires urgent action.

[0919] Output: Summary

[0920] Step 4: Sending and displaying summary electronic communications

[0921] The server generates a document from the summarized electronic communication and transmits it to the user's terminal.

[0922] Input: Abstract

[0923] Data processing: Generate documents in PDF or HTML format.

[0924] Output: A summary of the electronic communication

[0925] The user reviews the summarized electronic communication at the terminal.

[0926] Video summary function

[0927] Step 1: Get the video URL

[0928] The user enters their viewing history and a specified URL into the device, and the device then sends the URL to the server.

[0929] Input: Video URL

[0930] Output: URL data

[0931] Step 2: Analyze video information

[0932] The server accesses the specified URL and acquires the video data.

[0933] Input: URL data

[0934] The server downloads the video data using the YouTube API or similar.

[0935] Data processing: Uses automatic speech recognition (ASR) technology to convert voice into text and also analyzes video data.

[0936] Output: Analyzed text data and video data

[0937] Step 3: Summary of video information

[0938] The server generates a video digest based on the analysis results.

[0939] Input: Analyzed text data and video data

[0940] Data processing: Summarize important scenes and quotes into short clips.

[0941] Output: Summary clip

[0942] Step 4: Send and display summary video information

[0943] The server sends the summarized video file to the user's terminal.

[0944] Input: Summary clip

[0945] Data processing: Streaming playback becomes possible using a dedicated app.

[0946] Output: Summarized video file

[0947] Users can watch the summary video on their device and quickly grasp the important content.

[0948] (Application example 1)

[0949] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0950] In today's information-saturated world, users need to quickly and efficiently understand the vast amount of news articles, emails, and video information. It is particularly difficult to extract and provide users with only the most important information from each source. Furthermore, there is a lack of a way to easily summarize this information and provide it to users in an easily accessible format.

[0951] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0952] In this invention, the server includes means for collecting news articles, means for analyzing the text of the collected news articles and extracting key points, means for summarizing the news articles based on the key points, means for sending the summarized news to a user terminal, means for displaying the summarized news on the user terminal, and means for optimizing the summaries using a generative AI model and prompt sentences, thereby enabling users to quickly grasp only the important key points from a large amount of information.

[0953] A "news article" is a report or information about a current event or topic provided through online or offline media.

[0954] "Email" is a digital message sent or received over the Internet or other network.

[0955] A "video" is a multimedia file that combines audio and video and is played continuously.

[0956] A summary is a short summary of the important elements or key points of the original information.

[0957] A "user terminal" is a device used by a user, including a smartphone, tablet, or PC.

[0958] A "generative AI model" is an artificial intelligence model that uses machine learning algorithms to generate text and data.

[0959] A "prompt" is text input to a generative AI model that takes the form of an instruction or question to the model.

[0960] A "collection method" is a method or system for obtaining data or information from a particular source.

[0961] An "analysis tool" is a method or system for processing collected data or information and extracting meaningful elements.

[0962] A "transmission means" is a method or system for sending data or information from one system to another.

[0963] A "display means" is a method or system for visually presenting information to a user, and includes devices such as a screen.

[0964] This invention is an information summarization system for efficiently providing various information to users. This system summarizes information from three types of information sources: news, email, and video, and provides it to users concisely. To implement this, a server, a terminal, and user operations are required.

[0965] News article summary function

[0966] News article collection

[0967] The server accesses various news sites to collect the latest news articles, and then filters the articles by category, such as politics, economics, or sports, to extract the necessary news articles.

[0968] News article analysis

[0969] The server analyzes the collected news articles using natural language processing (NLP) algorithms, such as the BERT model, to extract the article's context and important keywords.

[0970] News article summaries

[0971] The server then summarises the news article based on the analysis, keeping the key points and keywords and removing redundant parts.

[0972] Sending and displaying news summaries

[0973] The server sends the summarized news to the user's terminal, where the user can check the concisely summarized news.

[0974] Email Summary Feature

[0975] Get email

[0976] The terminal accesses the user's email account and retrieves received emails.

[0977] Email Analysis

[0978] The device sends the email data to the server, which then uses NLP algorithms to analyze the email body and extract important information and keywords.

[0979] Email Summary

[0980] The server then uses this extracted data to summarize the email, concisely summarizing important schedules and information requiring urgent action.

[0981] Sending and viewing summary emails

[0982] The server sends the summarized email to the user's terminal, where the user can check the summarized email.

[0983] Video summary function

[0984] Get the video URL

[0985] The user inputs their viewing history and the specified URL into the device.

[0986] Video data analysis

[0987] The device sends the URL to the server, which retrieves the corresponding video. The audio data is converted into text using automatic speech recognition (ASR), and important scenes are extracted from the video data.

[0988] Video Summary

[0989] Based on the analysis data, the server generates a summary clip of the video containing important scenes and keywords.

[0990] Sending and displaying summary videos

[0991] The server sends the summarized video to the user's device, where the user can watch the summarized video in a short time.

[0992] Hardware and Software Used

[0993] The system's main hardware consists of a server and user devices (smartphones, tablets, and PCs). The software used includes Python, the BERT model, the Transformers library, the requests library, and speech recognition technology.

[0994] Examples and prompts

[0995] As a concrete example, consider a case where a user checks a news article summary on their smartphone. For example, the following prompt sentence is input to the generative AI model:

[0996] "Summarize the article: A new cabinet has been announced, with key changes."

[0997] An example output is:

[0998] "A new cabinet has been announced, with key members being changed."

[0999] Users can view this output on their smartphones.

[1000] This system allows users to quickly grasp only the important points from a large amount of information.

[1001] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1002] Step 1:

[1003] Collect news articles (server)

[1004] The server accesses various news sites and automatically collects the latest news articles. The input is the URL of the news site, and the output is the collected news article data. The server filters these articles by category and saves the collected data.

[1005] Step 2:

[1006] News article analysis (server)

[1007] The server analyzes the collected news article text using a natural language processing (NLP) algorithm. Specifically, it uses the BERT model to extract the article's context and important keywords. The input is the collected news article data, and the output is the analysis results: important keywords and key points.

[1008] Step 3:

[1009] News article summaries (server)

[1010] The server summarizes the news article based on the keywords and key points extracted in the previous step, eliminating redundant parts and picking out only the important information. The input is the analysis result, and the output is the summarized news article text.

[1011] Step 4:

[1012] Sending summary news (server)

[1013] The server sends the summarized news article to the user's terminal, where the input is the summarized news article text and the output is the summarized news sent to the user's terminal.

[1014] Step 5:

[1015] Displaying news summaries (terminal)

[1016] The user terminal displays the received summary news. The input is the summary news sent from the server, and the output is the news content displayed on the user interface.

[1017] Step 6:

[1018] Retrieving email (terminal)

[1019] The terminal accesses the user's email account and retrieves all received emails for one day. The input is the email account authentication information, and the output is the retrieved email data.

[1020] Step 7:

[1021] Email analysis (terminal)

[1022] The device sends the acquired email data to the server, which then uses NLP algorithms to analyze the content. The input is the email data, and the output is the extraction of important information and keywords.

[1023] Step 8:

[1024] Email Abstract (Server)

[1025] The server summarizes each email in a few lines based on the extracted important keywords and information. The input is the analysis result, and the output is the summarized email text.

[1026] Step 9:

[1027] Sending summary emails (server)

[1028] The server sends a list of summarized emails to the user terminal, where the input is the summarized email text and the output is the summarized email sent to the user terminal.

[1029] Step 10:

[1030] Display summary email (terminal)

[1031] The user terminal displays the received summary email. The input is the summary email sent from the server, and the output is the email content displayed on the user interface.

[1032] Step 11:

[1033] Get the video URL (device)

[1034] The user inputs their viewing history and the URL of the video they want to view into the device. The input is the URL of the video they specified, and the output is that URL information.

[1035] Step 12:

[1036] Video data analysis (server)

[1037] The device sends the URL of the specified video to the server. The server accesses the URL and retrieves the video data. The audio data is converted into text using speech recognition technology, and important scenes are extracted from the video data. The input is the video URL, and the output is the speech recognition results and the extraction of important scenes.

[1038] Step 13:

[1039] Video summary (server)

[1040] The server generates a video digest clip based on the extracted keypoints. The input is the speech recognition result and the extraction of important scenes, and the output is a summarized video clip.

[1041] Step 14:

[1042] Sending summary video (server)

[1043] The server sends the summarized video file to the user's terminal, where the input is the summarized video clip and the output is the summarized video file sent to the user's terminal.

[1044] Step 15:

[1045] Display summary video (device)

[1046] The user terminal plays the received summary video. The input is the summary video file sent from the server, and the output is the summary video that is played.

[1047] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1048] This invention combines a system that collects information from three types of information sources (news, email, and video), efficiently summarizes it, and provides it to users with an emotion engine that recognizes user emotions. Implementing this system requires a server that runs a program with the following functions, a terminal, and user operation.

[1049] News article summarization and sentiment engine

[1050] News article collection

[1051] The server has a mechanism to access various news sites on the Internet and automatically collect the latest news articles. For example, it filters by category, such as politics, economics, and sports, and extracts the necessary news articles.

[1052] News article analysis

[1053] The server analyzes the collected news article text using natural language processing (NLP) algorithms, using the latest NLP techniques such as BERT and GPT models to extract important keywords and key points from the articles.

[1054] News article summaries

[1055] Based on the results of the analysis, the server summarizes the news article in a few lines, leaving out the main points and important details of the news and eliminating redundant parts.

[1056] Emotion Recognition and Filtering

[1057] The server uses an emotion engine to recognize the user's current emotional state, for example, by analyzing the user's facial expressions and voice through a camera.

[1058] The server filters news articles appropriate for the user based on the perceived emotional state: for example, if the user is feeling stressed, it prioritizes positive news articles.

[1059] Sending and displaying news summaries

[1060] The server generates a document summarizing the summarized news and sends it to the user's terminal.

[1061] Users can easily check filtered news summaries on their devices.

[1062] Email summary and sentiment engine

[1063] Get email

[1064] The device accesses the user's email account (e.g., Gmail or Outlook) and retrieves all of the emails received for the day.

[1065] Email Analysis

[1066] The terminal sends the acquired email data (body, subject, sender information, etc.) to the server.

[1067] Based on the data sent, the server analyzes the content of each email using natural language processing (NLP) algorithms to extract important information and keywords.

[1068] Email Summary

[1069] The server extracts important keywords and key points from each email and then summarises them into a few lines, concisely summarising important meeting schedules or urgent requests.

[1070] Emotion recognition and priority display

[1071] The server uses an emotion engine to recognize the user's current emotional state.

[1072] The server prioritizes which emails to display based on the user's emotional state, for example, displaying less urgent emails to a stressed user first.

[1073] Sending and viewing summary emails

[1074] The server generates a list of summarized emails and sends them to the user's terminal.

[1075] Users can view emails on their devices summarized in order of importance according to their emotions.

[1076] Video summary function and emotion engine

[1077] Get the video URL

[1078] The user inputs their viewing history or a specified URL into the device.

[1079] Video data analysis

[1080] The terminal transmits the specified URL to the server, and the server accesses the URL to obtain the video data.

[1081] The server transcribes the audio from the captured video using automatic speech recognition (ASR) technology and analyzes the video data to identify important parts, for example, based on audio containing specific keywords or scene changes.

[1082] Video Summary

[1083] Based on the extracted key points, the server generates a digest video that summarizes the important scenes of the video in less than five minutes.

[1084] Emotion recognition and content regulation

[1085] The server uses an emotion engine to recognize the user's emotional state.

[1086] The server automatically selects and tailors the summary video to suit the user based on the recognized emotional state, for example prioritizing relaxing content if the user is tired.

[1087] Sending and displaying summary videos

[1088] The server transmits the digested video to the user's terminal.

[1089] Users can play the digest video generated on their device and quickly check the main content.

[1090] Specific examples

[1091] Specific news article examples

[1092] For example, if a user is in an emotional state where they want to relax after work, the server will recognize the user's current emotions and provide a summary of positive articles preferentially, allowing the user to obtain important information while reducing stress.

[1093] Specific examples of email

[1094] For example, if a user is checking email during busy morning hours, the server will recognize through its emotion engine that the user is feeling anxious or stressed. Based on this, the server will prioritize and summarize and display emails with high urgency.

[1095] Video examples

[1096] For example, if a user is tired and wants to relax, the server will recognize the user's current emotions and prioritize and summarize relaxing videos based on that emotion. The user can watch relaxing videos in a short amount of time.

[1097] In this way, by combining emotion engines, it becomes possible to provide information that is more in line with the user's needs, improving the user experience.

[1098] The processing flow will be explained below.

[1099] News article summarization and sentiment engine

[1100] Step 1: Gather news articles

[1101] The server accesses various news sites on the Internet and collects the URLs of the latest news articles, for example, by obtaining the necessary data from RSS feeds or APIs.

[1102] Step 2: Get news articles

[1103] The server retrieves the HTML data from the collected URLs and extracts the body of the news article, the title, and the date and time.

[1104] Step 3: Analyzing the news article

[1105] The server uses natural language processing (NLP) algorithms to analyze the text of the news article, extracting key keywords and gist information and identifying important information.

[1106] Step 4: Summarize the news article

[1107] The server then summarises the news article in a few lines based on the extracted key points and keywords, concisely summarising the main points.

[1108] Step 5: Emotion Recognition

[1109] The device captures the user's facial expressions and voice through a camera and microphone, and analyzes them with an emotion engine. For example, emotions are identified from the user's facial expressions and tone of voice.

[1110] Step 6: Filtering

[1111] The server then filters appropriate news articles based on the results of the emotion engine according to the user's emotional state. For example, if the user feels like relaxing, it prioritizes positive news.

[1112] Step 7: Submit your news summary

[1113] The server generates a document summarizing the filtered news articles and sends it to the user's terminal.

[1114] Step 8: Viewing Summary News

[1115] Users can easily check filtered news summaries on their devices.

[1116] Email summary and sentiment engine

[1117] Step 1: Get email

[1118] The device accesses the user's email account and retrieves all of the emails received in one day, using an email protocol (e.g., IMAP or POP3).

[1119] Step 2: Send email data

[1120] The terminal sends the acquired email data (body, subject, sender information, etc.) to the server.

[1121] Step 3: Parse the email

[1122] The server analyzes the email content using natural language processing (NLP) algorithms to extract important information and keywords, such as meeting schedules or urgent requests.

[1123] Step 4: Email Summary

[1124] The server then summarises each email in a few lines based on the extracted key keywords and information, concisely summarising the key points.

[1125] Step 5: Emotion Recognition

[1126] The device captures the user's facial expressions and voice through a camera and microphone, and analyzes them with an emotion engine. For example, emotions are identified from the user's facial expressions and tone of voice.

[1127] Step 6: Prioritize Display

[1128] The server then prioritizes which emails to display based on the results of the emotion engine and the user's emotional state. For example, if the user is feeling stressed, emails with a low level of urgency will be displayed first.

[1129] Step 7: Send a summary email

[1130] The server generates a prioritized summary mail list based on the emotional state and transmits it to the user's terminal.

[1131] Step 8: View the summary email

[1132] The user can check the emails summarized according to priority on the terminal.

[1133] Video summary function and emotion engine

[1134] Step 1: Get the video URL

[1135] The user inputs the viewing history or the URL of the specified video into the device.

[1136] Step 2: Sending video data

[1137] The terminal sends the specified URL to the server.

[1138] Step 3: Getting the video

[1139] The server accesses the transmitted URL and acquires the video data.

[1140] Step 4: Audio-visual analysis of the video

[1141] The server transcribes the audio from the captured video using automatic speech recognition (ASR) technology and analyzes the video data to identify important parts, such as audio containing key keywords and scene changes.

[1142] Step 5: Summarize your video

[1143] The server generates a video digest based on the extracted key points, summarizing important scenes into short clips.

[1144] Step 6: Emotion Recognition

[1145] The device captures the user's facial expressions and voice through a camera and microphone, and analyzes them with an emotion engine. For example, emotions are identified from the user's facial expressions and tone of voice.

[1146] Step 7: Adjust the content

[1147] Based on the results of the emotion engine, the server selects a summary video that best suits the user's emotional state and adjusts the content accordingly. For example, if the user wants to relax, it will prioritize videos with relaxing content.

[1148] Step 8: Submit your summary video

[1149] The server transmits the summarized digest video to the user's terminal.

[1150] Step 9: Displaying the Summary Video

[1151] Users can play the digest video generated on their device and quickly check the main content.

[1152] The above processing steps realize a system that efficiently summarizes news, email, and video information, and further adjusts the content provided based on the user's emotional state.

[1153] Example 2

[1154] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1155] Conventional summarization systems for news articles, emails, and videos provide information uniformly without considering the user's emotional state, making it difficult to provide content that adapts to the user's psychological state. Furthermore, efficient summarization of large amounts of information and appropriate provision to the user requires advanced natural language processing and emotion recognition technologies. Consequently, to improve the user experience, it is necessary to adjust the priority and content of information according to the user's emotional state.

[1156] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1157] In this invention, the server includes means for collecting news articles, means for analyzing the text of the collected news articles using a natural language processing algorithm and extracting key points, means for summarizing the news articles based on the key points, means for filtering the news articles based on the emotional state using an emotion engine that recognizes the emotional state of the user, means for transmitting the summarized news to a user terminal, and means for displaying the summarized news on the user terminal, thereby making it possible to provide news articles that are adapted to the emotional state of the user.

[1158] The server also includes means for acquiring emails, means for analyzing the contents of the acquired emails using a natural language processing algorithm and extracting important information, means for summarizing the emails based on the important information, means for determining the priority of the emails based on the emotional state having an emotion engine that recognizes the emotional state of the user, means for sending the summarized emails to the user terminal, and means for displaying the summarized emails on the user terminal, thereby enabling the importance of emails to be determined and provided in accordance with the emotional state of the user.

[1159] The server further includes a means for acquiring a URL of the video, a means for analyzing the audio and video of the acquired video using voice recognition technology and video analysis technology and extracting important parts, a means for summarizing the video based on the extracted important parts, a means having an emotion engine that recognizes the emotional state of the user and adjusting the content of the video based on the emotional state, a means for transmitting the summarized video to the user terminal, and a means for displaying the summarized video on the user terminal, thereby making it possible to provide video content adapted to the emotional state of the user.

[1160] These measures enable the provision of information according to the user's emotional state, improving the user experience.

[1161] The "means for collecting news articles" is a function that allows the server to access news sites on the Internet and automatically obtain the latest news articles.

[1162] "Natural language processing algorithms" are technologies for analyzing text data and understanding the meaning and structure of language, and include BERT and GPT models.

[1163] "Key point extraction" is a function that identifies and extracts important keywords and key information from the body of a news article or email.

[1164] The "means for summarizing news articles" is a function that summarizes the text of a news article in a concise format based on the extracted important information.

[1165] "Means for retrieving email" refers to a function that accesses a user's email account (e.g., Gmail or Outlook) and retrieves all received emails.

[1166] The "means for determining the priority of e-mails" is a function for determining the display order of received e-mails based on the emotional state of the user.

[1167] The "means for transmitting summarized e-mail to a user terminal" is a function for transmitting the contents of the summarized e-mail to a user terminal.

[1168] A "user terminal" is an electronic device such as a computer, smartphone, or tablet that a user uses as an interface.

[1169] The "emotion engine" is a technology for recognizing and analyzing the user's emotional state from their facial expressions and voice.

[1170] The "means for filtering based on emotional state" is a function that selects the content of news articles or emails to be displayed based on the recognized emotional state of the user.

[1171] "Means for obtaining a video URL" refers to a function for obtaining viewing history or a URL specified by the user.

[1172] "Speech recognition technology" is a technology for analyzing voice data and converting it into text.

[1173] "Video analysis technology" is a technology that analyzes video data to identify important scenes and keywords.

[1174] The "means for summarizing a video based on extracted important parts" is a function that extracts important parts of a video and generates a digest video that can be viewed in a short amount of time.

[1175] This system collects information from three sources: news articles, emails, and videos, and efficiently summarizes and provides it to users.The system incorporates an emotion engine that recognizes the user's emotions, and can provide content that adapts to the user's psychological state.

[1176] Hardware and software used

[1177] server

[1178] The server has the following features:

[1179] Gathering news articles: Accessing news sites on the Internet, specifically using Python's BeautifulSoup library and Scrapy framework.

[1180] Natural Language Processing: Using natural language processing algorithms like BERT and GPT models to parse the body of news articles and emails.

[1181] Speech Recognition: Use the Google Cloud Speech-to-Text API and PyDub library to analyze the audio in the video.

[1182] Video Analysis: Use FFmpeg and video analysis techniques to identify important scenes in the video.

[1183] Emotion Recognition: Uses technologies such as OpenCV and Dlib, Google Cloud Speech-to-Text API to analyze the user's facial expressions and voice.

[1184] Terminal

[1185] Accessing the user's email account (e.g., Gmail or Outlook) to retrieve email.

[1186] The acquired data is sent to the server.

[1187] It has the function of displaying summary news, summary emails, and summary videos sent from the server.

[1188] User

[1189] Use your device to check news, emails, and video summaries.

[1190] You can request a video summary by entering your viewing history or a specified URL into your device.

[1191] Data processing and calculation methods

[1192] News article collection

[1193] The server periodically accesses news sites to retrieve the latest news articles, which are then filtered by category and stored in a database.

[1194] News article analysis

[1195] The server analyzes the collected article text using natural language processing algorithms, such as BERT and GPT models, to extract key points and keywords from the article.

[1196] News article summaries

[1197] Based on the extracted key points, the server summarizes the news article, consolidating the important information into a few lines and eliminating redundant parts.

[1198] Emotion Recognition and Filtering

[1199] The emotion engine recognizes the user's current emotional state. It analyzes input data from the camera and microphone through facial expression recognition APIs and voice emotion analysis APIs. Based on the recognized emotional state, it selects appropriate news articles and emails for the user.

[1200] Email capture and analysis

[1201] The device accesses the user's email account and sends a day's worth of received emails to a server, which then analyzes the email content using natural language processing algorithms to extract important information.

[1202] Email Summary

[1203] Based on the extracted important information, the server summarizes the email, concisely summarizing urgent matters and important events.

[1204] Video analysis and summarization

[1205] The device sends the specified URL to the server, which then transcribes the audio from the video using speech recognition technology and analyzes the video data to identify important parts, generating a digest video based on key scenes.

[1206] Examples and prompts

[1207] Specific news article examples

[1208] For example, if a user is in an emotional state where they want to relax after work, the server can recognize the user's current emotions and provide a summary of positive articles preferentially, thus allowing the user to obtain important information while reducing stress.

[1209] Example prompt: "I'm in a relaxed mood today, so please summarize a positive news article for me."

[1210] Specific examples of email

[1211] For example, if a user is checking email during busy morning hours, the server will recognize through its emotion engine that the user is feeling anxious or stressed. Based on this, the server will prioritize and summarize and display emails with high urgency.

[1212] Example prompt: "It's a busy morning, so please prioritize and summarize the most urgent emails."

[1213] Video examples

[1214] For example, if a user is tired and wants to relax, the server will recognize the user's current emotions and prioritize and summarize relaxing videos based on that emotion. The user can watch relaxing videos in a short amount of time.

[1215] Example prompt: "I want to relax, so please summarize the video at the specified URL and create a digest video."

[1216] The above is a specific embodiment for carrying out the present invention, which allows for the provision of information according to the emotional state of the user, thereby improving the user experience.

[1217] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1218] News article summarization and sentiment engine

[1219] Step 1: Gather news articles

[1220] Input: List of news site URLs

[1221] Processing: The server uses Python's BeautifulSoup library and Scrapy framework to access news sites and retrieve the latest news articles.

[1222] Output: News article data (title, text, URL, publication date)

[1223] Specific operation: The server accesses the news site, extracts article data from the HTML document, and stores it in a database.

[1224] Step 2: Analyzing the news article

[1225] Input: The text of a news article stored in a database

[1226] Processing: The server uses BERT or GPT models to analyze the text of the news article and extract important keywords and key points.

[1227] Output: Extracted keywords and key points

[1228] Specific operation: Apply NLP model to analyze the text and temporarily save the extracted keywords and main points.

[1229] Step 3: Summarize the news article

[1230] Input: Extracted keywords and key points

[1231] Processing: The server uses the extracted information to summarize the news article in a few lines.

[1232] Output: A summarized news article

[1233] What it does: It summarizes news articles, keeping only the key points and removing redundant parts, and stores the summaries in a database.

[1234] Step 4: Emotion Recognition and Filtering

[1235] Input: User facial expression and voice data from camera and microphone

[1236] Processing: The server analyzes the user's emotions using OpenCV, Dlib, and the Google Cloud Speech-to-Text API.

[1237] Output: Perceived emotional state

[1238] Specific operation: Analyzes data collected through the camera and microphone to recognize the user's emotional state in real time.

[1239] Step 5: Filtering news articles

[1240] Input: Recognized emotional state, summarized news article

[1241] Processing: The server selects suitable news articles based on the recognized emotional state.

[1242] Output: Filtered summary news article

[1243] What it does: Select news articles that users find most appealing based on their emotional state.

[1244] Step 6: Send and display news summaries

[1245] Input: Filtered summary news articles

[1246] Processing: The server sends the summary news to the user's terminal, and the terminal displays the summary news.

[1247] Output: A summary news article displayed on the user's terminal

[1248] Specific operation: The device that receives the news article from the server displays it on the screen.

[1249] Email summary and sentiment engine

[1250] Step 1: Get email

[1251] Input: User's email account information

[1252] Processing: The device accesses the mail server (e.g., Gmail, Outlook) and retrieves one day's worth of received emails.

[1253] Output: Received email data

[1254] Specific operation: The device logs into the email account and downloads all received emails.

[1255] Step 2: Parse the email

[1256] Input: Received email data

[1257] Processing: The device sends the received email data to the server, which uses NLP algorithms to analyze the email content and extract important information.

[1258] Output: Extracted keywords and important information

[1259] What it does: The server analyzes the body of the email and extracts urgent matters and important schedules.

[1260] Step 3: Email Summary

[1261] Input: Extracted keywords and important information

[1262] Processing: The server uses this information to summarize each email.

[1263] Output: Abridged email

[1264] What it does: Summarizes emails by condensing the most important information into a few lines and removing redundant content.

[1265] Step 4: Emotion recognition and priority display

[1266] Input: User facial expression and voice data from camera and microphone

[1267] Processing: The server recognizes the user's emotional state in real time and prioritizes emails based on the emotional state.

[1268] Output: Prioritized emails

[1269] Specific operation: The server sets priorities based on the content of the email and the results of emotion recognition.

[1270] Step 5: Send and view summary emails

[1271] Input: Prioritized summary email

[1272] Processing: The server sends the summary email to the user's terminal, and the terminal displays the summary email.

[1273] Output: Summary email displayed on user terminal

[1274] Specific operation: The device that receives the email from the server displays the contents of the email on the screen.

[1275] Video summary function and emotion engine

[1276] Step 1: Get the video URL

[1277] Input: Viewing history or specified URL

[1278] Processing: The user inputs their viewing history or a specified URL into their device.

[1279] Output: The URL entered

[1280] Specific behavior: The user enters a URL into an input form on the device.

[1281] Step 2: Analyze the video data

[1282] Input: Entered URL

[1283] Processing: The device sends the specified URL to the server. The server accesses the URL and retrieves the video data. The server transcribes the audio using the Google Cloud Speech-to-Text API and analyzes the video data using FFmpeg.

[1284] Output: Transcribed audio data, analyzed video data

[1285] What it does: The server downloads the video from the URL and processes the audio and video separately.

[1286] Step 3: Summarize the video

[1287] Input: Analyzed audio and video data

[1288] Processing: The server extracts important scenes and creates a digest video of less than 5 minutes.

[1289] Output: Digest video

[1290] Specific actions: Edit important scenes into a short video.

[1291] Step 4: Emotion recognition and content adjustment

[1292] Input: User facial expression and voice data from camera and microphone

[1293] Processing: The server uses an emotion engine to recognize the user's emotional state and adjust the content of the summarized digest video.

[1294] Output: Adjusted digest video

[1295] Specific operation: The server adjusts and optimizes the content of the video according to the user's emotional state.

[1296] Step 5: Send and display summary video

[1297] Input: Adjusted digest video

[1298] Processing: The server sends the summary video to the user's terminal, and the terminal plays the summary video.

[1299] Output: Summary video displayed on the user's device

[1300] Specific operation: The device that receives the video from the server plays it.

[1301] Through these steps, the user can efficiently obtain appropriate information according to their emotional state.

[1302] (Application example 2)

[1303] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1304] In today's world, a vast amount of information is available via the Internet, making it difficult for users to efficiently obtain the information they need. Furthermore, few systems exist that provide optimal information based on the user's emotional state, and there is a need for systems that can quickly provide useful information to users in specific situations. Particularly during busy times or high-stress situations, there is a need for systems that can prioritize information that helps users relax or that is of high urgency.

[1305] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1306] In this invention, the server includes means for collecting news articles, means for analyzing the text of the collected news articles and extracting key points, means for summarizing the news articles based on the key points, means for transmitting the summarized news to a user terminal, means for displaying the summarized news on the user terminal, means for recognizing the emotional state of the user, and means for filtering the news articles based on the recognized emotional state and providing the user with the most appropriate summarized news. This makes it possible to provide the user with the most appropriate news article information according to their emotional state and improve the user's information acquisition efficiency.

[1307] A "news article" is written information about current events that is published on the Internet.

[1308] "Email" means an electronic message sent or received over the Internet.

[1309] "Video" is a digital medium that includes video and audio to convey information visually and aurally.

[1310] "Gist" refers to the main content or important parts of a news article, email, or video.

[1311] "Emotional state" refers to the user's current psychological response or mood, as recognized using the emotion engine.

[1312] "Filtering" is the process of selecting and providing relevant information based on a user's emotional state.

[1313] "Summarizing" is the act of shortening the content of a news article, email, or video, leaving out only the main points and expressing them concisely.

[1314] "Terminal" refers to a device that allows a user to receive and view information, such as a smartphone, smart glasses, or a head-mounted display.

[1315] An "emotion engine" is a technology that analyzes emotions from a user's facial expressions, voice, etc., and recognizes their emotional state.

[1316] A "user" is a person who uses an information service to efficiently retrieve and consume news articles, emails, and videos.

[1317] The present invention combines a system that collects information from three types of information sources - news articles, emails, and videos - and efficiently summarizes and provides it to users with an emotion engine that recognizes the user's emotions.

[1318] The main components of this system will be described.

[1319] News article summarization and sentiment engine

[1320] News article collection

[1321] The server accesses various news sites on the Internet and automatically collects the latest news articles. The articles are classified into categories for filtering purposes.

[1322] News article analysis

[1323] The server analyzes the text of collected news articles using natural language processing (NLP) algorithms, such as BERT or GPT models (using Hugging Face's Transformers library), to extract important keywords and key points from the articles.

[1324] News article summaries

[1325] Based on the extracted keywords and key points, the server summarizes the news article in a few lines, eliminating redundant parts and leaving only the main points.

[1326] Emotion Recognition and Filtering

[1327] The emotion engine analyzes the user's facial expressions and voice to recognize their current emotional state, using OpenFace, DeepFace, or the Microsoft Azure Emotion API.

[1328] Based on the user's emotional state, the server filters out appropriate news articles, for example, if the user is feeling stressed, it prioritizes positive news.

[1329] Sending and displaying news summaries

[1330] The summarized news is sent from the server to the user's device, where the user can easily check the news articles according to their emotions.

[1331] Email summary and sentiment engine

[1332] Get email

[1333] The device accesses the user's email account (e.g., Gmail or Outlook) and retrieves all of the emails received for the day.

[1334] Email Analysis

[1335] The captured email data (body, subject, sender information, etc.) is sent to a server and analyzed using NLP algorithms.

[1336] Email Summary

[1337] Extract key information and keywords and summarize each email in a few lines.

[1338] Emotion recognition and priority display

[1339] The emotion engine recognizes the user's emotional state and prioritizes which emails to display based on that state. For example, if the user is feeling stressed, emails with a low level of urgency will be prioritized.

[1340] Sending and viewing summary emails

[1341] The server generates a list of summarized emails and sends it to the user's device, where the user can view emails sorted by importance.

[1342] Video summary function and emotion engine

[1343] Get the video URL

[1344] The user inputs their viewing history and a specified URL into the device.

[1345] Video data analysis

[1346] The terminal transmits the specified URL to the server, and the server accesses the URL to obtain the video data.

[1347] The audio from the captured video is transcribed using ASR technology, and the video data is analyzed to identify important parts.

[1348] Video Summary

[1349] Based on the important parts, a digest video of the video is generated that is less than 5 minutes long.

[1350] Emotion recognition and content regulation

[1351] The emotion engine recognizes the user's emotional state and adjusts the content of the summary video based on that state: if the user is tired, relaxing content will be prioritized.

[1352] Sending and displaying summary videos

[1353] The summarized video is sent from the server to the user's device, allowing the user to quickly check the main content on the device.

[1354] Specific examples

[1355] Specific news article examples

[1356] For example, if a user is in an emotional state where they want to relax after work, the server will recognize the user's current emotions and prioritize providing positive articles.

[1357] Examples of prompt statements

[1358] "What news articles should be suggested if the user's emotional state is perceived as tired?"

[1359] The present invention allows users to acquire information more efficiently and provides appropriate information according to their emotional state, thereby improving the user experience and reducing stress and promoting relaxation.

[1360] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1361] Step 1:

[1362] News article collection

[1363] The server accesses various news sites on the Internet, automatically collects the latest news articles, filters them by category, and extracts the necessary news articles.

[1364] Input: A list of URLs for news sites on the Internet

[1365] Data processing: Extracting news article text using scraping technology

[1366] Output: A list of collected news articles

[1367] Step 2:

[1368] News article analysis

[1369] The server analyzes the collected news articles using natural language processing (NLP) algorithms, such as BERT and GPT models, to extract important keywords and key points from the articles.

[1370] Input: A list of collected news articles

[1371] Data processing: Analysis using NLP algorithms

[1372] Output: A list of extracted keywords and key points

[1373] Step 3:

[1374] News article summaries

[1375] The server then summarises the news article into a few lines based on the extracted keywords and key points, eliminating redundant parts and leaving only the main points.

[1376] Input: A list of extracted keywords and key points

[1377] Data processing: Compression by summarization algorithms

[1378] Output: A summarized news article

[1379] Step 4:

[1380] Recognition of emotional states

[1381] The server uses an emotion engine to recognize the user's emotional state, analyzing facial expression data and voice data sent from the user's device to identify the user's emotional state.

[1382] Input: facial expression data, voice data

[1383] Data processing: Analysis using emotion recognition algorithms

[1384] Output: User's emotional state

[1385] Step 5:

[1386] News article filtering

[1387] The server filters news articles appropriate for the user based on the user's perceived emotional state, prioritizing positive news for users who are feeling stressed.

[1388] Input: Summarized news article, user's emotional state

[1389] Data processing: Selection by filtering algorithm

[1390] Output: A list of news articles relevant to the user

[1391] Step 6:

[1392] Sending news summaries

[1393] The server transmits the summarized news to the user's terminal.

[1394] Input: A list of news articles relevant to the user

[1395] Data processing: Data transmission protocol (HTTP / Sockets)

[1396] Output: News received on the user's device

[1397] Step 7:

[1398] Viewing news summaries

[1399] The user's device displays the received news summary, allowing the user to easily check news articles according to their emotions.

[1400] Input: A summarized news article

[1401] Data processing: Display processing on the user interface

[1402] Output: User views of news articles

[1403] Next, similar steps are taken for email and video summaries and the emotion engine, but the specific steps are similar to the process for news articles: they are analyzed and summarized specifically for each source, filtered according to the user's emotional state, and then displayed on the device.

[1404] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1405] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1406] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1407] [Third embodiment]

[1408] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1409] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1410] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1411] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1412] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1413] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1414] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1415] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1416] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1417] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1418] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1419] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1420] This invention is a system that efficiently summarizes information from three types of information sources: news, email, and video, and provides it to users. To implement this system, a server that runs a program with the following functions, a terminal, and user operations are required.

[1421] News article summary function

[1422] News article collection

[1423] The server has a mechanism to access various news sites on the Internet and automatically collect the latest news articles. For example, it filters by category, such as politics, economics, and sports, and extracts the necessary news articles.

[1424] News article analysis

[1425] The server analyzes the collected news articles using natural language processing (NLP) algorithms. Using the latest NLP techniques, such as BERT and GPT models, it extracts important keywords and key points from the articles. This analysis determines the context and importance of the text and identifies the information needed for summarization.

[1426] News article summaries

[1427] Based on the results of the analysis, the server summarizes the news article in a few lines, leaving out the main points and important details of the news and eliminating redundant parts.

[1428] Sending and displaying news summaries

[1429] The server generates a document summarizing the summarized news and sends it to the user's terminal.

[1430] Users can easily check the summarized news on their devices.

[1431] Email Summary Feature

[1432] Get email

[1433] The device accesses the user's email account (e.g., Gmail or Outlook) and retrieves all of the emails received for the day.

[1434] Email Analysis

[1435] The terminal sends the acquired email data (body, subject, sender information, etc.) to the server.

[1436] Based on the data sent, the server analyzes the content of each email using natural language processing (NLP) algorithms to extract important information and keywords.

[1437] Email Summary

[1438] The server extracts important keywords and key points from each email and then summarises them into a few lines, concisely summarising important meeting schedules or urgent requests.

[1439] Sending and viewing summary emails

[1440] The server generates a list of summarized emails and sends them to the user's terminal.

[1441] The user can easily check the summarized email on the terminal.

[1442] Video summary function

[1443] Get the video URL

[1444] The user inputs their viewing history and a specified URL into the device.

[1445] Video data analysis

[1446] The terminal transmits the specified URL to the server, and the server accesses the URL to obtain the video data.

[1447] The server converts the audio of the video into text using automatic speech recognition (ASR) technology, and then analyzes the video data to extract important scenes, for example, by identifying important parts based on audio containing specific keywords or scene changes.

[1448] Video Summary

[1449] The server generates a video digest based on the extracted key points, summarizing important announcements and interesting moments into short clips.

[1450] Sending and displaying summary videos

[1451] The server transmits the summarized video file to the user's terminal.

[1452] The user can view the digest video generated on the terminal and quickly understand the main content.

[1453] Specific examples

[1454] News article summary examples

[1455] For example, if the server retrieves a "political news article" from a nationally known news site, it will analyze the text for the title "Cabinet reshuffle" using the BERT model, extracting "new cabinet members," "policy changes," "key statements," etc., and generate a summary based on this.

[1456] Example of an email summary

[1457] For example, a device retrieves an email with the subject "Meeting Schedule" from a user's email account and sends the contents to the server. The server extracts information such as the meeting date and time, names of participants, and main agenda items from the body of the email and generates a summary.

[1458] Video summary examples

[1459] For example, if a user selects the URL of a "sports highlight video" from their viewing history, the server retrieves the corresponding video and performs voice recognition and video analysis. As a result, it extracts "important goal scenes" and "interview highlights," and generates a summary clip based on these.

[1460] In this way, the present invention realizes a system that efficiently summarizes news, email, and video information and provides it to users, allowing them to quickly and concisely grasp the information they need.

[1461] The processing flow will be explained below.

[1462] News article summary function

[1463] Step 1: Gather news articles

[1464] The server accesses the specified news site and collects the URLs of the latest news articles, sometimes using the news site's API.

[1465] Step 2: Get news articles

[1466] The server retrieves HTML data from the collected URLs and converts it into a format that is easy to analyze. It extracts the news text, title, date and time, etc.

[1467] Step 3: Analyzing the news article

[1468] The server uses natural language processing (NLP) algorithms to analyze the text of retrieved news articles, using models such as BERT and GPT to extract key keywords and key points from each article.

[1469] Step 4: Summarize the news article

[1470] The server then summarises the article based on the analysis results, combining multiple algorithms to generate an optimal summary that always includes important information, such as important dates, people's names, and events.

[1471] Step 5: Submit your news summary

[1472] The server generates a document summarizing the summarized news and sends it to the user's terminal.

[1473] Step 6: Viewing News Summary

[1474] Users can easily view summaries of news on their devices, and can filter news by category and importance.

[1475] Email Summary Feature

[1476] Step 1: Get email

[1477] The device accesses the user's email account and retrieves all of the emails received in one day. It uses RestAPI or similar to retrieve data from the email server.

[1478] Step 2: Send email data

[1479] The terminal sends the acquired email data (subject, body, sender information, etc.) to the server.

[1480] Step 3: Parse the email

[1481] The server uses NLP algorithms to extract obviously important information, such as meeting schedules, project progress, and requests that require urgent attention.

[1482] Step 4: Email Summary

[1483] The server then summarises each email into a few lines based on the extracted keywords and key points, concisely summarising the key points so that the content can be understood at a glance.

[1484] Step 5: Send a summary email

[1485] The server generates a list of summarized mails and sends it to the user's terminal.

[1486] Step 6: View the summary email

[1487] Users can view summaries of emails compiled on their device, and it also includes the ability to filter out unnecessary emails and display only the important ones.

[1488] Video summary function

[1489] Step 1: Get the video URL

[1490] The user inputs the viewing history or the URL of the specified video into the device.

[1491] Step 2: Sending video data

[1492] The terminal sends the specified URL to the server.

[1493] Step 3: Getting the video

[1494] The server accesses the URL and retrieves the video data. YouTube API or Tver API is often used.

[1495] Step 4: Audio-visual analysis of the video

[1496] The server transcribes the audio from the captured video using automatic speech recognition (ASR) technology and performs scene analysis of the video to identify important parts, such as audio containing specific keywords or scene changes.

[1497] Step 5: Summarize your video

[1498] Based on the extracted key points, the server generates a digest video that summarizes the important scenes of the video in less than five minutes.

[1499] Step 6: Submit your summary video

[1500] The server transmits the digested video to the user terminal.

[1501] Step 7: Displaying the summary video

[1502] Users can play the digest video generated on their device and quickly check the main content.

[1503] The above processing steps realize a system that efficiently summarizes news, email, and video information, allowing users to quickly obtain the information they need.

[1504] Example 1

[1505] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1506] In modern society, as the amount of information increases, it is becoming increasingly difficult to efficiently grasp important information. In particular, there is a lack of methods to quickly retrieve, summarize, and present necessary information from various sources, such as news articles, emails, and videos. This forces users to manually find important parts from a vast amount of information, which is a time-consuming and labor-intensive process.

[1507] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1508] In this invention, the server includes a means for collecting news information, a means for analyzing the collected news information using a natural language processing algorithm to extract key points, and a means for summarizing the news information in a few lines based on the key points. This enables summarization of news articles. The server also includes a means for acquiring electronic communications, a means for analyzing the content of the acquired electronic communications to extract important information, and a means for summarizing the electronic communications in a few lines based on the important information. This enables summarization of emails. The server also includes a means for acquiring URLs of video information, a means for analyzing the audio and video of the acquired video information to extract important parts, and a means for summarizing the video information into short clips based on the extracted important parts. This enables summarization of videos. These functions allow users to efficiently acquire important information from different information sources and grasp it in a short amount of time.

[1509] "News information" refers to the content and articles of news sites published on the Internet.

[1510] "Natural language processing algorithms" refers to technologies that analyze text data and evaluate context and importance. Specifically, this includes models such as BERT and GPT.

[1511] "Key points" refer to important information or keywords found in news information, electronic communications, and video information.

[1512] "Summarizing" means compressing long pieces of information into a concise summary of only the main points and important information.

[1513] "Electronic communications" refers to communications such as emails exchanged over the Internet.

[1514] "Speech recognition technology" refers to technology that converts voice data into text data. Specifically, it includes ASR (automatic speech recognition) technology.

[1515] "Video information" refers to the content and images of videos that can be viewed on the Internet.

[1516] A "URL" is an address used to specify a specific resource on the Internet.

[1517] "Analyze" refers to analyzing data or information using algorithms to find specific patterns or important elements.

[1518] "User terminal" refers to a device used by a user to receive and display information, including, but not limited to, a smartphone, tablet, or computer.

[1519] "Document" refers to a digital document, such as a text file or PDF, that compiles summarized information.

[1520] "Generate" refers to creating data or information from scratch using a computer.

[1521] "Transmitting" refers to sending data or information from one location to another over a network.

[1522] "Display" refers to visually presenting data or information so that it can be read by a user.

[1523] A "short clip" refers to a short piece of video created by cutting out important parts from a longer video.

[1524] This invention is a system that collects information from three types of information sources: news information, electronic communications, and video information, and efficiently summarizes each of them and provides them to users. To implement this system, a server that runs a program with the following functions, a terminal, and user operations are required.

[1525] News article summary function

[1526] Gathering news information

[1527] The server sets a Cron job to access a news site, for example, every day at 9:00 a.m., to collect the latest news information. The server uses the news site's API to extract the news title and text data of the news body.

[1528] News information analysis

[1529] The server analyzes the collected news information using natural language processing algorithms. Specifically, it uses generative AI models such as BERT and GPT to tokenize the text data, evaluate its context and importance, extract important keywords and key points, and store the results.

[1530] News summary

[1531] The server summarizes the news information in a few lines based on the extracted keywords and key points, concisely retaining only the main points and eliminating redundant parts.

[1532] Sending and displaying summary news information

[1533] The server generates a document containing the summarized news information and sends it to the user's terminal, for example, in PDF or HTML format.

[1534] To enable a user to check summarized news information on a terminal and efficiently grasp the latest news.

[1535] Summary function of electronic communications

[1536] Obtaining Electronic Communications

[1537] The device accesses the user's email account using the IMAP protocol and retrieves electronic communications for a specified period of time. The connection is made using a secure authentication protocol (such as OAuth).

[1538] Analysis of electronic communications

[1539] The device sends captured electronic communications to a server, which then uses generative AI models such as BERT and GPT to analyze the email data and extract key information and keywords.

[1540] Electronic Communications Summary

[1541] The server summarizes electronic communications in a few lines based on extracted keywords and key points, concisely summarizing important meeting schedules and information requiring urgent action.

[1542] Summary of electronic communication transmission and display

[1543] The server generates a document from the summarized electronic communication and transmits it to the user's terminal.

[1544] The user can view summarized electronic communications on a terminal and quickly grasp important information.

[1545] Video summary function

[1546] Get the video URL

[1547] The user enters their viewing history and a specified URL into the device, and the device then sends the URL to the server.

[1548] Video information analysis

[1549] The server accesses the specified URL and retrieves the video data. The server then downloads the correct data using the YouTube API or similar.

[1550] The server uses automatic speech recognition (ASR) technology to convert the audio in the video into text and simultaneously analyzes the video data, for example, to detect lines containing important keywords or scene changes.

[1551] Video information summary

[1552] The server generates a video digest based on the analysis results, compiling important announcements and interesting moments into short clips.

[1553] Sending and displaying summary video information

[1554] The server sends the summarized video file to the user's device, where it can be streamed using a dedicated app.

[1555] Users can watch the summary video on their device and quickly grasp the important content.

[1556] Specific examples

[1557] Examples of news summaries

[1558] For example, the server retrieves "political news information" from a domestic news site and analyzes it using the BERT model. The analysis extracts "new cabinet members," "policy changes," "major statements," etc., and generates a summary based on this.

[1559] Examples of Electronic Communications Summaries

[1560] For example, a terminal retrieves an electronic message with the subject "Meeting Schedule" from a user's email account and sends it to a server. The server extracts information such as the meeting date and time, participant names, and main agenda items from the message body and generates a summary.

[1561] Example of video summary

[1562] For example, if a user selects the URL of a "sports highlight video" from their viewing history, the server retrieves and analyzes the corresponding video. As a result of voice recognition and video analysis, it extracts "goal scenes" and "interview highlights," and generates a summary clip based on these.

[1563] Prompt Sentence Examples

[1564] "Please summarize the latest political news article."

[1565] "Please tell me the highlights of the email I received today."

[1566] "Summarize the key scenes in this sports video."

[1567] In this way, the present invention realizes a system that efficiently summarizes news information, electronic communications, and video information and provides them to users, allowing them to quickly and concisely grasp the information they need.

[1568] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1569] News article summary function

[1570] Step 1: Gather news information

[1571] The server sets up a Cron job to access a news site on the Internet, for example, every day at 9:00 AM.

[1572] Input: List of news site URLs

[1573] The server accesses each URL and uses the news site's API to obtain the latest news information.

[1574] Data processing: Parse HTML to extract news titles and text data.

[1575] Output: A list of news articles in text format

[1576] Step 2: Analyzing the news information

[1577] The server analyzes the news information collected in the previous step using a natural language processing algorithm.

[1578] Input: News article list

[1579] The server uses generative AI models such as BERT or GPT to tokenize the text data and evaluate its context and importance.

[1580] Data calculation: Extract important keywords and key points.

[1581] Output: A dataset showing key keywords and key points

[1582] Step 3: Summarize the news information

[1583] The server summarizes the news information in a few lines based on the extracted keywords and key points.

[1584] Input: A dataset showing important keywords and key points

[1585] Data processing: Keep only the main points and remove redundant parts.

[1586] Output: Summary

[1587] Step 4: Send and display summary news information

[1588] The server generates a document containing summarized news information and transmits it to the user's terminal.

[1589] Input: Abstract

[1590] Data processing: Generate documents in PDF or HTML format.

[1591] Output: A summary of the news document

[1592] The user checks the summarized news information on the terminal.

[1593] Summary function of electronic communications

[1594] Step 1: Obtaining Electronic Communications

[1595] The device accesses the user's email account using the IMAP protocol.

[1596] Enter your email account information

[1597] The device captures electronic communications for a specified period of time.

[1598] Data processing: Read data from received emails.

[1599] Output: Electronic communication data

[1600] Step 2: Analyzing Electronic Communications

[1601] The terminal transmits the captured electronic communication to a server.

[1602] Input: Electronic communication data

[1603] The server analyzes the data using a generative AI model such as BERT or GPT.

[1604] Data calculations: Extracting important information and keywords.

[1605] Output: Data showing important information and keywords

[1606] Step 3: Summarizing Electronic Communications

[1607] The server summarizes the electronic communication in a few lines based on the extracted keywords and key points.

[1608] Input: Data that indicates important information or keywords

[1609] Data processing: Concisely summarize important meeting schedules and information that requires urgent action.

[1610] Output: Summary

[1611] Step 4: Sending and displaying summary electronic communications

[1612] The server generates a document from the summarized electronic communication and transmits it to the user's terminal.

[1613] Input: Abstract

[1614] Data processing: Generate documents in PDF or HTML format.

[1615] Output: A summary of the electronic communication

[1616] The user reviews the summarized electronic communication at the terminal.

[1617] Video summary function

[1618] Step 1: Get the video URL

[1619] The user enters their viewing history and a specified URL into the device, and the device then sends the URL to the server.

[1620] Input: Video URL

[1621] Output: URL data

[1622] Step 2: Analyze video information

[1623] The server accesses the specified URL and acquires the video data.

[1624] Input: URL data

[1625] The server downloads the video data using the YouTube API or similar.

[1626] Data processing: Uses automatic speech recognition (ASR) technology to convert voice into text and also analyzes video data.

[1627] Output: Analyzed text data and video data

[1628] Step 3: Summary of video information

[1629] The server generates a video digest based on the analysis results.

[1630] Input: Analyzed text data and video data

[1631] Data processing: Summarize important scenes and quotes into short clips.

[1632] Output: Summary clip

[1633] Step 4: Send and display summary video information

[1634] The server sends the summarized video file to the user's terminal.

[1635] Input: Summary clip

[1636] Data processing: Streaming playback becomes possible using a dedicated app.

[1637] Output: Summarized video file

[1638] Users can watch the summary video on their device and quickly grasp the important content.

[1639] (Application example 1)

[1640] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1641] In today's information-saturated world, users need to quickly and efficiently understand the vast amount of news articles, emails, and video information. It is particularly difficult to extract and provide users with only the most important information from each source. Furthermore, there is a lack of a way to easily summarize this information and provide it to users in an easily accessible format.

[1642] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1643] In this invention, the server includes means for collecting news articles, means for analyzing the text of the collected news articles and extracting key points, means for summarizing the news articles based on the key points, means for sending the summarized news to a user terminal, means for displaying the summarized news on the user terminal, and means for optimizing the summaries using a generative AI model and prompt sentences, thereby enabling users to quickly grasp only the important key points from a large amount of information.

[1644] A "news article" is a report or information about a current event or topic provided through online or offline media.

[1645] "Email" is a digital message sent or received over the Internet or other network.

[1646] A "video" is a multimedia file that combines audio and video and is played continuously.

[1647] A summary is a short summary of the important elements or key points of the original information.

[1648] A "user terminal" is a device used by a user, including a smartphone, tablet, or PC.

[1649] A "generative AI model" is an artificial intelligence model that uses machine learning algorithms to generate text and data.

[1650] A "prompt" is text input to a generative AI model that takes the form of an instruction or question to the model.

[1651] A "collection method" is a method or system for obtaining data or information from a particular source.

[1652] An "analysis tool" is a method or system for processing collected data or information and extracting meaningful elements.

[1653] A "transmission means" is a method or system for sending data or information from one system to another.

[1654] A "display means" is a method or system for visually presenting information to a user, and includes devices such as a screen.

[1655] This invention is an information summarization system for efficiently providing various information to users. This system summarizes information from three types of information sources: news, email, and video, and provides it to users concisely. To implement this, a server, a terminal, and user operations are required.

[1656] News article summary function

[1657] News article collection

[1658] The server accesses various news sites to collect the latest news articles, and then filters the articles by category, such as politics, economics, or sports, to extract the necessary news articles.

[1659] News article analysis

[1660] The server analyzes the collected news articles using natural language processing (NLP) algorithms, such as the BERT model, to extract the article's context and important keywords.

[1661] News article summaries

[1662] The server then summarises the news article based on the analysis, keeping the key points and keywords and removing redundant parts.

[1663] Sending and displaying news summaries

[1664] The server sends the summarized news to the user's terminal, where the user can check the concisely summarized news.

[1665] Email Summary Feature

[1666] Get email

[1667] The terminal accesses the user's email account and retrieves received emails.

[1668] Email Analysis

[1669] The device sends the email data to the server, which then uses NLP algorithms to analyze the email body and extract important information and keywords.

[1670] Email Summary

[1671] The server then uses this extracted data to summarize the email, concisely summarizing important schedules and information requiring urgent action.

[1672] Sending and viewing summary emails

[1673] The server sends the summarized email to the user's terminal, where the user can check the summarized email.

[1674] Video summary function

[1675] Get the video URL

[1676] The user inputs their viewing history and the specified URL into the device.

[1677] Video data analysis

[1678] The device sends the URL to the server, which retrieves the corresponding video. The audio data is converted into text using automatic speech recognition (ASR), and important scenes are extracted from the video data.

[1679] Video Summary

[1680] Based on the analysis data, the server generates a summary clip of the video containing important scenes and keywords.

[1681] Sending and displaying summary videos

[1682] The server sends the summarized video to the user's device, where the user can watch the summarized video in a short time.

[1683] Hardware and Software Used

[1684] The system's main hardware consists of a server and user devices (smartphones, tablets, and PCs). The software used includes Python, the BERT model, the Transformers library, the requests library, and speech recognition technology.

[1685] Examples and prompts

[1686] As a concrete example, consider a case where a user checks a news article summary on their smartphone. For example, the following prompt sentence is input to the generative AI model:

[1687] "Summarize the article: A new cabinet has been announced, with key changes."

[1688] An example output is:

[1689] "A new cabinet has been announced, with key members being changed."

[1690] Users can view this output on their smartphones.

[1691] This system allows users to quickly grasp only the important points from a large amount of information.

[1692] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1693] Step 1:

[1694] Collect news articles (server)

[1695] The server accesses various news sites and automatically collects the latest news articles. The input is the URL of the news site, and the output is the collected news article data. The server filters these articles by category and saves the collected data.

[1696] Step 2:

[1697] News article analysis (server)

[1698] The server analyzes the collected news article text using a natural language processing (NLP) algorithm. Specifically, it uses the BERT model to extract the article's context and important keywords. The input is the collected news article data, and the output is the analysis results: important keywords and key points.

[1699] Step 3:

[1700] News article summaries (server)

[1701] The server summarizes the news article based on the keywords and key points extracted in the previous step, eliminating redundant parts and picking out only the important information. The input is the analysis result, and the output is the summarized news article text.

[1702] Step 4:

[1703] Sending summary news (server)

[1704] The server sends the summarized news article to the user's terminal, where the input is the summarized news article text and the output is the summarized news sent to the user's terminal.

[1705] Step 5:

[1706] Displaying news summaries (terminal)

[1707] The user terminal displays the received summary news. The input is the summary news sent from the server, and the output is the news content displayed on the user interface.

[1708] Step 6:

[1709] Retrieving email (terminal)

[1710] The terminal accesses the user's email account and retrieves all received emails for one day. The input is the email account authentication information, and the output is the retrieved email data.

[1711] Step 7:

[1712] Email analysis (terminal)

[1713] The device sends the acquired email data to the server, which then uses NLP algorithms to analyze the content. The input is the email data, and the output is the extraction of important information and keywords.

[1714] Step 8:

[1715] Email Abstract (Server)

[1716] The server summarizes each email in a few lines based on the extracted important keywords and information. The input is the analysis result, and the output is the summarized email text.

[1717] Step 9:

[1718] Sending summary emails (server)

[1719] The server sends a list of summarized emails to the user terminal, where the input is the summarized email text and the output is the summarized email sent to the user terminal.

[1720] Step 10:

[1721] Display summary email (terminal)

[1722] The user terminal displays the received summary email. The input is the summary email sent from the server, and the output is the email content displayed on the user interface.

[1723] Step 11:

[1724] Get the video URL (device)

[1725] The user inputs their viewing history and the URL of the video they want to view into the device. The input is the URL of the video they specified, and the output is that URL information.

[1726] Step 12:

[1727] Video data analysis (server)

[1728] The device sends the URL of the specified video to the server. The server accesses the URL and retrieves the video data. The audio data is converted into text using speech recognition technology, and important scenes are extracted from the video data. The input is the video URL, and the output is the speech recognition results and the extraction of important scenes.

[1729] Step 13:

[1730] Video summary (server)

[1731] The server generates a video digest clip based on the extracted keypoints. The input is the speech recognition result and the extraction of important scenes, and the output is a summarized video clip.

[1732] Step 14:

[1733] Sending summary video (server)

[1734] The server sends the summarized video file to the user's terminal, where the input is the summarized video clip and the output is the summarized video file sent to the user's terminal.

[1735] Step 15:

[1736] Display summary video (device)

[1737] The user terminal plays the received summary video. The input is the summary video file sent from the server, and the output is the summary video that is played.

[1738] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1739] This invention combines a system that collects information from three types of information sources (news, email, and video), efficiently summarizes it, and provides it to users with an emotion engine that recognizes user emotions. Implementing this system requires a server that runs a program with the following functions, a terminal, and user operation.

[1740] News article summarization and sentiment engine

[1741] News article collection

[1742] The server has a mechanism to access various news sites on the Internet and automatically collect the latest news articles. For example, it filters by category, such as politics, economics, and sports, and extracts the necessary news articles.

[1743] News article analysis

[1744] The server analyzes the collected news article text using natural language processing (NLP) algorithms, using the latest NLP techniques such as BERT and GPT models to extract important keywords and key points from the articles.

[1745] News article summaries

[1746] Based on the results of the analysis, the server summarizes the news article in a few lines, leaving out the main points and important details of the news and eliminating redundant parts.

[1747] Emotion Recognition and Filtering

[1748] The server uses an emotion engine to recognize the user's current emotional state, for example, by analyzing the user's facial expressions and voice through a camera.

[1749] The server filters news articles appropriate for the user based on the perceived emotional state: for example, if the user is feeling stressed, it prioritizes positive news articles.

[1750] Sending and displaying news summaries

[1751] The server generates a document summarizing the summarized news and sends it to the user's terminal.

[1752] Users can easily check filtered news summaries on their devices.

[1753] Email summary and sentiment engine

[1754] Get email

[1755] The device accesses the user's email account (e.g., Gmail or Outlook) and retrieves all of the emails received for the day.

[1756] Email Analysis

[1757] The terminal sends the acquired email data (body, subject, sender information, etc.) to the server.

[1758] Based on the data sent, the server analyzes the content of each email using natural language processing (NLP) algorithms to extract important information and keywords.

[1759] Email Summary

[1760] The server extracts important keywords and key points from each email and then summarises them into a few lines, concisely summarising important meeting schedules or urgent requests.

[1761] Emotion recognition and priority display

[1762] The server uses an emotion engine to recognize the user's current emotional state.

[1763] The server prioritizes which emails to display based on the user's emotional state, for example, displaying less urgent emails to a stressed user first.

[1764] Sending and viewing summary emails

[1765] The server generates a list of summarized emails and sends them to the user's terminal.

[1766] Users can view emails on their devices summarized in order of importance according to their emotions.

[1767] Video summary function and emotion engine

[1768] Get the video URL

[1769] The user inputs their viewing history or a specified URL into the device.

[1770] Video data analysis

[1771] The terminal transmits the specified URL to the server, and the server accesses the URL to obtain the video data.

[1772] The server transcribes the audio from the captured video using automatic speech recognition (ASR) technology and analyzes the video data to identify important parts, for example, based on audio containing specific keywords or scene changes.

[1773] Video Summary

[1774] Based on the extracted key points, the server generates a digest video that summarizes the important scenes of the video in less than five minutes.

[1775] Emotion recognition and content regulation

[1776] The server uses an emotion engine to recognize the user's emotional state.

[1777] The server automatically selects and tailors the summary video to suit the user based on the recognized emotional state, for example prioritizing relaxing content if the user is tired.

[1778] Sending and displaying summary videos

[1779] The server transmits the digested video to the user's terminal.

[1780] Users can play the digest video generated on their device and quickly check the main content.

[1781] Specific examples

[1782] Specific news article examples

[1783] For example, if a user is in an emotional state where they want to relax after work, the server will recognize the user's current emotions and provide a summary of positive articles preferentially, allowing the user to obtain important information while reducing stress.

[1784] Specific examples of email

[1785] For example, if a user is checking email during busy morning hours, the server will recognize through its emotion engine that the user is feeling anxious or stressed. Based on this, the server will prioritize and summarize and display emails with high urgency.

[1786] Video examples

[1787] For example, if a user is tired and wants to relax, the server will recognize the user's current emotions and prioritize and summarize relaxing videos based on that emotion. The user can watch relaxing videos in a short amount of time.

[1788] In this way, by combining emotion engines, it becomes possible to provide information that is more in line with the user's needs, improving the user experience.

[1789] The processing flow will be explained below.

[1790] News article summarization and sentiment engine

[1791] Step 1: Gather news articles

[1792] The server accesses various news sites on the Internet and collects the URLs of the latest news articles, for example, by obtaining the necessary data from RSS feeds or APIs.

[1793] Step 2: Get news articles

[1794] The server retrieves the HTML data from the collected URLs and extracts the body of the news article, the title, and the date and time.

[1795] Step 3: Analyzing the news article

[1796] The server uses natural language processing (NLP) algorithms to analyze the text of the news article, extracting key keywords and gist information and identifying important information.

[1797] Step 4: Summarize the news article

[1798] The server then summarises the news article in a few lines based on the extracted key points and keywords, concisely summarising the main points.

[1799] Step 5: Emotion Recognition

[1800] The device captures the user's facial expressions and voice through a camera and microphone, and analyzes them with an emotion engine. For example, emotions are identified from the user's facial expressions and tone of voice.

[1801] Step 6: Filtering

[1802] The server then filters appropriate news articles based on the results of the emotion engine according to the user's emotional state. For example, if the user feels like relaxing, it prioritizes positive news.

[1803] Step 7: Submit your news summary

[1804] The server generates a document summarizing the filtered news articles and sends it to the user's terminal.

[1805] Step 8: Viewing Summary News

[1806] Users can easily check filtered news summaries on their devices.

[1807] Email summary and sentiment engine

[1808] Step 1: Get email

[1809] The device accesses the user's email account and retrieves all of the emails received in one day, using an email protocol (e.g., IMAP or POP3).

[1810] Step 2: Send email data

[1811] The terminal sends the acquired email data (body, subject, sender information, etc.) to the server.

[1812] Step 3: Parse the email

[1813] The server analyzes the email content using natural language processing (NLP) algorithms to extract important information and keywords, such as meeting schedules or urgent requests.

[1814] Step 4: Email Summary

[1815] The server then summarises each email in a few lines based on the extracted key keywords and information, concisely summarising the key points.

[1816] Step 5: Emotion Recognition

[1817] The device captures the user's facial expressions and voice through a camera and microphone, and analyzes them with an emotion engine. For example, emotions are identified from the user's facial expressions and tone of voice.

[1818] Step 6: Prioritize Display

[1819] The server then prioritizes which emails to display based on the results of the emotion engine and the user's emotional state. For example, if the user is feeling stressed, emails with a low level of urgency will be displayed first.

[1820] Step 7: Send a summary email

[1821] The server generates a prioritized summary mail list based on the emotional state and transmits it to the user's terminal.

[1822] Step 8: View the summary email

[1823] The user can check the emails summarized according to priority on the terminal.

[1824] Video summary function and emotion engine

[1825] Step 1: Get the video URL

[1826] The user inputs the viewing history or the URL of the specified video into the device.

[1827] Step 2: Sending video data

[1828] The terminal sends the specified URL to the server.

[1829] Step 3: Getting the video

[1830] The server accesses the transmitted URL and acquires the video data.

[1831] Step 4: Audio-visual analysis of the video

[1832] The server transcribes the audio from the captured video using automatic speech recognition (ASR) technology and analyzes the video data to identify important parts, such as audio containing key keywords and scene changes.

[1833] Step 5: Summarize your video

[1834] The server generates a video digest based on the extracted key points, summarizing important scenes into short clips.

[1835] Step 6: Emotion Recognition

[1836] The device captures the user's facial expressions and voice through a camera and microphone, and analyzes them with an emotion engine. For example, emotions are identified from the user's facial expressions and tone of voice.

[1837] Step 7: Adjust the content

[1838] Based on the results of the emotion engine, the server selects a summary video that best suits the user's emotional state and adjusts the content accordingly. For example, if the user wants to relax, it will prioritize videos with relaxing content.

[1839] Step 8: Submit your summary video

[1840] The server transmits the summarized digest video to the user's terminal.

[1841] Step 9: Displaying the Summary Video

[1842] Users can play the digest video generated on their device and quickly check the main content.

[1843] The above processing steps realize a system that efficiently summarizes news, email, and video information, and further adjusts the content provided based on the user's emotional state.

[1844] Example 2

[1845] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1846] Conventional summarization systems for news articles, emails, and videos provide information uniformly without considering the user's emotional state, making it difficult to provide content that adapts to the user's psychological state. Furthermore, efficient summarization of large amounts of information and appropriate provision to the user requires advanced natural language processing and emotion recognition technologies. Consequently, to improve the user experience, it is necessary to adjust the priority and content of information according to the user's emotional state.

[1847] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1848] In this invention, the server includes means for collecting news articles, means for analyzing the text of the collected news articles using a natural language processing algorithm and extracting key points, means for summarizing the news articles based on the key points, means for filtering the news articles based on the emotional state using an emotion engine that recognizes the emotional state of the user, means for transmitting the summarized news to a user terminal, and means for displaying the summarized news on the user terminal, thereby making it possible to provide news articles that are adapted to the emotional state of the user.

[1849] The server also includes means for acquiring emails, means for analyzing the contents of the acquired emails using a natural language processing algorithm and extracting important information, means for summarizing the emails based on the important information, means for determining the priority of the emails based on the emotional state having an emotion engine that recognizes the emotional state of the user, means for sending the summarized emails to the user terminal, and means for displaying the summarized emails on the user terminal, thereby enabling the importance of emails to be determined and provided in accordance with the emotional state of the user.

[1850] The server further includes a means for acquiring a URL of the video, a means for analyzing the audio and video of the acquired video using voice recognition technology and video analysis technology and extracting important parts, a means for summarizing the video based on the extracted important parts, a means having an emotion engine that recognizes the emotional state of the user and adjusting the content of the video based on the emotional state, a means for transmitting the summarized video to the user terminal, and a means for displaying the summarized video on the user terminal, thereby making it possible to provide video content adapted to the emotional state of the user.

[1851] These measures enable the provision of information according to the user's emotional state, improving the user experience.

[1852] The "means for collecting news articles" is a function that allows the server to access news sites on the Internet and automatically obtain the latest news articles.

[1853] "Natural language processing algorithms" are technologies for analyzing text data and understanding the meaning and structure of language, and include BERT and GPT models.

[1854] "Key point extraction" is a function that identifies and extracts important keywords and key information from the body of a news article or email.

[1855] The "means for summarizing news articles" is a function that summarizes the text of a news article in a concise format based on the extracted important information.

[1856] "Means for retrieving email" refers to a function that accesses a user's email account (e.g., Gmail or Outlook) and retrieves all received emails.

[1857] The "means for determining the priority of e-mails" is a function for determining the display order of received e-mails based on the emotional state of the user.

[1858] The "means for transmitting summarized e-mail to a user terminal" is a function for transmitting the contents of the summarized e-mail to a user terminal.

[1859] A "user terminal" is an electronic device such as a computer, smartphone, or tablet that a user uses as an interface.

[1860] The "emotion engine" is a technology for recognizing and analyzing the user's emotional state from their facial expressions and voice.

[1861] The "means for filtering based on emotional state" is a function that selects the content of news articles or emails to be displayed based on the recognized emotional state of the user.

[1862] "Means for obtaining a video URL" refers to a function for obtaining viewing history or a URL specified by the user.

[1863] "Speech recognition technology" is a technology for analyzing voice data and converting it into text.

[1864] "Video analysis technology" is a technology that analyzes video data to identify important scenes and keywords.

[1865] The "means for summarizing a video based on extracted important parts" is a function that extracts important parts of a video and generates a digest video that can be viewed in a short amount of time.

[1866] This system collects information from three sources: news articles, emails, and videos, and efficiently summarizes and provides it to users.The system incorporates an emotion engine that recognizes the user's emotions, and can provide content that adapts to the user's psychological state.

[1867] Hardware and software used

[1868] server

[1869] The server has the following features:

[1870] Gathering news articles: Accessing news sites on the Internet, specifically using Python's BeautifulSoup library and Scrapy framework.

[1871] Natural Language Processing: Using natural language processing algorithms like BERT and GPT models to parse the body of news articles and emails.

[1872] Speech Recognition: Use the Google Cloud Speech-to-Text API and PyDub library to analyze the audio in the video.

[1873] Video Analysis: Use FFmpeg and video analysis techniques to identify important scenes in the video.

[1874] Emotion Recognition: Uses technologies such as OpenCV and Dlib, Google Cloud Speech-to-Text API to analyze the user's facial expressions and voice.

[1875] Terminal

[1876] Accessing the user's email account (e.g., Gmail or Outlook) to retrieve email.

[1877] The acquired data is sent to the server.

[1878] It has the function of displaying summary news, summary emails, and summary videos sent from the server.

[1879] User

[1880] Use your device to check news, emails, and video summaries.

[1881] You can request a video summary by entering your viewing history or a specified URL into your device.

[1882] Data processing and calculation methods

[1883] News article collection

[1884] The server periodically accesses news sites to retrieve the latest news articles, which are then filtered by category and stored in a database.

[1885] News article analysis

[1886] The server analyzes the collected article text using natural language processing algorithms, such as BERT and GPT models, to extract key points and keywords from the article.

[1887] News article summaries

[1888] Based on the extracted key points, the server summarizes the news article, consolidating the important information into a few lines and eliminating redundant parts.

[1889] Emotion Recognition and Filtering

[1890] The emotion engine recognizes the user's current emotional state. It analyzes input data from the camera and microphone through facial expression recognition APIs and voice emotion analysis APIs. Based on the recognized emotional state, it selects appropriate news articles and emails for the user.

[1891] Email capture and analysis

[1892] The device accesses the user's email account and sends a day's worth of received emails to a server, which then analyzes the email content using natural language processing algorithms to extract important information.

[1893] Email Summary

[1894] Based on the extracted important information, the server summarizes the email, concisely summarizing urgent matters and important events.

[1895] Video analysis and summarization

[1896] The device sends the specified URL to the server, which then transcribes the audio from the video using speech recognition technology and analyzes the video data to identify important parts, generating a digest video based on key scenes.

[1897] Examples and prompts

[1898] Specific news article examples

[1899] For example, if a user is in an emotional state where they want to relax after work, the server can recognize the user's current emotions and provide a summary of positive articles preferentially, thus allowing the user to obtain important information while reducing stress.

[1900] Example prompt: "I'm in a relaxed mood today, so please summarize a positive news article for me."

[1901] Specific examples of email

[1902] For example, if a user is checking email during busy morning hours, the server will recognize through its emotion engine that the user is feeling anxious or stressed. Based on this, the server will prioritize and summarize and display emails with high urgency.

[1903] Example prompt: "It's a busy morning, so please prioritize and summarize the most urgent emails."

[1904] Video examples

[1905] For example, if a user is tired and wants to relax, the server will recognize the user's current emotions and prioritize and summarize relaxing videos based on that emotion. The user can watch relaxing videos in a short amount of time.

[1906] Example prompt: "I want to relax, so please summarize the video at the specified URL and create a digest video."

[1907] The above is a specific embodiment for carrying out the present invention, which allows for the provision of information according to the emotional state of the user, thereby improving the user experience.

[1908] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1909] News article summarization and sentiment engine

[1910] Step 1: Gather news articles

[1911] Input: List of news site URLs

[1912] Processing: The server uses Python's BeautifulSoup library and Scrapy framework to access news sites and retrieve the latest news articles.

[1913] Output: News article data (title, text, URL, publication date)

[1914] Specific operation: The server accesses the news site, extracts article data from the HTML document, and stores it in a database.

[1915] Step 2: Analyzing the news article

[1916] Input: The text of a news article stored in a database

[1917] Processing: The server uses BERT or GPT models to analyze the text of the news article and extract important keywords and key points.

[1918] Output: Extracted keywords and key points

[1919] Specific operation: Apply NLP model to analyze the text and temporarily save the extracted keywords and main points.

[1920] Step 3: Summarize the news article

[1921] Input: Extracted keywords and key points

[1922] Processing: The server uses the extracted information to summarize the news article in a few lines.

[1923] Output: A summarized news article

[1924] What it does: It summarizes news articles, keeping only the key points and removing redundant parts, and stores the summaries in a database.

[1925] Step 4: Emotion Recognition and Filtering

[1926] Input: User facial expression and voice data from camera and microphone

[1927] Processing: The server analyzes the user's emotions using OpenCV, Dlib, and the Google Cloud Speech-to-Text API.

[1928] Output: Perceived emotional state

[1929] Specific operation: Analyzes data collected through the camera and microphone to recognize the user's emotional state in real time.

[1930] Step 5: Filtering news articles

[1931] Input: Recognized emotional state, summarized news article

[1932] Processing: The server selects suitable news articles based on the recognized emotional state.

[1933] Output: Filtered summary news article

[1934] What it does: Select news articles that users find most appealing based on their emotional state.

[1935] Step 6: Send and display news summaries

[1936] Input: Filtered summary news articles

[1937] Processing: The server sends the summary news to the user's terminal, and the terminal displays the summary news.

[1938] Output: A summary news article displayed on the user's terminal

[1939] Specific operation: The device that receives the news article from the server displays it on the screen.

[1940] Email summary and sentiment engine

[1941] Step 1: Get email

[1942] Input: User's email account information

[1943] Processing: The device accesses the mail server (e.g., Gmail, Outlook) and retrieves one day's worth of received emails.

[1944] Output: Received email data

[1945] Specific operation: The device logs into the email account and downloads all received emails.

[1946] Step 2: Parse the email

[1947] Input: Received email data

[1948] Processing: The device sends the received email data to the server, which uses NLP algorithms to analyze the email content and extract important information.

[1949] Output: Extracted keywords and important information

[1950] What it does: The server analyzes the body of the email and extracts urgent matters and important schedules.

[1951] Step 3: Email Summary

[1952] Input: Extracted keywords and important information

[1953] Processing: The server uses this information to summarize each email.

[1954] Output: Abridged email

[1955] What it does: Summarizes emails by condensing the most important information into a few lines and removing redundant content.

[1956] Step 4: Emotion recognition and priority display

[1957] Input: User facial expression and voice data from camera and microphone

[1958] Processing: The server recognizes the user's emotional state in real time and prioritizes emails based on the emotional state.

[1959] Output: Prioritized emails

[1960] Specific operation: The server sets priorities based on the content of the email and the results of emotion recognition.

[1961] Step 5: Send and view summary emails

[1962] Input: Prioritized summary email

[1963] Processing: The server sends the summary email to the user's terminal, and the terminal displays the summary email.

[1964] Output: Summary email displayed on user terminal

[1965] Specific operation: The device that receives the email from the server displays the contents of the email on the screen.

[1966] Video summary function and emotion engine

[1967] Step 1: Get the video URL

[1968] Input: Viewing history or specified URL

[1969] Processing: The user inputs their viewing history or a specified URL into their device.

[1970] Output: The URL entered

[1971] Specific behavior: The user enters a URL into an input form on the device.

[1972] Step 2: Analyze the video data

[1973] Input: Entered URL

[1974] Processing: The device sends the specified URL to the server. The server accesses the URL and retrieves the video data. The server transcribes the audio using the Google Cloud Speech-to-Text API and analyzes the video data using FFmpeg.

[1975] Output: Transcribed audio data, analyzed video data

[1976] What it does: The server downloads the video from the URL and processes the audio and video separately.

[1977] Step 3: Summarize the video

[1978] Input: Analyzed audio and video data

[1979] Processing: The server extracts important scenes and creates a digest video of less than 5 minutes.

[1980] Output: Digest video

[1981] Specific actions: Edit important scenes into a short video.

[1982] Step 4: Emotion recognition and content adjustment

[1983] Input: User facial expression and voice data from camera and microphone

[1984] Processing: The server uses an emotion engine to recognize the user's emotional state and adjust the content of the summarized digest video.

[1985] Output: Adjusted digest video

[1986] Specific operation: The server adjusts and optimizes the content of the video according to the user's emotional state.

[1987] Step 5: Send and display summary video

[1988] Input: Adjusted digest video

[1989] Processing: The server sends the summary video to the user's terminal, and the terminal plays the summary video.

[1990] Output: Summary video displayed on the user's device

[1991] Specific operation: The device that receives the video from the server plays it.

[1992] Through these steps, the user can efficiently obtain appropriate information according to their emotional state.

[1993] (Application example 2)

[1994] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1995] In today's world, a vast amount of information is available via the Internet, making it difficult for users to efficiently obtain the information they need. Furthermore, few systems exist that provide optimal information based on the user's emotional state, and there is a need for systems that can quickly provide useful information to users in specific situations. Particularly during busy times or high-stress situations, there is a need for systems that can prioritize information that helps users relax or that is of high urgency.

[1996] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1997] In this invention, the server includes means for collecting news articles, means for analyzing the text of the collected news articles and extracting key points, means for summarizing the news articles based on the key points, means for transmitting the summarized news to a user terminal, means for displaying the summarized news on the user terminal, means for recognizing the emotional state of the user, and means for filtering the news articles based on the recognized emotional state and providing the user with the most appropriate summarized news. This makes it possible to provide the user with the most appropriate news article information according to their emotional state and improve the user's information acquisition efficiency.

[1998] A "news article" is written information about current events that is published on the Internet.

[1999] "Email" means an electronic message sent or received over the Internet.

[2000] "Video" is a digital medium that includes video and audio to convey information visually and aurally.

[2001] "Gist" refers to the main content or important parts of a news article, email, or video.

[2002] "Emotional state" refers to the user's current psychological response or mood, as recognized using the emotion engine.

[2003] "Filtering" is the process of selecting and providing relevant information based on a user's emotional state.

[2004] "Summarizing" is the act of shortening the content of a news article, email, or video, leaving out only the main points and expressing them concisely.

[2005] "Terminal" refers to a device that allows a user to receive and view information, such as a smartphone, smart glasses, or a head-mounted display.

[2006] An "emotion engine" is a technology that analyzes emotions from a user's facial expressions, voice, etc., and recognizes their emotional state.

[2007] A "user" is a person who uses an information service to efficiently retrieve and consume news articles, emails, and videos.

[2008] The present invention combines a system that collects information from three types of information sources - news articles, emails, and videos - and efficiently summarizes and provides it to users with an emotion engine that recognizes the user's emotions.

[2009] The main components of this system will be described.

[2010] News article summarization and sentiment engine

[2011] News article collection

[2012] The server accesses various news sites on the Internet and automatically collects the latest news articles. The articles are classified into categories for filtering purposes.

[2013] News article analysis

[2014] The server analyzes the text of collected news articles using natural language processing (NLP) algorithms, such as BERT or GPT models (using Hugging Face's Transformers library), to extract important keywords and key points from the articles.

[2015] News article summaries

[2016] Based on the extracted keywords and key points, the server summarizes the news article in a few lines, eliminating redundant parts and leaving only the main points.

[2017] Emotion Recognition and Filtering

[2018] The emotion engine analyzes the user's facial expressions and voice to recognize their current emotional state, using OpenFace, DeepFace, or the Microsoft Azure Emotion API.

[2019] Based on the user's emotional state, the server filters out appropriate news articles, for example, if the user is feeling stressed, it prioritizes positive news.

[2020] Sending and displaying news summaries

[2021] The summarized news is sent from the server to the user's device, where the user can easily check the news articles according to their emotions.

[2022] Email summary and sentiment engine

[2023] Get email

[2024] The device accesses the user's email account (e.g., Gmail or Outlook) and retrieves all of the emails received for the day.

[2025] Email Analysis

[2026] The captured email data (body, subject, sender information, etc.) is sent to a server and analyzed using NLP algorithms.

[2027] Email Summary

[2028] Extract key information and keywords and summarize each email in a few lines.

[2029] Emotion recognition and priority display

[2030] The emotion engine recognizes the user's emotional state and prioritizes which emails to display based on that state. For example, if the user is feeling stressed, emails with a low level of urgency will be prioritized.

[2031] Sending and viewing summary emails

[2032] The server generates a list of summarized emails and sends it to the user's device, where the user can view emails sorted by importance.

[2033] Video summary function and emotion engine

[2034] Get the video URL

[2035] The user inputs their viewing history and a specified URL into the device.

[2036] Video data analysis

[2037] The terminal transmits the specified URL to the server, and the server accesses the URL to obtain the video data.

[2038] The audio from the captured video is transcribed using ASR technology, and the video data is analyzed to identify important parts.

[2039] Video Summary

[2040] Based on the important parts, a digest video of the video is generated that is less than 5 minutes long.

[2041] Emotion recognition and content regulation

[2042] The emotion engine recognizes the user's emotional state and adjusts the content of the summary video based on that state: if the user is tired, relaxing content will be prioritized.

[2043] Sending and displaying summary videos

[2044] The summarized video is sent from the server to the user's device, allowing the user to quickly check the main content on the device.

[2045] Specific examples

[2046] Specific news article examples

[2047] For example, if a user is in an emotional state where they want to relax after work, the server will recognize the user's current emotions and prioritize providing positive articles.

[2048] Examples of prompt statements

[2049] "What news articles should be suggested if the user's emotional state is perceived as tired?"

[2050] The present invention allows users to acquire information more efficiently and provides appropriate information according to their emotional state, thereby improving the user experience and reducing stress and promoting relaxation.

[2051] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2052] Step 1:

[2053] News article collection

[2054] The server accesses various news sites on the Internet, automatically collects the latest news articles, filters them by category, and extracts the necessary news articles.

[2055] Input: A list of URLs for news sites on the Internet

[2056] Data processing: Extracting news article text using scraping technology

[2057] Output: A list of collected news articles

[2058] Step 2:

[2059] News article analysis

[2060] The server analyzes the collected news articles using natural language processing (NLP) algorithms, such as BERT and GPT models, to extract important keywords and key points from the articles.

[2061] Input: A list of collected news articles

[2062] Data processing: Analysis using NLP algorithms

[2063] Output: A list of extracted keywords and key points

[2064] Step 3:

[2065] News article summaries

[2066] The server then summarises the news article into a few lines based on the extracted keywords and key points, eliminating redundant parts and leaving only the main points.

[2067] Input: A list of extracted keywords and key points

[2068] Data processing: Compression by summarization algorithms

[2069] Output: A summarized news article

[2070] Step 4:

[2071] Recognition of emotional states

[2072] The server uses an emotion engine to recognize the user's emotional state, analyzing facial expression data and voice data sent from the user's device to identify the user's emotional state.

[2073] Input: facial expression data, voice data

[2074] Data processing: Analysis using emotion recognition algorithms

[2075] Output: User's emotional state

[2076] Step 5:

[2077] News article filtering

[2078] The server filters news articles appropriate for the user based on the user's perceived emotional state, prioritizing positive news for users who are feeling stressed.

[2079] Input: Summarized news article, user's emotional state

[2080] Data processing: Selection by filtering algorithm

[2081] Output: A list of news articles relevant to the user

[2082] Step 6:

[2083] Sending news summaries

[2084] The server transmits the summarized news to the user's terminal.

[2085] Input: A list of news articles relevant to the user

[2086] Data processing: Data transmission protocol (HTTP / Sockets)

[2087] Output: News received on the user's device

[2088] Step 7:

[2089] Viewing news summaries

[2090] The user's device displays the received news summary, allowing the user to easily check news articles according to their emotions.

[2091] Input: A summarized news article

[2092] Data processing: Display processing on the user interface

[2093] Output: User views of news articles

[2094] Next, similar steps are taken for email and video summaries and the emotion engine, but the specific steps are similar to the process for news articles: they are analyzed and summarized specifically for each source, filtered according to the user's emotional state, and then displayed on the device.

[2095] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[2096] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2097] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[2098] [Fourth embodiment]

[2099] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[2100] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[2101] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[2102] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[2103] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[2104] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[2105] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[2106] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[2107] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[2108] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[2109] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[2110] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[2111] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2112] This invention is a system that efficiently summarizes information from three types of information sources: news, email, and video, and provides it to users. To implement this system, a server that runs a program with the following functions, a terminal, and user operations are required.

[2113] News article summary function

[2114] News article collection

[2115] The server has a mechanism to access various news sites on the Internet and automatically collect the latest news articles. For example, it filters by category, such as politics, economics, and sports, and extracts the necessary news articles.

[2116] News article analysis

[2117] The server analyzes the collected news articles using natural language processing (NLP) algorithms. Using the latest NLP techniques, such as BERT and GPT models, it extracts important keywords and key points from the articles. This analysis determines the context and importance of the text and identifies the information needed for summarization.

[2118] News article summaries

[2119] Based on the results of the analysis, the server summarizes the news article in a few lines, leaving out the main points and important details of the news and eliminating redundant parts.

[2120] Sending and displaying news summaries

[2121] The server generates a document summarizing the summarized news and sends it to the user's terminal.

[2122] Users can easily check the summarized news on their devices.

[2123] Email Summary Feature

[2124] Get email

[2125] The device accesses the user's email account (e.g., Gmail or Outlook) and retrieves all of the emails received for the day.

[2126] Email Analysis

[2127] The terminal sends the acquired email data (body, subject, sender information, etc.) to the server.

[2128] Based on the data sent, the server analyzes the content of each email using natural language processing (NLP) algorithms to extract important information and keywords.

[2129] Email Summary

[2130] The server extracts important keywords and key points from each email and then summarises them into a few lines, concisely summarising important meeting schedules or urgent requests.

[2131] Sending and viewing summary emails

[2132] The server generates a list of summarized emails and sends them to the user's terminal.

[2133] The user can easily check the summarized email on the terminal.

[2134] Video summary function

[2135] Get the video URL

[2136] The user inputs their viewing history and a specified URL into the device.

[2137] Video data analysis

[2138] The terminal transmits the specified URL to the server, and the server accesses the URL to obtain the video data.

[2139] The server converts the audio of the video into text using automatic speech recognition (ASR) technology, and then analyzes the video data to extract important scenes, for example, by identifying important parts based on audio containing specific keywords or scene changes.

[2140] Video Summary

[2141] The server generates a video digest based on the extracted key points, summarizing important announcements and interesting moments into short clips.

[2142] Sending and displaying summary videos

[2143] The server transmits the summarized video file to the user's terminal.

[2144] The user can view the digest video generated on the terminal and quickly understand the main content.

[2145] Specific examples

[2146] News article summary examples

[2147] For example, if the server retrieves a "political news article" from a nationally known news site, it will analyze the text for the title "Cabinet reshuffle" using the BERT model, extracting "new cabinet members," "policy changes," "key statements," etc., and generate a summary based on this.

[2148] Example of an email summary

[2149] For example, a device retrieves an email with the subject "Meeting Schedule" from a user's email account and sends the contents to the server. The server extracts information such as the meeting date and time, names of participants, and main agenda items from the body of the email and generates a summary.

[2150] Video summary examples

[2151] For example, if a user selects the URL of a "sports highlight video" from their viewing history, the server retrieves the corresponding video and performs voice recognition and video analysis. As a result, it extracts "important goal scenes" and "interview highlights," and generates a summary clip based on these.

[2152] In this way, the present invention realizes a system that efficiently summarizes news, email, and video information and provides it to users, allowing them to quickly and concisely grasp the information they need.

[2153] The processing flow will be explained below.

[2154] News article summary function

[2155] Step 1: Gather news articles

[2156] The server accesses the specified news site and collects the URLs of the latest news articles, sometimes using the news site's API.

[2157] Step 2: Get news articles

[2158] The server retrieves HTML data from the collected URLs and converts it into a format that is easy to analyze. It extracts the news text, title, date and time, etc.

[2159] Step 3: Analyzing the news article

[2160] The server uses natural language processing (NLP) algorithms to analyze the text of retrieved news articles, using models such as BERT and GPT to extract key keywords and key points from each article.

[2161] Step 4: Summarize the news article

[2162] The server then summarises the article based on the analysis results, combining multiple algorithms to generate an optimal summary that always includes important information, such as important dates, people's names, and events.

[2163] Step 5: Submit your news summary

[2164] The server generates a document summarizing the summarized news and sends it to the user's terminal.

[2165] Step 6: Viewing News Summary

[2166] Users can easily view summaries of news on their devices, and can filter news by category and importance.

[2167] Email Summary Feature

[2168] Step 1: Get email

[2169] The device accesses the user's email account and retrieves all of the emails received in one day. It uses RestAPI or similar to retrieve data from the email server.

[2170] Step 2: Send email data

[2171] The terminal sends the acquired email data (subject, body, sender information, etc.) to the server.

[2172] Step 3: Parse the email

[2173] The server uses NLP algorithms to extract obviously important information, such as meeting schedules, project progress, and requests that require urgent attention.

[2174] Step 4: Email Summary

[2175] The server then summarises each email into a few lines based on the extracted keywords and key points, concisely summarising the key points so that the content can be understood at a glance.

[2176] Step 5: Send a summary email

[2177] The server generates a list of summarized mails and sends it to the user's terminal.

[2178] Step 6: View the summary email

[2179] Users can view summaries of emails compiled on their device, and it also includes the ability to filter out unnecessary emails and display only the important ones.

[2180] Video summary function

[2181] Step 1: Get the video URL

[2182] The user inputs the viewing history or the URL of the specified video into the device.

[2183] Step 2: Sending video data

[2184] The terminal sends the specified URL to the server.

[2185] Step 3: Getting the video

[2186] The server accesses the URL and retrieves the video data. YouTube API or Tver API is often used.

[2187] Step 4: Audio-visual analysis of the video

[2188] The server transcribes the audio from the captured video using automatic speech recognition (ASR) technology and performs scene analysis of the video to identify important parts, such as audio containing specific keywords or scene changes.

[2189] Step 5: Summarize your video

[2190] Based on the extracted key points, the server generates a digest video that summarizes the important scenes of the video in less than five minutes.

[2191] Step 6: Submit your summary video

[2192] The server transmits the digested video to the user terminal.

[2193] Step 7: Displaying the summary video

[2194] Users can play the digest video generated on their device and quickly check the main content.

[2195] The above processing steps realize a system that efficiently summarizes news, email, and video information, allowing users to quickly obtain the information they need.

[2196] Example 1

[2197] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2198] In modern society, as the amount of information increases, it is becoming increasingly difficult to efficiently grasp important information. In particular, there is a lack of methods to quickly retrieve, summarize, and present necessary information from various sources, such as news articles, emails, and videos. This forces users to manually find important parts from a vast amount of information, which is a time-consuming and labor-intensive process.

[2199] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[2200] In this invention, the server includes a means for collecting news information, a means for analyzing the collected news information using a natural language processing algorithm to extract key points, and a means for summarizing the news information in a few lines based on the key points. This enables summarization of news articles. The server also includes a means for acquiring electronic communications, a means for analyzing the content of the acquired electronic communications to extract important information, and a means for summarizing the electronic communications in a few lines based on the important information. This enables summarization of emails. The server also includes a means for acquiring URLs of video information, a means for analyzing the audio and video of the acquired video information to extract important parts, and a means for summarizing the video information into short clips based on the extracted important parts. This enables summarization of videos. These functions allow users to efficiently acquire important information from different information sources and grasp it in a short amount of time.

[2201] "News information" refers to the content and articles of news sites published on the Internet.

[2202] "Natural language processing algorithms" refers to technologies that analyze text data and evaluate context and importance. Specifically, this includes models such as BERT and GPT.

[2203] "Key points" refer to important information or keywords found in news information, electronic communications, and video information.

[2204] "Summarizing" means compressing long pieces of information into a concise summary of only the main points and important information.

[2205] "Electronic communications" refers to communications such as emails exchanged over the Internet.

[2206] "Speech recognition technology" refers to technology that converts voice data into text data. Specifically, it includes ASR (automatic speech recognition) technology.

[2207] "Video information" refers to the content and images of videos that can be viewed on the Internet.

[2208] A "URL" is an address used to specify a specific resource on the Internet.

[2209] "Analyze" refers to analyzing data or information using algorithms to find specific patterns or important elements.

[2210] "User terminal" refers to a device used by a user to receive and display information, including, but not limited to, a smartphone, tablet, or computer.

[2211] "Document" refers to a digital document, such as a text file or PDF, that compiles summarized information.

[2212] "Generate" refers to creating data or information from scratch using a computer.

[2213] "Transmitting" refers to sending data or information from one location to another over a network.

[2214] "Display" refers to visually presenting data or information so that it can be read by a user.

[2215] A "short clip" refers to a short piece of video created by cutting out important parts from a longer video.

[2216] This invention is a system that collects information from three types of information sources: news information, electronic communications, and video information, and efficiently summarizes each of them and provides them to users. To implement this system, a server that runs a program with the following functions, a terminal, and user operations are required.

[2217] News article summary function

[2218] Gathering news information

[2219] The server sets a Cron job to access a news site, for example, every day at 9:00 a.m., to collect the latest news information. The server uses the news site's API to extract the news title and text data of the news body.

[2220] News information analysis

[2221] The server analyzes the collected news information using natural language processing algorithms. Specifically, it uses generative AI models such as BERT and GPT to tokenize the text data, evaluate its context and importance, extract important keywords and key points, and store the results.

[2222] News summary

[2223] The server summarizes the news information in a few lines based on the extracted keywords and key points, concisely retaining only the main points and eliminating redundant parts.

[2224] Sending and displaying summary news information

[2225] The server generates a document containing the summarized news information and sends it to the user's terminal, for example, in PDF or HTML format.

[2226] To enable a user to check summarized news information on a terminal and efficiently grasp the latest news.

[2227] Summary function of electronic communications

[2228] Obtaining Electronic Communications

[2229] The device accesses the user's email account using the IMAP protocol and retrieves electronic communications for a specified period of time. The connection is made using a secure authentication protocol (such as OAuth).

[2230] Analysis of electronic communications

[2231] The device sends captured electronic communications to a server, which then uses generative AI models such as BERT and GPT to analyze the email data and extract key information and keywords.

[2232] Electronic Communications Summary

[2233] The server summarizes electronic communications in a few lines based on extracted keywords and key points, concisely summarizing important meeting schedules and information requiring urgent action.

[2234] Summary of electronic communication transmission and display

[2235] The server generates a document from the summarized electronic communication and transmits it to the user's terminal.

[2236] The user can view summarized electronic communications on a terminal and quickly grasp important information.

[2237] Video summary function

[2238] Get the video URL

[2239] The user enters their viewing history and a specified URL into the device, and the device then sends the URL to the server.

[2240] Video information analysis

[2241] The server accesses the specified URL and retrieves the video data. The server then downloads the correct data using the YouTube API or similar.

[2242] The server uses automatic speech recognition (ASR) technology to convert the audio in the video into text and simultaneously analyzes the video data, for example, to detect lines containing important keywords or scene changes.

[2243] Video information summary

[2244] The server generates a video digest based on the analysis results, compiling important announcements and interesting moments into short clips.

[2245] Sending and displaying summary video information

[2246] The server sends the summarized video file to the user's device, where it can be streamed using a dedicated app.

[2247] Users can watch the summary video on their device and quickly grasp the important content.

[2248] Specific examples

[2249] Examples of news summaries

[2250] For example, the server retrieves "political news information" from a domestic news site and analyzes it using the BERT model. The analysis extracts "new cabinet members," "policy changes," "major statements," etc., and generates a summary based on this.

[2251] Examples of Electronic Communications Summaries

[2252] For example, a terminal retrieves an electronic message with the subject "Meeting Schedule" from a user's email account and sends it to a server. The server extracts information such as the meeting date and time, participant names, and main agenda items from the message body and generates a summary.

[2253] Example of video summary

[2254] For example, if a user selects the URL of a "sports highlight video" from their viewing history, the server retrieves and analyzes the corresponding video. As a result of voice recognition and video analysis, it extracts "goal scenes" and "interview highlights," and generates a summary clip based on these.

[2255] Prompt Sentence Examples

[2256] "Please summarize the latest political news article."

[2257] "Please tell me the highlights of the email I received today."

[2258] "Summarize the key scenes in this sports video."

[2259] In this way, the present invention realizes a system that efficiently summarizes news information, electronic communications, and video information and provides them to users, allowing them to quickly and concisely grasp the information they need.

[2260] The flow of the identification process in the first embodiment will be described with reference to FIG.

[2261] News article summary function

[2262] Step 1: Gather news information

[2263] The server sets up a Cron job to access a news site on the Internet, for example, every day at 9:00 AM.

[2264] Input: List of news site URLs

[2265] The server accesses each URL and uses the news site's API to obtain the latest news information.

[2266] Data processing: Parse HTML to extract news titles and text data.

[2267] Output: A list of news articles in text format

[2268] Step 2: Analyzing the news information

[2269] The server analyzes the news information collected in the previous step using a natural language processing algorithm.

[2270] Input: News article list

[2271] The server uses generative AI models such as BERT or GPT to tokenize the text data and evaluate its context and importance.

[2272] Data calculation: Extract important keywords and key points.

[2273] Output: A dataset showing key keywords and key points

[2274] Step 3: Summarize the news information

[2275] The server summarizes the news information in a few lines based on the extracted keywords and key points.

[2276] Input: A dataset showing important keywords and key points

[2277] Data processing: Keep only the main points and remove redundant parts.

[2278] Output: Summary

[2279] Step 4: Send and display summary news information

[2280] The server generates a document containing summarized news information and transmits it to the user's terminal.

[2281] Input: Abstract

[2282] Data processing: Generate documents in PDF or HTML format.

[2283] Output: A summary of the news document

[2284] The user checks the summarized news information on the terminal.

[2285] Summary function of electronic communications

[2286] Step 1: Obtaining Electronic Communications

[2287] The device accesses the user's email account using the IMAP protocol.

[2288] Enter your email account information

[2289] The device captures electronic communications for a specified period of time.

[2290] Data processing: Read data from received emails.

[2291] Output: Electronic communication data

[2292] Step 2: Analyzing Electronic Communications

[2293] The terminal transmits the captured electronic communication to a server.

[2294] Input: Electronic communication data

[2295] The server analyzes the data using a generative AI model such as BERT or GPT.

[2296] Data calculations: Extracting important information and keywords.

[2297] Output: Data showing important information and keywords

[2298] Step 3: Summarizing Electronic Communications

[2299] The server summarizes the electronic communication in a few lines based on the extracted keywords and key points.

[2300] Input: Data that indicates important information or keywords

[2301] Data processing: Concisely summarize important meeting schedules and information that requires urgent action.

[2302] Output: Summary

[2303] Step 4: Sending and displaying summary electronic communications

[2304] The server generates a document from the summarized electronic communication and transmits it to the user's terminal.

[2305] Input: Abstract

[2306] Data processing: Generate documents in PDF or HTML format.

[2307] Output: A summary of the electronic communication

[2308] The user reviews the summarized electronic communication at the terminal.

[2309] Video summary function

[2310] Step 1: Get the video URL

[2311] The user enters their viewing history and a specified URL into the device, and the device then sends the URL to the server.

[2312] Input: Video URL

[2313] Output: URL data

[2314] Step 2: Analyze video information

[2315] The server accesses the specified URL and acquires the video data.

[2316] Input: URL data

[2317] The server downloads the video data using the YouTube API or similar.

[2318] Data processing: Uses automatic speech recognition (ASR) technology to convert voice into text and also analyzes video data.

[2319] Output: Analyzed text data and video data

[2320] Step 3: Summary of video information

[2321] The server generates a video digest based on the analysis results.

[2322] Input: Analyzed text data and video data

[2323] Data processing: Summarize important scenes and quotes into short clips.

[2324] Output: Summary clip

[2325] Step 4: Send and display summary video information

[2326] The server sends the summarized video file to the user's terminal.

[2327] Input: Summary clip

[2328] Data processing: Streaming playback becomes possible using a dedicated app.

[2329] Output: Summarized video file

[2330] Users can watch the summary video on their device and quickly grasp the important content.

[2331] (Application example 1)

[2332] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2333] In today's information-saturated world, users need to quickly and efficiently understand the vast amount of news articles, emails, and video information. It is particularly difficult to extract and provide users with only the most important information from each source. Furthermore, there is a lack of a way to easily summarize this information and provide it to users in an easily accessible format.

[2334] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[2335] In this invention, the server includes means for collecting news articles, means for analyzing the text of the collected news articles and extracting key points, means for summarizing the news articles based on the key points, means for sending the summarized news to a user terminal, means for displaying the summarized news on the user terminal, and means for optimizing the summaries using a generative AI model and prompt sentences, thereby enabling users to quickly grasp only the important key points from a large amount of information.

[2336] A "news article" is a report or information about a current event or topic provided through online or offline media.

[2337] "Email" is a digital message sent or received over the Internet or other network.

[2338] A "video" is a multimedia file that combines audio and video and is played continuously.

[2339] A summary is a short summary of the important elements or key points of the original information.

[2340] A "user terminal" is a device used by a user, including a smartphone, tablet, or PC.

[2341] A "generative AI model" is an artificial intelligence model that uses machine learning algorithms to generate text and data.

[2342] A "prompt" is text input to a generative AI model that takes the form of an instruction or question to the model.

[2343] A "collection method" is a method or system for obtaining data or information from a particular source.

[2344] An "analysis tool" is a method or system for processing collected data or information and extracting meaningful elements.

[2345] A "transmission means" is a method or system for sending data or information from one system to another.

[2346] A "display means" is a method or system for visually presenting information to a user, and includes devices such as a screen.

[2347] This invention is an information summarization system for efficiently providing various information to users. This system summarizes information from three types of information sources: news, email, and video, and provides it to users concisely. To implement this, a server, a terminal, and user operations are required.

[2348] News article summary function

[2349] News article collection

[2350] The server accesses various news sites to collect the latest news articles, and then filters the articles by category, such as politics, economics, or sports, to extract the necessary news articles.

[2351] News article analysis

[2352] The server analyzes the collected news articles using natural language processing (NLP) algorithms, such as the BERT model, to extract the article's context and important keywords.

[2353] News article summaries

[2354] The server then summarises the news article based on the analysis, keeping the key points and keywords and removing redundant parts.

[2355] Sending and displaying news summaries

[2356] The server sends the summarized news to the user's terminal, where the user can check the concisely summarized news.

[2357] Email Summary Feature

[2358] Get email

[2359] The terminal accesses the user's email account and retrieves received emails.

[2360] Email Analysis

[2361] The device sends the email data to the server, which then uses NLP algorithms to analyze the email body and extract important information and keywords.

[2362] Email Summary

[2363] The server then uses this extracted data to summarize the email, concisely summarizing important schedules and information requiring urgent action.

[2364] Sending and viewing summary emails

[2365] The server sends the summarized email to the user's terminal, where the user can check the summarized email.

[2366] Video summary function

[2367] Get the video URL

[2368] The user inputs their viewing history and the specified URL into the device.

[2369] Video data analysis

[2370] The device sends the URL to the server, which retrieves the corresponding video. The audio data is converted into text using automatic speech recognition (ASR), and important scenes are extracted from the video data.

[2371] Video Summary

[2372] Based on the analysis data, the server generates a summary clip of the video containing important scenes and keywords.

[2373] Sending and displaying summary videos

[2374] The server sends the summarized video to the user's device, where the user can watch the summarized video in a short time.

[2375] Hardware and Software Used

[2376] The system's main hardware consists of a server and user devices (smartphones, tablets, and PCs). The software used includes Python, the BERT model, the Transformers library, the requests library, and speech recognition technology.

[2377] Examples and prompts

[2378] As a concrete example, consider a case where a user checks a news article summary on their smartphone. For example, the following prompt sentence is input to the generative AI model:

[2379] "Summarize the article: A new cabinet has been announced, with key changes."

[2380] An example output is:

[2381] "A new cabinet has been announced, with key members being changed."

[2382] Users can view this output on their smartphones.

[2383] This system allows users to quickly grasp only the important points from a large amount of information.

[2384] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[2385] Step 1:

[2386] Collect news articles (server)

[2387] The server accesses various news sites and automatically collects the latest news articles. The input is the URL of the news site, and the output is the collected news article data. The server filters these articles by category and saves the collected data.

[2388] Step 2:

[2389] News article analysis (server)

[2390] The server analyzes the collected news article text using a natural language processing (NLP) algorithm. Specifically, it uses the BERT model to extract the article's context and important keywords. The input is the collected news article data, and the output is the analysis results: important keywords and key points.

[2391] Step 3:

[2392] News article summaries (server)

[2393] The server summarizes the news article based on the keywords and key points extracted in the previous step, eliminating redundant parts and picking out only the important information. The input is the analysis result, and the output is the summarized news article text.

[2394] Step 4:

[2395] Sending summary news (server)

[2396] The server sends the summarized news article to the user's terminal, where the input is the summarized news article text and the output is the summarized news sent to the user's terminal.

[2397] Step 5:

[2398] Displaying news summaries (terminal)

[2399] The user terminal displays the received summary news. The input is the summary news sent from the server, and the output is the news content displayed on the user interface.

[2400] Step 6:

[2401] Retrieving email (terminal)

[2402] The terminal accesses the user's email account and retrieves all received emails for one day. The input is the email account authentication information, and the output is the retrieved email data.

[2403] Step 7:

[2404] Email analysis (terminal)

[2405] The device sends the acquired email data to the server, which then uses NLP algorithms to analyze the content. The input is the email data, and the output is the extraction of important information and keywords.

[2406] Step 8:

[2407] Email Abstract (Server)

[2408] The server summarizes each email in a few lines based on the extracted important keywords and information. The input is the analysis result, and the output is the summarized email text.

[2409] Step 9:

[2410] Sending summary emails (server)

[2411] The server sends a list of summarized emails to the user terminal, where the input is the summarized email text and the output is the summarized email sent to the user terminal.

[2412] Step 10:

[2413] Display summary email (terminal)

[2414] The user terminal displays the received summary email. The input is the summary email sent from the server, and the output is the email content displayed on the user interface.

[2415] Step 11:

[2416] Get the video URL (device)

[2417] The user inputs their viewing history and the URL of the video they want to view into the device. The input is the URL of the video they specified, and the output is that URL information.

[2418] Step 12:

[2419] Video data analysis (server)

[2420] The device sends the URL of the specified video to the server. The server accesses the URL and retrieves the video data. The audio data is converted into text using speech recognition technology, and important scenes are extracted from the video data. The input is the video URL, and the output is the speech recognition results and the extraction of important scenes.

[2421] Step 13:

[2422] Video summary (server)

[2423] The server generates a video digest clip based on the extracted keypoints. The input is the speech recognition result and the extraction of important scenes, and the output is a summarized video clip.

[2424] Step 14:

[2425] Sending summary video (server)

[2426] The server sends the summarized video file to the user's terminal, where the input is the summarized video clip and the output is the summarized video file sent to the user's terminal.

[2427] Step 15:

[2428] Display summary video (device)

[2429] The user terminal plays the received summary video. The input is the summary video file sent from the server, and the output is the summary video that is played.

[2430] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[2431] This invention combines a system that collects information from three types of information sources (news, email, and video), efficiently summarizes it, and provides it to users with an emotion engine that recognizes user emotions. Implementing this system requires a server that runs a program with the following functions, a terminal, and user operation.

[2432] News article summarization and sentiment engine

[2433] News article collection

[2434] The server has a mechanism to access various news sites on the Internet and automatically collect the latest news articles. For example, it filters by category, such as politics, economics, and sports, and extracts the necessary news articles.

[2435] News article analysis

[2436] The server analyzes the collected news article text using natural language processing (NLP) algorithms, using the latest NLP techniques such as BERT and GPT models to extract important keywords and key points from the articles.

[2437] News article summaries

[2438] Based on the results of the analysis, the server summarizes the news article in a few lines, leaving out the main points and important details of the news and eliminating redundant parts.

[2439] Emotion Recognition and Filtering

[2440] The server uses an emotion engine to recognize the user's current emotional state, for example, by analyzing the user's facial expressions and voice through a camera.

[2441] The server filters news articles appropriate for the user based on the perceived emotional state: for example, if the user is feeling stressed, it prioritizes positive news articles.

[2442] Sending and displaying news summaries

[2443] The server generates a document summarizing the summarized news and sends it to the user's terminal.

[2444] Users can easily check filtered news summaries on their devices.

[2445] Email summary and sentiment engine

[2446] Get email

[2447] The device accesses the user's email account (e.g., Gmail or Outlook) and retrieves all of the emails received for the day.

[2448] Email Analysis

[2449] The terminal sends the acquired email data (body, subject, sender information, etc.) to the server.

[2450] Based on the data sent, the server analyzes the content of each email using natural language processing (NLP) algorithms to extract important information and keywords.

[2451] Email Summary

[2452] The server extracts important keywords and key points from each email and then summarises them into a few lines, concisely summarising important meeting schedules or urgent requests.

[2453] Emotion recognition and priority display

[2454] The server uses an emotion engine to recognize the user's current emotional state.

[2455] The server prioritizes which emails to display based on the user's emotional state, for example, displaying less urgent emails to a stressed user first.

[2456] Sending and viewing summary emails

[2457] The server generates a list of summarized emails and sends them to the user's terminal.

[2458] Users can view emails on their devices summarized in order of importance according to their emotions.

[2459] Video summary function and emotion engine

[2460] Get the video URL

[2461] The user inputs their viewing history or a specified URL into the device.

[2462] Video data analysis

[2463] The terminal transmits the specified URL to the server, and the server accesses the URL to obtain the video data.

[2464] The server transcribes the audio from the captured video using automatic speech recognition (ASR) technology and analyzes the video data to identify important parts, for example, based on audio containing specific keywords or scene changes.

[2465] Video Summary

[2466] Based on the extracted key points, the server generates a digest video that summarizes the important scenes of the video in less than five minutes.

[2467] Emotion recognition and content regulation

[2468] The server uses an emotion engine to recognize the user's emotional state.

[2469] The server automatically selects and tailors the summary video to suit the user based on the recognized emotional state, for example prioritizing relaxing content if the user is tired.

[2470] Sending and displaying summary videos

[2471] The server transmits the digested video to the user's terminal.

[2472] Users can play the digest video generated on their device and quickly check the main content.

[2473] Specific examples

[2474] Specific news article examples

[2475] For example, if a user is in an emotional state where they want to relax after work, the server will recognize the user's current emotions and provide a summary of positive articles preferentially, allowing the user to obtain important information while reducing stress.

[2476] Specific examples of email

[2477] For example, if a user is checking email during busy morning hours, the server will recognize through its emotion engine that the user is feeling anxious or stressed. Based on this, the server will prioritize and summarize and display emails with high urgency.

[2478] Video examples

[2479] For example, if a user is tired and wants to relax, the server will recognize the user's current emotions and prioritize and summarize relaxing videos based on that emotion. The user can watch relaxing videos in a short amount of time.

[2480] In this way, by combining emotion engines, it becomes possible to provide information that is more in line with the user's needs, improving the user experience.

[2481] The processing flow will be explained below.

[2482] News article summarization and sentiment engine

[2483] Step 1: Gather news articles

[2484] The server accesses various news sites on the Internet and collects the URLs of the latest news articles, for example, by obtaining the necessary data from RSS feeds or APIs.

[2485] Step 2: Get news articles

[2486] The server retrieves the HTML data from the collected URLs and extracts the body of the news article, the title, and the date and time.

[2487] Step 3: Analyzing the news article

[2488] The server uses natural language processing (NLP) algorithms to analyze the text of the news article, extracting key keywords and gist information and identifying important information.

[2489] Step 4: Summarize the news article

[2490] The server then summarises the news article in a few lines based on the extracted key points and keywords, concisely summarising the main points.

[2491] Step 5: Emotion Recognition

[2492] The device captures the user's facial expressions and voice through a camera and microphone, and analyzes them with an emotion engine. For example, emotions are identified from the user's facial expressions and tone of voice.

[2493] Step 6: Filtering

[2494] The server then filters appropriate news articles based on the results of the emotion engine according to the user's emotional state. For example, if the user feels like relaxing, it prioritizes positive news.

[2495] Step 7: Submit your news summary

[2496] The server generates a document summarizing the filtered news articles and sends it to the user's terminal.

[2497] Step 8: Viewing Summary News

[2498] Users can easily check filtered news summaries on their devices.

[2499] Email summary and sentiment engine

[2500] Step 1: Get email

[2501] The device accesses the user's email account and retrieves all of the emails received in one day, using an email protocol (e.g., IMAP or POP3).

[2502] Step 2: Send email data

[2503] The terminal sends the acquired email data (body, subject, sender information, etc.) to the server.

[2504] Step 3: Parse the email

[2505] The server analyzes the email content using natural language processing (NLP) algorithms to extract important information and keywords, such as meeting schedules or urgent requests.

[2506] Step 4: Email Summary

[2507] The server then summarises each email in a few lines based on the extracted key keywords and information, concisely summarising the key points.

[2508] Step 5: Emotion Recognition

[2509] The device captures the user's facial expressions and voice through a camera and microphone, and analyzes them with an emotion engine. For example, emotions are identified from the user's facial expressions and tone of voice.

[2510] Step 6: Prioritize Display

[2511] The server then prioritizes which emails to display based on the results of the emotion engine and the user's emotional state. For example, if the user is feeling stressed, emails with a low level of urgency will be displayed first.

[2512] Step 7: Send a summary email

[2513] The server generates a prioritized summary mail list based on the emotional state and transmits it to the user's terminal.

[2514] Step 8: View the summary email

[2515] The user can check the emails summarized according to priority on the terminal.

[2516] Video summary function and emotion engine

[2517] Step 1: Get the video URL

[2518] The user inputs the viewing history or the URL of the specified video into the device.

[2519] Step 2: Sending video data

[2520] The terminal sends the specified URL to the server.

[2521] Step 3: Getting the video

[2522] The server accesses the transmitted URL and acquires the video data.

[2523] Step 4: Audio-visual analysis of the video

[2524] The server transcribes the audio from the captured video using automatic speech recognition (ASR) technology and analyzes the video data to identify important parts, such as audio containing key keywords and scene changes.

[2525] Step 5: Summarize your video

[2526] The server generates a video digest based on the extracted key points, summarizing important scenes into short clips.

[2527] Step 6: Emotion Recognition

[2528] The device captures the user's facial expressions and voice through a camera and microphone, and analyzes them with an emotion engine. For example, emotions are identified from the user's facial expressions and tone of voice.

[2529] ...

Claims

1. a means of collecting news articles; A means for analyzing the text of collected news articles and extracting key points; A means of summarizing news articles based on key points; means for transmitting the summarized news to a user terminal; means for displaying a news summary on a user terminal; A system including:

2. a means for obtaining email; A means for analyzing the content of the captured email and extracting important information; A way to summarize emails based on key information; means for transmitting the summarized email to a user terminal; means for displaying the summarized email at the user terminal; The system of claim 1 , comprising:

3. A way to get the URL of a video from the viewing history, A means for analyzing the audio and video of the captured video and extracting important parts; A means for summarizing a video based on the extracted important parts; means for transmitting the summarized video to a user terminal; means for displaying the summary video on a user terminal; The system of claim 1 , comprising:

Images