system

Abstract

We provide the system. [Solution] A processing means for retrieving information related to a specific word or phrase selected by a user within an e-book from a database, A display means for providing the relevant information to the user in a pop-up display format, Processing means for summarizing the contents of a specified range, A display means for visually displaying the summary results to the user, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, and includes steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a character of the chatbot, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance as a response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Many e-book users may forget the details of characters and specific terms when reading novels or comics over a long period, and it takes time to review such information. Also, when searching the Internet to recall the content of the previous work when a sequel is released, there is a risk of spoilers. In such a situation, there is a need to provide an effective method for improving the reading experience.

Means for Solving the Problems

[0005] This invention solves the above problem by providing a means for users to select words or phrases within an e-book, thereby obtaining relevant information based on previously read sections and displaying it in a pop-up format. Furthermore, it includes a function to generate and display a summary of a section specified by the user, making it easy to recall the content even after a long period of time has passed since reading. This system provides information in a way that eliminates the risk of spoilers, and by providing an interface that allows users to flexibly set the level of detail and number of characters in the summary, it realizes reading support that users can use comfortably.

[0006] A "user" is an individual who uses the system to read ebooks.

[0007] An "eBook" is a book distributed in digital format and is typically viewed on a digital device.

[0008] "Words" refer to words or phrases found within a book.

[0009] "Selection" refers to the act of a user specifying a particular word or phrase through the device's interface.

[0010] "Information" refers to data related to the content of an e-book, including characters, terminology, or summaries of the content.

[0011] A "database" is a collection of information that systematically stores related information and manages it in a searchable format.

[0012] "Acquisition" refers to the action of accessing and obtaining necessary information.

[0013] "Pop-up display" refers to a method of showing information in a small window or box that appears on the screen.

[0014] The word "specify" refers to the act of clearly setting or selecting a specific action or condition.

[0015] "Abstract" is a concise summary of the content of an e - book, which compresses and presents the original content.

[0016] "Control means" is a device or method for managing and adjusting the operation and functions of a system.

[0017] "Interface" is the contact point or means for a user and a system to perform mutual operations and information exchanges.

Brief Description of Drawings

[0018] [Figure 1] It is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] It is a conceptual diagram showing an example of the main functions of a data processing device and a smart device according to the first embodiment. [Figure 3] It is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] It is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] It is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] It is a conceptual diagram showing an example of the main functions of a data processing device and a headset - type terminal according to the third embodiment. [Figure 7] It is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] It is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] It shows an emotion map to which a plurality of emotions are mapped. [Figure 10] It shows an emotion map to which a plurality of emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12]It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Embodiment 2 when combined with an emotion engine. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when combined with an emotion engine.

Mode for Carrying Out the Invention

[0019] Hereinafter, an example of an embodiment of the system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0020] First, the terms used in the following description will be explained.

[0021] In the following embodiments, the numbered processor (hereinafter simply referred to as "processor") may be one arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be one type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include CPU (Central Processing Unit), GPU (Graphics Processing Unit), GPGPU (General-Purpose computing on Graphics Processing Units), APU (Accelerated Processing Unit), and the like.

[0022] In the following embodiments, the numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0023] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0024] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0025] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0026] [First Embodiment]

[0027] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0028] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0029] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0030] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0031] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0032] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0033] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0034] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0035] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0036] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0037] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0038] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0039] This invention is a technology that enables users to efficiently understand and enjoy ebooks over a long period of time. This system consists of a terminal for using ebooks, a server for searching and processing information, and information display means via a user interface.

[0040] First, the content of the ebook is registered in a database as related information for each page. This information includes details about the characters, key points in the story, and explanations of specific terminology.

[0041] When a user reads an ebook and long-presses a specific word or phrase, the device detects this interaction and requests information about that word or phrase from the server. This information request also includes data on where the user performed the long-press.

[0042] The server receives the request, searches the database, and extracts relevant information. During this process, it considers the user's reading progress and performs filtering to prevent spoilers. The extracted information is sent to the device in a pop-up display format.

[0043] The device displays received information as a pop-up on the screen, allowing the user to review the relevant section. This pop-up display enables quick and concise information delivery without interrupting the reading flow.

[0044] Furthermore, users can utilize a summarization function. By operating the summarization button on their device, they request the server to generate a summary of the specified range of content. The server receives this summary generation request, compresses the specified content using natural language processing technology, and creates a summary according to the user's desired level of detail and character count. The completed summary is then provided to the user via their device.

[0045] As a concrete example, if a user reads Chapter 5 of an ebook and long-presses the term "Black King" that appears there, the device transmits this action to the server. The server collects information about "Black King" up to Chapter 5 and sends it to the device, without including spoilers. The device displays this information as a pop-up, allowing the user to immediately see details about "Black King."

[0046] This system is designed to deepen users' reading comprehension, allow them to absorb the maximum amount of information in a limited time, and improve their reading experience.

[0047] The following describes the processing flow.

[0048] Step 1:

[0049] The user long-presses on a word or phrase of interest while reading an ebook. This action is detected by the device.

[0050] Step 2:

[0051] The device collects information about the word or phrase the user long-presses and sends it to the server as a request. This request includes the text information of the long-pressed word or phrase and the context information of the page.

[0052] Step 3:

[0053] The server searches the database based on the received request. Based on the user's past reading history, it extracts relevant information about the specified words and filters it to avoid spoilers.

[0054] Step 4:

[0055] The server formats the extracted information into a pop-up display format and sends it to the terminal. This formatting includes a concise structure to allow the user to quickly understand the information.

[0056] Step 5:

[0057] The device displays information received from the server as a pop-up on the screen. Through this pop-up, the user can view details and related information about the word or phrase they long-pressed.

[0058] Step 6:

[0059] Users can optionally press a summary button to request a summary of a specific range of pages. The device detects this action and sends a summary generation request to the server.

[0060] Step 7:

[0061] The server generates a summary of the specified content using natural language processing. This summary is adjusted based on the user's settings for character count and level of detail.

[0062] Step 8:

[0063] The server sends the generated summary to the terminal.

[0064] Step 9:

[0065] The device displays the received summary to the user. This summary allows the user to easily review previously read content.

[0066] (Example 1)

[0067] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0068] When using e-books, there is a need for a system that allows users to quickly and accurately obtain relevant information about specific words or content, and to easily access that information, while also having a spoiler prevention function that adapts to the user's reading progress, and further allowing the user to adjust the level of detail and word count of the summary they specify.

[0069] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0070] In this invention, the server includes communication means for recording location information of specific words or phrases selected by the user within an e-book and retrieving information based on that information; information filtering means for the server to consider the user's reading progress and eliminate spoilers; and means for analyzing information using a generative AI model and filtering specific information. This enables the user to efficiently obtain relevant information and improve their reading experience.

[0071] An "e-book" is a book distributed in digital format and is an information medium that can be viewed using an electronic device.

[0072] A "word or phrase" is the smallest unit of language that carries meaning, and refers to words and phrases used in writing or conversation.

[0073] A "storage device" is a system that stores large amounts of data and allows for quick retrieval as needed, and includes databases and memory.

[0074] "Processing means" refers to functions or devices for executing some kind of process, and includes technical configurations for analyzing or manipulating information.

[0075] "Visual display format" refers to a method of providing information in a way that is visible to the user, and includes displays using display devices.

[0076] "Information processing means" refers to functions or devices that transform information into a form suitable for a specific purpose, and includes data summarization and transformation.

[0077] "Information filtering means" refers to functions or devices that select information based on specific conditions and remove unnecessary or harmful information.

[0078] A "generative AI model" is an algorithm or system that uses machine learning to analyze information and identify specific patterns or trends.

[0079] This invention is a system that makes it easier for users to efficiently use ebooks and obtain related information. To read ebooks, users use a compatible device. This device is equipped with a touchscreen and has the function of displaying the pages of the ebook.

[0080] When a user is reading an ebook on their device, they can long-press a specific word or phrase, allowing the device to acquire location information corresponding to that phrase. This information is then transmitted to a server via communication. The server retrieves data related to the phrase based on the information stored in the storage device. The retrieved data is then analyzed using a generative AI model. The server further monitors the user's reading progress and automatically performs necessary filtering to prevent spoilers.

[0081] The filtered information is sent to the device in a visual display format, which then displays it to the user as a pop-up. This display allows the user to access relevant information without interrupting their reading flow.

[0082] For example, if a user long-presses the phrase "Black King" in Chapter 5 of an ebook, the device transmits this action to the server. The server collects information about "Black King," removes spoilers, and provides it to the device. The device displays this information as a pop-up, allowing the user to immediately see details about "Black King."

[0083] The terminal also has a summary button, which allows the user to request the server to summarize a specified range of content. The server uses a generative AI model to summarize the specified text and provides the terminal with a summary result based on the desired level of detail and character count.

[0084] As a concrete example of a prompt, it is possible to input prompts that correspond to the information the user is looking for, such as, "Please tell me how to obtain detailed information about a specific character in Chapter 5 of the ebook by long-pressing, without giving away any spoilers."

[0085] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0086] Step 1:

[0087] The user long-presses a specific word or phrase within the ebook.

[0088] The input is the user's touch operation, and the output is the selected word or phrase and its location information. The device detects this information and uses sensors to determine the location.

[0089] Step 2:

[0090] The device sends the acquired words and location information to the server.

[0091] The input consists of the words and location information obtained in step 1, and the output is the request data sent to the server. Using a communication module, the terminal organizes the information into packets and transmits them over the network.

[0092] Step 3:

[0093] The server analyzes the received request data and retrieves relevant information from the storage device.

[0094] The input is request data sent from the terminal, and the output is a dataset of related information. The server executes a database query, filters, and extracts the relevant information.

[0095] Step 4:

[0096] The server uses a generative AI model to analyze the acquired information and perform filtering to prevent spoilers based on the user's reading progress.

[0097] The input is the information extracted in step 3, and the output is the information with spoilers removed. The generative AI model uses a text analysis algorithm to select important information.

[0098] Step 5:

[0099] The server sends filtered information to the terminal.

[0100] The input is the information filtered in step 4, and the output is the response data to the terminal. The server packets the data and sends it using the communication protocol.

[0101] Step 6:

[0102] The terminal displays information received from the server to the user in a pop-up window.

[0103] The input is response data sent from the server, and the output is visual information for the user. The terminal generates a popup window using a GUI drawing library.

[0104] Step 7:

[0105] When a user presses the summary button, the device requests a summary of the specified content from the server.

[0106] The input consists of user button presses and specified ranges, and the output is a summarized request. The terminal detects the operation, generates a request, and prepares to send it.

[0107] Step 8:

[0108] The server uses a generative AI model to summarize the content within a specified range and creates a summary that matches the user's desired level of detail and character count.

[0109] The input is a summary request, and the output is the summarized result. The generative AI model applies natural language processing techniques to summarize text and generates a summary of appropriate length and content.

[0110] Step 9:

[0111] The server sends the summary results to the terminal, and the terminal displays them to the user.

[0112] The input is the summarized result, and the output is the summarized information provided to the user. The terminal integrates the results into a GUI and presents them to the user visually.

[0113] (Application Example 1)

[0114] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0115] The problem that this invention aims to solve is enabling users to instantly understand specific information and efficiently acquire information while viewing ebooks and digital content. This is required to improve user learning efficiency and enrich the reading experience. Furthermore, it is crucial to obtain necessary information concisely within a limited time through a summarization function.

[0116] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0117] In this invention, the server includes processing means for acquiring data related to a specific item selected by the user on an electronic terminal to obtain information, visualization means for displaying the acquired data, and information analysis means for summarizing the content from the portion read by the user. This enables the user to efficiently obtain information while understanding the content without interruption.

[0118] An "electronic terminal" is a device used by users to view and manipulate information, and includes smartphones and tablets.

[0119] A "processing means" is an element within a system that has the function of acquiring, extracting, or analyzing specific information or data.

[0120] A "visualization means" is an element within a system that has a display function for showing acquired information to the user.

[0121] An "information analysis tool" is an element within a system that has the function of processing content within a range specified by the user and generating a summary.

[0122] An "interface" is a function or means that allows a user to directly interact with a system. Using this interface, users can adjust the amount of information and the display format.

[0123] The system for realizing this application is configured as follows: The user views ebooks using an electronic device such as a smartphone or tablet. When the user selects a specific item of interest while viewing, the device sends that selection information to the server.

[0124] The server includes processing mechanisms for retrieving data related to selected items. The server searches the database and extracts relevant information. This allows for the provision of appropriate information while considering the user's browsing history. Furthermore, it uses information analysis tools to generate a summary of the selection. This summary is performed using natural language processing technologies such as OpenAI's GPT model.

[0125] The terminal displays information sent from the server to the user using visualization tools. This employs a pop-up format to efficiently deliver information without disrupting the browsing flow. Furthermore, the user can adjust the amount of information and display format of the summary through the interface.

[0126] For example, when a user selects a word in the story that particularly interests them, information about its historical background and importance to the story immediately pops up. Furthermore, it's possible to download a summary of the story in advance, taking advantage of the limited time available on an airplane.

[0127] An example of a prompt to facilitate this process is, "Summarize the following text: '...(text)...' Identify three key points within the given range." By providing this prompt to the generating AI model, the information requested by the user can be efficiently obtained.

[0128] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0129] Step 1:

[0130] Users browse ebooks on their electronic devices and select specific items that interest them. This selection information is entered into the device. The device packages the selected items and their location information and sends this data to the server.

[0131] Step 2:

[0132] The server receives selection information sent from the terminal. Based on this input data, it searches the database and extracts information related to the relevant items. This information includes details and background information about the items. After data extraction, spoiler information is filtered and excluded depending on where the user is reading.

[0133] Step 3:

[0134] The server further processes the filtered results of the relevant information based on the generating AI model. If summarization is required, it uses natural language processing techniques to summarize the selected content. Specifically, it inputs prompt sentences into the generating AI model to obtain a summary of the specified range. The results are then formatted to the user's specified amount and format.

[0135] Step 4:

[0136] The server then transmits the final processed information to the terminal. The terminal uses a display mechanism to visualize this received data for the user. The information is displayed in a pop-up format, allowing for quick and uninterrupted communication of the content without disrupting the user's browsing flow.

[0137] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0138] This invention is a system for improving the user's reading experience, supporting the comprehension of e-book content and adapting information presentation according to the user's emotions. The system comprises a terminal for displaying e-books, a server for processing information, an engine for analyzing emotions, and means for presenting information via a user interface.

[0139] First, the content of the ebook is stored in a database, with detailed information about characters and terminology organized page by page. This database is used to quickly provide relevant information when a user selects a specific word or phrase.

[0140] Users can utilize a feature that allows them to retrieve information related to a specific word or phrase while reading, displaying it as a pop-up. The device detects this user action and sends a request to the server. The server receives the request, consults its database, filters the relevant information, and sends it to the device.

[0141] Furthermore, this system incorporates an emotion engine that analyzes the user's facial expressions and movements to identify their emotional state. Based on the user's emotions, the emotion engine optimizes how information is presented and summarized. For example, if the system detects that the user is confused, it adjusts to provide more detailed information to aid understanding.

[0142] For example, if a user long-presses the term "Dark King" in Chapter 4 of an ebook and then smiles, the emotion engine will determine that the user is enjoying the content. The server receives this information and displays relevant details in a pop-up to enhance the user experience. If the user shows a surprised expression, providing additional background information will be considered.

[0143] Thus, the present invention integrates information provision and emotion analysis to improve the user's reading experience, deepening their understanding of the reading content and providing information tailored to their emotions.

[0144] The following describes the processing flow.

[0145] Step 1:

[0146] The user long-presses on a word or phrase that interests them while reading an ebook. This action is detected by the device.

[0147] Step 2:

[0148] The device sends a request to the server containing information about the word or phrase the user long-pressed. This request includes the text information of the word or phrase and contextual information about the page on which the word or phrase appears.

[0149] Step 3:

[0150] The server analyzes the received request, searches the database, and extracts information related to the relevant words. During this process, a mechanism is applied to exclude spoilers based on the user's reading progress.

[0151] Step 4:

[0152] The server formats the information into a pop-up display format and sends it to the terminal. This information is summarized concisely so that the user can easily understand it.

[0153] Step 5:

[0154] The device quickly provides information about the long-pressed word or phrase by displaying the information received from the server to the user in a pop-up format.

[0155] Step 6:

[0156] The emotion engine analyzes data acquired from the device's camera and microphone to identify the user's current emotional state.

[0157] Step 7:

[0158] The server adjusts the timing and content of information presentation based on the analysis results of the emotion engine. If it determines that the user is confused, it adjusts to provide additional explanations.

[0159] Step 8:

[0160] When a user presses the summarize button, the device transmits the action to the server and requests the generation of a summary within the specified range.

[0161] Step 9:

[0162] The server uses natural language processing to summarize the content based on the specified range and adjusts it to suit the user's settings. The summary is sent to the terminal and displayed to the user.

[0163] Step 10:

[0164] The device visually presents summarized information to the user, making it easy to review past content.

[0165] This series of processes allows users to deepen their understanding of the content while reading ebooks, by providing information tailored to their emotions.

[0166] (Example 2)

[0167] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0168] For users reading electronic materials, there are challenges such as difficulty in quickly obtaining relevant information and background details, and a lack of information presentation tailored to their emotions and level of understanding. This invention aims to solve these problems and improve the user experience.

[0169] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0170] In this invention, the server includes means for retrieving relevant information from a storage device when a user selects a word or phrase in electronic material, means for presenting the retrieved information in a pop-up display format, means for analyzing the user's facial expressions and actions to identify their emotional state, and means for optimizing the information presentation method based on the emotional state. This makes it possible to present information according to the user's emotions and level of understanding.

[0171] A "user" is an individual who reads and manipulates electronic materials.

[0172] "Electronic resources" refer to a collection of information that is stored in digital format and can be viewed on electronic devices.

[0173] A "word or phrase" is a cohesive unit within electronic material that has meaning and is eligible for selection.

[0174] A "storage device" is hardware used to store and provide information in digital format.

[0175] "Popup display format" refers to a method of presenting information on the screen as a new window or box.

[0176] "Facial expressions and actions" refer to the facial expressions and body movements that a user makes in front of an electronic device.

[0177] "Emotional state" refers to the analysis results that indicate the user's psychological or emotional state.

[0178] "Optimization" is a method of adjusting information and processing according to a purpose to maximize efficiency and effectiveness.

[0179] This invention is a system for resolving information gaps and lack of emotionally relevant information provision that users may encounter while reading and understanding electronic materials more deeply. The system consists of an electronic display device, an information processing device, an emotion analysis engine, and means for presenting information through a user interface.

[0180] Users can read electronic materials on an electronic display device and obtain related information by long-pressing on words or phrases of interest. The electronic display device recognizes the selected words or phrases and transmits that information to an information processing device. This information processing device then refers to a storage device with high-speed search capabilities, extracts information related to the specific words or phrases, and sends it back to the electronic display device. Through this entire process, users can quickly access the necessary information through pop-up displays.

[0181] Furthermore, the emotion analysis engine monitors the user's facial expressions and movements to identify the user's current emotional state. Based on this information, the information processing device optimizes how information is presented according to the user's emotional state. For example, if the analysis indicates that the user is surprised, more background information will be provided; if the user is smiling and enjoying themselves, interesting content will be emphasized.

[0182] For example, when a user long-presses the term "Dark King," the information processing device will look up its historical background and immediately display details on the electronic display. If the emotion analysis engine determines that the user's facial expression indicates enjoyment, it will present additional related interesting anecdotes or academic materials. If the system determines that the user is confused, it will prioritize providing detailed explanations or easily understandable diagrams.

[0183] An example of a prompt using a generative AI model is, "Please tell me how to quickly provide relevant information about the phrase 'Dark King' contained in the following sentence." This prompt clearly demonstrates how the system can effectively provide information.

[0184] This invention significantly improves the user's reading experience through the immediacy and emotionally tailored presentation of information.

[0185] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0186] Step 1:

[0187] The user long-presses a specific word or phrase within the electronic document. This input is a user action, and the device detects this action using a sensor. Specifically, when the user selects a term such as "Dark King," its coordinates and text information are collected. The output is the detected word or phrase information.

[0188] Step 2:

[0189] The terminal sends a request to the server containing the detected phrase information. The input is the phrase information, which is the output of step 1, and the terminal transmits this to the server over the network. Specifically, the terminal uses the network interface to construct the request and forward it to the server. The output is the request received by the server.

[0190] Step 3:

[0191] Based on the received request, the server retrieves relevant information by referencing a database in its storage device. The input is the word information within the request. The specific data processing involves filtering the relevant information using a high-speed search algorithm and extracting the necessary data. The output is the relevant information data, which is then sent to the terminal.

[0192] Step 4:

[0193] The terminal prepares to present relevant information data received from the server to the user. The input is the relevant information data supplied from the server. Specific actions include generating pop-ups on the user interface and visualizing the information. The output is the displayed content presented to the user.

[0194] Step 5:

[0195] The emotion analysis engine analyzes the user's facial expressions and movements to identify their emotional state. The input consists of real-time video data and user interaction information. Machine learning algorithms are used for data calculation, estimating emotions from facial patterns and movements. The output is the identified emotional state.

[0196] Step 6:

[0197] The device adjusts the presented information based on the identified emotional state. The input consists of the emotional state obtained in step 5 and the display content from step 4. The specific actions include customizing the display content according to the emotion and presenting it in an easily understandable format. The output is the optimized information presentation.

[0198] (Application Example 2)

[0199] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0200] Information users desire personalized information on specific content or products, but current systems have a problem in that this information is not optimized according to individual emotions and circumstances. Furthermore, because information provision cannot be optimized based on emotions, there is a challenge in that the user experience is not sufficiently improved.

[0201] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0202] In this invention, the server includes processing means for obtaining information related to a specific word selected by the user within the information media from data storage means, display means for providing the information to the user in a pop-up display format, analysis means for analyzing the user's emotional state, and control means for adaptively optimizing the content of the information presented based on the analyzed emotions. This enables personalized and optimized information provision according to the user's emotional state.

[0203] "Information media" refers to platforms or devices for displaying, managing, or transmitting information in digital format.

[0204] A "word" is a string of characters within an information medium that can be selected by the user.

[0205] "Data storage means" refers to a physical or virtual device or system for storing information.

[0206] "Processing means" refers to a device or method that has the function of performing a specific operation to process information.

[0207] "Display means" refers to devices or systems for visually presenting information.

[0208] "Analysis means" refers to a process or system for analyzing data and information to derive specific results.

[0209] A "control means" is a system or device that has the function of managing and adjusting the operation of other means or devices.

[0210] An "information user" is an individual or organization that receives information through information media.

[0211] "Emotional state" refers to the emotional state expressed as a psychological or physiological response of the information user.

[0212] "Optimization" is the act of adjusting or improving something to obtain the most effective and efficient results for a particular purpose.

[0213] The system for implementing this invention primarily comprises a display device, a data processing server, and an emotion analysis engine to provide information tailored to the emotional state of the information user. First, the terminal detects a specific word selected by the information user and transmits that information to the server. The server receives this request and retrieves relevant information from the data storage means. A search algorithm is used to filter and retrieve the relevant information.

[0214] Next, the display device is equipped with an intuitive interface for providing information, visually presenting information to the user in a pop-up format. In this process, the display device analyzes the user's facial expressions using an emotion analysis engine such as Microsoft® Azure® Face API to identify their emotional state. If the analysis determines that the user is interested, relevant details and positive evaluations are prioritized. Conversely, if confusion or doubt can be read from the facial expression, detailed explanations and additional information are displayed to aid understanding.

[0215] As a concrete example, suppose an information user gazes at a product in a virtual store and smiles. In this situation, the server sends positive reviews and related product suggestions to the display device. On the other hand, if the information user shows a questioning expression, detailed product specifications will be provided. An example of a prompt message that would implement this behavior is, "Suggest detailed product information to be provided when the user looks at the product with a surprised expression."

[0216] In this way, the system provides information optimized according to the user's emotional state, delivering a personalized user experience.

[0217] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0218] Step 1:

[0219] The terminal detects a word selected by the information user. Input is the user's touch operation, and output is the data of the selected word. The terminal detects the character position and content of the selected word and prepares to send it to the next processing step.

[0220] Step 2:

[0221] The terminal sends information about the selected word to the server. The input is word data, and the output is a request to the server. The terminal packets the identification information of the selected word and sends it to the server over the network.

[0222] Step 3:

[0223] The server retrieves related information from the data storage system based on the word information it receives. The input is word information, and the output is related information. The server issues a query to the database, searches for information related to the selected word, filters it, and retrieves it.

[0224] Step 4:

[0225] The server sends the relevant information it has acquired to the terminal. The input is the relevant information, and the output is the transmission of data to the terminal. The server bundles the relevant information into data packets and sends them to the terminal via the network.

[0226] Step 5:

[0227] The terminal displays relevant information it has acquired in a pop-up window. The input is relevant information from the server, and the output is a visual display via a pop-up. The terminal generates a pop-up window on the screen and displays the relevant information to the user.

[0228] Step 6:

[0229] The device uses an emotion analysis engine to analyze the user's facial expressions. The input is a camera image, and the output is analyzed emotion data. The device captures the user's face through the camera, sends the image data to the emotion analysis API, and analyzes the emotional state.

[0230] Step 7:

[0231] The server optimizes the information provided based on sentiment data. The input is sentiment data, and the output is optimized information. Based on the sentiment analysis results, the server determines which information to emphasize and customizes the content.

[0232] Step 8:

[0233] The device redisplays optimized information to improve the user experience. The input is optimized information, and the output is a customized visual display. The device receives optimized information from the server and displays it on the screen to present it in a way that is easy for the user to understand and engage.

[0234] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0235] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0236] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0237] [Second Embodiment]

[0238] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0239] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0240] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0241] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0242] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0243] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0244] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0245] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0246] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0247] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0248] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0249] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0250] This invention is a technology that enables users to efficiently understand and enjoy ebooks over a long period of time. This system consists of a terminal for using ebooks, a server for searching and processing information, and information display means via a user interface.

[0251] First, the content of the ebook is registered in a database as related information for each page. This information includes details about the characters, key points in the story, and explanations of specific terminology.

[0252] When a user reads an ebook and long-presses a specific word or phrase, the device detects this interaction and requests information about that word or phrase from the server. This information request also includes data on where the user performed the long-press.

[0253] The server receives the request, searches the database, and extracts relevant information. During this process, it considers the user's reading progress and performs filtering to prevent spoilers. The extracted information is sent to the device in a pop-up display format.

[0254] The device displays received information as a pop-up on the screen, allowing the user to review the relevant section. This pop-up display enables quick and concise information delivery without interrupting the reading flow.

[0255] Furthermore, users can utilize a summarization function. By operating the summarization button on their device, they request the server to generate a summary of the specified range of content. The server receives this summary generation request, compresses the specified content using natural language processing technology, and creates a summary according to the user's desired level of detail and character count. The completed summary is then provided to the user via their device.

[0256] As a concrete example, if a user reads Chapter 5 of an ebook and long-presses the term "Black King" that appears there, the device transmits this action to the server. The server collects information about "Black King" up to Chapter 5 and sends it to the device, without including spoilers. The device displays this information as a pop-up, allowing the user to immediately see details about "Black King."

[0257] This system is designed to deepen users' reading comprehension, allow them to absorb the maximum amount of information in a limited time, and improve their reading experience.

[0258] The following describes the processing flow.

[0259] Step 1:

[0260] The user long-presses on a word or phrase of interest while reading an ebook. This action is detected by the device.

[0261] Step 2:

[0262] The device collects information about the word or phrase the user long-presses and sends it to the server as a request. This request includes the text information of the long-pressed word or phrase and the context information of the page.

[0263] Step 3:

[0264] The server searches the database based on the received request. Based on the user's past reading history, it extracts relevant information about the specified words and filters it to avoid spoilers.

[0265] Step 4:

[0266] The server formats the extracted information into a pop-up display format and sends it to the terminal. This formatting includes a concise structure to allow the user to quickly understand the information.

[0267] Step 5:

[0268] The device displays information received from the server as a pop-up on the screen. Through this pop-up, the user can view details and related information about the word or phrase they long-pressed.

[0269] Step 6:

[0270] Users can optionally press a summary button to request a summary of a specific range of pages. The device detects this action and sends a summary generation request to the server.

[0271] Step 7:

[0272] The server generates a summary of the specified content using natural language processing. This summary is adjusted based on the user's settings for character count and level of detail.

[0273] Step 8:

[0274] The server sends the generated summary to the terminal.

[0275] Step 9:

[0276] The device displays the received summary to the user. This summary allows the user to easily review previously read content.

[0277] (Example 1)

[0278] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0279] When using e-books, there is a need for a system that allows users to quickly and accurately obtain relevant information about specific words or content, and to easily access that information, while also having a spoiler prevention function that adapts to the user's reading progress, and further allowing the user to adjust the level of detail and word count of the summary they specify.

[0280] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0281] In this invention, the server includes communication means for recording location information of specific words or phrases selected by the user within an e-book and retrieving information based on that information; information filtering means for the server to consider the user's reading progress and eliminate spoilers; and means for analyzing information using a generative AI model and filtering specific information. This enables the user to efficiently obtain relevant information and improve their reading experience.

[0282] An "e-book" refers to a book distributed in digital format and is an information medium that can be viewed using an electronic device.

[0283] A "phrase" refers to the smallest unit with meaning in a language and is a word or phrase used in a sentence or conversation.

[0284] A "storage device" is a system that stores a large amount of data and can retrieve it quickly as needed, including databases and memories.

[0285] A "processing means" refers to the function or device for executing a certain process and refers to the technical configuration for analyzing and operating information.

[0286] A "visual display form" refers to a method of providing information in a visible form to the user's eyes and includes displays using display devices.

[0287] An "information processing means" refers to the function or device for converting information into a form suitable for a specific purpose and includes summarizing and converting data.

[0288] An "information filtering means" refers to the function or device for selecting information based on specific conditions and removing unnecessary or harmful information.

[0289] A "generative AI model" refers to an algorithm or system for analyzing information by machine learning and identifying specific patterns and trends.

[0290] This invention is a system that enables users to efficiently utilize e-books and easily obtain related information. To read an e-book, a user uses a corresponding terminal. This terminal has a touch screen and the function of displaying the pages of an e-book.

[0291] When a user is reading an ebook on their device, they can long-press a specific word or phrase, allowing the device to acquire location information corresponding to that phrase. This information is then transmitted to a server via communication. The server retrieves data related to the phrase based on the information stored in the storage device. The retrieved data is then analyzed using a generative AI model. The server further monitors the user's reading progress and automatically performs necessary filtering to prevent spoilers.

[0292] The filtered information is sent to the device in a visual display format, which then displays it to the user as a pop-up. This display allows the user to access relevant information without interrupting their reading flow.

[0293] For example, if a user long-presses the phrase "Black King" in Chapter 5 of an ebook, the device transmits this action to the server. The server collects information about "Black King," removes spoilers, and provides it to the device. The device displays this information as a pop-up, allowing the user to immediately see details about "Black King."

[0294] The terminal also has a summary button, which allows the user to request the server to summarize a specified range of content. The server uses a generative AI model to summarize the specified text and provides the terminal with a summary result based on the desired level of detail and character count.

[0295] As a concrete example of a prompt, it is possible to input prompts that correspond to the information the user is looking for, such as, "Please tell me how to obtain detailed information about a specific character in Chapter 5 of the ebook by long-pressing, without giving away any spoilers."

[0296] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0297] Step 1:

[0298] The user long - presses a specific phrase within the e - book.

[0299] The input is the user's touch operation, and the output is the selected phrase and its position information. The terminal detects this information and uses a sensor to identify the position information.

[0300] Step 2:

[0301] The terminal sends the obtained phrase and position information to the server.

[0302] The input is the phrase and position information obtained in Step 1, and the output is the request data to the server. Using the communication module, the terminal constructs the information as packets and transmits it via the network.

[0303] Step 3:

[0304] The server analyzes the received request data and obtains relevant information from the storage device.

[0305] The input is the request data sent from the terminal, and the output is a dataset of relevant information. The server executes a database query, filters and extracts the corresponding information.

[0306] Step 4:

[0307] The server uses a generative AI model to analyze the obtained information and perform filtering to prevent spoilers according to the user's reading progress.

[0308] The input is the information extracted in Step 3, and the output is the information with spoilers excluded. The generative AI model utilizes text analysis algorithms to select important information.

[0309] Step 5:

[0310] The server sends the filtered information to the terminal.

[0311] The input is the information filtered in step 4, and the output is the response data to the terminal. The server packets the data and sends it using the communication protocol.

[0312] Step 6:

[0313] The terminal displays information received from the server to the user in a pop-up window.

[0314] The input is response data sent from the server, and the output is visual information for the user. The terminal generates a popup window using a GUI drawing library.

[0315] Step 7:

[0316] When a user presses the summary button, the device requests a summary of the specified content from the server.

[0317] The input consists of user button presses and specified ranges, and the output is a summarized request. The terminal detects the operation, generates a request, and prepares to send it.

[0318] Step 8:

[0319] The server uses a generative AI model to summarize the content within a specified range and creates a summary that matches the user's desired level of detail and character count.

[0320] The input is a summary request, and the output is the summarized result. The generative AI model applies natural language processing techniques to summarize text and generates a summary of appropriate length and content.

[0321] Step 9:

[0322] The server sends the summary results to the terminal, and the terminal displays them to the user.

[0323] The input is the summarized result, and the output is the summarized information provided to the user. The terminal integrates the results into a GUI and presents them to the user visually.

[0324] (Application Example 1)

[0325] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0326] The problem that this invention aims to solve is enabling users to instantly understand specific information and efficiently acquire information while viewing ebooks and digital content. This is required to improve user learning efficiency and enrich the reading experience. Furthermore, it is crucial to obtain necessary information concisely within a limited time through a summarization function.

[0327] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0328] In this invention, the server includes processing means for acquiring data related to a specific item selected by the user on an electronic terminal to obtain information, visualization means for displaying the acquired data, and information analysis means for summarizing the content from the portion read by the user. This enables the user to efficiently obtain information while understanding the content without interruption.

[0329] An "electronic terminal" is a device used by users to view and manipulate information, and includes smartphones and tablets.

[0330] A "processing means" is an element within a system that has the function of acquiring, extracting, or analyzing specific information or data.

[0331] A "visualization means" is an element within a system that has a display function for showing acquired information to the user.

[0332] An "information analysis tool" is an element within a system that has the function of processing content within a range specified by the user and generating a summary.

[0333] An "interface" is a function or means that allows a user to directly interact with a system. Using this interface, users can adjust the amount of information and the display format.

[0334] The system for realizing this application is configured as follows: The user views ebooks using an electronic device such as a smartphone or tablet. When the user selects a specific item of interest while viewing, the device sends that selection information to the server.

[0335] The server includes processing mechanisms for retrieving data related to selected items. The server searches the database and extracts relevant information. This allows for the provision of appropriate information while considering the user's browsing history. Furthermore, it uses information analysis tools to generate a summary of the selection. This summary is performed using natural language processing technologies such as OpenAI's GPT model.

[0336] The terminal displays information sent from the server to the user using visualization tools. This employs a pop-up format to efficiently deliver information without disrupting the browsing flow. Furthermore, the user can adjust the amount of information and display format of the summary through the interface.

[0337] For example, when a user selects a word in the story that particularly interests them, information about its historical background and importance to the story immediately pops up. Furthermore, it's possible to download a summary of the story in advance, taking advantage of the limited time available on an airplane.

[0338] An example of a prompt to facilitate this process is, "Summarize the following text: '...(text)...' Identify three key points within the given range." By providing this prompt to the generating AI model, the information requested by the user can be efficiently obtained.

[0339] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0340] Step 1:

[0341] Users browse ebooks on their electronic devices and select specific items that interest them. This selection information is entered into the device. The device packages the selected items and their location information and sends this data to the server.

[0342] Step 2:

[0343] The server receives selection information sent from the terminal. Based on this input data, it searches the database and extracts information related to the relevant items. This information includes details and background information about the items. After data extraction, spoiler information is filtered and excluded depending on where the user is reading.

[0344] Step 3:

[0345] The server further processes the filtered results of the relevant information based on the generating AI model. If summarization is required, it uses natural language processing techniques to summarize the selected content. Specifically, it inputs prompt sentences into the generating AI model to obtain a summary of the specified range. The results are then formatted to the user's specified amount and format.

[0346] Step 4:

[0347] The server then transmits the final processed information to the terminal. The terminal uses a display mechanism to visualize this received data for the user. The information is displayed in a pop-up format, allowing for quick and uninterrupted communication of the content without disrupting the user's browsing flow.

[0348] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0349] This invention is a system for improving the user's reading experience, supporting the comprehension of e-book content and adapting information presentation according to the user's emotions. The system comprises a terminal for displaying e-books, a server for processing information, an engine for analyzing emotions, and means for presenting information via a user interface.

[0350] First, the content of the ebook is stored in a database, with detailed information about characters and terminology organized page by page. This database is used to quickly provide relevant information when a user selects a specific word or phrase.

[0351] Users can utilize a feature that allows them to retrieve information related to a specific word or phrase while reading, displaying it as a pop-up. The device detects this user action and sends a request to the server. The server receives the request, consults its database, filters the relevant information, and sends it to the device.

[0352] Furthermore, this system incorporates an emotion engine that analyzes the user's facial expressions and movements to identify their emotional state. Based on the user's emotions, the emotion engine optimizes how information is presented and summarized. For example, if the system detects that the user is confused, it adjusts to provide more detailed information to aid understanding.

[0353] For example, if a user long-presses the term "Dark King" in Chapter 4 of an ebook and then smiles, the emotion engine will determine that the user is enjoying the content. The server receives this information and displays relevant details in a pop-up to enhance the user experience. If the user shows a surprised expression, providing additional background information will be considered.

[0354] Thus, the present invention integrates information provision and emotion analysis to improve the user's reading experience, deepening their understanding of the reading content and providing information tailored to their emotions.

[0355] The following describes the processing flow.

[0356] Step 1:

[0357] The user long-presses on a word or phrase that interests them while reading an ebook. This action is detected by the device.

[0358] Step 2:

[0359] The device sends a request to the server containing information about the word or phrase the user long-pressed. This request includes the text information of the word or phrase and contextual information about the page on which the word or phrase appears.

[0360] Step 3:

[0361] The server analyzes the received request, searches the database, and extracts information related to the relevant words. During this process, a mechanism is applied to exclude spoilers based on the user's reading progress.

[0362] Step 4:

[0363] The server formats the information into a pop-up display format and sends it to the terminal. This information is summarized concisely so that the user can easily understand it.

[0364] Step 5:

[0365] The device quickly provides information about the long-pressed word or phrase by displaying the information received from the server to the user in a pop-up format.

[0366] Step 6:

[0367] The emotion engine analyzes data acquired from the device's camera and microphone to identify the user's current emotional state.

[0368] Step 7:

[0369] The server adjusts the timing and content of information presentation based on the analysis results of the emotion engine. If it determines that the user is confused, it adjusts to provide additional explanations.

[0370] Step 8:

[0371] When a user presses the summarize button, the device transmits the action to the server and requests the generation of a summary within the specified range.

[0372] Step 9:

[0373] The server uses natural language processing to summarize the content based on the specified range and adjusts it to suit the user's settings. The summary is sent to the terminal and displayed to the user.

[0374] Step 10:

[0375] The device visually presents summarized information to the user, making it easy to review past content.

[0376] This series of processes allows users to deepen their understanding of the content while reading ebooks, by providing information tailored to their emotions.

[0377] (Example 2)

[0378] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0379] For users reading electronic materials, there are challenges such as difficulty in quickly obtaining relevant information and background details, and a lack of information presentation tailored to their emotions and level of understanding. This invention aims to solve these problems and improve the user experience.

[0380] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0381] In this invention, the server includes means for retrieving relevant information from a storage device when a user selects a word or phrase in electronic material, means for presenting the retrieved information in a pop-up display format, means for analyzing the user's facial expressions and actions to identify their emotional state, and means for optimizing the information presentation method based on the emotional state. This makes it possible to present information according to the user's emotions and level of understanding.

[0382] A "user" is an individual who reads and manipulates electronic materials.

[0383] "Electronic resources" refer to a collection of information that is stored in digital format and can be viewed on electronic devices.

[0384] A "word or phrase" is a cohesive unit within electronic material that has meaning and is eligible for selection.

[0385] A "storage device" is hardware used to store and provide information in digital format.

[0386] "Popup display format" refers to a method of presenting information on the screen as a new window or box.

[0387] "Facial expressions and actions" refer to the facial expressions and body movements that a user makes in front of an electronic device.

[0388] "Emotional state" refers to the analysis results that indicate the user's psychological or emotional state.

[0389] "Optimization" is a method of adjusting information and processing according to a purpose to maximize efficiency and effectiveness.

[0390] This invention is a system for resolving information gaps and lack of emotionally relevant information provision that users may encounter while reading and understanding electronic materials more deeply. The system consists of an electronic display device, an information processing device, an emotion analysis engine, and means for presenting information through a user interface.

[0391] Users can read electronic materials on an electronic display device and obtain related information by long-pressing on words or phrases of interest. The electronic display device recognizes the selected words or phrases and transmits that information to an information processing device. This information processing device then refers to a storage device with high-speed search capabilities, extracts information related to the specific words or phrases, and sends it back to the electronic display device. Through this entire process, users can quickly access the necessary information through pop-up displays.

[0392] Furthermore, the emotion analysis engine monitors the user's facial expressions and movements to identify the user's current emotional state. Based on this information, the information processing device optimizes how information is presented according to the user's emotional state. For example, if the analysis indicates that the user is surprised, more background information will be provided; if the user is smiling and enjoying themselves, interesting content will be emphasized.

[0393] For example, when a user long-presses the term "Dark King," the information processing device will look up its historical background and immediately display details on the electronic display. If the emotion analysis engine determines that the user's facial expression indicates enjoyment, it will present additional related interesting anecdotes or academic materials. If the system determines that the user is confused, it will prioritize providing detailed explanations or easily understandable diagrams.

[0394] An example of a prompt using a generative AI model is, "Please tell me how to quickly provide relevant information about the phrase 'Dark King' contained in the following sentence." This prompt clearly demonstrates how the system can effectively provide information.

[0395] This invention significantly improves the user's reading experience through the immediacy and emotionally tailored presentation of information.

[0396] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0397] Step 1:

[0398] The user long-presses a specific word or phrase within the electronic document. This input is a user action, and the device detects this action using a sensor. Specifically, when the user selects a term such as "Dark King," its coordinates and text information are collected. The output is the detected word or phrase information.

[0399] Step 2:

[0400] The terminal sends a request to the server containing the detected phrase information. The input is the phrase information, which is the output of step 1, and the terminal transmits this to the server over the network. Specifically, the terminal uses the network interface to construct the request and forward it to the server. The output is the request received by the server.

[0401] Step 3:

[0402] Based on the received request, the server retrieves relevant information by referencing a database in its storage device. The input is the word information within the request. The specific data processing involves filtering the relevant information using a high-speed search algorithm and extracting the necessary data. The output is the relevant information data, which is then sent to the terminal.

[0403] Step 4:

[0404] The terminal prepares to present relevant information data received from the server to the user. The input is the relevant information data supplied from the server. Specific actions include generating pop-ups on the user interface and visualizing the information. The output is the displayed content presented to the user.

[0405] Step 5:

[0406] The emotion analysis engine analyzes the user's facial expressions and movements to identify their emotional state. The input consists of real-time video data and user interaction information. Machine learning algorithms are used for data calculation, estimating emotions from facial patterns and movements. The output is the identified emotional state.

[0407] Step 6:

[0408] The device adjusts the presented information based on the identified emotional state. The input consists of the emotional state obtained in step 5 and the display content from step 4. The specific actions include customizing the display content according to the emotion and presenting it in an easily understandable format. The output is the optimized information presentation.

[0409] (Application Example 2)

[0410] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0411] Information users desire personalized information on specific content or products, but current systems have a problem in that this information is not optimized according to individual emotions and circumstances. Furthermore, because information provision cannot be optimized based on emotions, there is a challenge in that the user experience is not sufficiently improved.

[0412] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0413] In this invention, the server includes processing means for obtaining information related to a specific word selected by the user within the information media from data storage means, display means for providing the information to the user in a pop-up display format, analysis means for analyzing the user's emotional state, and control means for adaptively optimizing the content of the information presented based on the analyzed emotions. This enables personalized and optimized information provision according to the user's emotional state.

[0414] "Information media" refers to platforms or devices for displaying, managing, or transmitting information in digital format.

[0415] A "word" is a string of characters within an information medium that can be selected by the user.

[0416] "Data storage means" refers to a physical or virtual device or system for storing information.

[0417] "Processing means" refers to a device or method that has the function of performing a specific operation to process information.

[0418] "Display means" refers to devices or systems for visually presenting information.

[0419] "Analysis means" refers to a process or system for analyzing data and information to derive specific results.

[0420] A "control means" is a system or device that has the function of managing and adjusting the operation of other means or devices.

[0421] An "information user" is an individual or organization that receives information through information media.

[0422] "Emotional state" refers to the emotional state expressed as a psychological or physiological response of the information user.

[0423] "Optimization" is the act of adjusting or improving something to obtain the most effective and efficient results for a particular purpose.

[0424] The system for implementing this invention primarily comprises a display device, a data processing server, and an emotion analysis engine to provide information tailored to the emotional state of the information user. First, the terminal detects a specific word selected by the information user and transmits that information to the server. The server receives this request and retrieves relevant information from the data storage means. A search algorithm is used to filter and retrieve the relevant information.

[0425] Next, the display device is equipped with an intuitive interface for providing information, visually presenting information to the user in a pop-up format. In this process, the display device uses an emotion analysis engine, such as the Microsoft Azure Face API, to analyze the user's facial expressions and identify their emotional state. If the analysis indicates the user is interested, relevant details and positive evaluations are prioritized. Conversely, if confusion or doubt can be read from the facial expression, detailed explanations and additional information are displayed to aid understanding.

[0426] As a concrete example, suppose an information user gazes at a product in a virtual store and smiles. In this situation, the server sends positive reviews and related product suggestions to the display device. On the other hand, if the information user shows a questioning expression, detailed product specifications will be provided. An example of a prompt message that would implement this behavior is, "Suggest detailed product information to be provided when the user looks at the product with a surprised expression."

[0427] In this way, the system provides information optimized according to the user's emotional state, delivering a personalized user experience.

[0428] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0429] Step 1:

[0430] The terminal detects a word selected by the information user. Input is the user's touch operation, and output is the data of the selected word. The terminal detects the character position and content of the selected word and prepares to send it to the next processing step.

[0431] Step 2:

[0432] The terminal sends information about the selected word to the server. The input is word data, and the output is a request to the server. The terminal packets the identification information of the selected word and sends it to the server over the network.

[0433] Step 3:

[0434] The server retrieves related information from the data storage system based on the word information it receives. The input is word information, and the output is related information. The server issues a query to the database, searches for information related to the selected word, filters it, and retrieves it.

[0435] Step 4:

[0436] The server sends the relevant information it has acquired to the terminal. The input is the relevant information, and the output is the transmission of data to the terminal. The server bundles the relevant information into data packets and sends them to the terminal via the network.

[0437] Step 5:

[0438] The terminal displays relevant information it has acquired in a pop-up window. The input is relevant information from the server, and the output is a visual display via a pop-up. The terminal generates a pop-up window on the screen and displays the relevant information to the user.

[0439] Step 6:

[0440] The device uses an emotion analysis engine to analyze the user's facial expressions. The input is a camera image, and the output is analyzed emotion data. The device captures the user's face through the camera, sends the image data to the emotion analysis API, and analyzes the emotional state.

[0441] Step 7:

[0442] The server optimizes the information provided based on sentiment data. The input is sentiment data, and the output is optimized information. Based on the sentiment analysis results, the server determines which information to emphasize and customizes the content.

[0443] Step 8:

[0444] The device redisplays optimized information to improve the user experience. The input is optimized information, and the output is a customized visual display. The device receives optimized information from the server and displays it on the screen to present it in a way that is easy for the user to understand and engage.

[0445] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0446] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0447] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0448] [Third Embodiment]

[0449] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0450] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0451] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0452] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0453] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0454] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0455] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0456] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0457] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0458] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0459] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0460] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0461] This invention is a technology that enables users to efficiently understand and enjoy ebooks over a long period of time. This system consists of a terminal for using ebooks, a server for searching and processing information, and information display means via a user interface.

[0462] First, the content of the ebook is registered in a database as related information for each page. This information includes details about the characters, key points in the story, and explanations of specific terminology.

[0463] When a user reads an ebook and long-presses a specific word or phrase, the device detects this interaction and requests information about that word or phrase from the server. This information request also includes data on where the user performed the long-press.

[0464] The server receives the request, searches the database, and extracts relevant information. During this process, it considers the user's reading progress and performs filtering to prevent spoilers. The extracted information is sent to the device in a pop-up display format.

[0465] The device displays received information as a pop-up on the screen, allowing the user to review the relevant section. This pop-up display enables quick and concise information delivery without interrupting the reading flow.

[0466] Furthermore, users can utilize a summarization function. By operating the summarization button on their device, they request the server to generate a summary of the specified range of content. The server receives this summary generation request, compresses the specified content using natural language processing technology, and creates a summary according to the user's desired level of detail and character count. The completed summary is then provided to the user via their device.

[0467] As a concrete example, if a user reads Chapter 5 of an ebook and long-presses the term "Black King" that appears there, the device transmits this action to the server. The server collects information about "Black King" up to Chapter 5 and sends it to the device, without including spoilers. The device displays this information as a pop-up, allowing the user to immediately see details about "Black King."

[0468] This system is designed to deepen users' reading comprehension, allow them to absorb the maximum amount of information in a limited time, and improve their reading experience.

[0469] The following describes the processing flow.

[0470] Step 1:

[0471] The user long-presses on a word or phrase of interest while reading an ebook. This action is detected by the device.

[0472] Step 2:

[0473] The device collects information about the word or phrase the user long-presses and sends it to the server as a request. This request includes the text information of the long-pressed word or phrase and the context information of the page.

[0474] Step 3:

[0475] The server searches the database based on the received request. Based on the user's past reading history, it extracts relevant information about the specified words and filters it to avoid spoilers.

[0476] Step 4:

[0477] The server formats the extracted information into a pop-up display format and sends it to the terminal. This formatting includes a concise structure to allow the user to quickly understand the information.

[0478] Step 5:

[0479] The device displays information received from the server as a pop-up on the screen. Through this pop-up, the user can view details and related information about the word or phrase they long-pressed.

[0480] Step 6:

[0481] Users can optionally press a summary button to request a summary of a specific range of pages. The device detects this action and sends a summary generation request to the server.

[0482] Step 7:

[0483] The server generates a summary of the specified content using natural language processing. This summary is adjusted based on the user's settings for character count and level of detail.

[0484] Step 8:

[0485] The server sends the generated summary to the terminal.

[0486] Step 9:

[0487] The device displays the received summary to the user. This summary allows the user to easily review previously read content.

[0488] (Example 1)

[0489] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0490] When using e-books, there is a need for a system that allows users to quickly and accurately obtain relevant information about specific words or content, and to easily access that information, while also having a spoiler prevention function that adapts to the user's reading progress, and further allowing the user to adjust the level of detail and word count of the summary they specify.

[0491] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0492] In this invention, the server includes communication means for recording location information of specific words or phrases selected by the user within an e-book and retrieving information based on that information; information filtering means for the server to consider the user's reading progress and eliminate spoilers; and means for analyzing information using a generative AI model and filtering specific information. This enables the user to efficiently obtain relevant information and improve their reading experience.

[0493] An "e-book" is a book distributed in digital format and is an information medium that can be viewed using an electronic device.

[0494] A "word or phrase" is the smallest unit of language that carries meaning, and refers to words and phrases used in writing or conversation.

[0495] A "storage device" is a system that stores large amounts of data and allows for quick retrieval as needed, and includes databases and memory.

[0496] "Processing means" refers to functions or devices for executing some kind of process, and includes technical configurations for analyzing or manipulating information.

[0497] "Visual display format" refers to a method of providing information in a way that is visible to the user, and includes displays using display devices.

[0498] "Information processing means" refers to functions or devices that transform information into a form suitable for a specific purpose, and includes data summarization and transformation.

[0499] "Information filtering means" refers to functions or devices that select information based on specific conditions and remove unnecessary or harmful information.

[0500] A "generative AI model" is an algorithm or system that uses machine learning to analyze information and identify specific patterns or trends.

[0501] This invention is a system that makes it easier for users to efficiently use ebooks and obtain related information. To read ebooks, users use a compatible device. This device is equipped with a touchscreen and has the function of displaying the pages of the ebook.

[0502] When a user is reading an ebook on their device, they can long-press a specific word or phrase, allowing the device to acquire location information corresponding to that phrase. This information is then transmitted to a server via communication. The server retrieves data related to the phrase based on the information stored in the storage device. The retrieved data is then analyzed using a generative AI model. The server further monitors the user's reading progress and automatically performs necessary filtering to prevent spoilers.

[0503] The filtered information is sent to the device in a visual display format, which then displays it to the user as a pop-up. This display allows the user to access relevant information without interrupting their reading flow.

[0504] For example, if a user long-presses the phrase "Black King" in Chapter 5 of an ebook, the device transmits this action to the server. The server collects information about "Black King," removes spoilers, and provides it to the device. The device displays this information as a pop-up, allowing the user to immediately see details about "Black King."

[0505] The terminal also has a summary button, which allows the user to request the server to summarize a specified range of content. The server uses a generative AI model to summarize the specified text and provides the terminal with a summary result based on the desired level of detail and character count.

[0506] As a concrete example of a prompt, it is possible to input prompts that correspond to the information the user is looking for, such as, "Please tell me how to obtain detailed information about a specific character in Chapter 5 of the ebook by long-pressing, without giving away any spoilers."

[0507] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0508] Step 1:

[0509] The user long-presses a specific word or phrase within the ebook.

[0510] The input is the user's touch operation, and the output is the selected word or phrase and its location information. The device detects this information and uses sensors to determine the location.

[0511] Step 2:

[0512] The device sends the acquired words and location information to the server.

[0513] The input consists of the words and location information obtained in step 1, and the output is the request data sent to the server. Using a communication module, the terminal organizes the information into packets and transmits them over the network.

[0514] Step 3:

[0515] The server analyzes the received request data and retrieves relevant information from the storage device.

[0516] The input is request data sent from the terminal, and the output is a dataset of related information. The server executes a database query, filters, and extracts the relevant information.

[0517] Step 4:

[0518] The server uses a generative AI model to analyze the acquired information and perform filtering to prevent spoilers based on the user's reading progress.

[0519] The input is the information extracted in step 3, and the output is the information with spoilers removed. The generative AI model uses a text analysis algorithm to select important information.

[0520] Step 5:

[0521] The server sends filtered information to the terminal.

[0522] The input is the information filtered in step 4, and the output is the response data to the terminal. The server packets the data and sends it using the communication protocol.

[0523] Step 6:

[0524] The terminal displays information received from the server to the user in a pop-up window.

[0525] The input is response data sent from the server, and the output is visual information for the user. The terminal generates a popup window using a GUI drawing library.

[0526] Step 7:

[0527] When a user presses the summary button, the device requests a summary of the specified content from the server.

[0528] The input consists of user button presses and specified ranges, and the output is a summarized request. The terminal detects the operation, generates a request, and prepares to send it.

[0529] Step 8:

[0530] The server uses a generative AI model to summarize the content within a specified range and creates a summary that matches the user's desired level of detail and character count.

[0531] The input is a summary request, and the output is the summarized result. The generative AI model applies natural language processing techniques to summarize text and generates a summary of appropriate length and content.

[0532] Step 9:

[0533] The server sends the summary results to the terminal, and the terminal displays them to the user.

[0534] The input is the summarized result, and the output is the summarized information provided to the user. The terminal integrates the results into a GUI and presents them to the user visually.

[0535] (Application Example 1)

[0536] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0537] The problem that this invention aims to solve is enabling users to instantly understand specific information and efficiently acquire information while viewing ebooks and digital content. This is required to improve user learning efficiency and enrich the reading experience. Furthermore, it is crucial to obtain necessary information concisely within a limited time through a summarization function.

[0538] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0539] In this invention, the server includes processing means for acquiring data related to a specific item selected by the user on an electronic terminal to obtain information, visualization means for displaying the acquired data, and information analysis means for summarizing the content from the portion read by the user. This enables the user to efficiently obtain information while understanding the content without interruption.

[0540] An "electronic terminal" is a device used by users to view and manipulate information, and includes smartphones and tablets.

[0541] A "processing means" is an element within a system that has the function of acquiring, extracting, or analyzing specific information or data.

[0542] A "visualization means" is an element within a system that has a display function for showing acquired information to the user.

[0543] An "information analysis tool" is an element within a system that has the function of processing content within a range specified by the user and generating a summary.

[0544] An "interface" is a function or means that allows a user to directly interact with a system. Using this interface, users can adjust the amount of information and the display format.

[0545] The system for realizing this application is configured as follows: The user views ebooks using an electronic device such as a smartphone or tablet. When the user selects a specific item of interest while viewing, the device sends that selection information to the server.

[0546] The server includes processing mechanisms for retrieving data related to selected items. The server searches the database and extracts relevant information. This allows for the provision of appropriate information while considering the user's browsing history. Furthermore, it uses information analysis tools to generate a summary of the selection. This summary is performed using natural language processing technologies such as OpenAI's GPT model.

[0547] The terminal displays information sent from the server to the user using visualization tools. This employs a pop-up format to efficiently deliver information without disrupting the browsing flow. Furthermore, the user can adjust the amount of information and display format of the summary through the interface.

[0548] For example, when a user selects a word in the story that particularly interests them, information about its historical background and importance to the story immediately pops up. Furthermore, it's possible to download a summary of the story in advance, taking advantage of the limited time available on an airplane.

[0549] An example of a prompt to facilitate this process is, "Summarize the following text: '...(text)...' Identify three key points within the given range." By providing this prompt to the generating AI model, the information requested by the user can be efficiently obtained.

[0550] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0551] Step 1:

[0552] Users browse ebooks on their electronic devices and select specific items that interest them. This selection information is entered into the device. The device packages the selected items and their location information and sends this data to the server.

[0553] Step 2:

[0554] The server receives selection information sent from the terminal. Based on this input data, it searches the database and extracts information related to the relevant items. This information includes details and background information about the items. After data extraction, spoiler information is filtered and excluded depending on where the user is reading.

[0555] Step 3:

[0556] The server further processes the filtered results of the relevant information based on the generating AI model. If summarization is required, it uses natural language processing techniques to summarize the selected content. Specifically, it inputs prompt sentences into the generating AI model to obtain a summary of the specified range. The results are then formatted to the user's specified amount and format.

[0557] Step 4:

[0558] The server then transmits the final processed information to the terminal. The terminal uses a display mechanism to visualize this received data for the user. The information is displayed in a pop-up format, allowing for quick and uninterrupted communication of the content without disrupting the user's browsing flow.

[0559] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0560] This invention is a system for improving the user's reading experience, supporting the comprehension of e-book content and adapting information presentation according to the user's emotions. The system comprises a terminal for displaying e-books, a server for processing information, an engine for analyzing emotions, and means for presenting information via a user interface.

[0561] First, the content of the ebook is stored in a database, with detailed information about characters and terminology organized page by page. This database is used to quickly provide relevant information when a user selects a specific word or phrase.

[0562] Users can utilize a feature that allows them to retrieve information related to a specific word or phrase while reading, displaying it as a pop-up. The device detects this user action and sends a request to the server. The server receives the request, consults its database, filters the relevant information, and sends it to the device.

[0563] Furthermore, this system incorporates an emotion engine that analyzes the user's facial expressions and movements to identify their emotional state. Based on the user's emotions, the emotion engine optimizes how information is presented and summarized. For example, if the system detects that the user is confused, it adjusts to provide more detailed information to aid understanding.

[0564] For example, if a user long-presses the term "Dark King" in Chapter 4 of an ebook and then smiles, the emotion engine will determine that the user is enjoying the content. The server receives this information and displays relevant details in a pop-up to enhance the user experience. If the user shows a surprised expression, providing additional background information will be considered.

[0565] Thus, the present invention integrates information provision and emotion analysis to improve the user's reading experience, deepening their understanding of the reading content and providing information tailored to their emotions.

[0566] The following describes the processing flow.

[0567] Step 1:

[0568] The user long-presses on a word or phrase that interests them while reading an ebook. This action is detected by the device.

[0569] Step 2:

[0570] The device sends a request to the server containing information about the word or phrase the user long-pressed. This request includes the text information of the word or phrase and contextual information about the page on which the word or phrase appears.

[0571] Step 3:

[0572] The server analyzes the received request, searches the database, and extracts information related to the relevant words. During this process, a mechanism is applied to exclude spoilers based on the user's reading progress.

[0573] Step 4:

[0574] The server formats the information into a pop-up display format and sends it to the terminal. This information is summarized concisely so that the user can easily understand it.

[0575] Step 5:

[0576] The device quickly provides information about the long-pressed word or phrase by displaying the information received from the server to the user in a pop-up format.

[0577] Step 6:

[0578] The emotion engine analyzes data acquired from the device's camera and microphone to identify the user's current emotional state.

[0579] Step 7:

[0580] The server adjusts the timing and content of information presentation based on the analysis results of the emotion engine. If it determines that the user is confused, it adjusts to provide additional explanations.

[0581] Step 8:

[0582] When a user presses the summarize button, the device transmits the action to the server and requests the generation of a summary within the specified range.

[0583] Step 9:

[0584] The server uses natural language processing to summarize the content based on the specified range and adjusts it to suit the user's settings. The summary is sent to the terminal and displayed to the user.

[0585] Step 10:

[0586] The device visually presents summarized information to the user, making it easy to review past content.

[0587] This series of processes allows users to deepen their understanding of the content while reading ebooks, by providing information tailored to their emotions.

[0588] (Example 2)

[0589] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0590] For users reading electronic materials, there are challenges such as difficulty in quickly obtaining relevant information and background details, and a lack of information presentation tailored to their emotions and level of understanding. This invention aims to solve these problems and improve the user experience.

[0591] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0592] In this invention, the server includes means for retrieving relevant information from a storage device when a user selects a word or phrase in electronic material, means for presenting the retrieved information in a pop-up display format, means for analyzing the user's facial expressions and actions to identify their emotional state, and means for optimizing the information presentation method based on the emotional state. This makes it possible to present information according to the user's emotions and level of understanding.

[0593] A "user" is an individual who reads and manipulates electronic materials.

[0594] "Electronic resources" refer to a collection of information that is stored in digital format and can be viewed on electronic devices.

[0595] A "word or phrase" is a cohesive unit within electronic material that has meaning and is eligible for selection.

[0596] A "storage device" is hardware used to store and provide information in digital format.

[0597] "Popup display format" refers to a method of presenting information on the screen as a new window or box.

[0598] "Facial expressions and actions" refer to the facial expressions and body movements that a user makes in front of an electronic device.

[0599] "Emotional state" refers to the analysis results that indicate the user's psychological or emotional state.

[0600] "Optimization" is a method of adjusting information and processing according to a purpose to maximize efficiency and effectiveness.

[0601] This invention is a system for resolving information gaps and lack of emotionally relevant information provision that users may encounter while reading and understanding electronic materials more deeply. The system consists of an electronic display device, an information processing device, an emotion analysis engine, and means for presenting information through a user interface.

[0602] Users can read electronic materials on an electronic display device and obtain related information by long-pressing on words or phrases of interest. The electronic display device recognizes the selected words or phrases and transmits that information to an information processing device. This information processing device then refers to a storage device with high-speed search capabilities, extracts information related to the specific words or phrases, and sends it back to the electronic display device. Through this entire process, users can quickly access the necessary information through pop-up displays.

[0603] Furthermore, the emotion analysis engine monitors the user's facial expressions and movements to identify the user's current emotional state. Based on this information, the information processing device optimizes how information is presented according to the user's emotional state. For example, if the analysis indicates that the user is surprised, more background information will be provided; if the user is smiling and enjoying themselves, interesting content will be emphasized.

[0604] For example, when a user long-presses the term "Dark King," the information processing device will look up its historical background and immediately display details on the electronic display. If the emotion analysis engine determines that the user's facial expression indicates enjoyment, it will present additional related interesting anecdotes or academic materials. If the system determines that the user is confused, it will prioritize providing detailed explanations or easily understandable diagrams.

[0605] An example of a prompt using a generative AI model is, "Please tell me how to quickly provide relevant information about the phrase 'Dark King' contained in the following sentence." This prompt clearly demonstrates how the system can effectively provide information.

[0606] This invention significantly improves the user's reading experience through the immediacy and emotionally tailored presentation of information.

[0607] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0608] Step 1:

[0609] The user long-presses a specific word or phrase within the electronic document. This input is a user action, and the device detects this action using a sensor. Specifically, when the user selects a term such as "Dark King," its coordinates and text information are collected. The output is the detected word or phrase information.

[0610] Step 2:

[0611] The terminal sends a request to the server containing the detected phrase information. The input is the phrase information, which is the output of step 1, and the terminal transmits this to the server over the network. Specifically, the terminal uses the network interface to construct the request and forward it to the server. The output is the request received by the server.

[0612] Step 3:

[0613] Based on the received request, the server retrieves relevant information by referencing a database in its storage device. The input is the word information within the request. The specific data processing involves filtering the relevant information using a high-speed search algorithm and extracting the necessary data. The output is the relevant information data, which is then sent to the terminal.

[0614] Step 4:

[0615] The terminal prepares to present relevant information data received from the server to the user. The input is the relevant information data supplied from the server. Specific actions include generating pop-ups on the user interface and visualizing the information. The output is the displayed content presented to the user.

[0616] Step 5:

[0617] The emotion analysis engine analyzes the user's facial expressions and movements to identify their emotional state. The input consists of real-time video data and user interaction information. Machine learning algorithms are used for data calculation, estimating emotions from facial patterns and movements. The output is the identified emotional state.

[0618] Step 6:

[0619] The device adjusts the presented information based on the identified emotional state. The input consists of the emotional state obtained in step 5 and the display content from step 4. The specific actions include customizing the display content according to the emotion and presenting it in an easily understandable format. The output is the optimized information presentation.

[0620] (Application Example 2)

[0621] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0622] Information users desire personalized information on specific content or products, but current systems have a problem in that this information is not optimized according to individual emotions and circumstances. Furthermore, because information provision cannot be optimized based on emotions, there is a challenge in that the user experience is not sufficiently improved.

[0623] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0624] In this invention, the server includes processing means for obtaining information related to a specific word selected by the user within the information media from data storage means, display means for providing the information to the user in a pop-up display format, analysis means for analyzing the user's emotional state, and control means for adaptively optimizing the content of the information presented based on the analyzed emotions. This enables personalized and optimized information provision according to the user's emotional state.

[0625] "Information media" refers to platforms or devices for displaying, managing, or transmitting information in digital format.

[0626] A "word" is a string of characters within an information medium that can be selected by the user.

[0627] "Data storage means" refers to a physical or virtual device or system for storing information.

[0628] "Processing means" refers to a device or method that has the function of performing a specific operation to process information.

[0629] "Display means" refers to devices or systems for visually presenting information.

[0630] "Analysis means" refers to a process or system for analyzing data and information to derive specific results.

[0631] A "control means" is a system or device that has the function of managing and adjusting the operation of other means or devices.

[0632] An "information user" is an individual or organization that receives information through information media.

[0633] "Emotional state" refers to the emotional state expressed as a psychological or physiological response of the information user.

[0634] "Optimization" is the act of adjusting or improving something to obtain the most effective and efficient results for a particular purpose.

[0635] The system for implementing this invention primarily comprises a display device, a data processing server, and an emotion analysis engine to provide information tailored to the emotional state of the information user. First, the terminal detects a specific word selected by the information user and transmits that information to the server. The server receives this request and retrieves relevant information from the data storage means. A search algorithm is used to filter and retrieve the relevant information.

[0636] Next, the display device is equipped with an intuitive interface for providing information, visually presenting information to the user in a pop-up format. In this process, the display device uses an emotion analysis engine, such as the Microsoft Azure Face API, to analyze the user's facial expressions and identify their emotional state. If the analysis indicates the user is interested, relevant details and positive evaluations are prioritized. Conversely, if confusion or doubt can be read from the facial expression, detailed explanations and additional information are displayed to aid understanding.

[0637] As a concrete example, suppose an information user gazes at a product in a virtual store and smiles. In this situation, the server sends positive reviews and related product suggestions to the display device. On the other hand, if the information user shows a questioning expression, detailed product specifications will be provided. An example of a prompt message that would implement this behavior is, "Suggest detailed product information to be provided when the user looks at the product with a surprised expression."

[0638] In this way, the system provides information optimized according to the user's emotional state, delivering a personalized user experience.

[0639] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0640] Step 1:

[0641] The terminal detects a word selected by the information user. Input is the user's touch operation, and output is the data of the selected word. The terminal detects the character position and content of the selected word and prepares to send it to the next processing step.

[0642] Step 2:

[0643] The terminal sends information about the selected word to the server. The input is word data, and the output is a request to the server. The terminal packets the identification information of the selected word and sends it to the server over the network.

[0644] Step 3:

[0645] The server retrieves related information from the data storage system based on the word information it receives. The input is word information, and the output is related information. The server issues a query to the database, searches for information related to the selected word, filters it, and retrieves it.

[0646] Step 4:

[0647] The server sends the relevant information it has acquired to the terminal. The input is the relevant information, and the output is the transmission of data to the terminal. The server bundles the relevant information into data packets and sends them to the terminal via the network.

[0648] Step 5:

[0649] The terminal displays relevant information it has acquired in a pop-up window. The input is relevant information from the server, and the output is a visual display via a pop-up. The terminal generates a pop-up window on the screen and displays the relevant information to the user.

[0650] Step 6:

[0651] The device uses an emotion analysis engine to analyze the user's facial expressions. The input is a camera image, and the output is analyzed emotion data. The device captures the user's face through the camera, sends the image data to the emotion analysis API, and analyzes the emotional state.

[0652] Step 7:

[0653] The server optimizes the information provided based on sentiment data. The input is sentiment data, and the output is optimized information. Based on the sentiment analysis results, the server determines which information to emphasize and customizes the content.

[0654] Step 8:

[0655] The device redisplays optimized information to improve the user experience. The input is optimized information, and the output is a customized visual display. The device receives optimized information from the server and displays it on the screen to present it in a way that is easy for the user to understand and engage.

[0656] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0657] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0658] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0659] [Fourth Embodiment]

[0660] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0661] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0662] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0663] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0664] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0665] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0666] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0667] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0668] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0669] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0670] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0671] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0672] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0673] This invention is a technology that enables users to efficiently understand and enjoy ebooks over a long period of time. This system consists of a terminal for using ebooks, a server for searching and processing information, and information display means via a user interface.

[0674] First, the content of the ebook is registered in a database as related information for each page. This information includes details about the characters, key points in the story, and explanations of specific terminology.

[0675] When a user reads an ebook and long-presses a specific word or phrase, the device detects this interaction and requests information about that word or phrase from the server. This information request also includes data on where the user performed the long-press.

[0676] The server receives the request, searches the database, and extracts relevant information. During this process, it considers the user's reading progress and performs filtering to prevent spoilers. The extracted information is sent to the device in a pop-up display format.

[0677] The device displays received information as a pop-up on the screen, allowing the user to review the relevant section. This pop-up display enables quick and concise information delivery without interrupting the reading flow.

[0678] Furthermore, users can utilize a summarization function. By operating the summarization button on their device, they request the server to generate a summary of the specified range of content. The server receives this summary generation request, compresses the specified content using natural language processing technology, and creates a summary according to the user's desired level of detail and character count. The completed summary is then provided to the user via their device.

[0679] As a concrete example, if a user reads Chapter 5 of an ebook and long-presses the term "Black King" that appears there, the device transmits this action to the server. The server collects information about "Black King" up to Chapter 5 and sends it to the device, without including spoilers. The device displays this information as a pop-up, allowing the user to immediately see details about "Black King."

[0680] This system is designed to deepen users' reading comprehension, allow them to absorb the maximum amount of information in a limited time, and improve their reading experience.

[0681] The following describes the processing flow.

[0682] Step 1:

[0683] The user long-presses on a word or phrase of interest while reading an ebook. This action is detected by the device.

[0684] Step 2:

[0685] The device collects information about the word or phrase the user long-presses and sends it to the server as a request. This request includes the text information of the long-pressed word or phrase and the context information of the page.

[0686] Step 3:

[0687] The server searches the database based on the received request. Based on the user's past reading history, it extracts relevant information about the specified words and filters it to avoid spoilers.

[0688] Step 4:

[0689] The server formats the extracted information into a pop-up display format and sends it to the terminal. This formatting includes a concise structure to allow the user to quickly understand the information.

[0690] Step 5:

[0691] The device displays information received from the server as a pop-up on the screen. Through this pop-up, the user can view details and related information about the word or phrase they long-pressed.

[0692] Step 6:

[0693] Users can optionally press a summary button to request a summary of a specific range of pages. The device detects this action and sends a summary generation request to the server.

[0694] Step 7:

[0695] The server generates a summary of the specified content using natural language processing. This summary is adjusted based on the user's settings for character count and level of detail.

[0696] Step 8:

[0697] The server sends the generated summary to the terminal.

[0698] Step 9:

[0699] The device displays the received summary to the user. This summary allows the user to easily review previously read content.

[0700] (Example 1)

[0701] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0702] When using e-books, there is a need for a system that allows users to quickly and accurately obtain relevant information about specific words or content, and to easily access that information, while also having a spoiler prevention function that adapts to the user's reading progress, and further allowing the user to adjust the level of detail and word count of the summary they specify.

[0703] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0704] In this invention, the server includes communication means for recording location information of specific words or phrases selected by the user within an e-book and retrieving information based on that information; information filtering means for the server to consider the user's reading progress and eliminate spoilers; and means for analyzing information using a generative AI model and filtering specific information. This enables the user to efficiently obtain relevant information and improve their reading experience.

[0705] An "e-book" is a book distributed in digital format and is an information medium that can be viewed using an electronic device.

[0706] A "word or phrase" is the smallest unit of language that carries meaning, and refers to words and phrases used in writing or conversation.

[0707] A "storage device" is a system that stores large amounts of data and allows for quick retrieval as needed, and includes databases and memory.

[0708] "Processing means" refers to functions or devices for executing some kind of process, and includes technical configurations for analyzing or manipulating information.

[0709] "Visual display format" refers to a method of providing information in a way that is visible to the user, and includes displays using display devices.

[0710] "Information processing means" refers to functions or devices that transform information into a form suitable for a specific purpose, and includes data summarization and transformation.

[0711] "Information filtering means" refers to functions or devices that select information based on specific conditions and remove unnecessary or harmful information.

[0712] A "generative AI model" is an algorithm or system that uses machine learning to analyze information and identify specific patterns or trends.

[0713] This invention is a system that makes it easier for users to efficiently use ebooks and obtain related information. To read ebooks, users use a compatible device. This device is equipped with a touchscreen and has the function of displaying the pages of the ebook.

[0714] When a user is reading an ebook on their device, they can long-press a specific word or phrase, allowing the device to acquire location information corresponding to that phrase. This information is then transmitted to a server via communication. The server retrieves data related to the phrase based on the information stored in the storage device. The retrieved data is then analyzed using a generative AI model. The server further monitors the user's reading progress and automatically performs necessary filtering to prevent spoilers.

[0715] The filtered information is sent to the device in a visual display format, which then displays it to the user as a pop-up. This display allows the user to access relevant information without interrupting their reading flow.

[0716] For example, if a user long-presses the phrase "Black King" in Chapter 5 of an ebook, the device transmits this action to the server. The server collects information about "Black King," removes spoilers, and provides it to the device. The device displays this information as a pop-up, allowing the user to immediately see details about "Black King."

[0717] The terminal also has a summary button, which allows the user to request the server to summarize a specified range of content. The server uses a generative AI model to summarize the specified text and provides the terminal with a summary result based on the desired level of detail and character count.

[0718] As a concrete example of a prompt, it is possible to input prompts that correspond to the information the user is looking for, such as, "Please tell me how to obtain detailed information about a specific character in Chapter 5 of the ebook by long-pressing, without giving away any spoilers."

[0719] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0720] Step 1:

[0721] The user long-presses a specific word or phrase within the ebook.

[0722] The input is the user's touch operation, and the output is the selected word or phrase and its location information. The device detects this information and uses sensors to determine the location.

[0723] Step 2:

[0724] The device sends the acquired words and location information to the server.

[0725] The input consists of the words and location information obtained in step 1, and the output is the request data sent to the server. Using a communication module, the terminal organizes the information into packets and transmits them over the network.

[0726] Step 3:

[0727] The server analyzes the received request data and retrieves relevant information from the storage device.

[0728] The input is request data sent from the terminal, and the output is a dataset of related information. The server executes a database query, filters, and extracts the relevant information.

[0729] Step 4:

[0730] The server uses a generative AI model to analyze the acquired information and perform filtering to prevent spoilers based on the user's reading progress.

[0731] The input is the information extracted in step 3, and the output is the information with spoilers removed. The generative AI model uses a text analysis algorithm to select important information.

[0732] Step 5:

[0733] The server sends filtered information to the terminal.

[0734] The input is the information filtered in step 4, and the output is the response data to the terminal. The server packets the data and sends it using the communication protocol.

[0735] Step 6:

[0736] The terminal displays information received from the server to the user in a pop-up window.

[0737] The input is response data sent from the server, and the output is visual information for the user. The terminal generates a popup window using a GUI drawing library.

[0738] Step 7:

[0739] When a user presses the summary button, the device requests a summary of the specified content from the server.

[0740] The input consists of user button presses and specified ranges, and the output is a summarized request. The terminal detects the operation, generates a request, and prepares to send it.

[0741] Step 8:

[0742] The server uses a generative AI model to summarize the content within a specified range and creates a summary that matches the user's desired level of detail and character count.

[0743] The input is a summary request, and the output is the summarized result. The generative AI model applies natural language processing techniques to summarize text and generates a summary of appropriate length and content.

[0744] Step 9:

[0745] The server sends the summary results to the terminal, and the terminal displays them to the user.

[0746] The input is the summarized result, and the output is the summarized information provided to the user. The terminal integrates the results into a GUI and presents them to the user visually.

[0747] (Application Example 1)

[0748] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0749] The problem that this invention aims to solve is enabling users to instantly understand specific information and efficiently acquire information while viewing ebooks and digital content. This is required to improve user learning efficiency and enrich the reading experience. Furthermore, it is crucial to obtain necessary information concisely within a limited time through a summarization function.

[0750] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0751] In this invention, the server includes processing means for acquiring data related to a specific item selected by the user on an electronic terminal to obtain information, visualization means for displaying the acquired data, and information analysis means for summarizing the content from the portion read by the user. This enables the user to efficiently obtain information while understanding the content without interruption.

[0752] An "electronic terminal" is a device used by users to view and manipulate information, and includes smartphones and tablets.

[0753] A "processing means" is an element within a system that has the function of acquiring, extracting, or analyzing specific information or data.

[0754] A "visualization means" is an element within a system that has a display function for showing acquired information to the user.

[0755] An "information analysis tool" is an element within a system that has the function of processing content within a range specified by the user and generating a summary.

[0756] An "interface" is a function or means that allows a user to directly interact with a system. Using this interface, users can adjust the amount of information and the display format.

[0757] The system for realizing this application is configured as follows: The user views ebooks using an electronic device such as a smartphone or tablet. When the user selects a specific item of interest while viewing, the device sends that selection information to the server.

[0758] The server includes processing mechanisms for retrieving data related to selected items. The server searches the database and extracts relevant information. This allows for the provision of appropriate information while considering the user's browsing history. Furthermore, it uses information analysis tools to generate a summary of the selection. This summary is performed using natural language processing technologies such as OpenAI's GPT model.

[0759] The terminal displays information sent from the server to the user using visualization tools. This employs a pop-up format to efficiently deliver information without disrupting the browsing flow. Furthermore, the user can adjust the amount of information and display format of the summary through the interface.

[0760] For example, when a user selects a word in the story that particularly interests them, information about its historical background and importance to the story immediately pops up. Furthermore, it's possible to download a summary of the story in advance, taking advantage of the limited time available on an airplane.

[0761] An example of a prompt to facilitate this process is, "Summarize the following text: '...(text)...' Identify three key points within the given range." By providing this prompt to the generating AI model, the information requested by the user can be efficiently obtained.

[0762] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0763] Step 1:

[0764] Users browse ebooks on their electronic devices and select specific items that interest them. This selection information is entered into the device. The device packages the selected items and their location information and sends this data to the server.

[0765] Step 2:

[0766] The server receives selection information sent from the terminal. Based on this input data, it searches the database and extracts information related to the relevant items. This information includes details and background information about the items. After data extraction, spoiler information is filtered and excluded depending on where the user is reading.

[0767] Step 3:

[0768] The server further processes the filtered results of the relevant information based on the generating AI model. If summarization is required, it uses natural language processing techniques to summarize the selected content. Specifically, it inputs prompt sentences into the generating AI model to obtain a summary of the specified range. The results are then formatted to the user's specified amount and format.

[0769] Step 4:

[0770] The server then transmits the final processed information to the terminal. The terminal uses a display mechanism to visualize this received data for the user. The information is displayed in a pop-up format, allowing for quick and uninterrupted communication of the content without disrupting the user's browsing flow.

[0771] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0772] This invention is a system for improving the user's reading experience, supporting the comprehension of e-book content and adapting information presentation according to the user's emotions. The system comprises a terminal for displaying e-books, a server for processing information, an engine for analyzing emotions, and means for presenting information via a user interface.

[0773] First, the content of the ebook is stored in a database, with detailed information about characters and terminology organized page by page. This database is used to quickly provide relevant information when a user selects a specific word or phrase.

[0774] Users can utilize a feature that allows them to retrieve information related to a specific word or phrase while reading, displaying it as a pop-up. The device detects this user action and sends a request to the server. The server receives the request, consults its database, filters the relevant information, and sends it to the device.

[0775] Furthermore, this system incorporates an emotion engine that analyzes the user's facial expressions and movements to identify their emotional state. Based on the user's emotions, the emotion engine optimizes how information is presented and summarized. For example, if the system detects that the user is confused, it adjusts to provide more detailed information to aid understanding.

[0776] For example, if a user long-presses the term "Dark King" in Chapter 4 of an ebook and then smiles, the emotion engine will determine that the user is enjoying the content. The server receives this information and displays relevant details in a pop-up to enhance the user experience. If the user shows a surprised expression, providing additional background information will be considered.

[0777] Thus, the present invention integrates information provision and emotion analysis to improve the user's reading experience, deepening their understanding of the reading content and providing information tailored to their emotions.

[0778] The following describes the processing flow.

[0779] Step 1:

[0780] The user long-presses on a word or phrase that interests them while reading an ebook. This action is detected by the device.

[0781] Step 2:

[0782] The device sends a request to the server containing information about the word or phrase the user long-pressed. This request includes the text information of the word or phrase and contextual information about the page on which the word or phrase appears.

[0783] Step 3:

[0784] The server analyzes the received request, searches the database, and extracts information related to the relevant words. During this process, a mechanism is applied to exclude spoilers based on the user's reading progress.

[0785] Step 4:

[0786] The server formats the information into a pop-up display format and sends it to the terminal. This information is summarized concisely so that the user can easily understand it.

[0787] Step 5:

[0788] The device quickly provides information about the long-pressed word or phrase by displaying the information received from the server to the user in a pop-up format.

[0789] Step 6:

[0790] The emotion engine analyzes data acquired from the device's camera and microphone to identify the user's current emotional state.

[0791] Step 7:

[0792] The server adjusts the timing and content of information presentation based on the analysis results of the emotion engine. If it determines that the user is confused, it adjusts to provide additional explanations.

[0793] Step 8:

[0794] When a user presses the summarize button, the device transmits the action to the server and requests the generation of a summary within the specified range.

[0795] Step 9:

[0796] The server uses natural language processing to summarize the content based on the specified range and adjusts it to suit the user's settings. The summary is sent to the terminal and displayed to the user.

[0797] Step 10:

[0798] The device visually presents summarized information to the user, making it easy to review past content.

[0799] This series of processes allows users to deepen their understanding of the content while reading ebooks, by providing information tailored to their emotions.

[0800] (Example 2)

[0801] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0802] For users reading electronic materials, there are challenges such as difficulty in quickly obtaining relevant information and background details, and a lack of information presentation tailored to their emotions and level of understanding. This invention aims to solve these problems and improve the user experience.

[0803] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0804] In this invention, the server includes means for retrieving relevant information from a storage device when a user selects a word or phrase in electronic material, means for presenting the retrieved information in a pop-up display format, means for analyzing the user's facial expressions and actions to identify their emotional state, and means for optimizing the information presentation method based on the emotional state. This makes it possible to present information according to the user's emotions and level of understanding.

[0805] A "user" is an individual who reads and manipulates electronic materials.

[0806] "Electronic resources" refer to a collection of information that is stored in digital format and can be viewed on electronic devices.

[0807] A "word or phrase" is a cohesive unit within electronic material that has meaning and is eligible for selection.

[0808] A "storage device" is hardware used to store and provide information in digital format.

[0809] "Popup display format" refers to a method of presenting information on the screen as a new window or box.

[0810] "Facial expressions and actions" refer to the facial expressions and body movements that a user makes in front of an electronic device.

[0811] "Emotional state" refers to the analysis results that indicate the user's psychological or emotional state.

[0812] "Optimization" is a method of adjusting information and processing according to a purpose to maximize efficiency and effectiveness.

[0813] This invention is a system for resolving information gaps and lack of emotionally relevant information provision that users may encounter while reading and understanding electronic materials more deeply. The system consists of an electronic display device, an information processing device, an emotion analysis engine, and means for presenting information through a user interface.

[0814] Users can read electronic materials on an electronic display device and obtain related information by long-pressing on words or phrases of interest. The electronic display device recognizes the selected words or phrases and transmits that information to an information processing device. This information processing device then refers to a storage device with high-speed search capabilities, extracts information related to the specific words or phrases, and sends it back to the electronic display device. Through this entire process, users can quickly access the necessary information through pop-up displays.

[0815] Furthermore, the emotion analysis engine monitors the user's facial expressions and movements to identify the user's current emotional state. Based on this information, the information processing device optimizes how information is presented according to the user's emotional state. For example, if the analysis indicates that the user is surprised, more background information will be provided; if the user is smiling and enjoying themselves, interesting content will be emphasized.

[0816] For example, when a user long-presses the term "Dark King," the information processing device will look up its historical background and immediately display details on the electronic display. If the emotion analysis engine determines that the user's facial expression indicates enjoyment, it will present additional related interesting anecdotes or academic materials. If the system determines that the user is confused, it will prioritize providing detailed explanations or easily understandable diagrams.

[0817] An example of a prompt using a generative AI model is, "Please tell me how to quickly provide relevant information about the phrase 'Dark King' contained in the following sentence." This prompt clearly demonstrates how the system can effectively provide information.

[0818] This invention significantly improves the user's reading experience through the immediacy and emotionally tailored presentation of information.

[0819] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0820] Step 1:

[0821] The user long-presses a specific word or phrase within the electronic document. This input is a user action, and the device detects this action using a sensor. Specifically, when the user selects a term such as "Dark King," its coordinates and text information are collected. The output is the detected word or phrase information.

[0822] Step 2:

[0823] The terminal sends a request to the server containing the detected phrase information. The input is the phrase information, which is the output of step 1, and the terminal transmits this to the server over the network. Specifically, the terminal uses the network interface to construct the request and forward it to the server. The output is the request received by the server.

[0824] Step 3:

[0825] Based on the received request, the server retrieves relevant information by referencing a database in its storage device. The input is the word information within the request. The specific data processing involves filtering the relevant information using a high-speed search algorithm and extracting the necessary data. The output is the relevant information data, which is then sent to the terminal.

[0826] Step 4:

[0827] The terminal prepares to present relevant information data received from the server to the user. The input is the relevant information data supplied from the server. Specific actions include generating pop-ups on the user interface and visualizing the information. The output is the displayed content presented to the user.

[0828] Step 5:

[0829] The emotion analysis engine analyzes the user's facial expressions and movements to identify their emotional state. The input consists of real-time video data and user interaction information. Machine learning algorithms are used for data calculation, estimating emotions from facial patterns and movements. The output is the identified emotional state.

[0830] Step 6:

[0831] The device adjusts the presented information based on the identified emotional state. The input consists of the emotional state obtained in step 5 and the display content from step 4. The specific actions include customizing the display content according to the emotion and presenting it in an easily understandable format. The output is the optimized information presentation.

[0832] (Application Example 2)

[0833] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0834] Information users desire personalized information on specific content or products, but current systems have a problem in that this information is not optimized according to individual emotions and circumstances. Furthermore, because information provision cannot be optimized based on emotions, there is a challenge in that the user experience is not sufficiently improved.

[0835] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0836] In this invention, the server includes processing means for obtaining information related to a specific word selected by the user within the information media from data storage means, display means for providing the information to the user in a pop-up display format, analysis means for analyzing the user's emotional state, and control means for adaptively optimizing the content of the information presented based on the analyzed emotions. This enables personalized and optimized information provision according to the user's emotional state.

[0837] "Information media" refers to platforms or devices for displaying, managing, or transmitting information in digital format.

[0838] A "word" is a string of characters within an information medium that can be selected by the user.

[0839] "Data storage means" refers to a physical or virtual device or system for storing information.

[0840] "Processing means" refers to a device or method that has the function of performing a specific operation to process information.

[0841] "Display means" refers to devices or systems for visually presenting information.

[0842] "Analysis means" refers to a process or system for analyzing data and information to derive specific results.

[0843] A "control means" is a system or device that has the function of managing and adjusting the operation of other means or devices.

[0844] An "information user" is an individual or organization that receives information through information media.

[0845] "Emotional state" refers to the emotional state expressed as a psychological or physiological response of the information user.

[0846] "Optimization" is the act of adjusting or improving something to obtain the most effective and efficient results for a particular purpose.

[0847] The system for implementing this invention primarily comprises a display device, a data processing server, and an emotion analysis engine to provide information tailored to the emotional state of the information user. First, the terminal detects a specific word selected by the information user and transmits that information to the server. The server receives this request and retrieves relevant information from the data storage means. A search algorithm is used to filter and retrieve the relevant information.

[0848] Next, the display device is equipped with an intuitive interface for providing information, visually presenting information to the user in a pop-up format. In this process, the display device uses an emotion analysis engine, such as the Microsoft Azure Face API, to analyze the user's facial expressions and identify their emotional state. If the analysis indicates the user is interested, relevant details and positive evaluations are prioritized. Conversely, if confusion or doubt can be read from the facial expression, detailed explanations and additional information are displayed to aid understanding.

[0849] As a concrete example, suppose an information user gazes at a product in a virtual store and smiles. In this situation, the server sends positive reviews and related product suggestions to the display device. On the other hand, if the information user shows a questioning expression, detailed product specifications will be provided. An example of a prompt message that would implement this behavior is, "Suggest detailed product information to be provided when the user looks at the product with a surprised expression."

[0850] In this way, the system provides information optimized according to the user's emotional state, delivering a personalized user experience.

[0851] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0852] Step 1:

[0853] The terminal detects a word selected by the information user. Input is the user's touch operation, and output is the data of the selected word. The terminal detects the character position and content of the selected word and prepares to send it to the next processing step.

[0854] Step 2:

[0855] The terminal sends information about the selected word to the server. The input is word data, and the output is a request to the server. The terminal packets the identification information of the selected word and sends it to the server over the network.

[0856] Step 3:

[0857] The server retrieves related information from the data storage system based on the word information it receives. The input is word information, and the output is related information. The server issues a query to the database, searches for information related to the selected word, filters it, and retrieves it.

[0858] Step 4:

[0859] The server sends the relevant information it has acquired to the terminal. The input is the relevant information, and the output is the transmission of data to the terminal. The server bundles the relevant information into data packets and sends them to the terminal via the network.

[0860] Step 5:

[0861] The terminal displays relevant information it has acquired in a pop-up window. The input is relevant information from the server, and the output is a visual display via a pop-up. The terminal generates a pop-up window on the screen and displays the relevant information to the user.

[0862] Step 6:

[0863] The device uses an emotion analysis engine to analyze the user's facial expressions. The input is a camera image, and the output is analyzed emotion data. The device captures the user's face through the camera, sends the image data to the emotion analysis API, and analyzes the emotional state.

[0864] Step 7:

[0865] The server optimizes the information provided based on sentiment data. The input is sentiment data, and the output is optimized information. Based on the sentiment analysis results, the server determines which information to emphasize and customizes the content.

[0866] Step 8:

[0867] The device redisplays optimized information to improve the user experience. The input is optimized information, and the output is a customized visual display. The device receives optimized information from the server and displays it on the screen to present it in a way that is easy for the user to understand and engage.

[0868] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0869] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0870] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0871] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0872] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0873] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0874] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0875] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0876] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0877] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0878] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0879] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0880] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0881] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0882] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0883] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0884] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0885] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0886] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0887] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0888] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted as being incorporated by reference.

[0889] The following is further disclosed regarding the embodiments described above.

[0890] (Claim 1)

[0891] A processing means for retrieving information related to a specific word or phrase selected by a user within an e-book from a database,

[0892] A display means for providing the relevant information to the user in a pop-up display format,

[0893] Processing means for summarizing the contents of a specified range,

[0894] A display means for visually displaying the summary results to the user,

[0895] A system that includes this.

[0896] (Claim 2)

[0897] The system according to claim 1, characterized by comprising a control means that provides information in a way that eliminates spoilers according to the user's reading progress.

[0898] (Claim 3)

[0899] The system according to claim 1, characterized by having an interface for adjusting the level of detail and number of characters in a summary that can be set by the user.

[0900] "Example 1"

[0901] (Claim 1)

[0902] A processing means for obtaining information related to a specific word or phrase from a storage device when a user selects that word or phrase within an e-book,

[0903] A display means for providing the relevant information to the user in a visual display format,

[0904] Information processing means for summarizing the content within a specified range,

[0905] Output means for visually providing the summary results to the user,

[0906] A communication means for recording location information when a user selects a specific word or phrase, and for obtaining information based on that information,

[0907] The server considers the user's reading progress and employs information filtering methods to eliminate spoilers.

[0908] A system that includes this.

[0909] (Claim 2)

[0910] The system according to claim 1, comprising an interface for adjusting the level of detail and number of characters in a summary, which can be set by the user.

[0911] (Claim 3)

[0912] The system according to claim 1, which analyzes information using a generative AI model and filters specific information.

[0913] "Application Example 1"

[0914] (Claim 1)

[0915] A processing means for obtaining data related to a specific item selected by the user on an electronic terminal to obtain information,

[0916] A visualization means for displaying the acquired data,

[0917] Information analysis tools for summarizing content from the portion read by the user,

[0918] A display means for visualizing summarized information to the user,

[0919] A system that includes this.

[0920] (Claim 2)

[0921] The system according to claim 1, characterized by comprising control means for dynamically adjusting displayed information based on the user's browsing history.

[0922] (Claim 3)

[0923] The system according to claim 1, characterized by having an interface for adjusting the amount of information and display format of a summary that can be specified by the user.

[0924] "Example 2 of combining an emotion engine"

[0925] (Claim 1)

[0926] A means for retrieving related information from storage when a user selects a word or phrase within an electronic document,

[0927] A means of presenting the acquired information in a pop-up display format,

[0928] A means of summarizing a specified range,

[0929] Means for visually displaying the summary results,

[0930] A means of analyzing the user's facial expressions and movements to identify their emotional state,

[0931] A means of optimizing the method of presenting information based on emotional state,

[0932] A system that includes this.

[0933] (Claim 2)

[0934] The system according to claim 1, further comprising control means for removing unnecessary information from the information provided according to the user's progress.

[0935] (Claim 3)

[0936] The system according to claim 1, comprising an interface for adjusting user-configurable summary details and character count.

[0937] "Application example 2 when combining with an emotional engine"

[0938] (Claim 1)

[0939] A processing means for obtaining information related to a specific word from a data storage means when a user selects a specific word within an information medium,

[0940] A display means for providing the relevant information to the user in a pop-up display format,

[0941] Processing means for summarizing the contents of a specified range,

[0942] A display means for visually displaying the summary results to the user,

[0943] An analytical method for analyzing the emotional state of users,

[0944] A control means for adaptively optimizing the content of information presentation based on analyzed emotions,

[0945] A system that includes this.

[0946] (Claim 2)

[0947] The system according to claim 1, characterized by comprising control means for selectively presenting product evaluation information according to the user's emotional state.

[0948] (Claim 3)

[0949] The system according to claim 1, further characterized by having an interface for adjusting the level of detail and number of characters in a summary that can be set by the user, and further enabling the setting of priority for information presentation according to emotional state. [Explanation of Symbols]

[0950] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A processing means for retrieving information related to a specific word or phrase selected by a user within an e-book from a database, A display means for providing the relevant information to the user in a pop-up display format, Processing means for summarizing the contents of a specified range, A display means for visually displaying the summary results to the user, A system that includes this.

2. The system according to claim 1, characterized by comprising a control means that provides information in a way that eliminates spoilers according to the user's reading progress.

3. The system according to claim 1, characterized by having an interface for adjusting the level of detail and number of characters in a summary that can be set by the user.

Images