system

Abstract

Provide a system. [Solution] In the information processing device, means for receiving a list of the input documents; means for generating summary information for each document in the list; a means for aggregating the generated summary information to create a single document; means for outputting the created document; A system including:

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In today's world, vast amounts of information exist in various formats, making it difficult for users to efficiently collect, properly manage, and summarize the information they need. Documents for business leaders and managers, as well as frequently used technical information (e.g., how to use Excel functions and spreadsheets), are particularly voluminous and complex, making it extremely time-consuming for individual users to manually organize and summarize this information. Therefore, there is a need for a system that can efficiently collect information and organize and summarize it in a way that allows users to easily access it. [Means for solving the problem]

[0005] The present invention provides an information processing device including: means for receiving a list of input documents; means for generating summary information for each document in the list; means for aggregating the generated summary information to create a single document; and means for outputting the created document. Specifically, the means for generating summary information summarizes the documents using natural language processing technology. The device further includes means for collecting related information from the Internet based on input category information, means for generating a guide based on the collected information, means for building the generated guide into an application, and means for outputting the application, thereby enabling users to efficiently collect, organize, and summarize the information they need.

[0006] An "information processing device" is a device that has the function of receiving, analyzing, processing, and outputting information.

[0007] A "document list" is a collection of documents arranged in a specific format and in an order.

[0008] "Summary information" is a short description or summary that succinctly presents the overall content of a document.

[0009] "Natural language processing technology" is a type of technology that understands and automatically processes the language used by humans.

[0010] "Category information" is classification information for indicating information related to a specific theme or field.

[0011] The "Internet" is a huge network that connects computers all over the world.

[0012] A "guide" is a document or resource intended to provide specific information or instructions.

[0013] An "application" is a software program designed to accomplish a specific task.

[0014] A "user" is a person who uses an information processing device or its services.

[0015] "Output means" refers to a method or device for providing processed information to a user. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0017] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0020] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0021] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0022] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0024] [First embodiment]

[0025] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0026] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0029] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0032] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0036] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0037] This invention relates to a system that uses an information processing device to efficiently collect, organize, and summarize information. This system receives a list of multiple documents entered by the user, generates and aggregates summary information for each document, and creates and outputs a single document, thereby reducing the burden on the user. It is also possible to collect information related to a specific category from the Internet, generate a guide, and provide it as an application.

[0038] Overall processing flow of the program

[0039] The system operates as follows.

[0040] 1. User enters CEO's book list

[0041] The user inputs the titles of multiple books related to the CEO from the terminal. This information is sent to the server. For example, the user inputs "CEO's Book 1" and "CEO's Book 2" and sends them to the server.

[0042] 2. The server receives the input data and starts processing it.

[0043] The server receives the list sent by the user and uses natural language processing technology to generate summary information for each book in the list. For each book title, the server calls a summary generation API to obtain summary information.

[0044] 3. The server aggregates the summary information

[0045] The server then compiles the obtained summaries into a single document. For example, it concatenates the summaries of "CEO Book 1" and "CEO Book 2" in order to create a final best-of document.

[0046] 4. The server delivers the final product to the user

[0047] The completed best-of document is converted into PDF or e-book format and provided to users, allowing them to see a summary without having to read each individual book.

[0048] Specific examples

[0049] Example 1: CEO Book Summary

[0050] The user enters the following list:

[0051] ["CEO Book 1", "CEO Book 2", "CEO Book 3"]

[0052] The server gets a summary from the chat AI for each book:

[0053] Summary 1: "CEO Book 1 Summary"

[0054] Summary 2: "CEO Book 2 Summary"

[0055] Summary 3: "CEO Book 3 Summary"

[0056] The server creates the final best version of the document:

[0057] Final Document: "CEO Book 1 Summary\n\nCEO Book 2 Summary\n\nCEO Book 3 Summary"

[0058] Example 2: Creating a guide on Excel functions

[0059] User selected categories:

[0060] "Excel Functions"

[0061] The server gathers relevant information from the Internet:

[0062] Information 1: "Basic Excel Functions"

[0063] Information 2: "Advanced Excel Functions"

[0064] Information 3: "How to use Excel functions"

[0065] The server generates a guide based on the information collected:

[0066] Guide: "Basic Excel Functions\n\nAdvanced Excel Functions\n\nHow to Use Excel Functions"

[0067] Build the server-generated guide into an application and serve it to users:

[0068] App: "Excel Function Guide App"

[0069] This allows users to easily refer to a comprehensive guide to Excel functions.

[0070] The processing flow will be explained below.

[0071] Program processing steps

[0072] Example 1: Compiling the CEO's top 10 books using chat generation AI

[0073] Step 1:

[0074] The user inputs a list of CEO's books from the terminal. For example, the user inputs titles such as "CEO's Book 1," "CEO's Book 2," and "CEO's Book 3."

[0075] Step 2:

[0076] The terminal sends the entered list to the server. The sent list is sent in JSON format, for example, as "["CEO's Book 1", "CEO's Book 2", "CEO's Book 3"]".

[0077] Step 3:

[0078] The server receives the user input, and upon receipt, performs data integrity checks and format validation.

[0079] Step 4:

[0080] The server sends a request to the chat generation AI to generate summary information for each document title in the received list. Specifically, it sends an API request to each title to obtain summary information.

[0081] Step 5:

[0082] The server receives the summary information returned by the chat generation AI. For example, the response may be something like "Summary of CEO's Book 1," "Summary of CEO's Book 2," or "Summary of CEO's Book 3."

[0083] Step 6:

[0084] The server aggregates the received summaries to create a single document, specifically by concatenating each summary in turn to create a final document that includes all summaries.

[0085] Step 7:

[0086] The server converts the final document into a PDF or e-book format, making it easily viewable by users.

[0087] Step 8:

[0088] The server provides the converted file to the user, who then downloads or streams the final document from their device.

[0089] Example 2: Picking out frequently used information and turning it into an app

[0090] Step 1:

[0091] The user selects a category of interest (e.g., Excel Functions) from their device, for example by using a drop-down menu or search bar to select "Excel Functions."

[0092] Step 2:

[0093] The terminal transmits the selected category information to the server. For example, the terminal transmits data indicating that the category is "Excel Functions."

[0094] Step 3:

[0095] The server receives the user's selection and begins collecting relevant information, specifically information about Excel functions from internet resources using techniques such as web scraping.

[0096] Step 4:

[0097] The server verifies and filters the collected information, eliminating duplicate information and selecting useful information.

[0098] Step 5:

[0099] The server generates a guide based on the useful information it collects. Specifically, it organizes the information and creates a comprehensive guide divided into chapters and sections.

[0100] Step 6:

[0101] The server-generated guide is then built into an application, for example, developed into a format that can be deployed as a mobile app or web app.

[0102] Step 7:

[0103] The server provides the completed application to the user, who receives a download link or a web app URL to use the application.

[0104] This allows the user to easily obtain and use detailed information about the specified category.

[0105] Example 1

[0106] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0107] Conventional information processing devices require users to individually read a large number of documents, which requires a significant amount of time and effort. Furthermore, there are insufficient methods for efficiently collecting and organizing information related to a specific category. Therefore, there is a need for a system that can efficiently collect, organize, and summarize information.

[0108] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0109] In this invention, the server includes means for receiving a list of input documents, means for generating summary information for each document in the list using a generative AI model, means for aggregating the generated summaries to create a single document, and means for converting the created document into a PDF or e-book format and outputting it, allowing users to efficiently collect information and check summaries without having to read each document individually.

[0110] An "information processing device" is a device that processes data, receives information input by a user, performs specific processing, and provides an output result.

[0111] The "means for receiving a list of documents" is a means having a function for taking in the titles and contents of a plurality of documents input by the user in list form and transmitting the list to the server.

[0112] A "generative AI model" is a model that uses artificial intelligence technology to analyze text data and generate summary information. Examples include generative AI models that use natural language processing technology.

[0113] The "means for generating summary information" is a means for concisely summarizing the contents of a document and extracting its essence.

[0114] The "means for aggregating summary information to create one document" refers to a means for aggregating multiple pieces of summary information, linking them, and organizing them into one unified document.

[0115] "Means of converting to PDF or e-book format and outputting" refers to the means of converting created documents into a format that is easy for users to view and providing them as electronic files. PDF and e-book formats are examples of such formats.

[0116] "Category information" refers to information related to a specific category, and is information for identifying themes or fields in which a user is interested.

[0117] The "means for collecting related information" refers to a means for searching and collecting related information from the Internet based on the input category information.

[0118] "Means for generating a guide" refers to means for organizing collected information and creating guides or manuals for specific purposes.

[0119] The "means for constructing as an application" refers to a means for converting the generated guide into an application in a form that can be used by the user and providing it to the user.

[0120] This invention relates to a system that uses an information processing device to efficiently collect, organize, and summarize information. This system receives a list of multiple documents entered by a user, generates summary information for each document using a generative AI model, aggregates them to create a single document, and converts and outputs the summary in PDF or e-book format, thereby reducing the burden on the user.

[0121] First, the user inputs a list of multiple documents using a terminal. For example, the user inputs the document titles such as "CEO's Book 1" and "CEO's Book 2" in a browser or dedicated application. The input documents are sent from the terminal to the server using an HTTP POST request.

[0122] The server receives the list of documents sent by the user and analyzes their contents. Next, it generates a summary for each document title using a generative AI model (e.g., OpenAI's GPT-3 (registered trademark), which uses natural language processing technology). The server sends a prompt to the generative AI model to obtain the summary.

[0123] As a concrete example, suppose the user inputs the following list:

[0124] ["CEO Book 1", "CEO Book 2", "CEO Book 3"]

[0125] In response, the server sends the following prompt to the generative AI model:

[0126] "Generate a summary for each of the following book titles:\n- CEO Book 1\n- CEO Book 2\n- CEO Book 3"

[0127] Each summary is received from the generative AI model, and a summary like the one below is returned.

[0128] Summary 1: "CEO Book 1 Summary"

[0129] Summary 2: "CEO Book 2 Summary"

[0130] Summary 3: "CEO Book 3 Summary"

[0131] The server aggregates these summaries into a single document and organizes them, for example, as follows:

[0132] Final Document: "CEO Book 1 Summary\n\nCEO Book 2 Summary\n\nCEO Book 3 Summary"

[0133] The server then converts the final document into a format that is easy for users to view, such as PDF or e-book format, using a PDF generation tool or e-book format conversion tool.

[0134] Finally, the server provides the converted document to the user, for example by sending the user an email containing a download link for the generated file, or by displaying the link on a dedicated download page.

[0135] This system allows users to efficiently collect large amounts of information and check summaries, significantly reducing the burden of information processing.It also has the function of automatically collecting, organizing, and summarizing information related to specific categories from the Internet, making it suitable for collecting a variety of information.

[0136] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0137] Step 1:

[0138] The user enters a list of documents

[0139] Input: A user uses a terminal to enter the titles of several documents, for example, "CEO Book 1," "CEO Book 2," "CEO Book 3," etc.

[0140] How it works: A user enters a list of documents into a form in a browser or dedicated application and presses the submit button.

[0141] Output: The input data is prepared as a JSON list and sent to the next processing step.

[0142] Step 2:

[0143] The device sends the data to the server

[0144] Input: A list of documents entered by the user.

[0145] How it works: The device sends JSON-formatted data to the server using an HTTP POST request.

[0146] Output: The data reaches the server and goes on to the next processing step.

[0147] Step 3:

[0148] The server receives the input data and starts processing it.

[0149] Input: Document list sent from the terminal.

[0150] Behavior: The server parses the received data and extracts a list of document titles.

[0151] Output: The extracted list of document titles is used to generate summaries in the next step.

[0152] Step 4:

[0153] The server makes a request to generate a summary

[0154] Input: A list of extracted document titles.

[0155] How it works: The server calls a generative AI model (e.g., OpenAI GPT-3) and creates a prompt for generating a summary. For example, "Generate a summary for the following book title: CEO Book 1." This prompt is then sent to the generative AI model.

[0156] Output: Summary information returned from the generative AI model.

[0157] Step 5:

[0158] The server retrieves the summary results

[0159] Input: Summary information from a generative AI model.

[0160] How it works: The server takes the summary information received from the generative AI model and organizes it by document title.

[0161] Output: Data to aggregate organized summary information into a single document.

[0162] Step 6:

[0163] The server aggregates the summary information

[0164] Input: Organized summary information.

[0165] What it does: Concatenates the summaries into a single document, organized by paragraph, for example, "Summary of CEO Book 1\n\nSummary of CEO Book 2\n\nSummary of CEO Book 3."

[0166] Output: The final summary document.

[0167] Step 7:

[0168] The server converts and outputs the final document

[0169] Input: Final summary document.

[0170] What it does: The server converts this document to PDF or e-book format using tools such as PDF generation tools and e-book format conversion tools.

[0171] Output: The converted document file (e.g. PDF), a download link provided to the user.

[0172] Step 8:

[0173] The server provides the final product to the user.

[0174] Input: The converted document file.

[0175] Action: The server initiates the steps to provide the generated file to the user, for example by sending an email containing a download link or by displaying the link on a dedicated download page.

[0176] Output: A user-accessible link or download page.

[0177] This system will enable users to efficiently collect information requested by the public and check summaries, thereby significantly reducing the burden of information processing.

[0178] (Application example 1)

[0179] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0180] Conventional information processing systems lack the means for users to efficiently collect, summarize, and use information from multiple documents. Furthermore, it is difficult to efficiently provide product information, particularly in brick-and-mortar stores, making it difficult for customers to quickly obtain the information they need.

[0181] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0182] In this invention, the server includes means for receiving a list of input documents, means for generating summary information for each document in the list, means for aggregating the generated summary information to create a single document, means for outputting the created document, means for receiving product-related information and generating summary information, and means for aggregating the generated summary information to create a single document and providing the document to the user. This makes it possible to efficiently collect, summarize, and provide information required by users and customers as a single document.

[0183] An "information processing device" is a combination of hardware and software for receiving, processing, generating, and outputting information.

[0184] The "means for receiving a list of documents" is a device having a function for receiving a list of titles and contents of a plurality of documents input by the user.

[0185] A "means for generating summary information" is an algorithm or software that extracts important information from an input document and generates a summary in a concise format.

[0186] The "means for aggregating summary information to create one document" refers to a device having a function for aggregating and integrating a plurality of pieces of generated summary information into one continuous document.

[0187] The "means for outputting a document" is a device that has the function of providing the created document to the user in a format such as PDF or e-book.

[0188] The "means for receiving information related to merchandise" is a device having a function for receiving merchandise information in a store and related market data.

[0189] The "means for providing to the user" is a device having a function for providing the generated information or document to the user in an accessible form.

[0190] The present invention relates to a system that uses an information processing device to efficiently collect, organize, and summarize information. This system receives a list of multiple documents entered by the user, generates and aggregates summary information for each document, and creates and outputs a single document, thereby reducing the burden on the user. It is also possible to collect information related to a specific category from the Internet, generate a guide, and provide it as an application. Furthermore, the system also has the function of efficiently collecting information related to products in physical stores and providing summary information.

[0191] The server has a means of receiving a list of multiple documents input by the user. This list consists of the titles and contents of documents that the user is interested in. The server then uses natural language processing techniques on these documents to generate summaries. The natural language processing techniques used include OpenAI APIs. Once the summaries are generated, the server aggregates them into a single document. Finally, the generated document is provided to the user in PDF or e-book format.

[0192] Next, the server receives information related to the product and has a means to generate summary information in the same way. This includes the process of collecting related information from the Internet. For example, when collecting the latest information and trend information about a product and generating summary information, the following prompt sentence is used for the generative AI model:

[0193] Example prompt sentence:

[0194] Provide updates and summaries on the "latest smartphones".

[0195] By inputting this prompt into a generative AI model, the latest information on the topic is obtained and a summary is generated, which is then compiled into a single document and ultimately provided to the user in PDF or e-book format.

[0196] For example, if a user enters a list such as "latest smartphones," "trendy fashion items," or "products on sale," the server collects and summarizes information about each item and creates a final, best-of document. This process allows users and customers to efficiently gather the information they need and obtain it in a summarized form.

[0197] In the embodiment of the invention, the input data is diverse, including information collected from the Internet, data entered by the user, data on products in the store, etc. This makes it possible to build a system that eliminates the complexity of information collection that users and customers face and can provide necessary information quickly and efficiently.

[0198] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0199] Step 1:

[0200] User enters a list of documents

[0201] A user inputs the titles and topics of multiple documents he or she wishes to summarize using a terminal, and the input list is sent to the server.

[0202] Specific operation: A user uses a smartphone or PC to enter the title of a document through a web interface or application, and then clicks the send button to send it to the server.

[0203] Step 2:

[0204] The server receives the input data

[0205] The server receives a list of documents submitted by the user, which are added to a processing queue.

[0206] Specific operation: The server receives the HTTP request, analyzes the sent data, and stores it in a database or queue.

[0207] Step 3:

[0208] The server generates the summary information

[0209] Based on the received list of documents, the server generates a summary of each document using a generative AI model. The summary information is obtained by inputting a prompt to the generative AI model.

[0210] Specific operation: The server calls OpenAI's API, sends the prompt sentence to the generative AI model, and obtains summary information.

[0211] Example prompt: Please provide an update and summary on "The Latest Smartphones."

[0212] Step 4:

[0213] The server aggregates the summary information

[0214] The generated summaries are compiled into a single document, which is then converted into a PDF or e-book format and prepared for delivery to users.

[0215] Specific operation: The server concatenates each summary information and converts it into PDF or e-book format using a document formatter.

[0216] Step 5:

[0217] The server provides the final product to the user

[0218] The generated document is sent to the user terminal, where it can be viewed or downloaded by the user.

[0219] Specific operation: The server transfers the file using the HTTP response so that the generated PDF or e-book file can be sent appropriately to the user's device.

[0220] Step 6:

[0221] The server receives product-related information

[0222] Information related to in-store products is collected from the internet and other data sources, and the collected information serves as input for generating summary information.

[0223] Specific operation: The server uses an API to obtain product information from a data source on the Internet.

[0224] Step 7:

[0225] The server generates a summary of the product information

[0226] Based on the collected product information, the server generates a summary using a generative AI model.

[0227] Specific operation: The server sends prompts to the generative AI model to summarize data including the latest product information and consumer reviews, and obtains summary information.

[0228] Step 8:

[0229] Aggregating and outputting summary information about products

[0230] The generated summaries are then compiled into a single document and provided to the user.

[0231] Specific operation: The server concatenates the summary information, converts it into PDF or e-book format, and sends it to the user's terminal to provide it to the user as a single document.

[0232] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0233] The present invention relates to a system that efficiently collects, organizes, and summarizes information using an information processing device. This system receives a list of multiple documents entered by the user, generates and consolidates summary information for each document, and creates and outputs a single document, thereby reducing the user's burden. It is also possible to collect information related to a specific category from the Internet, generate a guide, and provide it as an application. Furthermore, the present invention is equipped with an emotion engine that recognizes the user's emotions and has the function of adjusting the method of generating and providing information based on the user's emotions.

[0234] Overall processing flow of the program

[0235] The system operates as follows.

[0236] 1. The user inputs a document list

[0237] The user inputs a list of documents of interest from the terminal, for example, by entering titles such as "Document 1," "Document 2," and "Document 3." This information is sent to the server.

[0238] 2. The device acquires emotion data

[0239] The device collects user emotional data (facial expressions, voice, text input, etc.) using an emotion engine, which then uses the data to adjust the content and generation method of document summaries.

[0240] 3. The device sends the data to the server

[0241] The device sends both the document list and the emotion data to the server. The sent data is summarized in JSON format, for example, as follows: {"documents":["Document 1", "Document 2", "Document 3"], "emotions":{"happiness":0.8, "sadness":0.1}}.

[0242] 4. The server receives the document list and begins processing.

[0243] Based on the document list received by the server, natural language processing technology is used to generate summary information for each document. The server calls a summary generation API for the title of each document to obtain the summary information.

[0244] 5. The server analyzes the emotion data and adjusts the generated content

[0245] The server analyzes the received emotional data and adjusts the summary method according to the user's emotions. For example, if the user is excited, it will provide detailed summary information, but if the user is calm, it will provide a short summary of the main points.

[0246] 6. The server aggregates the summary information

[0247] The server then compiles the summaries into a single document, concatenating each summary piece in turn to create a final document that includes all the summaries.

[0248] 7. The server delivers the final product to the user

[0249] The completed best version document is converted into PDF or e-book format and provided to the user, who can then download or stream the final document to their device.

[0250] 8. Generating Guides and Applications

[0251] When a user selects a specific category (e.g., Excel functions), the server collects and filters related information from the internet to generate a comprehensive guide, which is then tailored based on the user's preferences and ultimately delivered as a mobile or web app.

[0252] Specific examples

[0253] Example 1: CEO Book Summary

[0254] The user enters the following list:

[0255] ["CEO Book 1", "CEO Book 2", "CEO Book 3"]

[0256] The device acquires emotion data from the user's facial expressions and voice, then sends it to the server:

[0257] {"documents":["CEO Book 1", "CEO Book 2", "CEO Book 3"], "emotions":{"happiness":0.7, "interest":0.9}}

[0258] The server gets a summary from the chat AI for each book and adjusts the summary method based on the emotion data:

[0259] Summary 1: "CEO Book 1 Summary"

[0260] Summary 2: "CEO Book 2 Summary"

[0261] Summary 3: "CEO Book 3 Summary"

[0262] The server creates the final best version of the document:

[0263] Final Document: "CEO Book 1 Summary\n\nCEO Book 2 Summary\n\nCEO Book 3 Summary"

[0264] Example 2: Creating a guide on Excel functions

[0265] User selected categories:

[0266] "Excel Functions"

[0267] The device acquires the user's emotion data and then sends it to the server:

[0268] {"category":"Excel Functions", "emotions":{"focus":0.8, "confidence":0.7}}

[0269] The server collects relevant information from the internet and filters it based on emotion data:

[0270] Information 1: "Basic Excel Functions"

[0271] Information 2: "Advanced Excel Functions"

[0272] Information 3: "How to use Excel functions"

[0273] The server generates guides based on collected information and adjusts the content based on the user's emotions:

[0274] Guide: "Basic Excel Functions\n\nAdvanced Excel Functions\n\nHow to Use Excel Functions"

[0275] Build the server-generated guide into an application and serve it to users:

[0276] App: "Excel Function Guide App"

[0277] This allows the user to easily obtain detailed information about the specified category and use the information in a way that best suits their own feelings.

[0278] The processing flow will be explained below.

[0279] Program processing steps

[0280] Example 1: Compiling the CEO's top 10 books using chat generation AI

[0281] Step 1:

[0282] The user inputs a list of CEO's books from the terminal. For example, the user inputs titles such as "CEO's Book 1," "CEO's Book 2," and "CEO's Book 3."

[0283] Step 2:

[0284] The device acquires emotion data from the user's facial expressions and voice. The emotion engine analyzes the user's camera footage and microphone input to detect their emotional state (e.g., happiness, excitement, etc.).

[0285] Step 3:

[0286] The terminal transmits the input document list and the acquired emotion data to the server. The transmitted data is in a format that includes the document list and emotion data.

[0287] Step 4:

[0288] The server receives the document list and begins the process of generating summary information for each title. The server uses natural language processing techniques to generate a summary for each document.

[0289] Step 5:

[0290] The server analyzes the emotion data and adjusts the summary content based on the user's emotion, for example, generating a detailed and comprehensive summary when the user is in a high excitement state, and a concise summary when the user is in a low excitement state.

[0291] Step 6:

[0292] The server aggregates the generated summaries into a single document, and then concatenates all the summaries to create a final, best-of document.

[0293] Step 7:

[0294] The server converts the final product into a PDF or e-book format, which makes it easy for users to view.

[0295] Step 8:

[0296] The server provides the converted file to the user, who then downloads or streams the final document from their device.

[0297] Example 2: Picking out frequently used information and turning it into an app

[0298] Step 1:

[0299] The user selects the category (e.g. Excel Functions) that he is interested in from the terminal. For example, he selects the category as "Excel Functions".

[0300] Step 2:

[0301] The device acquires emotion data from the user's facial expressions and voice. The emotion engine recognizes facial expressions and analyzes voice to detect the user's emotional state.

[0302] Step 3:

[0303] The terminal transmits the selected category information and emotion data to the server. The transmitted data includes the category information and emotion data.

[0304] Step 4:

[0305] The server receives the category information and collects related information from the Internet. Web scraping technology is used to gather useful information about Excel functions.

[0306] Step 5:

[0307] The server verifies and filters the information collected, eliminating duplicate information and selecting useful information.

[0308] Step 6:

[0309] The server analyzes the emotional data and adjusts the way the collected information is organized and the content of the guide based on the user's emotions. For example, if the user is feeling confident, it will include more advanced content, and conversely, if the user is feeling anxious, it will emphasize basic content.

[0310] Step 7:

[0311] The server generates the guide and builds it into an application. Based on the collected information, a comprehensive guide is created and deployed as a mobile or web app.

[0312] Step 8:

[0313] The server provides the completed application to the user, who receives a link to download or use the app on their device.

[0314] Through the above processing steps, the user can easily obtain detailed information about the specified category and can use the information in a way that is optimal for his or her own feelings.

[0315] Example 2

[0316] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0317] Conventional information processing systems require users to collect information, summarize each document individually, and compile it, requiring a great deal of effort. Furthermore, the summarized information is uniform and not tailored to the user's emotions or interests, preventing optimal information provision. Furthermore, it is difficult to provide comprehensive guidance related to specific categories of information in a format tailored to each individual user. To solve these problems, a new system is needed that can collect, summarize, aggregate, and tailor information based on the user's emotions.

[0318] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0319] In this invention, the server includes means for receiving a list of multiple documents input by a user, means for acquiring user emotion data, and means for transmitting the acquired emotion data and list to the server. This enables the generation and provision of optimal document summary information based on the user's emotion. The server also includes means for using natural language processing technology to generate document summary information and means for adjusting the generated summary information based on the user's emotion data. This enables the creation of highly accurate summary documents that reflect the user's emotion. The server also includes means for aggregating the adjusted summary information to create a single document and means for outputting the created document. This enables the information desired by the user to be provided as a single comprehensive document, thereby realizing efficient collection and use of information.

[0320] An "information processing device" is a device that processes data and organizes information using a combination of computer systems and software.

[0321] A "document list" is a list that compiles the titles and contents of multiple documents entered or selected by the user.

[0322] "Natural language processing technology" is a technology that enables computers to understand and process human language, and is used for generating summaries and semantic analysis.

[0323] "Emotion data" is data that indicates the emotional state of a user, obtained by analyzing the user's facial expression, voice, text input, and the like.

[0324] "Summary information" is information that shortens the content of the original document and succinctly summarizes the main points and content.

[0325] "Aggregation" means combining multiple elements or pieces of information into one.

[0326] "Document creation" refers to the process of composing multiple pieces of summary information into a single continuous document and converting it into an outputtable format.

[0327] "Output" refers to providing the generated data or document to the user, and includes means such as displaying it on the screen, generating a PDF, or providing a download link.

[0328] "Category information" is data that indicates information about a particular theme or field in which a user is interested.

[0329] A "guide" is a comprehensive instruction manual or explanatory document that provides information on a particular topic.

[0330] "Application" means a program that provides specific functions or services, including mobile applications and web applications.

[0331] This invention relates to a system that uses an information processing device to efficiently collect, organize, and summarize information. This system receives a list of multiple documents entered by the user, generates summary information for each document, and aggregates them to create and output a single document, thereby reducing the burden on the user. It is also possible to collect information related to a specific category from the Internet, generate a guide, and provide it as an application.

[0332] Hardware and software used

[0333] Hardware

[0334] User devices (PCs, smartphones, tablets, etc.)

[0335] Server (high-performance data processing system)

[0336] software

[0337] Natural Language Processing (NLP) engine (e.g., OpenAI GPT-4 (registered trademark))

[0338] Sentiment Analysis Engine

[0339] Generative AI Models

[0340] Data processing and calculation

[0341] 1. The user inputs a document list

[0342] The user inputs the titles of the documents they are interested in into the input form on their device. For example, they input them in a list format, such as "Document A," "Document B," "Document C," etc. This creates a document list and sends it to the server.

[0343] 2. The device acquires emotion data

[0344] The device uses a built-in emotion analysis engine to collect user emotional data in real time. For example, it uses the device's built-in camera and microphone to analyze the user's facial expressions and tone of voice. This allows it to quantify emotional parameters such as how happy or excited the user is.

[0345] 3. The device sends the data to the server

[0346] The device encodes the collected document list and emotion data into JSON format and sends it to the server. For example, the data is sent in the following format:

[0347] {"documents":["Document A", "Document B", "Document C"], "emotions":{"happiness":0.8, "sadness":0.1}}

[0348] 4. The server receives the document list and begins processing.

[0349] The server uses natural language processing (NLP) technology to generate summary information for each document based on the received document list. The NLP engine calls a summary generation API based on the title of each document to obtain the summary information.

[0350] 5. The server analyzes the emotion data and adjusts the generated content

[0351] The server adjusts the summarization method based on the received emotion data according to the user's emotional state, for example, providing a detailed summary when the user is excited and generating a concise summary when the user is calm.

[0352] 6. The server aggregates the summary information

[0353] The server aggregates the obtained summaries into a single document, and then sequentially concatenates the summaries to create a single final document containing all the summaries.

[0354] 7. The server delivers the final product to the user

[0355] The final product is converted into PDF or e-book format and provided to the user, who can then download or stream the final document to their device.

[0356] 8. The server generates the guide and application

[0357] When a user selects a specific category, the server collects and filters related information from the internet to generate a comprehensive guide, which is then tailored based on the user's sentiment and ultimately provided as a mobile or web app.

[0358] Specific examples

[0359] Example 1: CEO Book Summary

[0360] The user enters the following list:

[0361] ["CEO Book 1", "CEO Book 2", "CEO Book 3"]

[0362] The device acquires emotion data from the user's facial expressions and voice, then sends it to the server:

[0363] {"documents":["CEO Book 1", "CEO Book 2", "CEO Book 3"], "emotions":{"happiness":0.7, "interest":0.9}}

[0364] The server uses natural language processing to obtain a summary for each book and adjusts the summary method based on sentiment data:

[0365] Summary 1: "CEO Book 1 Summary"

[0366] Summary 2: "CEO Book 2 Summary"

[0367] Summary 3: "CEO Book 3 Summary"

[0368] The server creates the final document:

[0369] Final Document: "CEO Book 1 Summary\n\nCEO Book 2 Summary\n\nCEO Book 3 Summary"

[0370] Example 2: Creating a guide on Excel functions

[0371] User selected categories:

[0372] "Excel Functions"

[0373] The device acquires the user's emotion data and then sends it to the server:

[0374] {"category":"Excel Functions", "emotions":{"focus":0.8, "confidence":0.7}}

[0375] The server collects relevant information from the internet and filters it based on emotion data:

[0376] Information 1: "Basic Excel Functions"

[0377] Information 2: "Advanced Excel Functions"

[0378] Information 3: "How to use Excel functions"

[0379] The server generates guides based on collected information and adjusts the content based on the user's emotions:

[0380] Guide: "Basic Excel Functions\n\nAdvanced Excel Functions\n\nHow to Use Excel Functions"

[0381] Build the server-generated guide into an application and serve it to users:

[0382] App: "Excel Function Guide App"

[0383] By executing the above processing procedure, the present invention collects information, summarizes it, and generates guides optimally based on the user's emotions, thereby improving the user experience.

[0384] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0385] Step 1:

[0386] The user enters a document list

[0387] Input: A user inputs a list of document titles into their device, for example, "Document A," "Document B," "Document C," etc.

[0388] Specific operation: The user enters document titles one by one into the input form on the terminal and presses the "Send" button.

[0389] Output: The input document list is saved as local data on the device.

[0390] Step 2:

[0391] The device acquires emotion data

[0392] Input: Real-time data such as the user's facial image and voice.

[0393] How it works: The device's built-in camera and microphone capture the user's facial expressions and tone of voice, and an emotion analysis engine analyzes this data.

[0394] Data processing: The emotion analysis engine converts the user's emotional status into a numerical value, such as "happiness=0.8" or "sadness=0.1."

[0395] Output: Quantified emotion data is generated.

[0396] Step 3:

[0397] The device sends the data to the server

[0398] Input: The document list generated in step 1 and the emotion data generated in step 2.

[0399] Specific operation: The device encodes the document list and emotion data into JSON format and sends it to the server via an HTTP request.

[0400] Output: The data sent to the server looks like this:

[0401] {"documents":["Document A", "Document B", "Document C"], "emotions":{"happiness":0.8, "sadness":0.1}}

[0402] Step 4:

[0403] The server receives the document list and starts processing

[0404] Input: JSON data containing the document list sent in step 3.

[0405] Specific operation: The server receives the HTTP request and parses the document list.

[0406] Data calculation: Using a natural language processing (NLP) engine, a summary generation API is called for each document title to obtain summary information.

[0407] Output: A summary is generated for each document.

[0408] Step 5:

[0409] The server analyzes the emotional data and adjusts the generated content

[0410] Input: The summary information generated in step 4 and the emotion data sent in step 3.

[0411] Specific operation: The server analyzes the emotion data and adjusts each summary information according to the user's emotional state.

[0412] Data processing: For example, providing a detailed summary when the user is excited and adjusting the summary to a concise one when the user is calm.

[0413] Output: A tailored summary is generated.

[0414] Step 6:

[0415] The server aggregates the summary information

[0416] Input: Summary information refined in step 5.

[0417] Specific operation: The server concatenates each abstract and creates one final document that contains all the abstracts.

[0418] Data processing: Arrange the summary information according to a sequential linking algorithm.

[0419] Output: The final aggregated document is generated.

[0420] Step 7:

[0421] The server provides the final product to the user.

[0422] Input: The aggregate document generated in step 6.

[0423] Specific operation: The server converts the aggregate document into PDF format and provides a download link to the user's device.

[0424] Output: The final document in PDF format is provided to the user, who can download or view it on their device.

[0425] Step 8:

[0426] The server generates the guides and applications.

[0427] Input: Specific category information selected by the user.

[0428] Specific operation: The server collects relevant information from the Internet and filters the information based on emotion data.

[0429] Data Computing: Generate comprehensive guides based on collected information and adjust content based on user sentiment.

[0430] Output: A finished guide delivered as a mobile app or web app.

[0431] (Application example 2)

[0432] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0433] Currently, physical stores require staff to respond to customers quickly and appropriately, but it is often difficult to provide the optimal response and information for customer questions and requests.In addition, it is difficult to provide information based on the customer's emotions, which makes it difficult to improve customer satisfaction.

[0434] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving a list of multiple input documents, means for generating summary information for each document in the list, means for aggregating the generated summary information to create a single document, means for outputting the created document, means for acquiring related information based on the input question content and emotion data, and means for proposing the optimal response method based on the acquired information. This enables staff in physical stores to respond quickly and appropriately to customer questions and requests, and further improves customer satisfaction by providing optimal information based on customer emotions.

[0435] An "information processing device" is a device for analyzing and processing input information.

[0436] A "document list" is a list that includes the titles and contents of multiple documents.

[0437] "Summary information" is information that briefly summarizes the important content of the entire document.

[0438] "Emotional data" is data that indicates the user's emotional state and is generated based on facial expressions, voice, text input, etc.

[0439] "Natural language processing technology" is a technology that allows computers to understand and process human language.

[0440] "Means for collecting relevant information from the Internet" refers to methods for searching data on the Internet and collecting the necessary information.

[0441] The means for generating a "guide" is a method for creating a guide that is useful to users based on the collected information.

[0442] An "application" is a software program used for a specific function or purpose.

[0443] "Customer support" refers to providing support to store staff to provide appropriate support to customers.

[0444] "Means for providing optimal information" refers to a method for selecting and providing the most appropriate information based on user input and emotional data.

[0445] The present invention provides an information processing system for supporting customer service in brick-and-mortar stores. This system uses an information processing device to generate a summary of a document and provide appropriate information based on emotion data. A specific embodiment of this system is described below.

[0446] Hardware and software used

[0447] 1. Hardware:

[0448] Devices such as smartphones and smart glasses: These are devices that staff carry and use.

[0449] Server: Used for processing and storing data.

[0450] 2. Software:

[0451] Natural language processing technology: Technology for summarizing documents. For example, Python libraries such as NLTK and spaCy are used.

[0452] API: An interface for obtaining information corresponding to customer questions, for example, using a RESTful API.

[0453] Emotion engine: An engine for analyzing customer emotion data, for example, using OpenFace or Emotion API.

[0454] Processing flow

[0455] 1. User input:

[0456] The user (staff member) inputs customer questions via a smartphone or smart glasses, and this information is sent to the server, which is the starting point for processing.

[0457] 2. Collecting Emotional Data:

[0458] The device collects emotional data from the user's facial expressions and voice using a camera and microphone, and analyzes the data using an emotion engine. The collected emotional data is then sent to the server along with the questions asked.

[0459] 3. Document and information summary generation:

[0460] Based on the received question, the server retrieves appropriate document and product information from a database, and generates summary information from this information using natural language processing technology.

[0461] 4. Adjust based on sentiment data:

[0462] Based on the analyzed emotional data, the server adjusts the way it presents information, for example, providing detailed explanations if the customer is excited, or a concise summary if the customer is calm.

[0463] 5. Recommendations for the best response:

[0464] The server then proposes the optimal response based on the acquired information, which provides specific guidance on how staff should respond to the customer.

[0465] 6. Provision of Information:

[0466] The final generated information and response methods are displayed on the staff's terminal, allowing the staff to immediately provide the appropriate information to the customer.

[0467] Specific examples

[0468] For example, if a customer asks a store employee, "What is the material of this shirt?", the employee inputs the question through the smart glasses. The device captures emotional data of the customer's happiness from their facial expression and sends it to the server. The server then retrieves information from a database of related products and generates a detailed summary of the shirt's material. The server then analyzes the emotional data and, if it determines that the customer is interested in purchasing, suggests appropriate actions to encourage the purchase. Finally, this information and actions are displayed on the employee's smart glasses, allowing them to immediately explain the situation to the customer.

[0469] Prompt Sentence Examples

[0470] User: What is the material of this shirt?

[0471] Emotion Data: Customers are interested.

[0472] Requested: Product information presentation and customer support suggestions

[0473] This invention allows staff in physical stores to respond to customers quickly and optimally, thereby improving customer satisfaction.

[0474] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0475] Step 1:

[0476] The user uses an input device (such as a smartphone or smart glasses) to input questions or requests from customers. The information entered by the user is temporarily stored on the device. The input is made using a text input interface or a voice recognition system.

[0477] Input: Customer question or request (e.g., "What is the material of this shirt?")

[0478] Output: Text data of the question

[0479] Step 2:

[0480] The device analyzes the user's facial expressions and tone of voice to obtain emotional data. This is done using sensors such as a camera and microphone, and analysis is performed using an emotion engine (e.g., OpenFace or Emotion API). The analysis results are stored on the device as a quantified emotional state.

[0481] Input: User facial expressions and tone of voice

[0482] Output: Quantified emotion data (e.g., happiness, interest, etc.)

[0483] Step 3:

[0484] The device packages the question and emotion data and sends it to the server, typically in JSON format, via internet.

[0485] Input: Question text and sentiment data

[0486] Output: Packaged data (JSON format)

[0487] Step 4:

[0488] The server analyzes the received data and retrieves product information corresponding to the question from the database. At this time, the server retrieves information from the external database using a RESTful API.

[0489] Input: Packaged data (JSON format)

[0490] Output: Retrieved product information

[0491] Step 5:

[0492] The server generates a summary of the document based on the product information it has acquired. This process uses natural language processing techniques (e.g., NLTK and spaCy). The summary is generated in a concise and to-the-point format.

[0493] Input: Retrieved product information

[0494] Output: Summary information

[0495] Step 6:

[0496] The server analyzes the emotion data and adjusts the summary presentation, for example, including detailed information if the customer is interested, or providing a more concise summary if the customer is not. The adjustments based on emotion data are made using a generative AI model.

[0497] Input: Summary information and emotion data

[0498] Output: Adjusted summary information

[0499] Step 7:

[0500] The server proposes the optimal response method. To do this, it refers to past data and case studies and generates specific response methods and wording, etc. This is also done using a generative AI model.

[0501] Input: Adjusted summary information and emotion data

[0502] Output: Best Practices

[0503] Step 8:

[0504] The server then sends the final information (adjusted summary information and optimal response methods) to the terminal, which displays this information for immediate use by staff. The display can be done using a text display interface or a speech synthesis system.

[0505] Input: Final generated information

[0506] Output: Information displayed on the staff member's terminal

[0507] This series of processes enables staff in physical stores to provide customers with information and respond in the most prompt and optimal way.

[0508] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0509] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0510] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0511] [Second embodiment]

[0512] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0513] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0514] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0515] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0516] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0517] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0518] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0519] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0520] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0521] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0522] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0523] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0524] This invention relates to a system that uses an information processing device to efficiently collect, organize, and summarize information. This system receives a list of multiple documents entered by the user, generates and aggregates summary information for each document, and creates and outputs a single document, thereby reducing the burden on the user. It is also possible to collect information related to a specific category from the Internet, generate a guide, and provide it as an application.

[0525] Overall processing flow of the program

[0526] The system operates as follows.

[0527] 1. User enters CEO's book list

[0528] The user inputs the titles of multiple books related to the CEO from the terminal. This information is sent to the server. For example, the user inputs "CEO's Book 1" and "CEO's Book 2" and sends them to the server.

[0529] 2. The server receives the input data and starts processing it.

[0530] The server receives the list sent by the user and uses natural language processing technology to generate summary information for each book in the list. For each book title, the server calls a summary generation API to obtain summary information.

[0531] 3. The server aggregates the summary information

[0532] The server then compiles the obtained summaries into a single document. For example, it concatenates the summaries of "CEO Book 1" and "CEO Book 2" in order to create a final best-of document.

[0533] 4. The server delivers the final product to the user

[0534] The completed best-of document is converted into PDF or e-book format and provided to users, allowing them to see a summary without having to read each individual book.

[0535] Specific examples

[0536] Example 1: CEO Book Summary

[0537] The user enters the following list:

[0538] ["CEO Book 1", "CEO Book 2", "CEO Book 3"]

[0539] The server gets a summary from the chat AI for each book:

[0540] Summary 1: "CEO Book 1 Summary"

[0541] Summary 2: "CEO Book 2 Summary"

[0542] Summary 3: "CEO Book 3 Summary"

[0543] The server creates the final best version of the document:

[0544] Final Document: "CEO Book 1 Summary\n\nCEO Book 2 Summary\n\nCEO Book 3 Summary"

[0545] Example 2: Creating a guide on Excel functions

[0546] User selected categories:

[0547] "Excel Functions"

[0548] The server gathers relevant information from the Internet:

[0549] Information 1: "Basic Excel Functions"

[0550] Information 2: "Advanced Excel Functions"

[0551] Information 3: "How to use Excel functions"

[0552] The server generates a guide based on the information collected:

[0553] Guide: "Basic Excel Functions\n\nAdvanced Excel Functions\n\nHow to Use Excel Functions"

[0554] Build the server-generated guide into an application and serve it to users:

[0555] App: "Excel Function Guide App"

[0556] This allows users to easily refer to a comprehensive guide to Excel functions.

[0557] The processing flow will be explained below.

[0558] Program processing steps

[0559] Example 1: Compiling the CEO's top 10 books using chat generation AI

[0560] Step 1:

[0561] The user inputs a list of CEO's books from the terminal. For example, the user inputs titles such as "CEO's Book 1," "CEO's Book 2," and "CEO's Book 3."

[0562] Step 2:

[0563] The terminal sends the entered list to the server. The sent list is sent in JSON format, for example, as "["CEO's Book 1", "CEO's Book 2", "CEO's Book 3"]".

[0564] Step 3:

[0565] The server receives the user input, and upon receipt, performs data integrity checks and format validation.

[0566] Step 4:

[0567] The server sends a request to the chat generation AI to generate summary information for each document title in the received list. Specifically, it sends an API request to each title to obtain summary information.

[0568] Step 5:

[0569] The server receives the summary information returned by the chat generation AI. For example, the response may be something like "Summary of CEO's Book 1," "Summary of CEO's Book 2," or "Summary of CEO's Book 3."

[0570] Step 6:

[0571] The server aggregates the received summaries to create a single document, specifically by concatenating each summary in turn to create a final document that includes all summaries.

[0572] Step 7:

[0573] The server converts the final document into a PDF or e-book format, making it easily viewable by users.

[0574] Step 8:

[0575] The server provides the converted file to the user, who then downloads or streams the final document from their device.

[0576] Example 2: Picking out frequently used information and turning it into an app

[0577] Step 1:

[0578] The user selects a category of interest (e.g., Excel Functions) from their device, for example by using a drop-down menu or search bar to select "Excel Functions."

[0579] Step 2:

[0580] The terminal transmits the selected category information to the server. For example, the terminal transmits data indicating that the category is "Excel Functions."

[0581] Step 3:

[0582] The server receives the user's selection and begins collecting relevant information, specifically information about Excel functions from internet resources using techniques such as web scraping.

[0583] Step 4:

[0584] The server verifies and filters the collected information, eliminating duplicate information and selecting useful information.

[0585] Step 5:

[0586] The server generates a guide based on the useful information it collects. Specifically, it organizes the information and creates a comprehensive guide divided into chapters and sections.

[0587] Step 6:

[0588] The server-generated guide is then built into an application, for example, developed into a format that can be deployed as a mobile app or web app.

[0589] Step 7:

[0590] The server provides the completed application to the user, who receives a download link or a web app URL to use the application.

[0591] This allows the user to easily obtain and use detailed information about the specified category.

[0592] Example 1

[0593] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0594] Conventional information processing devices require users to individually read a large number of documents, which requires a significant amount of time and effort. Furthermore, there are insufficient methods for efficiently collecting and organizing information related to a specific category. Therefore, there is a need for a system that can efficiently collect, organize, and summarize information.

[0595] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0596] In this invention, the server includes means for receiving a list of input documents, means for generating summary information for each document in the list using a generative AI model, means for aggregating the generated summaries to create a single document, and means for converting the created document into a PDF or e-book format and outputting it, allowing users to efficiently collect information and check summaries without having to read each document individually.

[0597] An "information processing device" is a device that processes data, receives information input by a user, performs specific processing, and provides an output result.

[0598] The "means for receiving a list of documents" is a means having a function for taking in the titles and contents of a plurality of documents input by the user in list form and transmitting the list to the server.

[0599] A "generative AI model" is a model that uses artificial intelligence technology to analyze text data and generate summary information. Examples include generative AI models that use natural language processing technology.

[0600] The "means for generating summary information" is a means for concisely summarizing the contents of a document and extracting its essence.

[0601] The "means for aggregating summary information to create one document" refers to a means for aggregating multiple pieces of summary information, linking them, and organizing them into one unified document.

[0602] "Means of converting to PDF or e-book format and outputting" refers to the means of converting created documents into a format that is easy for users to view and providing them as electronic files. PDF and e-book formats are examples of such formats.

[0603] "Category information" refers to information related to a specific category, and is information for identifying themes or fields in which a user is interested.

[0604] The "means for collecting related information" refers to a means for searching and collecting related information from the Internet based on the input category information.

[0605] "Means for generating a guide" refers to means for organizing collected information and creating guides or manuals for specific purposes.

[0606] The "means for constructing as an application" refers to a means for converting the generated guide into an application in a form that can be used by the user and providing it to the user.

[0607] This invention relates to a system that uses an information processing device to efficiently collect, organize, and summarize information. This system receives a list of multiple documents entered by a user, generates summary information for each document using a generative AI model, aggregates them to create a single document, and converts and outputs the summary in PDF or e-book format, thereby reducing the burden on the user.

[0608] First, the user inputs a list of multiple documents using a terminal. For example, the user inputs the document titles such as "CEO's Book 1" and "CEO's Book 2" in a browser or dedicated application. The input documents are sent from the terminal to the server using an HTTP POST request.

[0609] The server receives the list of documents sent by the user and analyzes their contents. Next, it generates a summary for each document title using a generative AI model (e.g., OpenAI's GPT-3, which uses natural language processing technology). The server sends a prompt to the generative AI model to obtain the summary.

[0610] As a concrete example, suppose the user inputs the following list:

[0611] ["CEO Book 1", "CEO Book 2", "CEO Book 3"]

[0612] In response, the server sends the following prompt to the generative AI model:

[0613] "Generate a summary for each of the following book titles:\n- CEO Book 1\n- CEO Book 2\n- CEO Book 3"

[0614] Each summary is received from the generative AI model, and a summary like the one below is returned.

[0615] Summary 1: "CEO Book 1 Summary"

[0616] Summary 2: "CEO Book 2 Summary"

[0617] Summary 3: "CEO Book 3 Summary"

[0618] The server aggregates these summaries into a single document and organizes them, for example, as follows:

[0619] Final Document: "CEO Book 1 Summary\n\nCEO Book 2 Summary\n\nCEO Book 3 Summary"

[0620] The server then converts the final document into a format that is easy for users to view, such as PDF or e-book format, using a PDF generation tool or e-book format conversion tool.

[0621] Finally, the server provides the converted document to the user, for example by sending the user an email containing a download link for the generated file, or by displaying the link on a dedicated download page.

[0622] This system allows users to efficiently collect large amounts of information and check summaries, significantly reducing the burden of information processing.It also has the function of automatically collecting, organizing, and summarizing information related to specific categories from the Internet, making it suitable for collecting a variety of information.

[0623] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0624] Step 1:

[0625] The user enters a list of documents

[0626] Input: A user uses a terminal to enter the titles of several documents, for example, "CEO Book 1," "CEO Book 2," "CEO Book 3," etc.

[0627] How it works: A user enters a list of documents into a form in a browser or dedicated application and presses the submit button.

[0628] Output: The input data is prepared as a JSON list and sent to the next processing step.

[0629] Step 2:

[0630] The device sends the data to the server

[0631] Input: A list of documents entered by the user.

[0632] How it works: The device sends JSON-formatted data to the server using an HTTP POST request.

[0633] Output: The data reaches the server and goes on to the next processing step.

[0634] Step 3:

[0635] The server receives the input data and starts processing it.

[0636] Input: Document list sent from the terminal.

[0637] Behavior: The server parses the received data and extracts a list of document titles.

[0638] Output: The extracted list of document titles is used to generate summaries in the next step.

[0639] Step 4:

[0640] The server makes a request to generate a summary

[0641] Input: A list of extracted document titles.

[0642] How it works: The server calls a generative AI model (e.g., OpenAI GPT-3) and creates a prompt for generating a summary. For example, "Generate a summary for the following book title: CEO Book 1." This prompt is then sent to the generative AI model.

[0643] Output: Summary information returned from the generative AI model.

[0644] Step 5:

[0645] The server retrieves the summary results

[0646] Input: Summary information from a generative AI model.

[0647] How it works: The server takes the summary information received from the generative AI model and organizes it by document title.

[0648] Output: Data to aggregate organized summary information into a single document.

[0649] Step 6:

[0650] The server aggregates the summary information

[0651] Input: Organized summary information.

[0652] What it does: Concatenates the summaries into a single document, organized by paragraph, for example, "Summary of CEO Book 1\n\nSummary of CEO Book 2\n\nSummary of CEO Book 3."

[0653] Output: The final summary document.

[0654] Step 7:

[0655] The server converts and outputs the final document

[0656] Input: Final summary document.

[0657] What it does: The server converts this document to PDF or e-book format using tools such as PDF generation tools and e-book format conversion tools.

[0658] Output: The converted document file (e.g. PDF), a download link provided to the user.

[0659] Step 8:

[0660] The server provides the final product to the user.

[0661] Input: The converted document file.

[0662] Action: The server initiates the steps to provide the generated file to the user, for example by sending an email containing a download link or by displaying the link on a dedicated download page.

[0663] Output: A user-accessible link or download page.

[0664] This system will enable users to efficiently collect information requested by the public and check summaries, thereby significantly reducing the burden of information processing.

[0665] (Application example 1)

[0666] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0667] Conventional information processing systems lack the means for users to efficiently collect, summarize, and use information from multiple documents. Furthermore, it is difficult to efficiently provide product information, particularly in brick-and-mortar stores, making it difficult for customers to quickly obtain the information they need.

[0668] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0669] In this invention, the server includes means for receiving a list of input documents, means for generating summary information for each document in the list, means for aggregating the generated summary information to create a single document, means for outputting the created document, means for receiving product-related information and generating summary information, and means for aggregating the generated summary information to create a single document and providing the document to the user. This makes it possible to efficiently collect, summarize, and provide information required by users and customers as a single document.

[0670] An "information processing device" is a combination of hardware and software for receiving, processing, generating, and outputting information.

[0671] The "means for receiving a list of documents" is a device having a function for receiving a list of titles and contents of a plurality of documents input by the user.

[0672] A "means for generating summary information" is an algorithm or software that extracts important information from an input document and generates a summary in a concise format.

[0673] The "means for aggregating summary information to create one document" refers to a device having a function for aggregating and integrating a plurality of pieces of generated summary information into one continuous document.

[0674] The "means for outputting a document" is a device that has the function of providing the created document to the user in a format such as PDF or e-book.

[0675] The "means for receiving information related to merchandise" is a device having a function for receiving merchandise information in a store and related market data.

[0676] The "means for providing to the user" is a device having a function for providing the generated information or document to the user in an accessible form.

[0677] The present invention relates to a system that uses an information processing device to efficiently collect, organize, and summarize information. This system receives a list of multiple documents entered by the user, generates and aggregates summary information for each document, and creates and outputs a single document, thereby reducing the burden on the user. It is also possible to collect information related to a specific category from the Internet, generate a guide, and provide it as an application. Furthermore, the system also has the function of efficiently collecting information related to products in physical stores and providing summary information.

[0678] The server has a means of receiving a list of multiple documents input by the user. This list consists of the titles and contents of documents that the user is interested in. The server then uses natural language processing techniques on these documents to generate summaries. The natural language processing techniques used include OpenAI APIs. Once the summaries are generated, the server aggregates them into a single document. Finally, the generated document is provided to the user in PDF or e-book format.

[0679] Next, the server receives information related to the product and has a means to generate summary information in the same way. This includes the process of collecting related information from the Internet. For example, when collecting the latest information and trend information about a product and generating summary information, the following prompt sentence is used for the generative AI model:

[0680] Example prompt sentence:

[0681] Provide updates and summaries on the "latest smartphones".

[0682] By inputting this prompt into a generative AI model, the latest information on the topic is obtained and a summary is generated, which is then compiled into a single document and ultimately provided to the user in PDF or e-book format.

[0683] For example, if a user enters a list such as "latest smartphones," "trendy fashion items," or "products on sale," the server collects and summarizes information about each item and creates a final, best-of document. This process allows users and customers to efficiently gather the information they need and obtain it in a summarized form.

[0684] In the embodiment of the invention, the input data is diverse, including information collected from the Internet, data entered by the user, data on products in the store, etc. This makes it possible to build a system that eliminates the complexity of information collection that users and customers face and can provide necessary information quickly and efficiently.

[0685] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0686] Step 1:

[0687] User enters a list of documents

[0688] A user inputs the titles and topics of multiple documents he or she wishes to summarize using a terminal, and the input list is sent to the server.

[0689] Specific operation: A user uses a smartphone or PC to enter the title of a document through a web interface or application, and then clicks the send button to send it to the server.

[0690] Step 2:

[0691] The server receives the input data

[0692] The server receives a list of documents submitted by the user, which are added to a processing queue.

[0693] Specific operation: The server receives the HTTP request, analyzes the sent data, and stores it in a database or queue.

[0694] Step 3:

[0695] The server generates the summary information

[0696] Based on the received list of documents, the server generates a summary of each document using a generative AI model. The summary information is obtained by inputting a prompt to the generative AI model.

[0697] Specific operation: The server calls OpenAI's API, sends the prompt sentence to the generative AI model, and obtains summary information.

[0698] Example prompt: Please provide an update and summary on "The Latest Smartphones."

[0699] Step 4:

[0700] The server aggregates the summary information

[0701] The generated summaries are compiled into a single document, which is then converted into a PDF or e-book format and prepared for delivery to users.

[0702] Specific operation: The server concatenates each summary information and converts it into PDF or e-book format using a document formatter.

[0703] Step 5:

[0704] The server provides the final product to the user

[0705] The generated document is sent to the user terminal, where it can be viewed or downloaded by the user.

[0706] Specific operation: The server transfers the file using the HTTP response so that the generated PDF or e-book file can be sent appropriately to the user's device.

[0707] Step 6:

[0708] The server receives product-related information

[0709] Information related to in-store products is collected from the internet and other data sources, and the collected information serves as input for generating summary information.

[0710] Specific operation: The server uses an API to obtain product information from a data source on the Internet.

[0711] Step 7:

[0712] The server generates a summary of the product information

[0713] Based on the collected product information, the server generates a summary using a generative AI model.

[0714] Specific operation: The server sends prompts to the generative AI model to summarize data including the latest product information and consumer reviews, and obtains summary information.

[0715] Step 8:

[0716] Aggregating and outputting summary information about products

[0717] The generated summaries are then compiled into a single document and provided to the user.

[0718] Specific operation: The server concatenates the summary information, converts it into PDF or e-book format, and sends it to the user's terminal to provide it to the user as a single document.

[0719] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0720] The present invention relates to a system that efficiently collects, organizes, and summarizes information using an information processing device. This system receives a list of multiple documents entered by the user, generates and consolidates summary information for each document, and creates and outputs a single document, thereby reducing the user's burden. It is also possible to collect information related to a specific category from the Internet, generate a guide, and provide it as an application. Furthermore, the present invention is equipped with an emotion engine that recognizes the user's emotions and has the function of adjusting the method of generating and providing information based on the user's emotions.

[0721] Overall processing flow of the program

[0722] The system operates as follows.

[0723] 1. The user inputs a document list

[0724] The user inputs a list of documents of interest from the terminal, for example, by entering titles such as "Document 1," "Document 2," and "Document 3." This information is sent to the server.

[0725] 2. The device acquires emotion data

[0726] The device collects user emotional data (facial expressions, voice, text input, etc.) using an emotion engine, which then uses the data to adjust the content and generation method of document summaries.

[0727] 3. The device sends the data to the server

[0728] The device sends both the document list and the emotion data to the server. The sent data is summarized in JSON format, for example, as follows: {"documents":["Document 1", "Document 2", "Document 3"], "emotions":{"happiness":0.8, "sadness":0.1}}.

[0729] 4. The server receives the document list and begins processing.

[0730] Based on the document list received by the server, natural language processing technology is used to generate summary information for each document. The server calls a summary generation API for the title of each document to obtain the summary information.

[0731] 5. The server analyzes the emotion data and adjusts the generated content

[0732] The server analyzes the received emotional data and adjusts the summary method according to the user's emotions. For example, if the user is excited, it will provide detailed summary information, but if the user is calm, it will provide a short summary of the main points.

[0733] 6. The server aggregates the summary information

[0734] The server then compiles the summaries into a single document, concatenating each summary piece in turn to create a final document that includes all the summaries.

[0735] 7. The server delivers the final product to the user

[0736] The completed best version document is converted into PDF or e-book format and provided to the user, who can then download or stream the final document to their device.

[0737] 8. Generating Guides and Applications

[0738] When a user selects a specific category (e.g., Excel functions), the server collects and filters related information from the internet to generate a comprehensive guide, which is then tailored based on the user's preferences and ultimately delivered as a mobile or web app.

[0739] Specific examples

[0740] Example 1: CEO Book Summary

[0741] The user enters the following list:

[0742] ["CEO Book 1", "CEO Book 2", "CEO Book 3"]

[0743] The device acquires emotion data from the user's facial expressions and voice, then sends it to the server:

[0744] {"documents":["CEO Book 1", "CEO Book 2", "CEO Book 3"], "emotions":{"happiness":0.7, "interest":0.9}}

[0745] The server gets a summary from the chat AI for each book and adjusts the summary method based on the emotion data:

[0746] Summary 1: "CEO Book 1 Summary"

[0747] Summary 2: "CEO Book 2 Summary"

[0748] Summary 3: "CEO Book 3 Summary"

[0749] The server creates the final best version of the document:

[0750] Final Document: "CEO Book 1 Summary\n\nCEO Book 2 Summary\n\nCEO Book 3 Summary"

[0751] Example 2: Creating a guide on Excel functions

[0752] User selected categories:

[0753] "Excel Functions"

[0754] The device acquires the user's emotion data and then sends it to the server:

[0755] {"category":"Excel Functions", "emotions":{"focus":0.8, "confidence":0.7}}

[0756] The server collects relevant information from the internet and filters it based on emotion data:

[0757] Information 1: "Basic Excel Functions"

[0758] Information 2: "Advanced Excel Functions"

[0759] Information 3: "How to use Excel functions"

[0760] The server generates guides based on collected information and adjusts the content based on the user's emotions:

[0761] Guide: "Basic Excel Functions\n\nAdvanced Excel Functions\n\nHow to Use Excel Functions"

[0762] Build the server-generated guide into an application and serve it to users:

[0763] App: "Excel Function Guide App"

[0764] This allows the user to easily obtain detailed information about the specified category and use the information in a way that best suits their own feelings.

[0765] The processing flow will be explained below.

[0766] Program processing steps

[0767] Example 1: Compiling the CEO's top 10 books using chat generation AI

[0768] Step 1:

[0769] The user inputs a list of CEO's books from the terminal. For example, the user inputs titles such as "CEO's Book 1," "CEO's Book 2," and "CEO's Book 3."

[0770] Step 2:

[0771] The device acquires emotion data from the user's facial expressions and voice. The emotion engine analyzes the user's camera footage and microphone input to detect their emotional state (e.g., happiness, excitement, etc.).

[0772] Step 3:

[0773] The terminal transmits the input document list and the acquired emotion data to the server. The transmitted data is in a format that includes the document list and emotion data.

[0774] Step 4:

[0775] The server receives the document list and begins the process of generating summary information for each title. The server uses natural language processing techniques to generate a summary for each document.

[0776] Step 5:

[0777] The server analyzes the emotion data and adjusts the summary content based on the user's emotion, for example, generating a detailed and comprehensive summary when the user is in a high excitement state, and a concise summary when the user is in a low excitement state.

[0778] Step 6:

[0779] The server aggregates the generated summaries into a single document, and then concatenates all the summaries to create a final, best-of document.

[0780] Step 7:

[0781] The server converts the final product into a PDF or e-book format, which makes it easy for users to view.

[0782] Step 8:

[0783] The server provides the converted file to the user, who then downloads or streams the final document from their device.

[0784] Example 2: Picking out frequently used information and turning it into an app

[0785] Step 1:

[0786] The user selects the category (e.g. Excel Functions) that he is interested in from the terminal. For example, he selects the category as "Excel Functions".

[0787] Step 2:

[0788] The device acquires emotion data from the user's facial expressions and voice. The emotion engine recognizes facial expressions and analyzes voice to detect the user's emotional state.

[0789] Step 3:

[0790] The terminal transmits the selected category information and emotion data to the server. The transmitted data includes the category information and emotion data.

[0791] Step 4:

[0792] The server receives the category information and collects related information from the Internet. Web scraping technology is used to gather useful information about Excel functions.

[0793] Step 5:

[0794] The server verifies and filters the information collected, eliminating duplicate information and selecting useful information.

[0795] Step 6:

[0796] The server analyzes the emotional data and adjusts the way the collected information is organized and the content of the guide based on the user's emotions. For example, if the user is feeling confident, it will include more advanced content, and conversely, if the user is feeling anxious, it will emphasize basic content.

[0797] Step 7:

[0798] The server generates the guide and builds it into an application. Based on the collected information, a comprehensive guide is created and deployed as a mobile or web app.

[0799] Step 8:

[0800] The server provides the completed application to the user, who receives a link to download or use the app on their device.

[0801] Through the above processing steps, the user can easily obtain detailed information about the specified category and can use the information in a way that is optimal for his or her own feelings.

[0802] Example 2

[0803] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0804] Conventional information processing systems require users to collect information, summarize each document individually, and compile it, requiring a great deal of effort. Furthermore, the summarized information is uniform and not tailored to the user's emotions or interests, preventing optimal information provision. Furthermore, it is difficult to provide comprehensive guidance related to specific categories of information in a format tailored to each individual user. To solve these problems, a new system is needed that can collect, summarize, aggregate, and tailor information based on the user's emotions.

[0805] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0806] In this invention, the server includes means for receiving a list of multiple documents input by a user, means for acquiring user emotion data, and means for transmitting the acquired emotion data and list to the server. This enables the generation and provision of optimal document summary information based on the user's emotion. The server also includes means for using natural language processing technology to generate document summary information and means for adjusting the generated summary information based on the user's emotion data. This enables the creation of highly accurate summary documents that reflect the user's emotion. The server also includes means for aggregating the adjusted summary information to create a single document and means for outputting the created document. This enables the information desired by the user to be provided as a single comprehensive document, thereby realizing efficient collection and use of information.

[0807] An "information processing device" is a device that processes data and organizes information using a combination of computer systems and software.

[0808] A "document list" is a list that compiles the titles and contents of multiple documents entered or selected by the user.

[0809] "Natural language processing technology" is a technology that enables computers to understand and process human language, and is used for generating summaries and semantic analysis.

[0810] "Emotion data" is data that indicates the emotional state of a user, obtained by analyzing the user's facial expression, voice, text input, and the like.

[0811] "Summary information" is information that shortens the content of the original document and succinctly summarizes the main points and content.

[0812] "Aggregation" means combining multiple elements or pieces of information into one.

[0813] "Document creation" refers to the process of composing multiple pieces of summary information into a single continuous document and converting it into an outputtable format.

[0814] "Output" refers to providing the generated data or document to the user, and includes means such as displaying it on the screen, generating a PDF, or providing a download link.

[0815] "Category information" is data that indicates information about a particular theme or field in which a user is interested.

[0816] A "guide" is a comprehensive instruction manual or explanatory document that provides information on a particular topic.

[0817] "Application" means a program that provides specific functions or services, including mobile applications and web applications.

[0818] This invention relates to a system that uses an information processing device to efficiently collect, organize, and summarize information. This system receives a list of multiple documents entered by the user, generates summary information for each document, and aggregates them to create and output a single document, thereby reducing the burden on the user. It is also possible to collect information related to a specific category from the Internet, generate a guide, and provide it as an application.

[0819] Hardware and software used

[0820] Hardware

[0821] User devices (PCs, smartphones, tablets, etc.)

[0822] Server (high-performance data processing system)

[0823] software

[0824] Natural Language Processing (NLP) engines (e.g., OpenAI GPT-4)

[0825] Sentiment Analysis Engine

[0826] Generative AI Models

[0827] Data processing and calculation

[0828] 1. The user inputs a document list

[0829] The user inputs the titles of the documents they are interested in into the input form on their device. For example, they input them in a list format, such as "Document A," "Document B," "Document C," etc. This creates a document list and sends it to the server.

[0830] 2. The device acquires emotion data

[0831] The device uses a built-in emotion analysis engine to collect user emotional data in real time. For example, it uses the device's built-in camera and microphone to analyze the user's facial expressions and tone of voice. This allows it to quantify emotional parameters such as how happy or excited the user is.

[0832] 3. The device sends the data to the server

[0833] The device encodes the collected document list and emotion data into JSON format and sends it to the server. For example, the data is sent in the following format:

[0834] {"documents":["Document A", "Document B", "Document C"], "emotions":{"happiness":0.8, "sadness":0.1}}

[0835] 4. The server receives the document list and begins processing.

[0836] The server uses natural language processing (NLP) technology to generate summary information for each document based on the received document list. The NLP engine calls a summary generation API based on the title of each document to obtain the summary information.

[0837] 5. The server analyzes the emotion data and adjusts the generated content

[0838] The server adjusts the summarization method based on the received emotion data according to the user's emotional state, for example, providing a detailed summary when the user is excited and generating a concise summary when the user is calm.

[0839] 6. The server aggregates the summary information

[0840] The server aggregates the obtained summaries into a single document, and then sequentially concatenates the summaries to create a single final document containing all the summaries.

[0841] 7. The server delivers the final product to the user

[0842] The final product is converted into PDF or e-book format and provided to the user, who can then download or stream the final document to their device.

[0843] 8. The server generates the guide and application

[0844] When a user selects a specific category, the server collects and filters related information from the internet to generate a comprehensive guide, which is then tailored based on the user's sentiment and ultimately provided as a mobile or web app.

[0845] Specific examples

[0846] Example 1: CEO Book Summary

[0847] The user enters the following list:

[0848] ["CEO Book 1", "CEO Book 2", "CEO Book 3"]

[0849] The device acquires emotion data from the user's facial expressions and voice, then sends it to the server:

[0850] {"documents":["CEO Book 1", "CEO Book 2", "CEO Book 3"], "emotions":{"happiness":0.7, "interest":0.9}}

[0851] The server uses natural language processing to obtain a summary for each book and adjusts the summary method based on sentiment data:

[0852] Summary 1: "CEO Book 1 Summary"

[0853] Summary 2: "CEO Book 2 Summary"

[0854] Summary 3: "CEO Book 3 Summary"

[0855] The server creates the final document:

[0856] Final Document: "CEO Book 1 Summary\n\nCEO Book 2 Summary\n\nCEO Book 3 Summary"

[0857] Example 2: Creating a guide on Excel functions

[0858] User selected categories:

[0859] "Excel Functions"

[0860] The device acquires the user's emotion data and then sends it to the server:

[0861] {"category":"Excel Functions", "emotions":{"focus":0.8, "confidence":0.7}}

[0862] The server collects relevant information from the internet and filters it based on emotion data:

[0863] Information 1: "Basic Excel Functions"

[0864] Information 2: "Advanced Excel Functions"

[0865] Information 3: "How to use Excel functions"

[0866] The server generates guides based on collected information and adjusts the content based on the user's emotions:

[0867] Guide: "Basic Excel Functions\n\nAdvanced Excel Functions\n\nHow to Use Excel Functions"

[0868] Build the server-generated guide into an application and serve it to users:

[0869] App: "Excel Function Guide App"

[0870] By executing the above processing procedure, the present invention collects information, summarizes it, and generates guides optimally based on the user's emotions, thereby improving the user experience.

[0871] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0872] Step 1:

[0873] The user enters a document list

[0874] Input: A user inputs a list of document titles into their device, for example, "Document A," "Document B," "Document C," etc.

[0875] Specific operation: The user enters document titles one by one into the input form on the terminal and presses the "Send" button.

[0876] Output: The input document list is saved as local data on the device.

[0877] Step 2:

[0878] The device acquires emotion data

[0879] Input: Real-time data such as the user's facial image and voice.

[0880] How it works: The device's built-in camera and microphone capture the user's facial expressions and tone of voice, and an emotion analysis engine analyzes this data.

[0881] Data processing: The emotion analysis engine converts the user's emotional status into a numerical value, such as "happiness=0.8" or "sadness=0.1."

[0882] Output: Quantified emotion data is generated.

[0883] Step 3:

[0884] The device sends the data to the server

[0885] Input: The document list generated in step 1 and the emotion data generated in step 2.

[0886] Specific operation: The device encodes the document list and emotion data into JSON format and sends it to the server via an HTTP request.

[0887] Output: The data sent to the server looks like this:

[0888] {"documents":["Document A", "Document B", "Document C"], "emotions":{"happiness":0.8, "sadness":0.1}}

[0889] Step 4:

[0890] The server receives the document list and starts processing

[0891] Input: JSON data containing the document list sent in step 3.

[0892] Specific operation: The server receives the HTTP request and parses the document list.

[0893] Data calculation: Using a natural language processing (NLP) engine, a summary generation API is called for each document title to obtain summary information.

[0894] Output: A summary is generated for each document.

[0895] Step 5:

[0896] The server analyzes the emotional data and adjusts the generated content

[0897] Input: The summary information generated in step 4 and the emotion data sent in step 3.

[0898] Specific operation: The server analyzes the emotion data and adjusts each summary information according to the user's emotional state.

[0899] Data processing: For example, providing a detailed summary when the user is excited and adjusting the summary to a concise one when the user is calm.

[0900] Output: A tailored summary is generated.

[0901] Step 6:

[0902] The server aggregates the summary information

[0903] Input: Summary information refined in step 5.

[0904] Specific operation: The server concatenates each abstract and creates one final document that contains all the abstracts.

[0905] Data processing: Arrange the summary information according to a sequential linking algorithm.

[0906] Output: The final aggregated document is generated.

[0907] Step 7:

[0908] The server provides the final product to the user.

[0909] Input: The aggregate document generated in step 6.

[0910] Specific operation: The server converts the aggregate document into PDF format and provides a download link to the user's device.

[0911] Output: The final document in PDF format is provided to the user, who can download or view it on their device.

[0912] Step 8:

[0913] The server generates the guides and applications.

[0914] Input: Specific category information selected by the user.

[0915] Specific operation: The server collects relevant information from the Internet and filters the information based on emotion data.

[0916] Data Computing: Generate comprehensive guides based on collected information and adjust content based on user sentiment.

[0917] Output: A finished guide delivered as a mobile app or web app.

[0918] (Application example 2)

[0919] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0920] Currently, physical stores require staff to respond to customers quickly and appropriately, but it is often difficult to provide the optimal response and information for customer questions and requests.In addition, it is difficult to provide information based on the customer's emotions, which makes it difficult to improve customer satisfaction.

[0921] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving a list of multiple input documents, means for generating summary information for each document in the list, means for aggregating the generated summary information to create a single document, means for outputting the created document, means for acquiring related information based on the input question content and emotion data, and means for proposing the optimal response method based on the acquired information. This enables staff in physical stores to respond quickly and appropriately to customer questions and requests, and further improves customer satisfaction by providing optimal information based on customer emotions.

[0922] An "information processing device" is a device for analyzing and processing input information.

[0923] A "document list" is a list that includes the titles and contents of multiple documents.

[0924] "Summary information" is information that briefly summarizes the important content of the entire document.

[0925] "Emotional data" is data that indicates the user's emotional state and is generated based on facial expressions, voice, text input, etc.

[0926] "Natural language processing technology" is a technology that allows computers to understand and process human language.

[0927] "Means for collecting relevant information from the Internet" refers to methods for searching data on the Internet and collecting the necessary information.

[0928] The means for generating a "guide" is a method for creating a guide that is useful to users based on the collected information.

[0929] An "application" is a software program used for a specific function or purpose.

[0930] "Customer support" refers to providing support to store staff to provide appropriate support to customers.

[0931] "Means for providing optimal information" refers to a method for selecting and providing the most appropriate information based on user input and emotional data.

[0932] The present invention provides an information processing system for supporting customer service in brick-and-mortar stores. This system uses an information processing device to generate a summary of a document and provide appropriate information based on emotion data. A specific embodiment of this system is described below.

[0933] Hardware and software used

[0934] 1. Hardware:

[0935] Devices such as smartphones and smart glasses: These are devices that staff carry and use.

[0936] Server: Used for processing and storing data.

[0937] 2. Software:

[0938] Natural language processing technology: Technology for summarizing documents. For example, Python libraries such as NLTK and spaCy are used.

[0939] API: An interface for obtaining information corresponding to customer questions, for example, using a RESTful API.

[0940] Emotion engine: An engine for analyzing customer emotion data, for example, using OpenFace or Emotion API.

[0941] Processing flow

[0942] 1. User input:

[0943] The user (staff member) inputs customer questions via a smartphone or smart glasses, and this information is sent to the server, which is the starting point for processing.

[0944] 2. Collecting Emotional Data:

[0945] The device collects emotional data from the user's facial expressions and voice using a camera and microphone, and analyzes the data using an emotion engine. The collected emotional data is then sent to the server along with the questions asked.

[0946] 3. Document and information summary generation:

[0947] Based on the received question, the server retrieves appropriate document and product information from a database, and generates summary information from this information using natural language processing technology.

[0948] 4. Adjust based on sentiment data:

[0949] Based on the analyzed emotional data, the server adjusts the way it presents information, for example, providing detailed explanations if the customer is excited, or a concise summary if the customer is calm.

[0950] 5. Recommendations for the best response:

[0951] The server then proposes the optimal response based on the acquired information, which provides specific guidance on how staff should respond to the customer.

[0952] 6. Provision of Information:

[0953] The final generated information and response methods are displayed on the staff's terminal, allowing the staff to immediately provide the appropriate information to the customer.

[0954] Specific examples

[0955] For example, if a customer asks a store employee, "What is the material of this shirt?", the employee inputs the question through the smart glasses. The device captures emotional data of the customer's happiness from their facial expression and sends it to the server. The server then retrieves information from a database of related products and generates a detailed summary of the shirt's material. The server then analyzes the emotional data and, if it determines that the customer is interested in purchasing, suggests appropriate actions to encourage the purchase. Finally, this information and actions are displayed on the employee's smart glasses, allowing them to immediately explain the situation to the customer.

[0956] Prompt Sentence Examples

[0957] User: What is the material of this shirt?

[0958] Emotion Data: Customers are interested.

[0959] Requested: Product information presentation and customer support suggestions

[0960] This invention allows staff in physical stores to respond to customers quickly and optimally, thereby improving customer satisfaction.

[0961] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0962] Step 1:

[0963] The user uses an input device (such as a smartphone or smart glasses) to input questions or requests from customers. The information entered by the user is temporarily stored on the device. The input is made using a text input interface or a voice recognition system.

[0964] Input: Customer question or request (e.g., "What is the material of this shirt?")

[0965] Output: Text data of the question

[0966] Step 2:

[0967] The device analyzes the user's facial expressions and tone of voice to obtain emotional data. This is done using sensors such as a camera and microphone, and analysis is performed using an emotion engine (e.g., OpenFace or Emotion API). The analysis results are stored on the device as a quantified emotional state.

[0968] Input: User facial expressions and tone of voice

[0969] Output: Quantified emotion data (e.g., happiness, interest, etc.)

[0970] Step 3:

[0971] The device packages the question and emotion data and sends it to the server, typically in JSON format, via internet.

[0972] Input: Question text and sentiment data

[0973] Output: Packaged data (JSON format)

[0974] Step 4:

[0975] The server analyzes the received data and retrieves product information corresponding to the question from the database. At this time, the server retrieves information from the external database using a RESTful API.

[0976] Input: Packaged data (JSON format)

[0977] Output: Retrieved product information

[0978] Step 5:

[0979] The server generates a summary of the document based on the product information it has acquired. This process uses natural language processing techniques (e.g., NLTK and spaCy). The summary is generated in a concise and to-the-point format.

[0980] Input: Retrieved product information

[0981] Output: Summary information

[0982] Step 6:

[0983] The server analyzes the emotion data and adjusts the summary presentation, for example, including detailed information if the customer is interested, or providing a more concise summary if the customer is not. The adjustments based on emotion data are made using a generative AI model.

[0984] Input: Summary information and emotion data

[0985] Output: Adjusted summary information

[0986] Step 7:

[0987] The server proposes the optimal response method. To do this, it refers to past data and case studies and generates specific response methods and wording, etc. This is also done using a generative AI model.

[0988] Input: Adjusted summary information and emotion data

[0989] Output: Best Practices

[0990] Step 8:

[0991] The server then sends the final information (adjusted summary information and optimal response methods) to the terminal, which displays this information for immediate use by staff. The display can be done using a text display interface or a speech synthesis system.

[0992] Input: Final generated information

[0993] Output: Information displayed on the staff member's terminal

[0994] This series of processes enables staff in physical stores to provide customers with information and respond in the most prompt and optimal way.

[0995] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0996] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0997] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0998] [Third embodiment]

[0999] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1000] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1001] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1002] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1003] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1004] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1005] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1006] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1007] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1008] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1009] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1010] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1011] This invention relates to a system that uses an information processing device to efficiently collect, organize, and summarize information. This system receives a list of multiple documents entered by the user, generates and aggregates summary information for each document, and creates and outputs a single document, thereby reducing the burden on the user. It is also possible to collect information related to a specific category from the Internet, generate a guide, and provide it as an application.

[1012] Overall processing flow of the program

[1013] The system operates as follows.

[1014] 1. User enters CEO's book list

[1015] The user inputs the titles of multiple books related to the CEO from the terminal. This information is sent to the server. For example, the user inputs "CEO's Book 1" and "CEO's Book 2" and sends them to the server.

[1016] 2. The server receives the input data and starts processing it.

[1017] The server receives the list sent by the user and uses natural language processing technology to generate summary information for each book in the list. For each book title, the server calls a summary generation API to obtain summary information.

[1018] 3. The server aggregates the summary information

[1019] The server then compiles the obtained summaries into a single document. For example, it concatenates the summaries of "CEO Book 1" and "CEO Book 2" in order to create a final best-of document.

[1020] 4. The server delivers the final product to the user

[1021] The completed best-of document is converted into PDF or e-book format and provided to users, allowing them to see a summary without having to read each individual book.

[1022] Specific examples

[1023] Example 1: CEO Book Summary

[1024] The user enters the following list:

[1025] ["CEO Book 1", "CEO Book 2", "CEO Book 3"]

[1026] The server gets a summary from the chat AI for each book:

[1027] Summary 1: "CEO Book 1 Summary"

[1028] Summary 2: "CEO Book 2 Summary"

[1029] Summary 3: "CEO Book 3 Summary"

[1030] The server creates the final best version of the document:

[1031] Final Document: "CEO Book 1 Summary\n\nCEO Book 2 Summary\n\nCEO Book 3 Summary"

[1032] Example 2: Creating a guide on Excel functions

[1033] User selected categories:

[1034] "Excel Functions"

[1035] The server gathers relevant information from the Internet:

[1036] Information 1: "Basic Excel Functions"

[1037] Information 2: "Advanced Excel Functions"

[1038] Information 3: "How to use Excel functions"

[1039] The server generates a guide based on the information collected:

[1040] Guide: "Basic Excel Functions\n\nAdvanced Excel Functions\n\nHow to Use Excel Functions"

[1041] Build the server-generated guide into an application and serve it to users:

[1042] App: "Excel Function Guide App"

[1043] This allows users to easily refer to a comprehensive guide to Excel functions.

[1044] The processing flow will be explained below.

[1045] Program processing steps

[1046] Example 1: Compiling the CEO's top 10 books using chat generation AI

[1047] Step 1:

[1048] The user inputs a list of CEO's books from the terminal. For example, the user inputs titles such as "CEO's Book 1," "CEO's Book 2," and "CEO's Book 3."

[1049] Step 2:

[1050] The terminal sends the entered list to the server. The sent list is sent in JSON format, for example, as "["CEO's Book 1", "CEO's Book 2", "CEO's Book 3"]".

[1051] Step 3:

[1052] The server receives the user input, and upon receipt, performs data integrity checks and format validation.

[1053] Step 4:

[1054] The server sends a request to the chat generation AI to generate summary information for each document title in the received list. Specifically, it sends an API request to each title to obtain summary information.

[1055] Step 5:

[1056] The server receives the summary information returned by the chat generation AI. For example, the response may be something like "Summary of CEO's Book 1," "Summary of CEO's Book 2," or "Summary of CEO's Book 3."

[1057] Step 6:

[1058] The server aggregates the received summaries to create a single document, specifically by concatenating each summary in turn to create a final document that includes all summaries.

[1059] Step 7:

[1060] The server converts the final document into a PDF or e-book format, making it easily viewable by users.

[1061] Step 8:

[1062] The server provides the converted file to the user, who then downloads or streams the final document from their device.

[1063] Example 2: Picking out frequently used information and turning it into an app

[1064] Step 1:

[1065] The user selects a category of interest (e.g., Excel Functions) from their device, for example by using a drop-down menu or search bar to select "Excel Functions."

[1066] Step 2:

[1067] The terminal transmits the selected category information to the server. For example, the terminal transmits data indicating that the category is "Excel Functions."

[1068] Step 3:

[1069] The server receives the user's selection and begins collecting relevant information, specifically information about Excel functions from internet resources using techniques such as web scraping.

[1070] Step 4:

[1071] The server verifies and filters the collected information, eliminating duplicate information and selecting useful information.

[1072] Step 5:

[1073] The server generates a guide based on the useful information it collects. Specifically, it organizes the information and creates a comprehensive guide divided into chapters and sections.

[1074] Step 6:

[1075] The server-generated guide is then built into an application, for example, developed into a format that can be deployed as a mobile app or web app.

[1076] Step 7:

[1077] The server provides the completed application to the user, who receives a download link or a web app URL to use the application.

[1078] This allows the user to easily obtain and use detailed information about the specified category.

[1079] Example 1

[1080] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1081] Conventional information processing devices require users to individually read a large number of documents, which requires a significant amount of time and effort. Furthermore, there are insufficient methods for efficiently collecting and organizing information related to a specific category. Therefore, there is a need for a system that can efficiently collect, organize, and summarize information.

[1082] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1083] In this invention, the server includes means for receiving a list of input documents, means for generating summary information for each document in the list using a generative AI model, means for aggregating the generated summaries to create a single document, and means for converting the created document into a PDF or e-book format and outputting it, allowing users to efficiently collect information and check summaries without having to read each document individually.

[1084] An "information processing device" is a device that processes data, receives information input by a user, performs specific processing, and provides an output result.

[1085] The "means for receiving a list of documents" is a means having a function for taking in the titles and contents of a plurality of documents input by the user in list form and transmitting the list to the server.

[1086] A "generative AI model" is a model that uses artificial intelligence technology to analyze text data and generate summary information. Examples include generative AI models that use natural language processing technology.

[1087] The "means for generating summary information" is a means for concisely summarizing the contents of a document and extracting its essence.

[1088] The "means for aggregating summary information to create one document" refers to a means for aggregating multiple pieces of summary information, linking them, and organizing them into one unified document.

[1089] "Means of converting to PDF or e-book format and outputting" refers to the means of converting created documents into a format that is easy for users to view and providing them as electronic files. PDF and e-book formats are examples of such formats.

[1090] "Category information" refers to information related to a specific category, and is information for identifying themes or fields in which a user is interested.

[1091] The "means for collecting related information" refers to a means for searching and collecting related information from the Internet based on the input category information.

[1092] "Means for generating a guide" refers to means for organizing collected information and creating guides or manuals for specific purposes.

[1093] The "means for constructing as an application" refers to a means for converting the generated guide into an application in a form that can be used by the user and providing it to the user.

[1094] This invention relates to a system that uses an information processing device to efficiently collect, organize, and summarize information. This system receives a list of multiple documents entered by a user, generates summary information for each document using a generative AI model, aggregates them to create a single document, and converts and outputs the summary in PDF or e-book format, thereby reducing the burden on the user.

[1095] First, the user inputs a list of multiple documents using a terminal. For example, the user inputs the document titles such as "CEO's Book 1" and "CEO's Book 2" in a browser or dedicated application. The input documents are sent from the terminal to the server using an HTTP POST request.

[1096] The server receives the list of documents sent by the user and analyzes their contents. Next, it generates a summary for each document title using a generative AI model (e.g., OpenAI's GPT-3, which uses natural language processing technology). The server sends a prompt to the generative AI model to obtain the summary.

[1097] As a concrete example, suppose the user inputs the following list:

[1098] ["CEO Book 1", "CEO Book 2", "CEO Book 3"]

[1099] In response, the server sends the following prompt to the generative AI model:

[1100] "Generate a summary for each of the following book titles:\n- CEO Book 1\n- CEO Book 2\n- CEO Book 3"

[1101] Each summary is received from the generative AI model, and a summary like the one below is returned.

[1102] Summary 1: "CEO Book 1 Summary"

[1103] Summary 2: "CEO Book 2 Summary"

[1104] Summary 3: "CEO Book 3 Summary"

[1105] The server aggregates these summaries into a single document and organizes them, for example, as follows:

[1106] Final Document: "CEO Book 1 Summary\n\nCEO Book 2 Summary\n\nCEO Book 3 Summary"

[1107] The server then converts the final document into a format that is easy for users to view, such as PDF or e-book format, using a PDF generation tool or e-book format conversion tool.

[1108] Finally, the server provides the converted document to the user, for example by sending the user an email containing a download link for the generated file, or by displaying the link on a dedicated download page.

[1109] This system allows users to efficiently collect large amounts of information and check summaries, significantly reducing the burden of information processing.It also has the function of automatically collecting, organizing, and summarizing information related to specific categories from the Internet, making it suitable for collecting a variety of information.

[1110] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1111] Step 1:

[1112] The user enters a list of documents

[1113] Input: A user uses a terminal to enter the titles of several documents, for example, "CEO Book 1," "CEO Book 2," "CEO Book 3," etc.

[1114] How it works: A user enters a list of documents into a form in a browser or dedicated application and presses the submit button.

[1115] Output: The input data is prepared as a JSON list and sent to the next processing step.

[1116] Step 2:

[1117] The device sends the data to the server

[1118] Input: A list of documents entered by the user.

[1119] How it works: The device sends JSON-formatted data to the server using an HTTP POST request.

[1120] Output: The data reaches the server and goes on to the next processing step.

[1121] Step 3:

[1122] The server receives the input data and starts processing it.

[1123] Input: Document list sent from the terminal.

[1124] Behavior: The server parses the received data and extracts a list of document titles.

[1125] Output: The extracted list of document titles is used to generate summaries in the next step.

[1126] Step 4:

[1127] The server makes a request to generate a summary

[1128] Input: A list of extracted document titles.

[1129] How it works: The server calls a generative AI model (e.g., OpenAI GPT-3) and creates a prompt for generating a summary. For example, "Generate a summary for the following book title: CEO Book 1." This prompt is then sent to the generative AI model.

[1130] Output: Summary information returned from the generative AI model.

[1131] Step 5:

[1132] The server retrieves the summary results

[1133] Input: Summary information from a generative AI model.

[1134] How it works: The server takes the summary information received from the generative AI model and organizes it by document title.

[1135] Output: Data to aggregate organized summary information into a single document.

[1136] Step 6:

[1137] The server aggregates the summary information

[1138] Input: Organized summary information.

[1139] What it does: Concatenates the summaries into a single document, organized by paragraph, for example, "Summary of CEO Book 1\n\nSummary of CEO Book 2\n\nSummary of CEO Book 3."

[1140] Output: The final summary document.

[1141] Step 7:

[1142] The server converts and outputs the final document

[1143] Input: Final summary document.

[1144] What it does: The server converts this document to PDF or e-book format using tools such as PDF generation tools and e-book format conversion tools.

[1145] Output: The converted document file (e.g. PDF), a download link provided to the user.

[1146] Step 8:

[1147] The server provides the final product to the user.

[1148] Input: The converted document file.

[1149] Action: The server initiates the steps to provide the generated file to the user, for example by sending an email containing a download link or by displaying the link on a dedicated download page.

[1150] Output: A user-accessible link or download page.

[1151] This system will enable users to efficiently collect information requested by the public and check summaries, thereby significantly reducing the burden of information processing.

[1152] (Application example 1)

[1153] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1154] Conventional information processing systems lack the means for users to efficiently collect, summarize, and use information from multiple documents. Furthermore, it is difficult to efficiently provide product information, particularly in brick-and-mortar stores, making it difficult for customers to quickly obtain the information they need.

[1155] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1156] In this invention, the server includes means for receiving a list of input documents, means for generating summary information for each document in the list, means for aggregating the generated summary information to create a single document, means for outputting the created document, means for receiving product-related information and generating summary information, and means for aggregating the generated summary information to create a single document and providing the document to the user. This makes it possible to efficiently collect, summarize, and provide information required by users and customers as a single document.

[1157] An "information processing device" is a combination of hardware and software for receiving, processing, generating, and outputting information.

[1158] The "means for receiving a list of documents" is a device having a function for receiving a list of titles and contents of a plurality of documents input by the user.

[1159] A "means for generating summary information" is an algorithm or software that extracts important information from an input document and generates a summary in a concise format.

[1160] The "means for aggregating summary information to create one document" refers to a device having a function for aggregating and integrating a plurality of pieces of generated summary information into one continuous document.

[1161] The "means for outputting a document" is a device that has the function of providing the created document to the user in a format such as PDF or e-book.

[1162] The "means for receiving information related to merchandise" is a device having a function for receiving merchandise information in a store and related market data.

[1163] The "means for providing to the user" is a device having a function for providing the generated information or document to the user in an accessible form.

[1164] The present invention relates to a system that uses an information processing device to efficiently collect, organize, and summarize information. This system receives a list of multiple documents entered by the user, generates and aggregates summary information for each document, and creates and outputs a single document, thereby reducing the burden on the user. It is also possible to collect information related to a specific category from the Internet, generate a guide, and provide it as an application. Furthermore, the system also has the function of efficiently collecting information related to products in physical stores and providing summary information.

[1165] The server has a means of receiving a list of multiple documents input by the user. This list consists of the titles and contents of documents that the user is interested in. The server then uses natural language processing techniques on these documents to generate summaries. The natural language processing techniques used include OpenAI APIs. Once the summaries are generated, the server aggregates them into a single document. Finally, the generated document is provided to the user in PDF or e-book format.

[1166] Next, the server receives information related to the product and has a means to generate summary information in the same way. This includes the process of collecting related information from the Internet. For example, when collecting the latest information and trend information about a product and generating summary information, the following prompt sentence is used for the generative AI model:

[1167] Example prompt sentence:

[1168] Provide updates and summaries on the "latest smartphones".

[1169] By inputting this prompt into a generative AI model, the latest information on the topic is obtained and a summary is generated, which is then compiled into a single document and ultimately provided to the user in PDF or e-book format.

[1170] For example, if a user enters a list such as "latest smartphones," "trendy fashion items," or "products on sale," the server collects and summarizes information about each item and creates a final, best-of document. This process allows users and customers to efficiently gather the information they need and obtain it in a summarized form.

[1171] In the embodiment of the invention, the input data is diverse, including information collected from the Internet, data entered by the user, data on products in the store, etc. This makes it possible to build a system that eliminates the complexity of information collection that users and customers face and can provide necessary information quickly and efficiently.

[1172] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1173] Step 1:

[1174] User enters a list of documents

[1175] A user inputs the titles and topics of multiple documents he or she wishes to summarize using a terminal, and the input list is sent to the server.

[1176] Specific operation: A user uses a smartphone or PC to enter the title of a document through a web interface or application, and then clicks the send button to send it to the server.

[1177] Step 2:

[1178] The server receives the input data

[1179] The server receives a list of documents submitted by the user, which are added to a processing queue.

[1180] Specific operation: The server receives the HTTP request, analyzes the sent data, and stores it in a database or queue.

[1181] Step 3:

[1182] The server generates the summary information

[1183] Based on the received list of documents, the server generates a summary of each document using a generative AI model. The summary information is obtained by inputting a prompt to the generative AI model.

[1184] Specific operation: The server calls OpenAI's API, sends the prompt sentence to the generative AI model, and obtains summary information.

[1185] Example prompt: Please provide an update and summary on "The Latest Smartphones."

[1186] Step 4:

[1187] The server aggregates the summary information

[1188] The generated summaries are compiled into a single document, which is then converted into a PDF or e-book format and prepared for delivery to users.

[1189] Specific operation: The server concatenates each summary information and converts it into PDF or e-book format using a document formatter.

[1190] Step 5:

[1191] The server provides the final product to the user

[1192] The generated document is sent to the user terminal, where it can be viewed or downloaded by the user.

[1193] Specific operation: The server transfers the file using the HTTP response so that the generated PDF or e-book file can be sent appropriately to the user's device.

[1194] Step 6:

[1195] The server receives product-related information

[1196] Information related to in-store products is collected from the internet and other data sources, and the collected information serves as input for generating summary information.

[1197] Specific operation: The server uses an API to obtain product information from a data source on the Internet.

[1198] Step 7:

[1199] The server generates a summary of the product information

[1200] Based on the collected product information, the server generates a summary using a generative AI model.

[1201] Specific operation: The server sends prompts to the generative AI model to summarize data including the latest product information and consumer reviews, and obtains summary information.

[1202] Step 8:

[1203] Aggregating and outputting summary information about products

[1204] The generated summaries are then compiled into a single document and provided to the user.

[1205] Specific operation: The server concatenates the summary information, converts it into PDF or e-book format, and sends it to the user's terminal to provide it to the user as a single document.

[1206] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1207] The present invention relates to a system that efficiently collects, organizes, and summarizes information using an information processing device. This system receives a list of multiple documents entered by the user, generates and consolidates summary information for each document, and creates and outputs a single document, thereby reducing the user's burden. It is also possible to collect information related to a specific category from the Internet, generate a guide, and provide it as an application. Furthermore, the present invention is equipped with an emotion engine that recognizes the user's emotions and has the function of adjusting the method of generating and providing information based on the user's emotions.

[1208] Overall processing flow of the program

[1209] The system operates as follows.

[1210] 1. The user inputs a document list

[1211] The user inputs a list of documents of interest from the terminal, for example, by entering titles such as "Document 1," "Document 2," and "Document 3." This information is sent to the server.

[1212] 2. The device acquires emotion data

[1213] The device collects user emotional data (facial expressions, voice, text input, etc.) using an emotion engine, which then uses the data to adjust the content and generation method of document summaries.

[1214] 3. The device sends the data to the server

[1215] The device sends both the document list and the emotion data to the server. The sent data is summarized in JSON format, for example, as follows: {"documents":["Document 1", "Document 2", "Document 3"], "emotions":{"happiness":0.8, "sadness":0.1}}.

[1216] 4. The server receives the document list and begins processing.

[1217] Based on the document list received by the server, natural language processing technology is used to generate summary information for each document. The server calls a summary generation API for the title of each document to obtain the summary information.

[1218] 5. The server analyzes the emotion data and adjusts the generated content

[1219] The server analyzes the received emotional data and adjusts the summary method according to the user's emotions. For example, if the user is excited, it will provide detailed summary information, but if the user is calm, it will provide a short summary of the main points.

[1220] 6. The server aggregates the summary information

[1221] The server then compiles the summaries into a single document, concatenating each summary piece in turn to create a final document that includes all the summaries.

[1222] 7. The server delivers the final product to the user

[1223] The completed best version document is converted into PDF or e-book format and provided to the user, who can then download or stream the final document to their device.

[1224] 8. Generating Guides and Applications

[1225] When a user selects a specific category (e.g., Excel functions), the server collects and filters related information from the internet to generate a comprehensive guide, which is then tailored based on the user's preferences and ultimately delivered as a mobile or web app.

[1226] Specific examples

[1227] Example 1: CEO Book Summary

[1228] The user enters the following list:

[1229] ["CEO Book 1", "CEO Book 2", "CEO Book 3"]

[1230] The device acquires emotion data from the user's facial expressions and voice, then sends it to the server:

[1231] {"documents":["CEO Book 1", "CEO Book 2", "CEO Book 3"], "emotions":{"happiness":0.7, "interest":0.9}}

[1232] The server gets a summary from the chat AI for each book and adjusts the summary method based on the emotion data:

[1233] Summary 1: "CEO Book 1 Summary"

[1234] Summary 2: "CEO Book 2 Summary"

[1235] Summary 3: "CEO Book 3 Summary"

[1236] The server creates the final best version of the document:

[1237] Final Document: "CEO Book 1 Summary\n\nCEO Book 2 Summary\n\nCEO Book 3 Summary"

[1238] Example 2: Creating a guide on Excel functions

[1239] User selected categories:

[1240] "Excel Functions"

[1241] The device acquires the user's emotion data and then sends it to the server:

[1242] {"category":"Excel Functions", "emotions":{"focus":0.8, "confidence":0.7}}

[1243] The server collects relevant information from the internet and filters it based on emotion data:

[1244] Information 1: "Basic Excel Functions"

[1245] Information 2: "Advanced Excel Functions"

[1246] Information 3: "How to use Excel functions"

[1247] The server generates guides based on collected information and adjusts the content based on the user's emotions:

[1248] Guide: "Basic Excel Functions\n\nAdvanced Excel Functions\n\nHow to Use Excel Functions"

[1249] Build the server-generated guide into an application and serve it to users:

[1250] App: "Excel Function Guide App"

[1251] This allows the user to easily obtain detailed information about the specified category and use the information in a way that best suits their own feelings.

[1252] The processing flow will be explained below.

[1253] Program processing steps

[1254] Example 1: Compiling the CEO's top 10 books using chat generation AI

[1255] Step 1:

[1256] The user inputs a list of CEO's books from the terminal. For example, the user inputs titles such as "CEO's Book 1," "CEO's Book 2," and "CEO's Book 3."

[1257] Step 2:

[1258] The device acquires emotion data from the user's facial expressions and voice. The emotion engine analyzes the user's camera footage and microphone input to detect their emotional state (e.g., happiness, excitement, etc.).

[1259] Step 3:

[1260] The terminal transmits the input document list and the acquired emotion data to the server. The transmitted data is in a format that includes the document list and emotion data.

[1261] Step 4:

[1262] The server receives the document list and begins the process of generating summary information for each title. The server uses natural language processing techniques to generate a summary for each document.

[1263] Step 5:

[1264] The server analyzes the emotion data and adjusts the summary content based on the user's emotion, for example, generating a detailed and comprehensive summary when the user is in a high excitement state, and a concise summary when the user is in a low excitement state.

[1265] Step 6:

[1266] The server aggregates the generated summaries into a single document, and then concatenates all the summaries to create a final, best-of document.

[1267] Step 7:

[1268] The server converts the final product into a PDF or e-book format, which makes it easy for users to view.

[1269] Step 8:

[1270] The server provides the converted file to the user, who then downloads or streams the final document from their device.

[1271] Example 2: Picking out frequently used information and turning it into an app

[1272] Step 1:

[1273] The user selects the category (e.g. Excel Functions) that he is interested in from the terminal. For example, he selects the category as "Excel Functions".

[1274] Step 2:

[1275] The device acquires emotion data from the user's facial expressions and voice. The emotion engine recognizes facial expressions and analyzes voice to detect the user's emotional state.

[1276] Step 3:

[1277] The terminal transmits the selected category information and emotion data to the server. The transmitted data includes the category information and emotion data.

[1278] Step 4:

[1279] The server receives the category information and collects related information from the Internet. Web scraping technology is used to gather useful information about Excel functions.

[1280] Step 5:

[1281] The server verifies and filters the information collected, eliminating duplicate information and selecting useful information.

[1282] Step 6:

[1283] The server analyzes the emotional data and adjusts the way the collected information is organized and the content of the guide based on the user's emotions. For example, if the user is feeling confident, it will include more advanced content, and conversely, if the user is feeling anxious, it will emphasize basic content.

[1284] Step 7:

[1285] The server generates the guide and builds it into an application. Based on the collected information, a comprehensive guide is created and deployed as a mobile or web app.

[1286] Step 8:

[1287] The server provides the completed application to the user, who receives a link to download or use the app on their device.

[1288] Through the above processing steps, the user can easily obtain detailed information about the specified category and can use the information in a way that is optimal for his or her own feelings.

[1289] Example 2

[1290] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1291] Conventional information processing systems require users to collect information, summarize each document individually, and compile it, requiring a great deal of effort. Furthermore, the summarized information is uniform and not tailored to the user's emotions or interests, preventing optimal information provision. Furthermore, it is difficult to provide comprehensive guidance related to specific categories of information in a format tailored to each individual user. To solve these problems, a new system is needed that can collect, summarize, aggregate, and tailor information based on the user's emotions.

[1292] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1293] In this invention, the server includes means for receiving a list of multiple documents input by a user, means for acquiring user emotion data, and means for transmitting the acquired emotion data and list to the server. This enables the generation and provision of optimal document summary information based on the user's emotion. The server also includes means for using natural language processing technology to generate document summary information and means for adjusting the generated summary information based on the user's emotion data. This enables the creation of highly accurate summary documents that reflect the user's emotion. The server also includes means for aggregating the adjusted summary information to create a single document and means for outputting the created document. This enables the information desired by the user to be provided as a single comprehensive document, thereby realizing efficient collection and use of information.

[1294] An "information processing device" is a device that processes data and organizes information using a combination of computer systems and software.

[1295] A "document list" is a list that compiles the titles and contents of multiple documents entered or selected by the user.

[1296] "Natural language processing technology" is a technology that enables computers to understand and process human language, and is used for generating summaries and semantic analysis.

[1297] "Emotion data" is data that indicates the emotional state of a user, obtained by analyzing the user's facial expression, voice, text input, and the like.

[1298] "Summary information" is information that shortens the content of the original document and succinctly summarizes the main points and content.

[1299] "Aggregation" means combining multiple elements or pieces of information into one.

[1300] "Document creation" refers to the process of composing multiple pieces of summary information into a single continuous document and converting it into an outputtable format.

[1301] "Output" refers to providing the generated data or document to the user, and includes means such as displaying it on the screen, generating a PDF, or providing a download link.

[1302] "Category information" is data that indicates information about a particular theme or field in which a user is interested.

[1303] A "guide" is a comprehensive instruction manual or explanatory document that provides information on a particular topic.

[1304] "Application" means a program that provides specific functions or services, including mobile applications and web applications.

[1305] This invention relates to a system that uses an information processing device to efficiently collect, organize, and summarize information. This system receives a list of multiple documents entered by the user, generates summary information for each document, and aggregates them to create and output a single document, thereby reducing the burden on the user. It is also possible to collect information related to a specific category from the Internet, generate a guide, and provide it as an application.

[1306] Hardware and software used

[1307] Hardware

[1308] User devices (PCs, smartphones, tablets, etc.)

[1309] Server (high-performance data processing system)

[1310] software

[1311] Natural Language Processing (NLP) engines (e.g., OpenAI GPT-4)

[1312] Sentiment Analysis Engine

[1313] Generative AI Models

[1314] Data processing and calculation

[1315] 1. The user inputs a document list

[1316] The user inputs the titles of the documents they are interested in into the input form on their device. For example, they input them in a list format, such as "Document A," "Document B," "Document C," etc. This creates a document list and sends it to the server.

[1317] 2. The device acquires emotion data

[1318] The device uses a built-in emotion analysis engine to collect user emotional data in real time. For example, it uses the device's built-in camera and microphone to analyze the user's facial expressions and tone of voice. This allows it to quantify emotional parameters such as how happy or excited the user is.

[1319] 3. The device sends the data to the server

[1320] The device encodes the collected document list and emotion data into JSON format and sends it to the server. For example, the data is sent in the following format:

[1321] {"documents":["Document A", "Document B", "Document C"], "emotions":{"happiness":0.8, "sadness":0.1}}

[1322] 4. The server receives the document list and begins processing.

[1323] The server uses natural language processing (NLP) technology to generate summary information for each document based on the received document list. The NLP engine calls a summary generation API based on the title of each document to obtain the summary information.

[1324] 5. The server analyzes the emotion data and adjusts the generated content

[1325] The server adjusts the summarization method based on the received emotion data according to the user's emotional state, for example, providing a detailed summary when the user is excited and generating a concise summary when the user is calm.

[1326] 6. The server aggregates the summary information

[1327] The server aggregates the obtained summaries into a single document, and then sequentially concatenates the summaries to create a single final document containing all the summaries.

[1328] 7. The server delivers the final product to the user

[1329] The final product is converted into PDF or e-book format and provided to the user, who can then download or stream the final document to their device.

[1330] 8. The server generates the guide and application

[1331] When a user selects a specific category, the server collects and filters related information from the internet to generate a comprehensive guide, which is then tailored based on the user's sentiment and ultimately provided as a mobile or web app.

[1332] Specific examples

[1333] Example 1: CEO Book Summary

[1334] The user enters the following list:

[1335] ["CEO Book 1", "CEO Book 2", "CEO Book 3"]

[1336] The device acquires emotion data from the user's facial expressions and voice, then sends it to the server:

[1337] {"documents":["CEO Book 1", "CEO Book 2", "CEO Book 3"], "emotions":{"happiness":0.7, "interest":0.9}}

[1338] The server uses natural language processing to obtain a summary for each book and adjusts the summary method based on sentiment data:

[1339] Summary 1: "CEO Book 1 Summary"

[1340] Summary 2: "CEO Book 2 Summary"

[1341] Summary 3: "CEO Book 3 Summary"

[1342] The server creates the final document:

[1343] Final Document: "CEO Book 1 Summary\n\nCEO Book 2 Summary\n\nCEO Book 3 Summary"

[1344] Example 2: Creating a guide on Excel functions

[1345] User selected categories:

[1346] "Excel Functions"

[1347] The device acquires the user's emotion data and then sends it to the server:

[1348] {"category":"Excel Functions", "emotions":{"focus":0.8, "confidence":0.7}}

[1349] The server collects relevant information from the internet and filters it based on emotion data:

[1350] Information 1: "Basic Excel Functions"

[1351] Information 2: "Advanced Excel Functions"

[1352] Information 3: "How to use Excel functions"

[1353] The server generates guides based on collected information and adjusts the content based on the user's emotions:

[1354] Guide: "Basic Excel Functions\n\nAdvanced Excel Functions\n\nHow to Use Excel Functions"

[1355] Build the server-generated guide into an application and serve it to users:

[1356] App: "Excel Function Guide App"

[1357] By executing the above processing procedure, the present invention collects information, summarizes it, and generates guides optimally based on the user's emotions, thereby improving the user experience.

[1358] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1359] Step 1:

[1360] The user enters a document list

[1361] Input: A user inputs a list of document titles into their device, for example, "Document A," "Document B," "Document C," etc.

[1362] Specific operation: The user enters document titles one by one into the input form on the terminal and presses the "Send" button.

[1363] Output: The input document list is saved as local data on the device.

[1364] Step 2:

[1365] The device acquires emotion data

[1366] Input: Real-time data such as the user's facial image and voice.

[1367] How it works: The device's built-in camera and microphone capture the user's facial expressions and tone of voice, and an emotion analysis engine analyzes this data.

[1368] Data processing: The emotion analysis engine converts the user's emotional status into a numerical value, such as "happiness=0.8" or "sadness=0.1."

[1369] Output: Quantified emotion data is generated.

[1370] Step 3:

[1371] The device sends the data to the server

[1372] Input: The document list generated in step 1 and the emotion data generated in step 2.

[1373] Specific operation: The device encodes the document list and emotion data into JSON format and sends it to the server via an HTTP request.

[1374] Output: The data sent to the server looks like this:

[1375] {"documents":["Document A", "Document B", "Document C"], "emotions":{"happiness":0.8, "sadness":0.1}}

[1376] Step 4:

[1377] The server receives the document list and starts processing

[1378] Input: JSON data containing the document list sent in step 3.

[1379] Specific operation: The server receives the HTTP request and parses the document list.

[1380] Data calculation: Using a natural language processing (NLP) engine, a summary generation API is called for each document title to obtain summary information.

[1381] Output: A summary is generated for each document.

[1382] Step 5:

[1383] The server analyzes the emotional data and adjusts the generated content

[1384] Input: The summary information generated in step 4 and the emotion data sent in step 3.

[1385] Specific operation: The server analyzes the emotion data and adjusts each summary information according to the user's emotional state.

[1386] Data processing: For example, providing a detailed summary when the user is excited and adjusting the summary to a concise one when the user is calm.

[1387] Output: A tailored summary is generated.

[1388] Step 6:

[1389] The server aggregates the summary information

[1390] Input: Summary information refined in step 5.

[1391] Specific operation: The server concatenates each abstract and creates one final document that contains all the abstracts.

[1392] Data processing: Arrange the summary information according to a sequential linking algorithm.

[1393] Output: The final aggregated document is generated.

[1394] Step 7:

[1395] The server provides the final product to the user.

[1396] Input: The aggregate document generated in step 6.

[1397] Specific operation: The server converts the aggregate document into PDF format and provides a download link to the user's device.

[1398] Output: The final document in PDF format is provided to the user, who can download or view it on their device.

[1399] Step 8:

[1400] The server generates the guides and applications.

[1401] Input: Specific category information selected by the user.

[1402] Specific operation: The server collects relevant information from the Internet and filters the information based on emotion data.

[1403] Data Computing: Generate comprehensive guides based on collected information and adjust content based on user sentiment.

[1404] Output: A finished guide delivered as a mobile app or web app.

[1405] (Application example 2)

[1406] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1407] Currently, physical stores require staff to respond to customers quickly and appropriately, but it is often difficult to provide the optimal response and information for customer questions and requests.In addition, it is difficult to provide information based on the customer's emotions, which makes it difficult to improve customer satisfaction.

[1408] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving a list of multiple input documents, means for generating summary information for each document in the list, means for aggregating the generated summary information to create a single document, means for outputting the created document, means for acquiring related information based on the input question content and emotion data, and means for proposing the optimal response method based on the acquired information. This enables staff in physical stores to respond quickly and appropriately to customer questions and requests, and further improves customer satisfaction by providing optimal information based on customer emotions.

[1409] An "information processing device" is a device for analyzing and processing input information.

[1410] A "document list" is a list that includes the titles and contents of multiple documents.

[1411] "Summary information" is information that briefly summarizes the important content of the entire document.

[1412] "Emotional data" is data that indicates the user's emotional state and is generated based on facial expressions, voice, text input, etc.

[1413] "Natural language processing technology" is a technology that allows computers to understand and process human language.

[1414] "Means for collecting relevant information from the Internet" refers to methods for searching data on the Internet and collecting the necessary information.

[1415] The means for generating a "guide" is a method for creating a guide that is useful to users based on the collected information.

[1416] An "application" is a software program used for a specific function or purpose.

[1417] "Customer support" refers to providing support to store staff to provide appropriate support to customers.

[1418] "Means for providing optimal information" refers to a method for selecting and providing the most appropriate information based on user input and emotional data.

[1419] The present invention provides an information processing system for supporting customer service in brick-and-mortar stores. This system uses an information processing device to generate a summary of a document and provide appropriate information based on emotion data. A specific embodiment of this system is described below.

[1420] Hardware and software used

[1421] 1. Hardware:

[1422] Devices such as smartphones and smart glasses: These are devices that staff carry and use.

[1423] Server: Used for processing and storing data.

[1424] 2. Software:

[1425] Natural language processing technology: Technology for summarizing documents. For example, Python libraries such as NLTK and spaCy are used.

[1426] API: An interface for obtaining information corresponding to customer questions, for example, using a RESTful API.

[1427] Emotion engine: An engine for analyzing customer emotion data, for example, using OpenFace or Emotion API.

[1428] Processing flow

[1429] 1. User input:

[1430] The user (staff member) inputs customer questions via a smartphone or smart glasses, and this information is sent to the server, which is the starting point for processing.

[1431] 2. Collecting Emotional Data:

[1432] The device collects emotional data from the user's facial expressions and voice using a camera and microphone, and analyzes the data using an emotion engine. The collected emotional data is then sent to the server along with the questions asked.

[1433] 3. Document and information summary generation:

[1434] Based on the received question, the server retrieves appropriate document and product information from a database, and generates summary information from this information using natural language processing technology.

[1435] 4. Adjust based on sentiment data:

[1436] Based on the analyzed emotional data, the server adjusts the way it presents information, for example, providing detailed explanations if the customer is excited, or a concise summary if the customer is calm.

[1437] 5. Recommendations for the best response:

[1438] The server then proposes the optimal response based on the acquired information, which provides specific guidance on how staff should respond to the customer.

[1439] 6. Provision of Information:

[1440] The final generated information and response methods are displayed on the staff's terminal, allowing the staff to immediately provide the appropriate information to the customer.

[1441] Specific examples

[1442] For example, if a customer asks a store employee, "What is the material of this shirt?", the employee inputs the question through the smart glasses. The device captures emotional data of the customer's happiness from their facial expression and sends it to the server. The server then retrieves information from a database of related products and generates a detailed summary of the shirt's material. The server then analyzes the emotional data and, if it determines that the customer is interested in purchasing, suggests appropriate actions to encourage the purchase. Finally, this information and actions are displayed on the employee's smart glasses, allowing them to immediately explain the situation to the customer.

[1443] Prompt Sentence Examples

[1444] User: What is the material of this shirt?

[1445] Emotion Data: Customers are interested.

[1446] Requested: Product information presentation and customer support suggestions

[1447] This invention allows staff in physical stores to respond to customers quickly and optimally, thereby improving customer satisfaction.

[1448] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1449] Step 1:

[1450] The user uses an input device (such as a smartphone or smart glasses) to input questions or requests from customers. The information entered by the user is temporarily stored on the device. The input is made using a text input interface or a voice recognition system.

[1451] Input: Customer question or request (e.g., "What is the material of this shirt?")

[1452] Output: Text data of the question

[1453] Step 2:

[1454] The device analyzes the user's facial expressions and tone of voice to obtain emotional data. This is done using sensors such as a camera and microphone, and analysis is performed using an emotion engine (e.g., OpenFace or Emotion API). The analysis results are stored on the device as a quantified emotional state.

[1455] Input: User facial expressions and tone of voice

[1456] Output: Quantified emotion data (e.g., happiness, interest, etc.)

[1457] Step 3:

[1458] The device packages the question and emotion data and sends it to the server, typically in JSON format, via internet.

[1459] Input: Question text and sentiment data

[1460] Output: Packaged data (JSON format)

[1461] Step 4:

[1462] The server analyzes the received data and retrieves product information corresponding to the question from the database. At this time, the server retrieves information from the external database using a RESTful API.

[1463] Input: Packaged data (JSON format)

[1464] Output: Retrieved product information

[1465] Step 5:

[1466] The server generates a summary of the document based on the product information it has acquired. This process uses natural language processing techniques (e.g., NLTK and spaCy). The summary is generated in a concise and to-the-point format.

[1467] Input: Retrieved product information

[1468] Output: Summary information

[1469] Step 6:

[1470] The server analyzes the emotion data and adjusts the summary presentation, for example, including detailed information if the customer is interested, or providing a more concise summary if the customer is not. The adjustments based on emotion data are made using a generative AI model.

[1471] Input: Summary information and emotion data

[1472] Output: Adjusted summary information

[1473] Step 7:

[1474] The server proposes the optimal response method. To do this, it refers to past data and case studies and generates specific response methods and wording, etc. This is also done using a generative AI model.

[1475] Input: Adjusted summary information and emotion data

[1476] Output: Best Practices

[1477] Step 8:

[1478] The server then sends the final information (adjusted summary information and optimal response methods) to the terminal, which displays this information for immediate use by staff. The display can be done using a text display interface or a speech synthesis system.

[1479] Input: Final generated information

[1480] Output: Information displayed on the staff member's terminal

[1481] This series of processes enables staff in physical stores to provide customers with information and respond in the most prompt and optimal way.

[1482] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1483] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1484] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1485] [Fourth embodiment]

[1486] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1487] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1488] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1489] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1490] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1491] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1492] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1493] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1494] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1495] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1496] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1497] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1498] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1499] This invention relates to a system that uses an information processing device to efficiently collect, organize, and summarize information. This system receives a list of multiple documents entered by the user, generates and aggregates summary information for each document, and creates and outputs a single document, thereby reducing the burden on the user. It is also possible to collect information related to a specific category from the Internet, generate a guide, and provide it as an application.

[1500] Overall processing flow of the program

[1501] The system operates as follows.

[1502] 1. User enters CEO's book list

[1503] The user inputs the titles of multiple books related to the CEO from the terminal. This information is sent to the server. For example, the user inputs "CEO's Book 1" and "CEO's Book 2" and sends them to the server.

[1504] 2. The server receives the input data and starts processing it.

[1505] The server receives the list sent by the user and uses natural language processing technology to generate summary information for each book in the list. For each book title, the server calls a summary generation API to obtain summary information.

[1506] 3. The server aggregates the summary information

[1507] The server then compiles the obtained summaries into a single document. For example, it concatenates the summaries of "CEO Book 1" and "CEO Book 2" in order to create a final best-of document.

[1508] 4. The server delivers the final product to the user

[1509] The completed best-of document is converted into PDF or e-book format and provided to users, allowing them to see a summary without having to read each individual book.

[1510] Specific examples

[1511] Example 1: CEO Book Summary

[1512] The user enters the following list:

[1513] ["CEO Book 1", "CEO Book 2", "CEO Book 3"]

[1514] The server gets a summary from the chat AI for each book:

[1515] Summary 1: "CEO Book 1 Summary"

[1516] Summary 2: "CEO Book 2 Summary"

[1517] Summary 3: "CEO Book 3 Summary"

[1518] The server creates the final best version of the document:

[1519] Final Document: "CEO Book 1 Summary\n\nCEO Book 2 Summary\n\nCEO Book 3 Summary"

[1520] Example 2: Creating a guide on Excel functions

[1521] User selected categories:

[1522] "Excel Functions"

[1523] The server gathers relevant information from the Internet:

[1524] Information 1: "Basic Excel Functions"

[1525] Information 2: "Advanced Excel Functions"

[1526] Information 3: "How to use Excel functions"

[1527] The server generates a guide based on the information collected:

[1528] Guide: "Basic Excel Functions\n\nAdvanced Excel Functions\n\nHow to Use Excel Functions"

[1529] Build the server-generated guide into an application and serve it to users:

[1530] App: "Excel Function Guide App"

[1531] This allows users to easily refer to a comprehensive guide to Excel functions.

[1532] The processing flow will be explained below.

[1533] Program processing steps

[1534] Example 1: Compiling the CEO's top 10 books using chat generation AI

[1535] Step 1:

[1536] The user inputs a list of CEO's books from the terminal. For example, the user inputs titles such as "CEO's Book 1," "CEO's Book 2," and "CEO's Book 3."

[1537] Step 2:

[1538] The terminal sends the entered list to the server. The sent list is sent in JSON format, for example, as "["CEO's Book 1", "CEO's Book 2", "CEO's Book 3"]".

[1539] Step 3:

[1540] The server receives the user input, and upon receipt, performs data integrity checks and format validation.

[1541] Step 4:

[1542] The server sends a request to the chat generation AI to generate summary information for each document title in the received list. Specifically, it sends an API request to each title to obtain summary information.

[1543] Step 5:

[1544] The server receives the summary information returned by the chat generation AI. For example, the response may be something like "Summary of CEO's Book 1," "Summary of CEO's Book 2," or "Summary of CEO's Book 3."

[1545] Step 6:

[1546] The server aggregates the received summaries to create a single document, specifically by concatenating each summary in turn to create a final document that includes all summaries.

[1547] Step 7:

[1548] The server converts the final document into a PDF or e-book format, making it easily viewable by users.

[1549] Step 8:

[1550] The server provides the converted file to the user, who then downloads or streams the final document from their device.

[1551] Example 2: Picking out frequently used information and turning it into an app

[1552] Step 1:

[1553] The user selects a category of interest (e.g., Excel Functions) from their device, for example by using a drop-down menu or search bar to select "Excel Functions."

[1554] Step 2:

[1555] The terminal transmits the selected category information to the server. For example, the terminal transmits data indicating that the category is "Excel Functions."

[1556] Step 3:

[1557] The server receives the user's selection and begins collecting relevant information, specifically information about Excel functions from internet resources using techniques such as web scraping.

[1558] Step 4:

[1559] The server verifies and filters the collected information, eliminating duplicate information and selecting useful information.

[1560] Step 5:

[1561] The server generates a guide based on the useful information it collects. Specifically, it organizes the information and creates a comprehensive guide divided into chapters and sections.

[1562] Step 6:

[1563] The server-generated guide is then built into an application, for example, developed into a format that can be deployed as a mobile app or web app.

[1564] Step 7:

[1565] The server provides the completed application to the user, who receives a download link or a web app URL to use the application.

[1566] This allows the user to easily obtain and use detailed information about the specified category.

[1567] Example 1

[1568] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1569] Conventional information processing devices require users to individually read a large number of documents, which requires a significant amount of time and effort. Furthermore, there are insufficient methods for efficiently collecting and organizing information related to a specific category. Therefore, there is a need for a system that can efficiently collect, organize, and summarize information.

[1570] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1571] In this invention, the server includes means for receiving a list of input documents, means for generating summary information for each document in the list using a generative AI model, means for aggregating the generated summaries to create a single document, and means for converting the created document into a PDF or e-book format and outputting it, allowing users to efficiently collect information and check summaries without having to read each document individually.

[1572] An "information processing device" is a device that processes data, receives information input by a user, performs specific processing, and provides an output result.

[1573] The "means for receiving a list of documents" is a means having a function for taking in the titles and contents of a plurality of documents input by the user in list form and transmitting the list to the server.

[1574] A "generative AI model" is a model that uses artificial intelligence technology to analyze text data and generate summary information. Examples include generative AI models that use natural language processing technology.

[1575] The "means for generating summary information" is a means for concisely summarizing the contents of a document and extracting its essence.

[1576] The "means for aggregating summary information to create one document" refers to a means for aggregating multiple pieces of summary information, linking them, and organizing them into one unified document.

[1577] "Means of converting to PDF or e-book format and outputting" refers to the means of converting created documents into a format that is easy for users to view and providing them as electronic files. PDF and e-book formats are examples of such formats.

[1578] "Category information" refers to information related to a specific category, and is information for identifying themes or fields in which a user is interested.

[1579] The "means for collecting related information" refers to a means for searching and collecting related information from the Internet based on the input category information.

[1580] "Means for generating a guide" refers to means for organizing collected information and creating guides or manuals for specific purposes.

[1581] The "means for constructing as an application" refers to a means for converting the generated guide into an application in a form that can be used by the user and providing it to the user.

[1582] This invention relates to a system that uses an information processing device to efficiently collect, organize, and summarize information. This system receives a list of multiple documents entered by a user, generates summary information for each document using a generative AI model, aggregates them to create a single document, and converts and outputs the summary in PDF or e-book format, thereby reducing the burden on the user.

[1583] First, the user inputs a list of multiple documents using a terminal. For example, the user inputs the document titles such as "CEO's Book 1" and "CEO's Book 2" in a browser or dedicated application. The input documents are sent from the terminal to the server using an HTTP POST request.

[1584] The server receives the list of documents sent by the user and analyzes their contents. Next, it generates a summary for each document title using a generative AI model (e.g., OpenAI's GPT-3, which uses natural language processing technology). The server sends a prompt to the generative AI model to obtain the summary.

[1585] As a concrete example, suppose the user inputs the following list:

[1586] ["CEO Book 1", "CEO Book 2", "CEO Book 3"]

[1587] In response, the server sends the following prompt to the generative AI model:

[1588] "Generate a summary for each of the following book titles:\n- CEO Book 1\n- CEO Book 2\n- CEO Book 3"

[1589] Each summary is received from the generative AI model, and a summary like the one below is returned.

[1590] Summary 1: "CEO Book 1 Summary"

[1591] Summary 2: "CEO Book 2 Summary"

[1592] Summary 3: "CEO Book 3 Summary"

[1593] The server aggregates these summaries into a single document and organizes them, for example, as follows:

[1594] Final Document: "CEO Book 1 Summary\n\nCEO Book 2 Summary\n\nCEO Book 3 Summary"

[1595] The server then converts the final document into a format that is easy for users to view, such as PDF or e-book format, using a PDF generation tool or e-book format conversion tool.

[1596] Finally, the server provides the converted document to the user, for example by sending the user an email containing a download link for the generated file, or by displaying the link on a dedicated download page.

[1597] This system allows users to efficiently collect large amounts of information and check summaries, significantly reducing the burden of information processing.It also has the function of automatically collecting, organizing, and summarizing information related to specific categories from the Internet, making it suitable for collecting a variety of information.

[1598] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1599] Step 1:

[1600] The user enters a list of documents

[1601] Input: A user uses a terminal to enter the titles of several documents, for example, "CEO Book 1," "CEO Book 2," "CEO Book 3," etc.

[1602] How it works: A user enters a list of documents into a form in a browser or dedicated application and presses the submit button.

[1603] Output: The input data is prepared as a JSON list and sent to the next processing step.

[1604] Step 2:

[1605] The device sends the data to the server

[1606] Input: A list of documents entered by the user.

[1607] How it works: The device sends JSON-formatted data to the server using an HTTP POST request.

[1608] Output: The data reaches the server and goes on to the next processing step.

[1609] Step 3:

[1610] The server receives the input data and starts processing it.

[1611] Input: Document list sent from the terminal.

[1612] Behavior: The server parses the received data and extracts a list of document titles.

[1613] Output: The extracted list of document titles is used to generate summaries in the next step.

[1614] Step 4:

[1615] The server makes a request to generate a summary

[1616] Input: A list of extracted document titles.

[1617] How it works: The server calls a generative AI model (e.g., OpenAI GPT-3) and creates a prompt for generating a summary. For example, "Generate a summary for the following book title: CEO Book 1." This prompt is then sent to the generative AI model.

[1618] Output: Summary information returned from the generative AI model.

[1619] Step 5:

[1620] The server retrieves the summary results

[1621] Input: Summary information from a generative AI model.

[1622] How it works: The server takes the summary information received from the generative AI model and organizes it by document title.

[1623] Output: Data to aggregate organized summary information into a single document.

[1624] Step 6:

[1625] The server aggregates the summary information

[1626] Input: Organized summary information.

[1627] What it does: Concatenates the summaries into a single document, organized by paragraph, for example, "Summary of CEO Book 1\n\nSummary of CEO Book 2\n\nSummary of CEO Book 3."

[1628] Output: The final summary document.

[1629] Step 7:

[1630] The server converts and outputs the final document

[1631] Input: Final summary document.

[1632] What it does: The server converts this document to PDF or e-book format using tools such as PDF generation tools and e-book format conversion tools.

[1633] Output: The converted document file (e.g. PDF), a download link provided to the user.

[1634] Step 8:

[1635] The server provides the final product to the user.

[1636] Input: The converted document file.

[1637] Action: The server initiates the steps to provide the generated file to the user, for example by sending an email containing a download link or by displaying the link on a dedicated download page.

[1638] Output: A user-accessible link or download page.

[1639] This system will enable users to efficiently collect information requested by the public and check summaries, thereby significantly reducing the burden of information processing.

[1640] (Application example 1)

[1641] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1642] Conventional information processing systems lack the means for users to efficiently collect, summarize, and use information from multiple documents. Furthermore, it is difficult to efficiently provide product information, particularly in brick-and-mortar stores, making it difficult for customers to quickly obtain the information they need.

[1643] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1644] In this invention, the server includes means for receiving a list of input documents, means for generating summary information for each document in the list, means for aggregating the generated summary information to create a single document, means for outputting the created document, means for receiving product-related information and generating summary information, and means for aggregating the generated summary information to create a single document and providing the document to the user. This makes it possible to efficiently collect, summarize, and provide information required by users and customers as a single document.

[1645] An "information processing device" is a combination of hardware and software for receiving, processing, generating, and outputting information.

[1646] The "means for receiving a list of documents" is a device having a function for receiving a list of titles and contents of a plurality of documents input by the user.

[1647] A "means for generating summary information" is an algorithm or software that extracts important information from an input document and generates a summary in a concise format.

[1648] The "means for aggregating summary information to create one document" refers to a device having a function for aggregating and integrating a plurality of pieces of generated summary information into one continuous document.

[1649] The "means for outputting a document" is a device that has the function of providing the created document to the user in a format such as PDF or e-book.

[1650] The "means for receiving information related to merchandise" is a device having a function for receiving merchandise information in a store and related market data.

[1651] The "means for providing to the user" is a device having a function for providing the generated information or document to the user in an accessible form.

[1652] The present invention relates to a system that uses an information processing device to efficiently collect, organize, and summarize information. This system receives a list of multiple documents entered by the user, generates and aggregates summary information for each document, and creates and outputs a single document, thereby reducing the burden on the user. It is also possible to collect information related to a specific category from the Internet, generate a guide, and provide it as an application. Furthermore, the system also has the function of efficiently collecting information related to products in physical stores and providing summary information.

[1653] The server has a means of receiving a list of multiple documents input by the user. This list consists of the titles and contents of documents that the user is interested in. The server then uses natural language processing techniques on these documents to generate summaries. The natural language processing techniques used include OpenAI APIs. Once the summaries are generated, the server aggregates them into a single document. Finally, the generated document is provided to the user in PDF or e-book format.

[1654] Next, the server receives information related to the product and has a means to generate summary information in the same way. This includes the process of collecting related information from the Internet. For example, when collecting the latest information and trend information about a product and generating summary information, the following prompt sentence is used for the generative AI model:

[1655] Example prompt sentence:

[1656] Provide updates and summaries on the "latest smartphones".

[1657] By inputting this prompt into a generative AI model, the latest information on the topic is obtained and a summary is generated, which is then compiled into a single document and ultimately provided to the user in PDF or e-book format.

[1658] For example, if a user enters a list such as "latest smartphones," "trendy fashion items," or "products on sale," the server collects and summarizes information about each item and creates a final, best-of document. This process allows users and customers to efficiently gather the information they need and obtain it in a summarized form.

[1659] In the embodiment of the invention, the input data is diverse, including information collected from the Internet, data entered by the user, data on products in the store, etc. This makes it possible to build a system that eliminates the complexity of information collection that users and customers face and can provide necessary information quickly and efficiently.

[1660] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1661] Step 1:

[1662] User enters a list of documents

[1663] A user inputs the titles and topics of multiple documents he or she wishes to summarize using a terminal, and the input list is sent to the server.

[1664] Specific operation: A user uses a smartphone or PC to enter the title of a document through a web interface or application, and then clicks the send button to send it to the server.

[1665] Step 2:

[1666] The server receives the input data

[1667] The server receives a list of documents submitted by the user, which are added to a processing queue.

[1668] Specific operation: The server receives the HTTP request, analyzes the sent data, and stores it in a database or queue.

[1669] Step 3:

[1670] The server generates the summary information

[1671] Based on the received list of documents, the server generates a summary of each document using a generative AI model. The summary information is obtained by inputting a prompt to the generative AI model.

[1672] Specific operation: The server calls OpenAI's API, sends the prompt sentence to the generative AI model, and obtains summary information.

[1673] Example prompt: Please provide an update and summary on "The Latest Smartphones."

[1674] Step 4:

[1675] The server aggregates the summary information

[1676] The generated summaries are compiled into a single document, which is then converted into a PDF or e-book format and prepared for delivery to users.

[1677] Specific operation: The server concatenates each summary information and converts it into PDF or e-book format using a document formatter.

[1678] Step 5:

[1679] The server provides the final product to the user

[1680] The generated document is sent to the user terminal, where it can be viewed or downloaded by the user.

[1681] Specific operation: The server transfers the file using the HTTP response so that the generated PDF or e-book file can be sent appropriately to the user's device.

[1682] Step 6:

[1683] The server receives product-related information

[1684] Information related to in-store products is collected from the internet and other data sources, and the collected information serves as input for generating summary information.

[1685] Specific operation: The server uses an API to obtain product information from a data source on the Internet.

[1686] Step 7:

[1687] The server generates a summary of the product information

[1688] Based on the collected product information, the server generates a summary using a generative AI model.

[1689] Specific operation: The server sends prompts to the generative AI model to summarize data including the latest product information and consumer reviews, and obtains summary information.

[1690] Step 8:

[1691] Aggregating and outputting summary information about products

[1692] The generated summaries are then compiled into a single document and provided to the user.

[1693] Specific operation: The server concatenates the summary information, converts it into PDF or e-book format, and sends it to the user's terminal to provide it to the user as a single document.

[1694] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1695] The present invention relates to a system that efficiently collects, organizes, and summarizes information using an information processing device. This system receives a list of multiple documents entered by the user, generates and consolidates summary information for each document, and creates and outputs a single document, thereby reducing the user's burden. It is also possible to collect information related to a specific category from the Internet, generate a guide, and provide it as an application. Furthermore, the present invention is equipped with an emotion engine that recognizes the user's emotions and has the function of adjusting the method of generating and providing information based on the user's emotions.

[1696] Overall processing flow of the program

[1697] The system operates as follows.

[1698] 1. The user inputs a document list

[1699] The user inputs a list of documents of interest from the terminal, for example, by entering titles such as "Document 1," "Document 2," and "Document 3." This information is sent to the server.

[1700] 2. The device acquires emotion data

[1701] The device collects user emotional data (facial expressions, voice, text input, etc.) using an emotion engine, which then uses the data to adjust the content and generation method of document summaries.

[1702] 3. The device sends the data to the server

[1703] The device sends both the document list and the emotion data to the server. The sent data is summarized in JSON format, for example, as follows: {"documents":["Document 1", "Document 2", "Document 3"], "emotions":{"happiness":0.8, "sadness":0.1}}.

[1704] 4. The server receives the document list and begins processing.

[1705] Based on the document list received by the server, natural language processing technology is used to generate summary information for each document. The server calls a summary generation API for the title of each document to obtain the summary information.

[1706] 5. The server analyzes the emotion data and adjusts the generated content

[1707] The server analyzes the received emotional data and adjusts the summary method according to the user's emotions. For example, if the user is excited, it will provide detailed summary information, but if the user is calm, it will provide a short summary of the main points.

[1708] 6. The server aggregates the summary information

[1709] The server then compiles the summaries into a single document, concatenating each summary piece in turn to create a final document that includes all the summaries.

[1710] 7. The server delivers the final product to the user

[1711] The completed best version document is converted into PDF or e-book format and provided to the user, who can then download or stream the final document to their device.

[1712] 8. Generating Guides and Applications

[1713] When a user selects a specific category (e.g., Excel functions), the server collects and filters related information from the internet to generate a comprehensive guide, which is then tailored based on the user's preferences and ultimately delivered as a mobile or web app.

[1714] Specific examples

[1715] Example 1: CEO Book Summary

[1716] The user enters the following list:

[1717] ["CEO Book 1", "CEO Book 2", "CEO Book 3"]

[1718] The device acquires emotion data from the user's facial expressions and voice, then sends it to the server:

[1719] {"documents":["CEO Book 1", "CEO Book 2", "CEO Book 3"], "emotions":{"happiness":0.7, "interest":0.9}}

[1720] The server gets a summary from the chat AI for each book and adjusts the summary method based on the emotion data:

[1721] Summary 1: "CEO Book 1 Summary"

[1722] Summary 2: "CEO Book 2 Summary"

[1723] Summary 3: "CEO Book 3 Summary"

[1724] The server creates the final best version of the document:

[1725] Final Document: "CEO Book 1 Summary\n\nCEO Book 2 Summary\n\nCEO Book 3 Summary"

[1726] Example 2: Creating a guide on Excel functions

[1727] User selected categories:

[1728] "Excel Functions"

[1729] The device acquires the user's emotion data and then sends it to the server:

[1730] {"category":"Excel Functions", "emotions":{"focus":0.8, "confidence":0.7}}

[1731] The server collects relevant information from the internet and filters it based on emotion data:

[1732] Information 1: "Basic Excel Functions"

[1733] Information 2: "Advanced Excel Functions"

[1734] Information 3: "How to use Excel functions"

[1735] The server generates guides based on collected information and adjusts the content based on the user's emotions:

[1736] Guide: "Basic Excel Functions\n\nAdvanced Excel Functions\n\nHow to Use Excel Functions"

[1737] Build the server-generated guide into an application and serve it to users:

[1738] App: "Excel Function Guide App"

[1739] This allows the user to easily obtain detailed information about the specified category and use the information in a way that best suits their own feelings.

[1740] The processing flow will be explained below.

[1741] Program processing steps

[1742] Example 1: Compiling the CEO's top 10 books using chat generation AI

[1743] Step 1:

[1744] The user inputs a list of CEO's books from the terminal. For example, the user inputs titles such as "CEO's Book 1," "CEO's Book 2," and "CEO's Book 3."

[1745] Step 2:

[1746] The device acquires emotion data from the user's facial expressions and voice. The emotion engine analyzes the user's camera footage and microphone input to detect their emotional state (e.g., happiness, excitement, etc.).

[1747] Step 3:

[1748] The terminal transmits the input document list and the acquired emotion data to the server. The transmitted data is in a format that includes the document list and emotion data.

[1749] Step 4:

[1750] The server receives the document list and begins the process of generating summary information for each title. The server uses natural language processing techniques to generate a summary for each document.

[1751] Step 5:

[1752] The server analyzes the emotion data and adjusts the summary content based on the user's emotion, for example, generating a detailed and comprehensive summary when the user is in a high excitement state, and a concise summary when the user is in a low excitement state.

[1753] Step 6:

[1754] The server aggregates the generated summaries into a single document, and then concatenates all the summaries to create a final, best-of document.

[1755] Step 7:

[1756] The server converts the final product into a PDF or e-book format, which makes it easy for users to view.

[1757] Step 8:

[1758] The server provides the converted file to the user, who then downloads or streams the final document from their device.

[1759] Example 2: Picking out frequently used information and turning it into an app

[1760] Step 1:

[1761] The user selects the category (e.g. Excel Functions) that he is interested in from the terminal. For example, he selects the category as "Excel Functions".

[1762] Step 2:

[1763] The device acquires emotion data from the user's facial expressions and voice. The emotion engine recognizes facial expressions and analyzes voice to detect the user's emotional state.

[1764] Step 3:

[1765] The terminal transmits the selected category information and emotion data to the server. The transmitted data includes the category information and emotion data.

[1766] Step 4:

[1767] The server receives the category information and collects related information from the Internet. Web scraping technology is used to gather useful information about Excel functions.

[1768] Step 5:

[1769] The server verifies and filters the information collected, eliminating duplicate information and selecting useful information.

[1770] Step 6:

[1771] The server analyzes the emotional data and adjusts the way the collected information is organized and the content of the guide based on the user's emotions. For example, if the user is feeling confident, it will include more advanced content, and conversely, if the user is feeling anxious, it will emphasize basic content.

[1772] Step 7:

[1773] The server generates the guide and builds it into an application. Based on the collected information, a comprehensive guide is created and deployed as a mobile or web app.

[1774] Step 8:

[1775] The server provides the completed application to the user, who receives a link to download or use the app on their device.

[1776] Through the above processing steps, the user can easily obtain detailed information about the specified category and can use the information in a way that is optimal for his or her own feelings.

[1777] Example 2

[1778] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1779] Conventional information processing systems require users to collect information, summarize each document individually, and compile it, requiring a great deal of effort. Furthermore, the summarized information is uniform and not tailored to the user's emotions or interests, preventing optimal information provision. Furthermore, it is difficult to provide comprehensive guidance related to specific categories of information in a format tailored to each individual user. To solve these problems, a new system is needed that can collect, summarize, aggregate, and tailor information based on the user's emotions.

[1780] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1781] In this invention, the server includes means for receiving a list of multiple documents input by a user, means for acquiring user emotion data, and means for transmitting the acquired emotion data and list to the server. This enables the generation and provision of optimal document summary information based on the user's emotion. The server also includes means for using natural language processing technology to generate document summary information and means for adjusting the generated summary information based on the user's emotion data. This enables the creation of highly accurate summary documents that reflect the user's emotion. The server also includes means for aggregating the adjusted summary information to create a single document and means for outputting the created document. This enables the information desired by the user to be provided as a single comprehensive document, thereby realizing efficient collection and use of information.

[1782] An "information processing device" is a device that processes data and organizes information using a combination of computer systems and software.

[1783] A "document list" is a list that compiles the titles and contents of multiple documents entered or selected by the user.

[1784] "Natural language processing technology" is a technology that enables computers to understand and process human language, and is used for generating summaries and semantic analysis.

[1785] "Emotion data" is data that indicates the emotional state of a user, obtained by analyzing the user's facial expression, voice, text input, and the like.

[1786] "Summary information" is information that shortens the content of the original document and succinctly summarizes the main points and content.

[1787] "Aggregation" means combining multiple elements or pieces of information into one.

[1788] "Document creation" refers to the process of composing multiple pieces of summary information into a single continuous document and converting it into an outputtable format.

[1789] "Output" refers to providing the generated data or document to the user, and includes means such as displaying it on the screen, generating a PDF, or providing a download link.

[1790] "Category information" is data that indicates information about a particular theme or field in which a user is interested.

[1791] A "guide" is a comprehensive instruction manual or explanatory document that provides information on a particular topic.

[1792] "Application" means a program that provides specific functions or services, including mobile applications and web applications.

[1793] This invention relates to a system that uses an information processing device to efficiently collect, organize, and summarize information. This system receives a list of multiple documents entered by the user, generates summary information for each document, and aggregates them to create and output a single document, thereby reducing the burden on the user. It is also possible to collect information related to a specific category from the Internet, generate a guide, and provide it as an application.

[1794] Hardware and software used

[1795] Hardware

[1796] User devices (PCs, smartphones, tablets, etc.)

[1797] Server (high-performance data processing system)

[1798] software

[1799] Natural Language Processing (NLP) engines (e.g., OpenAI GPT-4)

[1800] Sentiment Analysis Engine

[1801] Generative AI Models

[1802] Data processing and calculation

[1803] 1. The user inputs a document list

[1804] The user inputs the titles of the documents they are interested in into the input form on their device. For example, they input them in a list format, such as "Document A," "Document B," "Document C," etc. This creates a document list and sends it to the server.

[1805] 2. The device acquires emotion data

[1806] The device uses a built-in emotion analysis engine to collect user emotional data in real time. For example, it uses the device's built-in camera and microphone to analyze the user's facial expressions and tone of voice. This allows it to quantify emotional parameters such as how happy or excited the user is.

[1807] 3. The device sends the data to the server

[1808] The device encodes the collected document list and emotion data into JSON format and sends it to the server. For example, the data is sent in the following format:

[1809] {"documents":["Document A", "Document B", "Document C"], "emotions":{"happiness":0.8, "sadness":0.1}}

[1810] 4. The server receives the document list and begins processing.

[1811] The server uses natural language processing (NLP) technology to generate summary information for each document based on the received document list. The NLP engine calls a summary generation API based on the title of each document to obtain the summary information.

[1812] 5. The server analyzes the emotion data and adjusts the generated content

[1813] The server adjusts the summarization method based on the received emotion data according to the user's emotional state, for example, providing a detailed summary when the user is excited and generating a concise summary when the user is calm.

[1814] 6. The server aggregates the summary information

[1815] The server aggregates the obtained summaries into a single document, and then sequentially concatenates the summaries to create a single final document containing all the summaries.

[1816] 7. The server delivers the final product to the user

[1817] The final product is converted into PDF or e-book format and provided to the user, who can then download or stream the final document to their device.

[1818] 8. The server generates the guide and application

[1819] When a user selects a specific category, the server collects and filters related information from the internet to generate a comprehensive guide, which is then tailored based on the user's sentiment and ultimately provided as a mobile or web app.

[1820] Specific examples

[1821] Example 1: CEO Book Summary

[1822] The user enters the following list:

[1823] ["CEO Book 1", "CEO Book 2", "CEO Book 3"]

[1824] The device acquires emotion data from the user's facial expressions and voice, then sends it to the server:

[1825] {"documents":["CEO Book 1", "CEO Book 2", "CEO Book 3"], "emotions":{"happiness":0.7, "interest":0.9}}

[1826] The server uses natural language processing to obtain a summary for each book and adjusts the summary method based on sentiment data:

[1827] Summary 1: "CEO Book 1 Summary"

[1828] Summary 2: "CEO Book 2 Summary"

[1829] Summary 3: "CEO Book 3 Summary"

[1830] The server creates the final document:

[1831] Final Document: "CEO Book 1 Summary\n\nCEO Book 2 Summary\n\nCEO Book 3 Summary"

[1832] Example 2: Creating a guide on Excel functions

[1833] User selected categories:

[1834] "Excel Functions"

[1835] The device acquires the user's emotion data and then sends it to the server:

[1836] {"category":"Excel Functions", "emotions":{"focus":0.8, "confidence":0.7}}

[1837] The server collects relevant information from the internet and filters it based on emotion data:

[1838] Information 1: "Basic Excel Functions"

[1839] Information 2: "Advanced Excel Functions"

[1840] Information 3: "How to use Excel functions"

[1841] The server generates guides based on collected information and adjusts the content based on the user's emotions:

[1842] Guide: "Basic Excel Functions\n\nAdvanced Excel Functions\n\nHow to Use Excel Functions"

[1843] Build the server-generated guide into an application and serve it to users:

[1844] App: "Excel Function Guide App"

[1845] By executing the above processing procedure, the present invention collects information, summarizes it, and generates guides optimally based on the user's emotions, thereby improving the user experience.

[1846] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1847] Step 1:

[1848] The user enters a document list

[1849] Input: A user inputs a list of document titles into their device, for example, "Document A," "Document B," "Document C," etc.

[1850] Specific operation: The user enters document titles one by one into the input form on the terminal and presses the "Send" button.

[1851] Output: The input document list is saved as local data on the device.

[1852] Step 2:

[1853] The device acquires emotion data

[1854] Input: Real-time data such as the user's facial image and voice.

[1855] How it works: The device's built-in camera and microphone capture the user's facial expressions and tone of voice, and an emotion analysis engine analyzes this data.

[1856] Data processing: The emotion analysis engine converts the user's emotional status into a numerical value, such as "happiness=0.8" or "sadness=0.1."

[1857] Output: Quantified emotion data is generated.

[1858] Step 3:

[1859] The device sends the data to the server

[1860] Input: The document list generated in step 1 and the emotion data generated in step 2.

[1861] Specific operation: The device encodes the document list and emotion data into JSON format and sends it to the server via an HTTP request.

[1862] Output: The data sent to the server looks like this:

[1863] {"documents":["Document A", "Document B", "Document C"], "emotions":{"happiness":0.8, "sadness":0.1}}

[1864] Step 4:

[1865] The server receives the document list and starts processing

[1866] Input: JSON data containing the document list sent in step 3.

[1867] Specific operation: The server receives the HTTP request and parses the document list.

[1868] Data calculation: Using a natural language processing (NLP) engine, a summary generation API is called for each document title to obtain summary information.

[1869] Output: A summary is generated for each document.

[1870] Step 5:

[1871] The server analyzes the emotional data and adjusts the generated content

[1872] Input: The summary information generated in step 4 and the emotion data sent in step 3.

[1873] Specific operation: The server analyzes the emotion data and adjusts each summary information according to the user's emotional state.

[1874] Data processing: For example, providing a detailed summary when the user is excited and adjusting the summary to a concise one when the user is calm.

[1875] Output: A tailored summary is generated.

[1876] Step 6:

[1877] The server aggregates the summary information

[1878] Input: Summary information refined in step 5.

[1879] Specific operation: The server concatenates each abstract and creates one final document that contains all the abstracts.

[1880] Data processing: Arrange the summary information according to a sequential linking algorithm.

[1881] Output: The final aggregated document is generated.

[1882] Step 7:

[1883] The server provides the final product to the user.

[1884] Input: The aggregate document generated in step 6.

[1885] Specific operation: The server converts the aggregate document into PDF format and provides a download link to the user's device.

[1886] Output: The final document in PDF format is provided to the user, who can download or view it on their device.

[1887] Step 8:

[1888] The server generates the guides and applications.

[1889] Input: Specific category information selected by the user.

[1890] Specific operation: The server collects relevant information from the Internet and filters the information based on emotion data.

[1891] Data Computing: Generate comprehensive guides based on collected information and adjust content based on user sentiment.

[1892] Output: A finished guide delivered as a mobile app or web app.

[1893] (Application example 2)

[1894] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1895] Currently, physical stores require staff to respond to customers quickly and appropriately, but it is often difficult to provide the optimal response and information for customer questions and requests.In addition, it is difficult to provide information based on the customer's emotions, which makes it difficult to improve customer satisfaction.

[1896] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving a list of multiple input documents, means for generating summary information for each document in the list, means for aggregating the generated summary information to create a single document, means for outputting the created document, means for acquiring related information based on the input question content and emotion data, and means for proposing the optimal response method based on the acquired information. This enables staff in physical stores to respond quickly and appropriately to customer questions and requests, and further improves customer satisfaction by providing optimal information based on customer emotions.

[1897] An "information processing device" is a device for analyzing and processing input information.

[1898] A "document list" is a list that includes the titles and contents of multiple documents.

[1899] "Summary information" is information that briefly summarizes the important content of the entire document.

[1900] "Emotional data" is data that indicates the user's emotional state and is generated based on facial expressions, voice, text input, etc.

[1901] "Natural language processing technology" is a technology that allows computers to understand and process human language.

[1902] "Means for collecting relevant information from the Internet" refers to methods for searching data on the Internet and collecting the necessary information.

[1903] The means for generating a "guide" is a method for creating a guide that is useful to users based on the collected information.

[1904] An "application" is a software program used for a specific function or purpose.

[1905] "Customer support" refers to providing support to store staff to provide appropriate support to customers.

[1906] "Means for providing optimal information" refers to a method for selecting and providing the most appropriate information based on user input and emotional data.

[1907] The present invention provides an information processing system for supporting customer service in brick-and-mortar stores. This system uses an information processing device to generate a summary of a document and provide appropriate information based on emotion data. A specific embodiment of this system is described below.

[1908] Hardware and software used

[1909] 1. Hardware:

[1910] Devices such as smartphones and smart glasses: These are devices that staff carry and use.

[1911] Server: Used for processing and storing data.

[1912] 2. Software:

[1913] Natural language processing technology: Technology for summarizing documents. For example, Python libraries such as NLTK and spaCy are used.

[1914] API: An interface for obtaining information corresponding to customer questions, for example, using a RESTful API.

[1915] Emotion engine: An engine for analyzing customer emotion data, for example, using OpenFace or Emotion API.

[1916] Processing flow

[1917] 1. User input:

[1918] The user (staff member) inputs customer questions via a smartphone or smart glasses, and this information is sent to the server, which is the starting point for processing.

[1919] 2. Collecting Emotional Data:

[1920] The device collects emotional data from the user's facial expressions and voice using a camera and microphone, and analyzes the data using an emotion engine. The collected emotional data is then sent to the server along with the questions asked.

[1921] 3. Document and information summary generation:

[1922] Based on the received question, the server retrieves appropriate document and product information from a database, and generates summary information from this information using natural language processing technology.

[1923] 4. Adjust based on sentiment data:

[1924] Based on the analyzed emotional data, the server adjusts the way it presents information, for example, providing detailed explanations if the customer is excited, or a concise summary if the customer is calm.

[1925] 5. Recommendations for the best response:

[1926] The server then proposes the optimal response based on the acquired information, which provides specific guidance on how staff should respond to the customer.

[1927] 6. Provision of Information:

[1928] The final generated information and response methods are displayed on the staff's terminal, allowing the staff to immediately provide the appropriate information to the customer.

[1929] Specific examples

[1930] For example, if a customer asks a store employee, "What is the material of this shirt?", the employee inputs the question through the smart glasses. The device captures emotional data of the customer's happiness from their facial expression and sends it to the server. The server then retrieves information from a database of related products and generates a detailed summary of the shirt's material. The server then analyzes the emotional data and, if it determines that the customer is interested in purchasing, suggests appropriate actions to encourage the purchase. Finally, this information and actions are displayed on the employee's smart glasses, allowing them to immediately explain the situation to the customer.

[1931] Prompt Sentence Examples

[1932] User: What is the material of this shirt?

[1933] Emotion Data: Customers are interested.

[1934] Requested: Product information presentation and customer support suggestions

[1935] This invention allows staff in physical stores to respond to customers quickly and optimally, thereby improving customer satisfaction.

[1936] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1937] Step 1:

[1938] The user uses an input device (such as a smartphone or smart glasses) to input questions or requests from customers. The information entered by the user is temporarily stored on the device. The input is made using a text input interface or a voice recognition system.

[1939] Input: Customer question or request (e.g., "What is the material of this shirt?")

[1940] Output: Text data of the question

[1941] Step 2:

[1942] The device analyzes the user's facial expressions and tone of voice to obtain emotional data. This is done using sensors such as a camera and microphone, and analysis is performed using an emotion engine (e.g., OpenFace or Emotion API). The analysis results are stored on the device as a quantified emotional state.

[1943] Input: User facial expressions and tone of voice

[1944] Output: Quantified emotion data (e.g., happiness, interest, etc.)

[1945] Step 3:

[1946] The device packages the question and emotion data and sends it to the server, typically in JSON format, via internet.

[1947] Input: Question text and sentiment data

[1948] Output: Packaged data (JSON format)

[1949] Step 4:

[1950] The server analyzes the received data and retrieves product information corresponding to the question from the database. At this time, the server retrieves information from the external database using a RESTful API.

[1951] Input: Packaged data (JSON format)

[1952] Output: Retrieved product information

[1953] Step 5:

[1954] The server generates a summary of the document based on the product information it has acquired. This process uses natural language processing techniques (e.g., NLTK and spaCy). The summary is generated in a concise and to-the-point format.

[1955] Input: Retrieved product information

[1956] Output: Summary information

[1957] Step 6:

[1958] The server analyzes the emotion data and adjusts the summary presentation, for example, including detailed information if the customer is interested, or providing a more concise summary if the customer is not. The adjustments based on emotion data are made using a generative AI model.

[1959] Input: Summary information and emotion data

[1960] Output: Adjusted summary information

[1961] Step 7:

[1962] The server proposes the optimal response method. To do this, it refers to past data and case studies and generates specific response methods and wording, etc. This is also done using a generative AI model.

[1963] Input: Adjusted summary information and emotion data

[1964] Output: Best Practices

[1965] Step 8:

[1966] The server then sends the final information (adjusted summary information and optimal response methods) to the terminal, which displays this information for immediate use by staff. The display can be done using a text display interface or a speech synthesis system.

[1967] Input: Final generated information

[1968] Output: Information displayed on the staff member's terminal

[1969] This series of processes enables staff in physical stores to provide customers with information and respond in the most prompt and optimal way.

[1970] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1971] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1972] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1973] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1974] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1975] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1976] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1977] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1978] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1979] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1980] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1981] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1982] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1983] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1984] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1985] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1986] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1987] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1988] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1989] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1990] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1991] The following is further disclosed regarding the above embodiment.

[1992] (Claim 1)

[1993] In the information processing device,

[1994] means for receiving a list of the input documents;

[1995] means for generating summary information for each document in the list;

[1996] a means for aggregating the generated summary information to create a single document;

[1997] means for outputting the created document;

[1998] A system including:

[1999] (Claim 2)

[2000] 2. The system according to claim 1, wherein the means for generating summary information uses natural language processing technology to summarize the document.

[2001] (Claim 3)

[2002] A means for collecting related information from the Internet based on the input category information;

[2003] A means for generating a guide based on the collected information;

[2004] A means for building the generated guide into an application;

[2005] means for outputting the application;

[2006] The system of claim 1 further comprising:

[2007] "Example 1"

[2008] (Claim 1)

[2009] In the information processing device,

[2010] means for receiving a list of the input documents;

[2011] means for generating summary information for each document in the list using a generative AI model;

[2012] a means for aggregating the generated summary information to create a single document;

[2013] A means for converting the created document into a PDF or e-book format and outputting it;

[2014] A system including:

[2015] (Claim 2)

[2016] 2. The system according to claim 1, wherein the means for generating summary information uses natural language processing technology to summarize the document.

[2017] (Claim 3)

[2018] A means for collecting related information from the Internet based on the input category information;

[2019] A means for generating a guide based on the collected information;

[2020] A means for constructing and outputting the generated guide as an application;

[2021] The system of claim 1 further comprising:

[2022] "Application Example 1"

[2023] (Claim 1)

[2024] means for receiving a list of the input documents;

[2025] means for generating summary information for each document in the list;

[2026] a means for aggregating the generated summary information to create a single document;

[2027] means for outputting the created document;

[2028] means for receiving information related to the product and generating summary information;

[2029] a means for aggregating the generated summary information to create a single document and providing the document to a user;

[2030] A system including:

[2031] (Claim 2)

[2032] 2. The system according to claim 1, wherein the means for generating summary information uses natural language processing technology to summarize the document.

[2033] (Claim 3)

[2034] A means for collecting related information from the Internet based on the input category information;

[2035] A means for generating a guide based on the collected information;

[2036] A means for building the generated guide into an application;

[2037] means for outputting the application;

[2038] means for collecting summary information and providing summary information of products in the store;

[2039] The system of claim 1 further comprising:

[2040] "Example 2: Combining Emotion Engines"

[2041] (Claim 1)

[2042] means for receiving a list of a plurality of documents input by a user;

[2043] A means for acquiring user emotion data;

[2044] A means for transmitting the acquired emotion data and list to a server;

[2045] means for using natural language processing techniques to generate summary information for a document;

[2046] means for adjusting the generated summary information based on emotion data of the user;

[2047] a means for aggregating the adjusted summary information into a single document;

[2048] means for outputting the created document;

[2049] A system including:

[2050] (Claim 2)

[2051] 2. The system according to claim 1, wherein the means for generating summary information uses natural language processing technology to summarize the document, and adjusts the summarization method based on user emotion data.

[2052] (Claim 3)

[2053] A means for collecting related information from the Internet based on the input category information;

[2054] a means for generating a guide adjusted based on emotion data from the collected information;

[2055] A means for building the generated guide into an application;

[2056] means for outputting the application;

[2057] The system of claim 1 further comprising:

[2058] "Application example 2 when combining emotion engines"

[2059] (Claim 1)

[2060] In the information processing device,

[2061] means for receiving a list of the input documents;

[2062] means for generating summary information for each document in the list;

[2063] a means for aggregating the generated summary information to create a single document;

[2064] means for outputting the created document;

[2065] A means for acquiring related information based on the input question content and emotion data;

[2066] A means to propose the best response method based on the acquired information,

[2067] A system including:

[2068] (Claim 2)

[2069] The means for generating summary information uses natural language processing technology to summarize documents,

[2070] 2. The system according to claim 1, wherein the information presentation method is adjusted based on the emotion data.

[2071] (Claim 3)

[2072] A means for collecting related information from the Internet based on the input category information;

[2073] A means for generating a guide based on the collected information;

[2074] A means for building the generated guide into an application;

[2075] means for outputting the application;

[2076] 2. The system according to claim 1, wherein the application is used to support customer service in a physical store. [Explanation of symbols]

[2077] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. In the information processing device, means for receiving a list of the input documents; means for generating summary information for each document in the list; a means for aggregating the generated summary information to create a single document; means for outputting the created document; A system including:

2. 2. The system according to claim 1, wherein the means for generating summary information uses natural language processing technology to summarize the document.

3. A means for collecting related information from the Internet based on the input category information; A means for generating a guide based on the collected information; A means for building the generated guide into an application; means for outputting the application; The system of claim 1 further comprising:

Images