System

Abstract

A system is provided.SOLUTION: This system is provided with a means for collecting the past minutes, materials and comments of an upper layer, a means for analyzing collected contents by using a natural language processing technology, a means for registering the analyzed result in a database, a means for receiving and analyzing the query of a user, and for retrieving proper information, and a means for summarizing and displaying the retrieved result to the user.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Due to the complexity of the work, there is a need to quickly check the contents of past meetings and the opinions of upper management to improve work efficiency. In particular, a system is needed that can centrally manage minutes, materials, and comments from upper management across multiple projects and efficiently search and reference this information. However, conventional systems make it difficult to efficiently manage and search this information, preventing work efficiency from being realized. [Means for solving the problem]

[0005] The present invention provides a system that includes a means for collecting past minutes, documents, and comments from senior management, a means for analyzing the collected content using natural language processing technology, a means for registering the analysis results in a database, a means for receiving and analyzing user queries and searching for appropriate information, and a means for summarizing and displaying the search results to the user. The system also includes a means for summarizing the search results for the queries using a generative AI model and a means for storing the analyzed minutes and documents in a database in metadata format, thereby solving these problems. This allows users to easily search and reference past minutes, documents, and senior management's comments, thereby improving work efficiency and speeding up decision-making.

[0006] A "minutes" is a document that records the contents of a meeting or discussion, and includes a detailed description of what was said and what decisions were made.

[0007] "Materials" refers to documents and files containing information and data related to a meeting or project, such as reference materials and proposals.

[0008] "Senior Management Comments" are records of opinions and feedback from senior management (management and executives) related to projects and meetings.

[0009] "Natural language processing technology" is a technology that processes and analyzes human language using a computer, and includes text analysis and keyword extraction.

[0010] A "database" is a system for efficiently storing, searching, and managing large amounts of data.

[0011] A "query" is a user-entered inquiry to retrieve information from a database.

[0012] A "generative AI model" is a model that uses artificial intelligence techniques to generate, summarize, and transform data, presenting information to users in an easily understandable format.

[0013] The "metadata format" is a data format that includes additional information for managing data (for example, meeting date and time, participants, agenda, etc.). [Brief explanation of the drawings]

[0014] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0015] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0016] First, the terms used in the following description will be explained.

[0017] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0018] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0019] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0020] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0021] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0022] [First embodiment]

[0023] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0024] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0025] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0026] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0027] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0028] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0029] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0030] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0031] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0032] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0033] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0034] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0035] The present invention is a system that collects past minutes, documents, and comments from senior management, analyzes them using natural language processing technology, and registers the analysis results in a database. The system searches for the analysis results based on queries from users, summarizes the search results, and displays them to the user. A specific embodiment of this system is described below.

[0036] System configuration

[0037] The system consists of the following main components:

[0038] 1. Data Collection Components

[0039] Terminal: Provides an interface for users to upload electronic files such as minutes, documents, and comments from senior management into the system.

[0040] Server: Receives uploaded files and prepares them for analysis.

[0041] 2. Data Analysis Component

[0042] Server: Analyzes the uploaded text data using natural language processing technology, including part-of-speech analysis, keyword extraction, and semantic understanding.

[0043] Server: Structures the analysis results and generates metadata such as meeting date and time, participants, agenda, and content of comments.

[0044] 3. Database Component

[0045] Server: The parsed data and generated metadata are registered in a database.

[0046] 4. Query Processing Component

[0047] Terminal: Provides an interface for users to enter natural language queries.

[0048] Server: Analyzes the received query and searches for relevant information from a database.

[0049] 5. Search Results Summary Component

[0050] Server: Uses a generative AI model to summarize search results and convert them into an easy-to-understand format.

[0051] Terminal: Displays summarized search results to the user.

[0052] System operation example

[0053] 1. Data Collection Example

[0054] Terminal: The user selects project minutes, documents, and comments from upper management and uploads them to the system.

[0055] Server: Receives uploaded files and temporarily stores their contents.

[0056] 2. Data Analysis Example

[0057] Server: Uses a natural language processing engine to analyze the text of the uploaded minutes, extracting information such as "The Project ABC meeting took place on May 3, 2022, and the main topic was budget adjustment."

[0058] Server: Based on the analysis results, generate metadata such as the meeting date and time (May 3, 2022), participants (speaker list), and main agenda item (budget adjustment).

[0059] 3. Example of database registration

[0060] Server: The parsed text data and metadata are stored in a database, allowing for efficient search and reference later.

[0061] 4. Query Processing Example

[0062] User: For example, enter a query such as "I want to see the minutes of the project ABC held in May of last year" into the terminal.

[0063] Terminal: Sends queries to the server.

[0064] Server: Parses the query using a natural language processing engine and searches the database for relevant minutes.

[0065] 5. Example of summary and display of search results

[0066] Server: Uses a generative AI model to summarize the search results, for example, "The Project ABC meeting took place on May 3, 2022, and the main topic was budget adjustment."

[0067] Terminal: Displays summarized search results to the user.

[0068] In this way, users can quickly and easily search and refer to past meeting minutes, documents, and comments from upper management, which improves work efficiency and speeds up decision-making.

[0069] The processing flow will be explained below.

[0070] Step 1:

[0071] Terminal: The user prepares the minutes, materials, and comments from senior management as electronic files. The user uses the system interface to select the files to upload.

[0072] Step 2:

[0073] Terminal: The user sends the selected file to the server. The file is sent to the server through the system's upload function.

[0074] Step 3:

[0075] Server: The received file is stored in a temporary location before analysis, and preparations for analysis are made.

[0076] Step 4:

[0077] Server: The natural language processing engine is launched to analyze the text data in the uploaded file. During the analysis process, parts of speech analysis, keyword extraction, context understanding, etc. are performed.

[0078] Step 5:

[0079] Server: Based on the analysis results, metadata such as the meeting date and time, participants, agenda, and content of remarks is extracted from the text data. This information is formatted as structured data.

[0080] Step 6:

[0081] Server: The analyzed text data and generated metadata are registered in a database, allowing for efficient search and reference later.

[0082] Step 7:

[0083] User: Enters a query through the system interface to search for specific information, such as "I want to see the minutes of the project ABC meeting held last May."

[0084] Step 8:

[0085] Terminal: Sends queries entered by the user to the server.

[0086] Step 9:

[0087] Server: Analyzes the received query using natural language processing to understand the meaning of the sentence, allowing it to efficiently search for relevant information in a database.

[0088] Step 10:

[0089] Server: Searches the database for meeting minutes, documents, comments from senior management, etc. that correspond to the query.

[0090] Step 11:

[0091] Server: Using a generative AI model, the search results are summarized and converted into an easy-to-understand format.

[0092] Step 12:

[0093] Server: Sends summarized search results to the terminal.

[0094] Step 13:

[0095] Terminal: Displays the search results received from the server to the user, allowing the user to quickly check related information.

[0096] Step 14:

[0097] Users: Optionally, click on the resulting links to view detailed minutes, materials, and comments from senior management.

[0098] Example 1

[0099] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0100] Until now, it has been difficult to efficiently collect and analyze past meeting records, documents, and comments from senior management, and to quickly obtain the information users need. In particular, the process of searching for appropriate information from large amounts of data takes time and effort, and it is not easy to summarize the search results in an easy-to-understand format. Therefore, there is a need for a system that can easily search past meeting records and documents and provide easy-to-understand summaries.

[0101] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0102] In this invention, the server includes means for collecting past meeting records, materials, and comments from senior managers, means for analyzing the collected content using natural language processing technology, means for registering the analysis results in a database, means for receiving and analyzing user queries and searching for appropriate information, and means for summarizing the search results and displaying them to the user, thereby enabling the user to quickly and efficiently search past meeting records and materials and obtain easy-to-understand summaries.

[0103] "Past meeting records" refers to minutes and notes taken during meetings and discussions.

[0104] "Materials" refers to documents such as reports, presentations, and explanations used in meetings and discussions.

[0105] "Comments from senior management" refers to opinions or instructions left by senior management of a company or organization in meetings or in writing.

[0106] "Natural language processing technology" refers to technology that allows computers to analyze and understand human language.

[0107] "Analysis results" refers to information and data extracted from text data using natural language processing technology.

[0108] "Means for registering in a database" refers to a method or system for storing analysis results in a database.

[0109] "User query" refers to a search request or question entered by a user into the system.

[0110] "Means for retrieving relevant information" refers to techniques and methods for retrieving relevant information from a database based on a user's query.

[0111] "Means for summarizing search results and displaying them to the user" refers to techniques and methods for summarizing the searched information and providing it to the user in an easy-to-understand format.

[0112] A "generative artificial intelligence model" refers to a technology that uses artificial intelligence learned from large amounts of data to analyze and generate data.

[0113] "Metadata format" refers to a structured data format for describing the content and attributes of data.

[0114] This invention is a system that collects and analyzes past meeting records, documents, and comments from senior managers, registers the analysis results in a database, searches based on user queries, and displays summaries. This system allows users to quickly and efficiently obtain the information they need.

[0115] System configuration

[0116] The system mainly consists of the following main components:

[0117] 1. Data Collection Components

[0118] Terminal: Serves as an interface for users to upload electronic files such as meeting records, materials, senior management comments, etc. Specifically, the user selects a file using a file selection dialog and clicks the "Upload" button.

[0119] Server: Receives and temporarily stores files sent from the device. For example, it uses a cloud storage service such as Amazon S3.

[0120] 2. Data Analysis Component

[0121] Server: Analyzes the uploaded text data using natural language processing techniques. Specifically, it uses natural language processing libraries such as NLTK and spaCy to perform part-of-speech analysis, keyword extraction, and semantic understanding. For example, it analyzes the sentence "The Project ABC meeting was held on May 3, 2022, and the main topic was budget adjustment."

[0122] Server: Based on the analysis results, generate metadata (meeting date and time, participants, agenda, content of remarks, etc.).

[0123] 3. Database Component

[0124] Server: The parsed text data and generated metadata are registered in a database. Specifically, a database such as MySQL or PostgreSQL is used to execute SQL queries and store the data.

[0125] 4. Query Processing Component

[0126] Terminal: Provides an interface for users to enter queries in natural language. Users enter queries in natural language into a search box and click a "Search" button.

[0127] Server: Analyzes the received query and searches for relevant information from the database. Specifically, it analyzes the query using natural language processing models such as BERT and GPT, and generates and executes the appropriate SQL query.

[0128] 5. Search Results Summary Component

[0129] Server: Using a generative AI model (e.g., GPT-3), the search results are summarized. The summary is then converted into a format that is easy for users to understand. For example, a summary is generated: "The Project ABC meeting took place on May 3, 2022, and the main agenda item was budget adjustment."

[0130] Terminal: Displaying summarized search results to the user. The summarized results are displayed on the terminal interface, allowing the user to easily browse the information.

[0131] Specific examples and prompts

[0132] Data Collection Example

[0133] Terminal: The user uploads "Project ABC Meeting Minutes for May 3, 2022" to the system.

[0134] Server: Receives and temporarily stores files.

[0135] Data analysis example

[0136] Server: The uploaded minutes are analyzed using a natural language processing engine. For example, the sentence "The Project ABC meeting was held on May 3, 2022, and the main topic was budget adjustment" is analyzed.

[0137] Server: From the analysis results, generate metadata such as "Meeting date and time: May 3, 2022," "Participants: List," and "Agenda: Budget adjustment."

[0138] Example queries and search results

[0139] User: In the search box, type "Show me the meeting minutes for project ABC from last May."

[0140] Terminal: Sends input to the server.

[0141] Server: Uses GPT-3 to parse the query and search and summarize "Project ABC meeting on May 3, 2022" from the database.

[0142] Device: Display the summarized search result "Budget adjustment is the main topic."

[0143] Example prompt sentence:

[0144] "What are the main topics for the Project ABC meeting on May 3, 2022?"

[0145] With the above-described procedure and configuration, the present invention provides a system that enables a user to efficiently and quickly obtain the information he or she requires.

[0146] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0147] Program processing steps

[0148] Step 1: Collect data

[0149] Terminal - The user uses the terminal interface to select electronic files such as meeting notes, materials, and senior management comments, and clicks the upload button.

[0150] Input: Electronic file selected by the user

[0151] Output: File upload request from the device to the server

[0152] What happens: The user opens the file selection dialog, selects the appropriate file, and clicks "Upload."

[0153] Step 2: Receiving and temporarily saving the file

[0154] Server - The server receives the file sent from the device and temporarily stores it.

[0155] Input: Electronic file sent from the terminal

[0156] Output: File saved in temporary directory

[0157] Specific operation: The server saves the received file in the specified directory (e.g., / uploads) and records the file metadata (file name, upload time, etc.).

[0158] Step 3: Data analysis using natural language processing

[0159] Server - Analyzes the stored text data using natural language processing techniques.

[0160] Input: A file saved in the temporary directory

[0161] Output: Analysis results (part of speech analysis results, keyword list, semantic understanding results)

[0162] Specific operation: The server uses libraries such as NLTK and spaCy to perform part-of-speech analysis, keyword extraction, and semantic understanding of text data. For example, it parses the sentence "The Project ABC meeting was held on May 3, 2022, and the main topic was budget adjustment."

[0163] Step 4: Generate Metadata

[0164] Server - Generates metadata (meeting date and time, participants, agenda, content of comments, etc.) based on the analysis results.

[0165] Input: Analysis results from natural language processing

[0166] Output: Metadata (meeting date and time, participants, agenda, and comments)

[0167] Specific operation: Extract necessary information from the analysis results and generate metadata such as "Meeting date and time: May 3, 2022," "Participants: List," and "Agenda: Budget adjustment."

[0168] Step 5: Register with the database

[0169] Server - The parsed text data and generated metadata are registered in a database.

[0170] Input: Parsed text data and metadata

[0171] Output: Analysis results and metadata stored in a database

[0172] What happens next: The server runs SQL queries against a database like MySQL or PostgreSQL and inserts the data into the appropriate tables.

[0173] Step 6: Enter and submit your query

[0174] User - The user types a query in natural language and submits it through the terminal interface.

[0175] Input: A query entered by a user (e.g., "I want to see the minutes of the meeting on project ABC from last May")

[0176] Output: Query request from the terminal to the server

[0177] Specific behavior: The user enters a query in the search box and clicks the "Search" button.

[0178] Step 7: Parsing the query and searching

[0179] Server - Parses the incoming query and retrieves the relevant information from the database.

[0180] Input: The query submitted by the user

[0181] Output: Search results (related meeting records and materials)

[0182] What it does: The server uses natural language processing models such as BERT or GPT to analyze the query and generate and execute the appropriate SQL query.

[0183] Step 8: Summarizing search results

[0184] Server - Summarizes search results using a generative AI model.

[0185] Input: Search results from the database

[0186] Output: Summarized search results

[0187] What it does: The server uses a generative AI model (e.g., GPT-3) to summarize the search results and convert them into an understandable format, such as generating a summary like "The Project ABC meeting took place on May 3, 2022, and the main topic was budget adjustment."

[0188] Step 9: View summary results

[0189] Terminal - Display summarized search results to the user.

[0190] Input: Abstracted search results from the server

[0191] Output: Summary results displayed to the user on their screen

[0192] Specific operation: The summary results are displayed on the terminal interface in a format that can be easily viewed by the user.

[0193] Through the above steps, users can efficiently search past meeting records and materials and obtain easy-to-understand summaries.

[0194] (Application example 1)

[0195] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0196] It is difficult to efficiently analyze large amounts of data, such as past meeting minutes, documents, and comments from senior management, and provide an environment where users can easily access important information. Furthermore, there is a need to instantly summarize and present relevant information in response to user queries, which requires the use of advanced natural language processing technology and generative AI models. However, current systems often do not adequately meet these requirements.

[0197] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0198] In this invention, the server includes means for collecting past minutes, documents, and comments from upper management, means for analyzing the collected content using natural language processing technology, means for registering the analysis results in a database, means for receiving and analyzing user queries and searching for appropriate information, means for summarizing the search results and displaying them to the user, and means for generating summaries of relevant minutes and documents using a generative AI model when a user inputs a specific query. This makes it possible to efficiently extract necessary information from large amounts of past data, quickly and appropriately summarize it, and provide it to the user.

[0199] "Past minutes" are documents that record the contents of past meetings and discussions.

[0200] "Materials" refers to documents and digital data used to communicate, store, and analyze information.

[0201] "Comments from upper management" refers to opinions and instructions issued by managers or executives of a company or organization.

[0202] "Natural language processing technology" refers to technology that allows computers to understand, analyze, and generate human language.

[0203] A "generative AI model" is an algorithm that uses artificial intelligence to generate new content from data such as text and images.

[0204] "Metadata format" is additional information for describing the attributes of data, and clearly indicates the type and structure of the data.

[0205] A "query" is a question or request to a database or system to retrieve information.

[0206] A "server" is a computer that provides data over a network, and is responsible for storing, analyzing, and providing data.

[0207] A specific embodiment of the present invention will be described. The present invention relates to a system that collects past minutes, documents, and comments from senior management, analyzes them using natural language processing technology, and registers the results in a database. It has the function of searching for analysis results based on user queries and summarizing and displaying the results using a generative AI model.

[0208] System configuration

[0209] 1. Data Collection Components

[0210] Terminal: Provides an interface for users to upload electronic files such as minutes, documents, and comments from senior management into the system.

[0211] Server: Receives uploaded files and prepares them for analysis.

[0212] 2. Data Analysis Component

[0213] Server: Analyzes the uploaded text data using natural language processing technology. The analysis uses Spacey's natural language processing engine and includes part-of-speech analysis, keyword extraction, and semantic understanding.

[0214] Server: Structures the analysis results and generates metadata such as meeting date and time, participants, agenda, and content of comments.

[0215] 3. Database Component

[0216] Server: The analyzed data and generated metadata are registered in a database built on cloud storage.

[0217] 4. Query Processing Component

[0218] Terminal: Provides an interface for users to enter natural language queries.

[0219] Server: Analyzes the received query and searches for relevant information from a database.

[0220] 5. Search Results Summary Component

[0221] Server: Uses a generative AI model to summarize search results. Leverages Hugging Face's transformers library.

[0222] Terminal: Displays summarized search results to the user.

[0223] Explanation of system behavior

[0224] 1. Data Collection

[0225] Users use their devices to select project minutes, documents, and comments from senior management and upload them to the system. The server receives the uploaded files and temporarily stores their contents.

[0226] 2. Data Analysis

[0227] The server uses Spacey's natural language processing engine to analyze the text of the uploaded minutes, extracting information such as, "The Project ABC meeting took place on May 3, 2022, and the main topic was budget adjustment." It then generates metadata based on the analysis results, including the meeting date and time, participants, and main topic.

[0228] 3. Registering in the database

[0229] The server stores the analyzed text data and metadata in a database, allowing for efficient search and reference later.

[0230] 4. Query Processing

[0231] A user inputs a query into a terminal, such as "I want to know the details of the meeting about strengthening security." The terminal sends the query to the server, which then analyzes the query using a natural language processing engine and searches for relevant minutes from the database.

[0232] 5. Summarizing and displaying search results

[0233] The server uses the generative AI model to summarize the search results, for example, "The Project ABC meeting took place on May 3, 2022, and the main topic was budget adjustment." The device then displays the summarized search results to the user.

[0234] Examples of concrete examples and prompts

[0235] The user inputs a query on their device, such as "Please summarize the contents of the meeting about strengthening security." The server analyzes this query, extracts relevant minutes and materials from the database, and generates a summary using a generative AI model.

[0236] Example prompt sentence:

[0237] Please summarize the meeting on strengthening security.

[0238] In this way, the system enables users to efficiently extract necessary information from large amounts of past data, quickly and appropriately summarize it, and provide it to the users.

[0239] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0240] Step 1:

[0241] Uploading data

[0242] The user uses a terminal to select project minutes, documents, and comments from upper management and upload them to the system. The input is the electronic file selected by the user, which is sent to the server. The server receives the uploaded file and temporarily stores it. This stored data is then analyzed in the next step.

[0243] Step 2:

[0244] Data analysis

[0245] The server uses Spacy, a natural language processing engine, to analyze the uploaded text data. The input includes text data from temporarily saved electronic files. The server performs part-of-speech analysis, keyword extraction, and semantic understanding to extract information such as the meeting date and time, participants, agenda, and content of remarks. The analysis results and metadata are generated as output.

[0246] Step 3:

[0247] Registering in the database

[0248] The server registers the analyzed text data and generated metadata in a database on cloud storage. The analysis results and metadata generated in step 2 are used as input. Once registration in the database is complete, subsequent search processing and reference can be performed efficiently. The output is an update of the database.

[0249] Step 4:

[0250] Entering and submitting a query

[0251] A user uses a terminal to input a query in natural language. For example, a query such as "I want to know the details of the conference on strengthening security" is input. The input is the user's query text, which is sent to the server via the terminal. The output is the query text passed to the server.

[0252] Step 5:

[0253] Query Parsing and Search

[0254] The server uses a natural language processing engine to analyze the received query. The input includes the user's query text. Based on this analysis, it searches the database for relevant information. The output is the text data of the relevant minutes and documents.

[0255] Step 6:

[0256] Generate summary of search results

[0257] The server uses a generative AI model (Hugging Face transformers) to summarize the search results. The input is the text data of the minutes and documents extracted in step 5. The generative AI model summarizes the results and generates a summary text in an easy-to-understand format as output.

[0258] Step 7:

[0259] Viewing summary results

[0260] The terminal displays the summary text received from the server to the user. The input includes the summary text sent from the server. The user can check this summary and obtain the necessary information. The output is displayed visually to the user.

[0261] These steps enable users to efficiently extract necessary information from large amounts of past data and obtain information that is quickly and appropriately summarized.

[0262] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0263] The present invention is a system that collects past meeting minutes, documents, and comments from senior management, analyzes them using natural language processing technology and an emotion engine, and registers the analysis results in a database. It recognizes user queries and their associated emotions, customizes search results, summarizes them, and displays them to the user. A specific embodiment of this system is described below.

[0264] System configuration

[0265] The system consists of the following main components:

[0266] 1. Data Collection Components

[0267] Terminal: Provides an interface for users to upload electronic files such as minutes, documents, and comments from senior management into the system.

[0268] Server: Receives uploaded files and prepares them for analysis.

[0269] 2. Data Analysis Component

[0270] Server: Analyzes the uploaded text data using natural language processing technology. The analysis process involves parts of speech analysis, keyword extraction, and context understanding.

[0271] Server: Based on the analysis results, metadata such as the meeting date and time, participants, agenda, and content of remarks is extracted from the text data. This information is formatted as structured data.

[0272] 3. Database Component

[0273] Server: The analyzed text data and generated metadata are registered in a database, allowing for efficient search and reference later.

[0274] 4. Query Processing Component

[0275] Terminal: Provides an interface for users to enter natural language queries.

[0276] Server: Analyzes the received query and searches for relevant information from a database.

[0277] 5. Emotion Recognition Component

[0278] Terminal: Provides an interface for recognizing the user's emotional state from their facial expressions and voice when entering a query.

[0279] Server: Uses an emotion engine to analyze user emotions and use them as data to customize search results.

[0280] 6. Search Results Summary Component

[0281] Server: Uses a generative AI model to summarize search results and convert them into an easy-to-understand format.

[0282] Terminal: Displays summarized search results to the user.

[0283] 7. Emotional Feedback Component

[0284] Server: Collects user emotional feedback, analyzes and stores the data, and uses it to improve the accuracy of the system.

[0285] System operation example

[0286] 1. Data Collection Example

[0287] Terminal: The user selects project minutes, documents, and comments from upper management and uploads them to the system.

[0288] Server: Receives uploaded files and temporarily stores their contents.

[0289] 2. Data Analysis Example

[0290] Server: Uses a natural language processing engine to analyze the text of the uploaded minutes, extracting information such as "The Project ABC meeting took place on May 3, 2022, and the main topic was budget adjustment."

[0291] Server: Based on the analysis results, generate metadata such as the meeting date and time (May 3, 2022), participants (speaker list), and main agenda item (budget adjustment).

[0292] 3. Example of database registration

[0293] Server: The parsed text data and metadata are stored in a database, allowing for efficient search and reference later.

[0294] 4. Query Processing Example

[0295] User: For example, enter a query such as "I want to see the minutes of the project ABC held in May of last year" into the terminal.

[0296] Device: The emotion engine analyzes the user's facial expressions and voice when entering a query.

[0297] Terminal: Sends queries and emotion data to the server.

[0298] Server: Parses the query using a natural language processing engine and searches the database for relevant minutes.

[0299] 5. Example of customizing, summarizing and displaying search results

[0300] Server: Customize search results based on emotional data, for example, by providing a more concise summary if the user is feeling stressed.

[0301] Server: Uses a generative AI model to summarize the search results, for example, "The Project ABC meeting took place on May 3, 2022, and the main topic was budget adjustment."

[0302] Terminal: Displays summarized search results to the user, allowing the user to quickly view relevant information.

[0303] 6. Collecting Emotional Feedback

[0304] Terminal: Records the user's emotional feedback on search results.

[0305] Server: Collects emotional feedback to help improve the system's accuracy.

[0306] In this way, users can quickly and easily search and refer to past minutes, documents, and comments from upper management, improving work efficiency and speeding up decision-making.In addition, by using an emotion engine, it is possible to provide even more user-friendly and customized services.

[0307] The processing flow will be explained below.

[0308] Step 1:

[0309] Terminal: The user prepares the minutes, materials, and comments from senior management as electronic files. The user uses the system interface to select the files to upload.

[0310] Step 2:

[0311] Terminal: The user sends the selected file to the server. The file is sent to the server through the system's upload function.

[0312] Step 3:

[0313] Server: The received file is stored in a temporary location before analysis, and preparations for analysis are made.

[0314] Step 4:

[0315] Server: The natural language processing engine is launched to analyze the text data in the uploaded file. During the analysis process, parts of speech analysis, keyword extraction, context understanding, etc. are performed.

[0316] Step 5:

[0317] Server: Based on the analysis results, metadata such as the meeting date and time, participants, agenda, and content of remarks is extracted from the text data. This information is formatted as structured data.

[0318] Step 6:

[0319] Server: The analyzed text data and metadata are registered in a database, allowing for efficient search and reference later.

[0320] Step 7:

[0321] User: Enters a query through the system interface to search for specific information, such as "I want to see the minutes of the project ABC meeting held last May."

[0322] Step 8:

[0323] On the device: When a user enters a query, the emotion engine analyzes the user's facial expressions and voice. The emotion engine uses facial recognition and voice analysis technologies to assess the user's emotional state.

[0324] Step 9:

[0325] Terminal: Sends queries and emotion data to the server.

[0326] Step 10:

[0327] Server: Analyzes the received query using natural language processing to understand the meaning of the sentence, allowing it to efficiently search for relevant information in a database.

[0328] Step 11:

[0329] Server: Searches the database for meeting minutes, documents, comments from senior management, etc. that correspond to the query.

[0330] Step 12:

[0331] Server: Customize search results based on emotional data, for example, making summaries more concise if the user is stressed.

[0332] Step 13:

[0333] Server: Using a generative AI model, the search results are summarized. For example, the summary might be something like, "The Project ABC meeting took place on May 3, 2022, and the main topic was budget adjustment."

[0334] Step 14:

[0335] Server: Sends summarized search results to the terminal.

[0336] Step 15:

[0337] Terminal: Displays the search results received from the server to the user, allowing the user to quickly check related information.

[0338] Step 16:

[0339] Users: Optionally, click on the resulting links to view detailed minutes, materials, and comments from senior management.

[0340] Step 17:

[0341] Terminal: Records the user's emotional feedback on the search results. For example, it provides an interface that asks for a simple rating to see if the user is satisfied with the results.

[0342] Step 18:

[0343] Server: Collects emotional feedback to help improve the system's accuracy. The collected data is used to optimize the user experience and improve the emotion engine.

[0344] Example 2

[0345] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0346] Conventional systems that collect and analyze minutes, documents, and comments from senior management face the challenge of efficiently searching and referencing vast amounts of information and providing customized information that reflects the user's emotions. In particular, if users are unable to quickly obtain the information they desire, this can lead to reduced operational efficiency and delays in decision-making. Furthermore, there are few systems that take user emotions into account, making it difficult to improve the user experience.

[0347] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for collecting past minutes, documents, and comments from management, means for analyzing the collected content using natural language processing technology, means for registering the analysis results in a database, means for receiving and analyzing user queries and searching for appropriate information, means for summarizing search results and displaying them to the user, means for analyzing the user's emotions at the time of query input, means for customizing search results based on the user's emotions, and means for collecting emotional feedback and using it to improve the accuracy of the system. This allows users to quickly and appropriately search and refer to past minutes, documents, and comments from upper management, and to obtain information customized according to their emotions.

[0348] "Past minutes" refers to documents and data that record what was discussed at meetings and discussions.

[0349] "Document" means any digital or paper material that records information in textual, graphical or other form.

[0350] "Management comments" refers to documents or data that record opinions or instructions issued by senior managers or executives of a company or organization.

[0351] "Natural language processing technology" refers to technology that enables computers to understand and process human language, and includes parts of speech analysis, keyword extraction, and context understanding.

[0352] "Analysis results" refers to information obtained after analyzing text data using natural language processing technology, and includes metadata and important content.

[0353] "Database" refers to a digital storage system designed to efficiently store, search, and reference information.

[0354] "Query" refers to a natural language question or command that a user enters into a system to search for specific information.

[0355] "Search Results" refers to relevant information extracted from a database based on a query.

[0356] A "generative AI model" is an artificial intelligence model trained using large datasets, and is used specifically for natural language generation and summarization.

[0357] "Emotion analysis" refers to the technology of determining a user's emotional state at any given time from data such as facial expressions and voice.

[0358] "Emotional feedback" refers to the collection of data on users' emotional reactions to search results and information provided by the system.

[0359] System Overview

[0360] This system collects past minutes, documents, and management comments, analyzes them using natural language processing technology and a sentiment analysis engine, and registers the analysis results in a database. It recognizes user queries and their associated sentiment, customizes search results, summarizes them, and displays them to the user.

[0361] Component Configuration

[0362] The system consists of the following main components:

[0363] 1. Data Collection Components

[0364] Terminal: Provides an interface for users to upload electronic files such as minutes, documents, and management comments to the system.

[0365] Server: Receives the uploaded file and stores it in a temporary directory.

[0366] 2. Data Analysis Component

[0367] Server: Specifically, it analyzes the uploaded text data using natural language processing technologies such as SpaCy and NLTK. The analysis process involves part-of-speech analysis, keyword extraction, and context understanding.

[0368] Server: Based on the analysis results, metadata such as the meeting date and time, participants, agenda, and content of remarks is generated from the text data.

[0369] 3. Database Component

[0370] Server: The parsed text data and generated metadata are registered in a database, typically MySQL or PostgreSQL.

[0371] 4. Query Processing Component

[0372] Terminal: Provides an interface for users to enter natural language queries, including a text box and voice input capabilities.

[0373] Server: Analyzes queries using a natural language processing engine and searches for relevant information from a database.

[0374] 5. Emotion Recognition Component

[0375] Terminal: Provides an interface for recognizing the user's emotional state from their facial expressions and voice when entering a query.

[0376] Server: Uses a sentiment analysis engine to analyze user sentiment and use it as data to customize search results.

[0377] 6. Search Results Summary Component

[0378] Server: Uses a generative AI model (such as OpenAI's GPT-3) to summarize and convert search results into an understandable format.

[0379] Terminal: Displays summarized search results to the user.

[0380] 7. Emotional Feedback Component

[0381] Terminal: Records the user's emotional feedback on search results.

[0382] Server: Collects emotional feedback to help improve the system's accuracy.

[0383] Specific operation example

[0384] Data Collection Example

[0385] Terminal: The user selects project minutes, documents, and management comments and uploads them to the system.

[0386] Server: Receives the uploaded file and temporarily stores it in the / tmp / uploads / directory.

[0387] Data analysis example

[0388] Server: Uses a natural language processing engine to analyze the text of the uploaded minutes, extracting information such as "The Project ABC meeting took place on May 3, 2022, and the main topic was budget adjustment."

[0389] Server: Based on the analysis results, generate metadata such as "Date and time: May 3, 2022," "Participants: Tanaka, Sato, Suzuki," and "Agenda: Budget adjustment."

[0390] Database registration example

[0391] Server: The parsed text data and metadata are stored in a MySQL or PostgreSQL database.

[0392] Query Processing Example

[0393] User: Enters query "I want to see the minutes of the project ABC meeting held last May."

[0394] Device: When a user enters a query, the facial recognition camera analyzes the user's facial expression and sends it to the server along with the query.

[0395] Server: Parses the query and retrieves relevant information from a database.

[0396] Customizing search results, summarizing and display examples

[0397] Server: Adjusts search results based on user sentiment data and creates a summary: "The Project ABC meeting took place on May 3, 2022, and the main topic was budget adjustments."

[0398] Terminal: Displays summaries, allowing users to quickly review relevant information.

[0399] Example of collecting emotional feedback

[0400] Device: Collects the user's facial expressions and voice again in response to the displayed search results and records emotional feedback.

[0401] Server: Receives emotional feedback sent from the device and stores it as data to improve the accuracy of the system.

[0402] Prompt Sentence Examples

[0403] "Analyze the minutes data and extract the main topics and participants."

[0404] "Generate a summary of the meeting about project ABC last May"

[0405] This system allows users to quickly and appropriately search and refer to past minutes, documents, and management comments, and has the advantage of being able to obtain customized information based on sentiment.

[0406] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0407] Step 1:

[0408] Terminal: Provides a file upload interface to the user. Allows the user to select a file to upload and receives file input. For example, if the user selects the file "Meeting2022_05_03.docx", the path to this file becomes the input data.

[0409] Step 2:

[0410] User: Clicks the upload button on the terminal to send the selected file to the system. Specifically, the contents of the file are transferred from the terminal to the server as binary data.

[0411] Step 3:

[0412] Server: Receives binary data sent from the terminal and saves the file in a temporary storage directory (e.g. / tmp / uploads / ). The input is the binary data, and the output is the saved file path.

[0413] Step 4:

[0414] Server: The temporarily saved file is passed to a natural language processing engine (e.g., SpaCy or NLTK) to analyze the text. The input is the saved file path, and the output is the analyzed text data. Specifically, the file is read, the character code is converted, and the text is extracted.

[0415] Step 5:

[0416] Server: The extracted text data is passed to a natural language processing engine, where it is analyzed for parts of speech, keyword extraction, and contextual understanding. The input is text data, and the output is metadata (e.g., meeting date and time, participants, agenda, and content of remarks) as the analysis results.

[0417] Step 6:

[0418] Server: The parsed metadata is registered in a database (e.g., MySQL, PostgreSQL). The input is the metadata, and the output is a confirmation of registration in the database. Specifically, the metadata is inserted into the appropriate tables in the database.

[0419] Step 7:

[0420] Terminal: Provides a query input interface to users and accepts query input in natural language. The input is a query entered by the user (e.g., "I want to see the minutes of the project ABC held in May of last year").

[0421] Step 8:

[0422] Terminal: When a user enters a query, the device recognizes their facial expression and voice and passes them to the emotion analysis engine. The input is the user's facial image and voice data, and the output is analyzed emotion data.

[0423] Step 9:

[0424] Terminal: Sends query and emotion data to the server. The input is the query and emotion data, and the output is data transmission to the server.

[0425] Step 10:

[0426] Server: Analyzes the received query using a natural language processing engine and searches for relevant information from a database. The input is the query data, and the output is the search result data.

[0427] Step 11:

[0428] Server: Customize search results using emotional data. The input is search results and emotional data, and the output is customized search results. For example, if the user is feeling stressed, the summary will be made shorter.

[0429] Step 12:

[0430] Server: Summarizes search results using a generative AI model (e.g., OpenAI GPT-3). The input is the customized search results, and the output is the summarized text.

[0431] Step 13:

[0432] Terminal: displays summarized search results to the user. The input is the summarized text and the output is the displayed information.

[0433] Step 14:

[0434] Terminal: After displaying the image, the user's facial expressions and voice are collected again and recorded as emotional feedback. The input is the user's facial expressions and voice, and the output is emotional feedback data.

[0435] Step 15:

[0436] Server: Receives emotional feedback sent from the device and accumulates the data to be used to improve the accuracy of the system. The input is emotional feedback data, and the output is updated system learning data.

[0437] (Application example 2)

[0438] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0439] In recent years, with the increasing automation of customer service, brick-and-mortar stores are also in need of systems that can respond to customer questions effectively and quickly. However, conventional systems have limitations in information search and summarization functions using natural language processing, making it particularly difficult to respond to customer questions while taking emotions into account. Furthermore, they lack a summary function for visually displaying search results concisely, and a customization function based on the user's emotions, which prevents sufficient improvement of customer satisfaction. To solve these issues, a system with emotion recognition and customization functions is needed.

[0440] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0441] In this invention, the server includes means for collecting past minutes, documents, and comments from upper management, means for analyzing the collected content using natural language processing technology, means for registering the analysis results in a database, means for receiving and analyzing user queries and searching for appropriate information, means for analyzing customer voice data and recognizing emotions, means for summarizing search results and displaying them to the user, means for summarizing related information using a generative AI model, and means for collecting user emotional feedback and improving the accuracy of the system. This enables appropriate responses to customer questions that take emotions into consideration.

[0442] Minutes are documents that record the contents of meetings and discussions.

[0443] "Materials" are documents, data files, etc. that contain information about a particular topic or project.

[0444] "Comments from senior management" is a document that compiles opinions and instructions from senior management, such as managers and executives, of a company or organization.

[0445] "Natural language processing" is a technology that allows computers to understand, analyze, and process human language.

[0446] A "database" is a collection of data that makes it easy to search, organize, and manage.

[0447] A "query" is a request or question to a database to retrieve information.

[0448] "Emotion recognition" is a technology that identifies a person's emotional state from voice, text, facial expressions, etc.

[0449] A "generative AI model" is an artificial intelligence model that generates new text based on input data.

[0450] "Emotional feedback" is the process of collecting and analyzing a user's emotional state and reactions.

[0451] This invention is a system that collects past minutes, documents, and comments from upper management, analyzes them using natural language processing technology and an emotion engine, and registers the analysis results in a database. Here, a specific embodiment of the invention will be described.

[0452] System configuration

[0453] The system consists of the following main components:

[0454] Data Collection Component

[0455] Terminal: Provides an interface for users to upload electronic files such as minutes, documents, and comments from senior management into the system.

[0456] Server: Receives uploaded files and prepares them for analysis.

[0457] Data Analysis Components

[0458] Server: Analyzes the uploaded text data using natural language processing technology. The analysis process involves parts of speech analysis, keyword extraction, and context understanding.

[0459] Server: Based on the analysis results, metadata such as the meeting date and time, participants, agenda, and content of remarks is extracted from the text data. This information is formatted as structured data.

[0460] Database Components

[0461] Server: The analyzed text data and generated metadata are registered in a database, allowing for efficient search and reference later.

[0462] Query Processing Component

[0463] Terminal: Provides an interface for users to enter natural language queries.

[0464] Server: Analyzes the received query and searches for relevant information from a database.

[0465] Emotion Recognition Component

[0466] Terminal: Provides an interface for analyzing the customer's voice and text and recognizing their emotional state.

[0467] Server: Uses an emotion engine to analyze user emotions and use them as data to customize search results.

[0468] Search Results Summary Component

[0469] Server: Uses a generative AI model to summarize search results and convert them into an easy-to-understand format.

[0470] Terminal: Displays summarized search results to the user.

[0471] Emotional Feedback Component

[0472] Server: Collects user emotional feedback, analyzes and stores the data, and uses it to improve the accuracy of the system.

[0473] Specific examples

[0474] For example, if a store staff member enters a query into the system such as "Please tell me your recent feedback on customer service at the store," the system operates as follows: First, the server receives the query and analyzes it using natural language processing technology. Next, based on the analysis results, it searches for relevant feedback from the database and summarizes it using a generative AI model. In addition, it optimizes the search results based on the results of sentiment analysis of the voice data acquired by the device. Finally, the summary result and additional information that takes sentiment into consideration are displayed on the device.

[0475] Prompt Sentence Examples

[0476] "What feedback have you received recently about customer service at your store?"

[0477] In this way, store staff can quickly and easily view relevant information and improve the quality of customer service.

[0478] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0479] Step 1:

[0480] Terminal: The user uploads electronic files such as minutes, documents, and comments from upper management to the system. This input data is sent to the server. The specific operation for uploading is to press the file selection button, select the specified file, and click the upload button.

[0481] Input: minutes, materials, comments from upper management

[0482] Output: Collected data

[0483] Step 2:

[0484] Server: Receives and temporarily stores uploaded files, which contain detailed information such as meeting date and time, participants, agenda, and comments.

[0485] Input: Collected data

[0486] Output: Saved data

[0487] Step 3:

[0488] Server: Analyzes the uploaded text data using a natural language processing engine, specifically by analyzing parts of speech, extracting keywords, and understanding context to extract important metadata.

[0489] Input: Saved data

[0490] Output: Analysis data

[0491] Step 4:

[0492] Server: Based on the analysis results, metadata such as the meeting date and time, participants, agenda, and content of remarks is extracted from the text data and formatted as structured data.

[0493] Input: Analysis data

[0494] Output: Metadata

[0495] Step 5:

[0496] Server: The generated metadata is registered in a database, allowing for efficient search and reference later.

[0497] Input: Metadata

[0498] Output: Database update

[0499] Step 6:

[0500] Terminal: The user inputs a query such as a question or request in natural language. For example, "Please tell me the minutes of the latest meeting." The input query is sent to the server.

[0501] Input: User query

[0502] Output: Query data

[0503] Step 7:

[0504] Server: Analyzes the received query and searches the database for relevant information, using natural language processing technology to accurately understand the meaning of the query.

[0505] Input: Query data

[0506] Output: Search results

[0507] Step 8:

[0508] Terminal: Collects customer voice data and analyzes it using emotion recognition technology. Specifically, the device captures the voice file, converts it into text using voice recognition software, and analyzes emotions using an emotion engine.

[0509] Input: Audio data

[0510] Output: Emotion data

[0511] Step 9:

[0512] Server: Customize search results based on emotional data. For example, if a customer is stressed, tailor search results to display them in a concise and easy-to-understand format.

[0513] Input: Emotion data, search results

[0514] Output: Customized search results

[0515] Step 10:

[0516] Server: Uses a generative AI model to summarize search results and present them to the user in a more concise format. Generates summaries using prompts.

[0517] Input: Customized search results

[0518] Output: Summary data

[0519] Step 11:

[0520] Terminal: Display summarized search results to users, allowing them to quickly check relevant information through their terminal.

[0521] Input: Summary data

[0522] Output: Display data

[0523] Step 12:

[0524] Terminal: Record the user's emotional feedback on the summary results. For example, provide a feedback button or a simple survey form to collect the user's satisfaction and additional comments.

[0525] Input: User feedback

[0526] Output: Feedback data

[0527] Step 13:

[0528] Server: Collects emotional feedback and stores it as data to improve the accuracy of the system. Specifically, it analyzes the collected feedback and identifies areas for improvement in the system.

[0529] Input: Feedback data

[0530] Output: Improvement data

[0531] This process makes it possible to provide appropriate information in response to user questions while taking into consideration their emotions, thereby improving the quality of customer service.

[0532] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0533] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0534] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0535] [Second embodiment]

[0536] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0537] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0538] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0539] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0540] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0541] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0542] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0543] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0544] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0545] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0546] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0547] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0548] The present invention is a system that collects past minutes, documents, and comments from senior management, analyzes them using natural language processing technology, and registers the analysis results in a database. The system searches for the analysis results based on queries from users, summarizes the search results, and displays them to the user. A specific embodiment of this system is described below.

[0549] System configuration

[0550] The system consists of the following main components:

[0551] 1. Data Collection Components

[0552] Terminal: Provides an interface for users to upload electronic files such as minutes, documents, and comments from senior management into the system.

[0553] Server: Receives uploaded files and prepares them for analysis.

[0554] 2. Data Analysis Component

[0555] Server: Analyzes the uploaded text data using natural language processing technology, including part-of-speech analysis, keyword extraction, and semantic understanding.

[0556] Server: Structures the analysis results and generates metadata such as meeting date and time, participants, agenda, and content of comments.

[0557] 3. Database Component

[0558] Server: The parsed data and generated metadata are registered in a database.

[0559] 4. Query Processing Component

[0560] Terminal: Provides an interface for users to enter natural language queries.

[0561] Server: Analyzes the received query and searches for relevant information from a database.

[0562] 5. Search Results Summary Component

[0563] Server: Uses a generative AI model to summarize search results and convert them into an easy-to-understand format.

[0564] Terminal: Displays summarized search results to the user.

[0565] System operation example

[0566] 1. Data Collection Example

[0567] Terminal: The user selects project minutes, documents, and comments from upper management and uploads them to the system.

[0568] Server: Receives uploaded files and temporarily stores their contents.

[0569] 2. Data Analysis Example

[0570] Server: Uses a natural language processing engine to analyze the text of the uploaded minutes, extracting information such as "The Project ABC meeting took place on May 3, 2022, and the main topic was budget adjustment."

[0571] Server: Based on the analysis results, generate metadata such as the meeting date and time (May 3, 2022), participants (speaker list), and main agenda item (budget adjustment).

[0572] 3. Example of database registration

[0573] Server: The parsed text data and metadata are stored in a database, allowing for efficient search and reference later.

[0574] 4. Query Processing Example

[0575] User: For example, enter a query such as "I want to see the minutes of the project ABC held in May of last year" into the terminal.

[0576] Terminal: Sends queries to the server.

[0577] Server: Parses the query using a natural language processing engine and searches the database for relevant minutes.

[0578] 5. Example of summary and display of search results

[0579] Server: Uses a generative AI model to summarize the search results, for example, "The Project ABC meeting took place on May 3, 2022, and the main topic was budget adjustment."

[0580] Terminal: Displays summarized search results to the user.

[0581] In this way, users can quickly and easily search and refer to past meeting minutes, documents, and comments from upper management, which improves work efficiency and speeds up decision-making.

[0582] The processing flow will be explained below.

[0583] Step 1:

[0584] Terminal: The user prepares the minutes, materials, and comments from senior management as electronic files. The user uses the system interface to select the files to upload.

[0585] Step 2:

[0586] Terminal: The user sends the selected file to the server. The file is sent to the server through the system's upload function.

[0587] Step 3:

[0588] Server: The received file is stored in a temporary location before analysis, and preparations for analysis are made.

[0589] Step 4:

[0590] Server: The natural language processing engine is launched to analyze the text data in the uploaded file. During the analysis process, parts of speech analysis, keyword extraction, context understanding, etc. are performed.

[0591] Step 5:

[0592] Server: Based on the analysis results, metadata such as the meeting date and time, participants, agenda, and content of remarks is extracted from the text data. This information is formatted as structured data.

[0593] Step 6:

[0594] Server: The analyzed text data and generated metadata are registered in a database, allowing for efficient search and reference later.

[0595] Step 7:

[0596] User: Enters a query through the system interface to search for specific information, such as "I want to see the minutes of the project ABC meeting held last May."

[0597] Step 8:

[0598] Terminal: Sends queries entered by the user to the server.

[0599] Step 9:

[0600] Server: Analyzes the received query using natural language processing to understand the meaning of the sentence, allowing it to efficiently search for relevant information in a database.

[0601] Step 10:

[0602] Server: Searches the database for meeting minutes, documents, comments from senior management, etc. that correspond to the query.

[0603] Step 11:

[0604] Server: Using a generative AI model, the search results are summarized and converted into an easy-to-understand format.

[0605] Step 12:

[0606] Server: Sends summarized search results to the terminal.

[0607] Step 13:

[0608] Terminal: Displays the search results received from the server to the user, allowing the user to quickly check related information.

[0609] Step 14:

[0610] Users: Optionally, click on the resulting links to view detailed minutes, materials, and comments from senior management.

[0611] Example 1

[0612] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0613] Until now, it has been difficult to efficiently collect and analyze past meeting records, documents, and comments from senior management, and to quickly obtain the information users need. In particular, the process of searching for appropriate information from large amounts of data takes time and effort, and it is not easy to summarize the search results in an easy-to-understand format. Therefore, there is a need for a system that can easily search past meeting records and documents and provide easy-to-understand summaries.

[0614] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0615] In this invention, the server includes means for collecting past meeting records, materials, and comments from senior managers, means for analyzing the collected content using natural language processing technology, means for registering the analysis results in a database, means for receiving and analyzing user queries and searching for appropriate information, and means for summarizing the search results and displaying them to the user, thereby enabling the user to quickly and efficiently search past meeting records and materials and obtain easy-to-understand summaries.

[0616] "Past meeting records" refers to minutes and notes taken during meetings and discussions.

[0617] "Materials" refers to documents such as reports, presentations, and explanations used in meetings and discussions.

[0618] "Comments from senior management" refers to opinions or instructions left by senior management of a company or organization in meetings or in writing.

[0619] "Natural language processing technology" refers to technology that allows computers to analyze and understand human language.

[0620] "Analysis results" refers to information and data extracted from text data using natural language processing technology.

[0621] "Means for registering in a database" refers to a method or system for storing analysis results in a database.

[0622] "User query" refers to a search request or question entered by a user into the system.

[0623] "Means for retrieving relevant information" refers to techniques and methods for retrieving relevant information from a database based on a user's query.

[0624] "Means for summarizing search results and displaying them to the user" refers to techniques and methods for summarizing the searched information and providing it to the user in an easy-to-understand format.

[0625] A "generative artificial intelligence model" refers to a technology that uses artificial intelligence learned from large amounts of data to analyze and generate data.

[0626] "Metadata format" refers to a structured data format for describing the content and attributes of data.

[0627] This invention is a system that collects and analyzes past meeting records, documents, and comments from senior managers, registers the analysis results in a database, searches based on user queries, and displays summaries. This system allows users to quickly and efficiently obtain the information they need.

[0628] System configuration

[0629] The system mainly consists of the following main components:

[0630] 1. Data Collection Components

[0631] Terminal: Serves as an interface for users to upload electronic files such as meeting records, materials, senior management comments, etc. Specifically, the user selects a file using a file selection dialog and clicks the "Upload" button.

[0632] Server: Receives and temporarily stores files sent from the device. For example, it uses a cloud storage service such as Amazon S3.

[0633] 2. Data Analysis Component

[0634] Server: Analyzes the uploaded text data using natural language processing techniques. Specifically, it uses natural language processing libraries such as NLTK and spaCy to perform part-of-speech analysis, keyword extraction, and semantic understanding. For example, it analyzes the sentence "The Project ABC meeting was held on May 3, 2022, and the main topic was budget adjustment."

[0635] Server: Based on the analysis results, generate metadata (meeting date and time, participants, agenda, content of remarks, etc.).

[0636] 3. Database Component

[0637] Server: The parsed text data and generated metadata are registered in a database. Specifically, a database such as MySQL or PostgreSQL is used to execute SQL queries and store the data.

[0638] 4. Query Processing Component

[0639] Terminal: Provides an interface for users to enter queries in natural language. Users enter queries in natural language into a search box and click a "Search" button.

[0640] Server: Analyzes the received query and searches for relevant information from the database. Specifically, it analyzes the query using natural language processing models such as BERT and GPT, and generates and executes the appropriate SQL query.

[0641] 5. Search Results Summary Component

[0642] Server: Using a generative AI model (e.g., GPT-3), the search results are summarized. The summary is then converted into a format that is easy for users to understand. For example, a summary is generated: "The Project ABC meeting took place on May 3, 2022, and the main agenda item was budget adjustment."

[0643] Terminal: Displaying summarized search results to the user. The summarized results are displayed on the terminal interface, allowing the user to easily browse the information.

[0644] Specific examples and prompts

[0645] Data Collection Example

[0646] Terminal: The user uploads "Project ABC Meeting Minutes for May 3, 2022" to the system.

[0647] Server: Receives and temporarily stores files.

[0648] Data analysis example

[0649] Server: The uploaded minutes are analyzed using a natural language processing engine. For example, the sentence "The Project ABC meeting was held on May 3, 2022, and the main topic was budget adjustment" is analyzed.

[0650] Server: From the analysis results, generate metadata such as "Meeting date and time: May 3, 2022," "Participants: List," and "Agenda: Budget adjustment."

[0651] Example queries and search results

[0652] User: In the search box, type "Show me the meeting minutes for project ABC from last May."

[0653] Terminal: Sends input to the server.

[0654] Server: Uses GPT-3 to parse the query and search and summarize "Project ABC meeting on May 3, 2022" from the database.

[0655] Device: Display the summarized search result "Budget adjustment is the main topic."

[0656] Example prompt sentence:

[0657] "What are the main topics for the Project ABC meeting on May 3, 2022?"

[0658] With the above-described procedure and configuration, the present invention provides a system that enables a user to efficiently and quickly obtain the information he or she requires.

[0659] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0660] Program processing steps

[0661] Step 1: Collect data

[0662] Terminal - The user uses the terminal interface to select electronic files such as meeting notes, materials, and senior management comments, and clicks the upload button.

[0663] Input: Electronic file selected by the user

[0664] Output: File upload request from the device to the server

[0665] What happens: The user opens the file selection dialog, selects the appropriate file, and clicks "Upload."

[0666] Step 2: Receiving and temporarily saving the file

[0667] Server - The server receives the file sent from the device and temporarily stores it.

[0668] Input: Electronic file sent from the terminal

[0669] Output: File saved in temporary directory

[0670] Specific operation: The server saves the received file in the specified directory (e.g., / uploads) and records the file metadata (file name, upload time, etc.).

[0671] Step 3: Data analysis using natural language processing

[0672] Server - Analyzes the stored text data using natural language processing techniques.

[0673] Input: A file saved in the temporary directory

[0674] Output: Analysis results (part of speech analysis results, keyword list, semantic understanding results)

[0675] Specific operation: The server uses libraries such as NLTK and spaCy to perform part-of-speech analysis, keyword extraction, and semantic understanding of text data. For example, it parses the sentence "The Project ABC meeting was held on May 3, 2022, and the main topic was budget adjustment."

[0676] Step 4: Generate Metadata

[0677] Server - Generates metadata (meeting date and time, participants, agenda, content of comments, etc.) based on the analysis results.

[0678] Input: Analysis results from natural language processing

[0679] Output: Metadata (meeting date and time, participants, agenda, and comments)

[0680] Specific operation: Extract necessary information from the analysis results and generate metadata such as "Meeting date and time: May 3, 2022," "Participants: List," and "Agenda: Budget adjustment."

[0681] Step 5: Register with the database

[0682] Server - The parsed text data and generated metadata are registered in a database.

[0683] Input: Parsed text data and metadata

[0684] Output: Analysis results and metadata stored in a database

[0685] What happens next: The server runs SQL queries against a database like MySQL or PostgreSQL and inserts the data into the appropriate tables.

[0686] Step 6: Enter and submit your query

[0687] User - The user types a query in natural language and submits it through the terminal interface.

[0688] Input: A query entered by a user (e.g., "I want to see the minutes of the meeting on project ABC from last May")

[0689] Output: Query request from the terminal to the server

[0690] Specific behavior: The user enters a query in the search box and clicks the "Search" button.

[0691] Step 7: Parsing the query and searching

[0692] Server - Parses the incoming query and retrieves the relevant information from the database.

[0693] Input: The query submitted by the user

[0694] Output: Search results (related meeting records and materials)

[0695] What it does: The server uses natural language processing models such as BERT or GPT to analyze the query and generate and execute the appropriate SQL query.

[0696] Step 8: Summarizing search results

[0697] Server - Summarizes search results using a generative AI model.

[0698] Input: Search results from the database

[0699] Output: Summarized search results

[0700] What it does: The server uses a generative AI model (e.g., GPT-3) to summarize the search results and convert them into an understandable format, such as generating a summary like "The Project ABC meeting took place on May 3, 2022, and the main topic was budget adjustment."

[0701] Step 9: View summary results

[0702] Terminal - Display summarized search results to the user.

[0703] Input: Abstracted search results from the server

[0704] Output: Summary results displayed to the user on their screen

[0705] Specific operation: The summary results are displayed on the terminal interface in a format that can be easily viewed by the user.

[0706] Through the above steps, users can efficiently search past meeting records and materials and obtain easy-to-understand summaries.

[0707] (Application example 1)

[0708] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0709] It is difficult to efficiently analyze large amounts of data, such as past meeting minutes, documents, and comments from senior management, and provide an environment where users can easily access important information. Furthermore, there is a need to instantly summarize and present relevant information in response to user queries, which requires the use of advanced natural language processing technology and generative AI models. However, current systems often do not adequately meet these requirements.

[0710] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0711] In this invention, the server includes means for collecting past minutes, documents, and comments from upper management, means for analyzing the collected content using natural language processing technology, means for registering the analysis results in a database, means for receiving and analyzing user queries and searching for appropriate information, means for summarizing the search results and displaying them to the user, and means for generating summaries of relevant minutes and documents using a generative AI model when a user inputs a specific query. This makes it possible to efficiently extract necessary information from large amounts of past data, quickly and appropriately summarize it, and provide it to the user.

[0712] "Past minutes" are documents that record the contents of past meetings and discussions.

[0713] "Materials" refers to documents and digital data used to communicate, store, and analyze information.

[0714] "Comments from upper management" refers to opinions and instructions issued by managers or executives of a company or organization.

[0715] "Natural language processing technology" refers to technology that allows computers to understand, analyze, and generate human language.

[0716] A "generative AI model" is an algorithm that uses artificial intelligence to generate new content from data such as text and images.

[0717] "Metadata format" is additional information for describing the attributes of data, and clearly indicates the type and structure of the data.

[0718] A "query" is a question or request to a database or system to retrieve information.

[0719] A "server" is a computer that provides data over a network, and is responsible for storing, analyzing, and providing data.

[0720] A specific embodiment of the present invention will be described. The present invention relates to a system that collects past minutes, documents, and comments from senior management, analyzes them using natural language processing technology, and registers the results in a database. It has the function of searching for analysis results based on user queries and summarizing and displaying the results using a generative AI model.

[0721] System configuration

[0722] 1. Data Collection Components

[0723] Terminal: Provides an interface for users to upload electronic files such as minutes, documents, and comments from senior management into the system.

[0724] Server: Receives uploaded files and prepares them for analysis.

[0725] 2. Data Analysis Component

[0726] Server: Analyzes the uploaded text data using natural language processing technology. The analysis uses Spacey's natural language processing engine and includes part-of-speech analysis, keyword extraction, and semantic understanding.

[0727] Server: Structures the analysis results and generates metadata such as meeting date and time, participants, agenda, and content of comments.

[0728] 3. Database Component

[0729] Server: The analyzed data and generated metadata are registered in a database built on cloud storage.

[0730] 4. Query Processing Component

[0731] Terminal: Provides an interface for users to enter natural language queries.

[0732] Server: Analyzes the received query and searches for relevant information from a database.

[0733] 5. Search Results Summary Component

[0734] Server: Uses a generative AI model to summarize search results. Leverages Hugging Face's transformers library.

[0735] Terminal: Displays summarized search results to the user.

[0736] Explanation of system behavior

[0737] 1. Data Collection

[0738] Users use their devices to select project minutes, documents, and comments from senior management and upload them to the system. The server receives the uploaded files and temporarily stores their contents.

[0739] 2. Data Analysis

[0740] The server uses Spacey's natural language processing engine to analyze the text of the uploaded minutes, extracting information such as, "The Project ABC meeting took place on May 3, 2022, and the main topic was budget adjustment." It then generates metadata based on the analysis results, including the meeting date and time, participants, and main topic.

[0741] 3. Registering in the database

[0742] The server stores the analyzed text data and metadata in a database, allowing for efficient search and reference later.

[0743] 4. Query Processing

[0744] A user inputs a query into a terminal, such as "I want to know the details of the meeting about strengthening security." The terminal sends the query to the server, which then analyzes the query using a natural language processing engine and searches for relevant minutes from the database.

[0745] 5. Summarizing and displaying search results

[0746] The server uses the generative AI model to summarize the search results, for example, "The Project ABC meeting took place on May 3, 2022, and the main topic was budget adjustment." The device then displays the summarized search results to the user.

[0747] Examples of concrete examples and prompts

[0748] The user inputs a query on their device, such as "Please summarize the contents of the meeting about strengthening security." The server analyzes this query, extracts relevant minutes and materials from the database, and generates a summary using a generative AI model.

[0749] Example prompt sentence:

[0750] Please summarize the meeting on strengthening security.

[0751] In this way, the system enables users to efficiently extract necessary information from large amounts of past data, quickly and appropriately summarize it, and provide it to the users.

[0752] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0753] Step 1:

[0754] Uploading data

[0755] The user uses a terminal to select project minutes, documents, and comments from upper management and upload them to the system. The input is the electronic file selected by the user, which is sent to the server. The server receives the uploaded file and temporarily stores it. This stored data is then analyzed in the next step.

[0756] Step 2:

[0757] Data analysis

[0758] The server uses Spacy, a natural language processing engine, to analyze the uploaded text data. The input includes text data from temporarily saved electronic files. The server performs part-of-speech analysis, keyword extraction, and semantic understanding to extract information such as the meeting date and time, participants, agenda, and content of remarks. The analysis results and metadata are generated as output.

[0759] Step 3:

[0760] Registering in the database

[0761] The server registers the analyzed text data and generated metadata in a database on cloud storage. The analysis results and metadata generated in step 2 are used as input. Once registration in the database is complete, subsequent search processing and reference can be performed efficiently. The output is an update of the database.

[0762] Step 4:

[0763] Entering and submitting a query

[0764] A user uses a terminal to input a query in natural language. For example, a query such as "I want to know the details of the conference on strengthening security" is input. The input is the user's query text, which is sent to the server via the terminal. The output is the query text passed to the server.

[0765] Step 5:

[0766] Query Parsing and Search

[0767] The server uses a natural language processing engine to analyze the received query. The input includes the user's query text. Based on this analysis, it searches the database for relevant information. The output is the text data of the relevant minutes and documents.

[0768] Step 6:

[0769] Generate summary of search results

[0770] The server uses a generative AI model (Hugging Face transformers) to summarize the search results. The input is the text data of the minutes and documents extracted in step 5. The generative AI model summarizes the results and generates a summary text in an easy-to-understand format as output.

[0771] Step 7:

[0772] Viewing summary results

[0773] The terminal displays the summary text received from the server to the user. The input includes the summary text sent from the server. The user can check this summary and obtain the necessary information. The output is displayed visually to the user.

[0774] These steps enable users to efficiently extract necessary information from large amounts of past data and obtain information that is quickly and appropriately summarized.

[0775] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0776] The present invention is a system that collects past meeting minutes, documents, and comments from senior management, analyzes them using natural language processing technology and an emotion engine, and registers the analysis results in a database. It recognizes user queries and their associated emotions, customizes search results, summarizes them, and displays them to the user. A specific embodiment of this system is described below.

[0777] System configuration

[0778] The system consists of the following main components:

[0779] 1. Data Collection Components

[0780] Terminal: Provides an interface for users to upload electronic files such as minutes, documents, and comments from senior management into the system.

[0781] Server: Receives uploaded files and prepares them for analysis.

[0782] 2. Data Analysis Component

[0783] Server: Analyzes the uploaded text data using natural language processing technology. The analysis process involves parts of speech analysis, keyword extraction, and context understanding.

[0784] Server: Based on the analysis results, metadata such as the meeting date and time, participants, agenda, and content of remarks is extracted from the text data. This information is formatted as structured data.

[0785] 3. Database Component

[0786] Server: The analyzed text data and generated metadata are registered in a database, allowing for efficient search and reference later.

[0787] 4. Query Processing Component

[0788] Terminal: Provides an interface for users to enter natural language queries.

[0789] Server: Analyzes the received query and searches for relevant information from a database.

[0790] 5. Emotion Recognition Component

[0791] Terminal: Provides an interface for recognizing the user's emotional state from their facial expressions and voice when entering a query.

[0792] Server: Uses an emotion engine to analyze user emotions and use them as data to customize search results.

[0793] 6. Search Results Summary Component

[0794] Server: Uses a generative AI model to summarize search results and convert them into an easy-to-understand format.

[0795] Terminal: Displays summarized search results to the user.

[0796] 7. Emotional Feedback Component

[0797] Server: Collects user emotional feedback, analyzes and stores the data, and uses it to improve the accuracy of the system.

[0798] System operation example

[0799] 1. Data Collection Example

[0800] Terminal: The user selects project minutes, documents, and comments from upper management and uploads them to the system.

[0801] Server: Receives uploaded files and temporarily stores their contents.

[0802] 2. Data Analysis Example

[0803] Server: Uses a natural language processing engine to analyze the text of the uploaded minutes, extracting information such as "The Project ABC meeting took place on May 3, 2022, and the main topic was budget adjustment."

[0804] Server: Based on the analysis results, generate metadata such as the meeting date and time (May 3, 2022), participants (speaker list), and main agenda item (budget adjustment).

[0805] 3. Example of database registration

[0806] Server: The parsed text data and metadata are stored in a database, allowing for efficient search and reference later.

[0807] 4. Query Processing Example

[0808] User: For example, enter a query such as "I want to see the minutes of the project ABC held in May of last year" into the terminal.

[0809] Device: The emotion engine analyzes the user's facial expressions and voice when entering a query.

[0810] Terminal: Sends queries and emotion data to the server.

[0811] Server: Parses the query using a natural language processing engine and searches the database for relevant minutes.

[0812] 5. Example of customizing, summarizing and displaying search results

[0813] Server: Customize search results based on emotional data, for example, by providing a more concise summary if the user is feeling stressed.

[0814] Server: Uses a generative AI model to summarize the search results, for example, "The Project ABC meeting took place on May 3, 2022, and the main topic was budget adjustment."

[0815] Terminal: Displays summarized search results to the user, allowing the user to quickly view relevant information.

[0816] 6. Collecting Emotional Feedback

[0817] Terminal: Records the user's emotional feedback on search results.

[0818] Server: Collects emotional feedback to help improve the system's accuracy.

[0819] In this way, users can quickly and easily search and refer to past minutes, documents, and comments from upper management, improving work efficiency and speeding up decision-making.In addition, by using an emotion engine, it is possible to provide even more user-friendly and customized services.

[0820] The processing flow will be explained below.

[0821] Step 1:

[0822] Terminal: The user prepares the minutes, materials, and comments from senior management as electronic files. The user uses the system interface to select the files to upload.

[0823] Step 2:

[0824] Terminal: The user sends the selected file to the server. The file is sent to the server through the system's upload function.

[0825] Step 3:

[0826] Server: The received file is stored in a temporary location before analysis, and preparations for analysis are made.

[0827] Step 4:

[0828] Server: The natural language processing engine is launched to analyze the text data in the uploaded file. During the analysis process, parts of speech analysis, keyword extraction, context understanding, etc. are performed.

[0829] Step 5:

[0830] Server: Based on the analysis results, metadata such as the meeting date and time, participants, agenda, and content of remarks is extracted from the text data. This information is formatted as structured data.

[0831] Step 6:

[0832] Server: The analyzed text data and metadata are registered in a database, allowing for efficient search and reference later.

[0833] Step 7:

[0834] User: Enters a query through the system interface to search for specific information, such as "I want to see the minutes of the project ABC meeting held last May."

[0835] Step 8:

[0836] On the device: When a user enters a query, the emotion engine analyzes the user's facial expressions and voice. The emotion engine uses facial recognition and voice analysis technologies to assess the user's emotional state.

[0837] Step 9:

[0838] Terminal: Sends queries and emotion data to the server.

[0839] Step 10:

[0840] Server: Analyzes the received query using natural language processing to understand the meaning of the sentence, allowing it to efficiently search for relevant information in a database.

[0841] Step 11:

[0842] Server: Searches the database for meeting minutes, documents, comments from senior management, etc. that correspond to the query.

[0843] Step 12:

[0844] Server: Customize search results based on emotional data, for example, making summaries more concise if the user is stressed.

[0845] Step 13:

[0846] Server: Using a generative AI model, the search results are summarized. For example, the summary might be something like, "The Project ABC meeting took place on May 3, 2022, and the main topic was budget adjustment."

[0847] Step 14:

[0848] Server: Sends summarized search results to the terminal.

[0849] Step 15:

[0850] Terminal: Displays the search results received from the server to the user, allowing the user to quickly check related information.

[0851] Step 16:

[0852] Users: Optionally, click on the resulting links to view detailed minutes, materials, and comments from senior management.

[0853] Step 17:

[0854] Terminal: Records the user's emotional feedback on the search results. For example, it provides an interface that asks for a simple rating to see if the user is satisfied with the results.

[0855] Step 18:

[0856] Server: Collects emotional feedback to help improve the system's accuracy. The collected data is used to optimize the user experience and improve the emotion engine.

[0857] Example 2

[0858] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0859] Conventional systems that collect and analyze minutes, documents, and comments from senior management face the challenge of efficiently searching and referencing vast amounts of information and providing customized information that reflects the user's emotions. In particular, if users are unable to quickly obtain the information they desire, this can lead to reduced operational efficiency and delays in decision-making. Furthermore, there are few systems that take user emotions into account, making it difficult to improve the user experience.

[0860] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for collecting past minutes, documents, and comments from management, means for analyzing the collected content using natural language processing technology, means for registering the analysis results in a database, means for receiving and analyzing user queries and searching for appropriate information, means for summarizing search results and displaying them to the user, means for analyzing the user's emotions at the time of query input, means for customizing search results based on the user's emotions, and means for collecting emotional feedback and using it to improve the accuracy of the system. This allows users to quickly and appropriately search and refer to past minutes, documents, and comments from upper management, and to obtain information customized according to their emotions.

[0861] "Past minutes" refers to documents and data that record what was discussed at meetings and discussions.

[0862] "Document" means any digital or paper material that records information in textual, graphical or other form.

[0863] "Management comments" refers to documents or data that record opinions or instructions issued by senior managers or executives of a company or organization.

[0864] "Natural language processing technology" refers to technology that enables computers to understand and process human language, and includes parts of speech analysis, keyword extraction, and context understanding.

[0865] "Analysis results" refers to information obtained after analyzing text data using natural language processing technology, and includes metadata and important content.

[0866] "Database" refers to a digital storage system designed to efficiently store, search, and reference information.

[0867] "Query" refers to a natural language question or command that a user enters into a system to search for specific information.

[0868] "Search Results" refers to relevant information extracted from a database based on a query.

[0869] A "generative AI model" is an artificial intelligence model trained using large datasets, and is used specifically for natural language generation and summarization.

[0870] "Emotion analysis" refers to the technology of determining a user's emotional state at any given time from data such as facial expressions and voice.

[0871] "Emotional feedback" refers to the collection of data on users' emotional reactions to search results and information provided by the system.

[0872] System Overview

[0873] This system collects past minutes, documents, and management comments, analyzes them using natural language processing technology and a sentiment analysis engine, and registers the analysis results in a database. It recognizes user queries and their associated sentiment, customizes search results, summarizes them, and displays them to the user.

[0874] Component Configuration

[0875] The system consists of the following main components:

[0876] 1. Data Collection Components

[0877] Terminal: Provides an interface for users to upload electronic files such as minutes, documents, and management comments to the system.

[0878] Server: Receives the uploaded file and stores it in a temporary directory.

[0879] 2. Data Analysis Component

[0880] Server: Specifically, it analyzes the uploaded text data using natural language processing technologies such as SpaCy and NLTK. The analysis process involves part-of-speech analysis, keyword extraction, and context understanding.

[0881] Server: Based on the analysis results, metadata such as the meeting date and time, participants, agenda, and content of remarks is generated from the text data.

[0882] 3. Database Component

[0883] Server: The parsed text data and generated metadata are registered in a database, typically MySQL or PostgreSQL.

[0884] 4. Query Processing Component

[0885] Terminal: Provides an interface for users to enter natural language queries, including a text box and voice input capabilities.

[0886] Server: Analyzes queries using a natural language processing engine and searches for relevant information from a database.

[0887] 5. Emotion Recognition Component

[0888] Terminal: Provides an interface for recognizing the user's emotional state from their facial expressions and voice when entering a query.

[0889] Server: Uses a sentiment analysis engine to analyze user sentiment and use it as data to customize search results.

[0890] 6. Search Results Summary Component

[0891] Server: Uses a generative AI model (such as OpenAI's GPT-3) to summarize and convert search results into an understandable format.

[0892] Terminal: Displays summarized search results to the user.

[0893] 7. Emotional Feedback Component

[0894] Terminal: Records the user's emotional feedback on search results.

[0895] Server: Collects emotional feedback to help improve the system's accuracy.

[0896] Specific operation example

[0897] Data Collection Example

[0898] Terminal: The user selects project minutes, documents, and management comments and uploads them to the system.

[0899] Server: Receives the uploaded file and temporarily stores it in the / tmp / uploads / directory.

[0900] Data analysis example

[0901] Server: Uses a natural language processing engine to analyze the text of the uploaded minutes, extracting information such as "The Project ABC meeting took place on May 3, 2022, and the main topic was budget adjustment."

[0902] Server: Based on the analysis results, generate metadata such as "Date and time: May 3, 2022," "Participants: Tanaka, Sato, Suzuki," and "Agenda: Budget adjustment."

[0903] Database registration example

[0904] Server: The parsed text data and metadata are stored in a MySQL or PostgreSQL database.

[0905] Query Processing Example

[0906] User: Enters query "I want to see the minutes of the project ABC meeting held last May."

[0907] Device: When a user enters a query, the facial recognition camera analyzes the user's facial expression and sends it to the server along with the query.

[0908] Server: Parses the query and retrieves relevant information from a database.

[0909] Customizing search results, summarizing and display examples

[0910] Server: Adjusts search results based on user sentiment data and creates a summary: "The Project ABC meeting took place on May 3, 2022, and the main topic was budget adjustments."

[0911] Terminal: Displays summaries, allowing users to quickly review relevant information.

[0912] Example of collecting emotional feedback

[0913] Device: Collects the user's facial expressions and voice again in response to the displayed search results and records emotional feedback.

[0914] Server: Receives emotional feedback sent from the device and stores it as data to improve the accuracy of the system.

[0915] Prompt Sentence Examples

[0916] "Analyze the minutes data and extract the main topics and participants."

[0917] "Generate a summary of the meeting about project ABC last May"

[0918] This system allows users to quickly and appropriately search and refer to past minutes, documents, and management comments, and has the advantage of being able to obtain customized information based on sentiment.

[0919] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0920] Step 1:

[0921] Terminal: Provides a file upload interface to the user. Allows the user to select a file to upload and receives file input. For example, if the user selects the file "Meeting2022_05_03.docx", the path to this file becomes the input data.

[0922] Step 2:

[0923] User: Clicks the upload button on the terminal to send the selected file to the system. Specifically, the contents of the file are transferred from the terminal to the server as binary data.

[0924] Step 3:

[0925] Server: Receives binary data sent from the terminal and saves the file in a temporary storage directory (e.g. / tmp / uploads / ). The input is the binary data, and the output is the saved file path.

[0926] Step 4:

[0927] Server: The temporarily saved file is passed to a natural language processing engine (e.g., SpaCy or NLTK) to analyze the text. The input is the saved file path, and the output is the analyzed text data. Specifically, the file is read, the character code is converted, and the text is extracted.

[0928] Step 5:

[0929] Server: The extracted text data is passed to a natural language processing engine, where it is analyzed for parts of speech, keyword extraction, and contextual understanding. The input is text data, and the output is metadata (e.g., meeting date and time, participants, agenda, and content of remarks) as the analysis results.

[0930] Step 6:

[0931] Server: The parsed metadata is registered in a database (e.g., MySQL, PostgreSQL). The input is the metadata, and the output is a confirmation of registration in the database. Specifically, the metadata is inserted into the appropriate tables in the database.

[0932] Step 7:

[0933] Terminal: Provides a query input interface to users and accepts query input in natural language. The input is a query entered by the user (e.g., "I want to see the minutes of the project ABC held in May of last year").

[0934] Step 8:

[0935] Terminal: When a user enters a query, the device recognizes their facial expression and voice and passes them to the emotion analysis engine. The input is the user's facial image and voice data, and the output is analyzed emotion data.

[0936] Step 9:

[0937] Terminal: Sends query and emotion data to the server. The input is the query and emotion data, and the output is data transmission to the server.

[0938] Step 10:

[0939] Server: Analyzes the received query using a natural language processing engine and searches for relevant information from a database. The input is the query data, and the output is the search result data.

[0940] Step 11:

[0941] Server: Customize search results using emotional data. The input is search results and emotional data, and the output is customized search results. For example, if the user is feeling stressed, the summary will be made shorter.

[0942] Step 12:

[0943] Server: Summarizes search results using a generative AI model (e.g., OpenAI GPT-3). The input is the customized search results, and the output is the summarized text.

[0944] Step 13:

[0945] Terminal: displays summarized search results to the user. The input is the summarized text and the output is the displayed information.

[0946] Step 14:

[0947] Terminal: After displaying the image, the user's facial expressions and voice are collected again and recorded as emotional feedback. The input is the user's facial expressions and voice, and the output is emotional feedback data.

[0948] Step 15:

[0949] Server: Receives emotional feedback sent from the device and accumulates the data to be used to improve the accuracy of the system. The input is emotional feedback data, and the output is updated system learning data.

[0950] (Application example 2)

[0951] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0952] In recent years, with the increasing automation of customer service, brick-and-mortar stores are also in need of systems that can respond to customer questions effectively and quickly. However, conventional systems have limitations in information search and summarization functions using natural language processing, making it particularly difficult to respond to customer questions while taking emotions into account. Furthermore, they lack a summary function for visually displaying search results concisely, and a customization function based on the user's emotions, which prevents sufficient improvement of customer satisfaction. To solve these issues, a system with emotion recognition and customization functions is needed.

[0953] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0954] In this invention, the server includes means for collecting past minutes, documents, and comments from upper management, means for analyzing the collected content using natural language processing technology, means for registering the analysis results in a database, means for receiving and analyzing user queries and searching for appropriate information, means for analyzing customer voice data and recognizing emotions, means for summarizing search results and displaying them to the user, means for summarizing related information using a generative AI model, and means for collecting user emotional feedback and improving the accuracy of the system. This enables appropriate responses to customer questions that take emotions into consideration.

[0955] Minutes are documents that record the contents of meetings and discussions.

[0956] "Materials" are documents, data files, etc. that contain information about a particular topic or project.

[0957] "Comments from senior management" is a document that compiles opinions and instructions from senior management, such as managers and executives, of a company or organization.

[0958] "Natural language processing" is a technology that allows computers to understand, analyze, and process human language.

[0959] A "database" is a collection of data that makes it easy to search, organize, and manage.

[0960] A "query" is a request or question to a database to retrieve information.

[0961] "Emotion recognition" is a technology that identifies a person's emotional state from voice, text, facial expressions, etc.

[0962] A "generative AI model" is an artificial intelligence model that generates new text based on input data.

[0963] "Emotional feedback" is the process of collecting and analyzing a user's emotional state and reactions.

[0964] This invention is a system that collects past minutes, documents, and comments from upper management, analyzes them using natural language processing technology and an emotion engine, and registers the analysis results in a database. Here, a specific embodiment of the invention will be described.

[0965] System configuration

[0966] The system consists of the following main components:

[0967] Data Collection Component

[0968] Terminal: Provides an interface for users to upload electronic files such as minutes, documents, and comments from senior management into the system.

[0969] Server: Receives uploaded files and prepares them for analysis.

[0970] Data Analysis Components

[0971] Server: Analyzes the uploaded text data using natural language processing technology. The analysis process involves parts of speech analysis, keyword extraction, and context understanding.

[0972] Server: Based on the analysis results, metadata such as the meeting date and time, participants, agenda, and content of remarks is extracted from the text data. This information is formatted as structured data.

[0973] Database Components

[0974] Server: The analyzed text data and generated metadata are registered in a database, allowing for efficient search and reference later.

[0975] Query Processing Component

[0976] Terminal: Provides an interface for users to enter natural language queries.

[0977] Server: Analyzes the received query and searches for relevant information from a database.

[0978] Emotion Recognition Component

[0979] Terminal: Provides an interface for analyzing the customer's voice and text and recognizing their emotional state.

[0980] Server: Uses an emotion engine to analyze user emotions and use them as data to customize search results.

[0981] Search Results Summary Component

[0982] Server: Uses a generative AI model to summarize search results and convert them into an easy-to-understand format.

[0983] Terminal: Displays summarized search results to the user.

[0984] Emotional Feedback Component

[0985] Server: Collects user emotional feedback, analyzes and stores the data, and uses it to improve the accuracy of the system.

[0986] Specific examples

[0987] For example, if a store staff member enters a query into the system such as "Please tell me your recent feedback on customer service at the store," the system operates as follows: First, the server receives the query and analyzes it using natural language processing technology. Next, based on the analysis results, it searches for relevant feedback from the database and summarizes it using a generative AI model. In addition, it optimizes the search results based on the results of sentiment analysis of the voice data acquired by the device. Finally, the summary result and additional information that takes sentiment into consideration are displayed on the device.

[0988] Prompt Sentence Examples

[0989] "What feedback have you received recently about customer service at your store?"

[0990] In this way, store staff can quickly and easily view relevant information and improve the quality of customer service.

[0991] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0992] Step 1:

[0993] Terminal: The user uploads electronic files such as minutes, documents, and comments from upper management to the system. This input data is sent to the server. The specific operation for uploading is to press the file selection button, select the specified file, and click the upload button.

[0994] Input: minutes, materials, comments from upper management

[0995] Output: Collected data

[0996] Step 2:

[0997] Server: Receives and temporarily stores uploaded files, which contain detailed information such as meeting date and time, participants, agenda, and comments.

[0998] Input: Collected data

[0999] Output: Saved data

[1000] Step 3:

[1001] Server: Analyzes the uploaded text data using a natural language processing engine, specifically by analyzing parts of speech, extracting keywords, and understanding context to extract important metadata.

[1002] Input: Saved data

[1003] Output: Analysis data

[1004] Step 4:

[1005] Server: Based on the analysis results, metadata such as the meeting date and time, participants, agenda, and content of remarks is extracted from the text data and formatted as structured data.

[1006] Input: Analysis data

[1007] Output: Metadata

[1008] Step 5:

[1009] Server: The generated metadata is registered in a database, allowing for efficient search and reference later.

[1010] Input: Metadata

[1011] Output: Database update

[1012] Step 6:

[1013] Terminal: The user inputs a query such as a question or request in natural language. For example, "Please tell me the minutes of the latest meeting." The input query is sent to the server.

[1014] Input: User query

[1015] Output: Query data

[1016] Step 7:

[1017] Server: Analyzes the received query and searches the database for relevant information, using natural language processing technology to accurately understand the meaning of the query.

[1018] Input: Query data

[1019] Output: Search results

[1020] Step 8:

[1021] Terminal: Collects customer voice data and analyzes it using emotion recognition technology. Specifically, the device captures the voice file, converts it into text using voice recognition software, and analyzes emotions using an emotion engine.

[1022] Input: Audio data

[1023] Output: Emotion data

[1024] Step 9:

[1025] Server: Customize search results based on emotional data. For example, if a customer is stressed, tailor search results to display them in a concise and easy-to-understand format.

[1026] Input: Emotion data, search results

[1027] Output: Customized search results

[1028] Step 10:

[1029] Server: Uses a generative AI model to summarize search results and present them to the user in a more concise format. Generates summaries using prompts.

[1030] Input: Customized search results

[1031] Output: Summary data

[1032] Step 11:

[1033] Terminal: Display summarized search results to users, allowing them to quickly check relevant information through their terminal.

[1034] Input: Summary data

[1035] Output: Display data

[1036] Step 12:

[1037] Terminal: Record the user's emotional feedback on the summary results. For example, provide a feedback button or a simple survey form to collect the user's satisfaction and additional comments.

[1038] Input: User feedback

[1039] Output: Feedback data

[1040] Step 13:

[1041] Server: Collects emotional feedback and stores it as data to improve the accuracy of the system. Specifically, it analyzes the collected feedback and identifies areas for improvement in the system.

[1042] Input: Feedback data

[1043] Output: Improvement data

[1044] This process makes it possible to provide appropriate information in response to user questions while taking into consideration their emotions, thereby improving the quality of customer service.

[1045] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1046] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1047] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1048] [Third embodiment]

[1049] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1050] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1051] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1052] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1053] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1054] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1055] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1056] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1057] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1058] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1059] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1060] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1061] The present invention is a system that collects past minutes, documents, and comments from senior management, analyzes them using natural language processing technology, and registers the analysis results in a database. The system searches for the analysis results based on queries from users, summarizes the search results, and displays them to the user. A specific embodiment of this system is described below.

[1062] System configuration

[1063] The system consists of the following main components:

[1064] 1. Data Collection Components

[1065] Terminal: Provides an interface for users to upload electronic files such as minutes, documents, and comments from senior management into the system.

[1066] Server: Receives uploaded files and prepares them for analysis.

[1067] 2. Data Analysis Component

[1068] Server: Analyzes the uploaded text data using natural language processing technology, including part-of-speech analysis, keyword extraction, and semantic understanding.

[1069] Server: Structures the analysis results and generates metadata such as meeting date and time, participants, agenda, and content of comments.

[1070] 3. Database Component

[1071] Server: The parsed data and generated metadata are registered in a database.

[1072] 4. Query Processing Component

[1073] Terminal: Provides an interface for users to enter natural language queries.

[1074] Server: Analyzes the received query and searches for relevant information from a database.

[1075] 5. Search Results Summary Component

[1076] Server: Uses a generative AI model to summarize search results and convert them into an easy-to-understand format.

[1077] Terminal: Displays summarized search results to the user.

[1078] System operation example

[1079] 1. Data Collection Example

[1080] Terminal: The user selects project minutes, documents, and comments from upper management and uploads them to the system.

[1081] Server: Receives uploaded files and temporarily stores their contents.

[1082] 2. Data Analysis Example

[1083] Server: Uses a natural language processing engine to analyze the text of the uploaded minutes, extracting information such as "The Project ABC meeting took place on May 3, 2022, and the main topic was budget adjustment."

[1084] Server: Based on the analysis results, generate metadata such as the meeting date and time (May 3, 2022), participants (speaker list), and main agenda item (budget adjustment).

[1085] 3. Example of database registration

[1086] Server: The parsed text data and metadata are stored in a database, allowing for efficient search and reference later.

[1087] 4. Query Processing Example

[1088] User: For example, enter a query such as "I want to see the minutes of the project ABC held in May of last year" into the terminal.

[1089] Terminal: Sends queries to the server.

[1090] Server: Parses the query using a natural language processing engine and searches the database for relevant minutes.

[1091] 5. Example of summary and display of search results

[1092] Server: Uses a generative AI model to summarize the search results, for example, "The Project ABC meeting took place on May 3, 2022, and the main topic was budget adjustment."

[1093] Terminal: Displays summarized search results to the user.

[1094] In this way, users can quickly and easily search and refer to past meeting minutes, documents, and comments from upper management, which improves work efficiency and speeds up decision-making.

[1095] The processing flow will be explained below.

[1096] Step 1:

[1097] Terminal: The user prepares the minutes, materials, and comments from senior management as electronic files. The user uses the system interface to select the files to upload.

[1098] Step 2:

[1099] Terminal: The user sends the selected file to the server. The file is sent to the server through the system's upload function.

[1100] Step 3:

[1101] Server: The received file is stored in a temporary location before analysis, and preparations for analysis are made.

[1102] Step 4:

[1103] Server: The natural language processing engine is launched to analyze the text data in the uploaded file. During the analysis process, parts of speech analysis, keyword extraction, context understanding, etc. are performed.

[1104] Step 5:

[1105] Server: Based on the analysis results, metadata such as the meeting date and time, participants, agenda, and content of remarks is extracted from the text data. This information is formatted as structured data.

[1106] Step 6:

[1107] Server: The analyzed text data and generated metadata are registered in a database, allowing for efficient search and reference later.

[1108] Step 7:

[1109] User: Enters a query through the system interface to search for specific information, such as "I want to see the minutes of the project ABC meeting held last May."

[1110] Step 8:

[1111] Terminal: Sends queries entered by the user to the server.

[1112] Step 9:

[1113] Server: Analyzes the received query using natural language processing to understand the meaning of the sentence, allowing it to efficiently search for relevant information in a database.

[1114] Step 10:

[1115] Server: Searches the database for meeting minutes, documents, comments from senior management, etc. that correspond to the query.

[1116] Step 11:

[1117] Server: Using a generative AI model, the search results are summarized and converted into an easy-to-understand format.

[1118] Step 12:

[1119] Server: Sends summarized search results to the terminal.

[1120] Step 13:

[1121] Terminal: Displays the search results received from the server to the user, allowing the user to quickly check related information.

[1122] Step 14:

[1123] Users: Optionally, click on the resulting links to view detailed minutes, materials, and comments from senior management.

[1124] Example 1

[1125] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1126] Until now, it has been difficult to efficiently collect and analyze past meeting records, documents, and comments from senior management, and to quickly obtain the information users need. In particular, the process of searching for appropriate information from large amounts of data takes time and effort, and it is not easy to summarize the search results in an easy-to-understand format. Therefore, there is a need for a system that can easily search past meeting records and documents and provide easy-to-understand summaries.

[1127] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1128] In this invention, the server includes means for collecting past meeting records, materials, and comments from senior managers, means for analyzing the collected content using natural language processing technology, means for registering the analysis results in a database, means for receiving and analyzing user queries and searching for appropriate information, and means for summarizing the search results and displaying them to the user, thereby enabling the user to quickly and efficiently search past meeting records and materials and obtain easy-to-understand summaries.

[1129] "Past meeting records" refers to minutes and notes taken during meetings and discussions.

[1130] "Materials" refers to documents such as reports, presentations, and explanations used in meetings and discussions.

[1131] "Comments from senior management" refers to opinions or instructions left by senior management of a company or organization in meetings or in writing.

[1132] "Natural language processing technology" refers to technology that allows computers to analyze and understand human language.

[1133] "Analysis results" refers to information and data extracted from text data using natural language processing technology.

[1134] "Means for registering in a database" refers to a method or system for storing analysis results in a database.

[1135] "User query" refers to a search request or question entered by a user into the system.

[1136] "Means for retrieving relevant information" refers to techniques and methods for retrieving relevant information from a database based on a user's query.

[1137] "Means for summarizing search results and displaying them to the user" refers to techniques and methods for summarizing the searched information and providing it to the user in an easy-to-understand format.

[1138] A "generative artificial intelligence model" refers to a technology that uses artificial intelligence learned from large amounts of data to analyze and generate data.

[1139] "Metadata format" refers to a structured data format for describing the content and attributes of data.

[1140] This invention is a system that collects and analyzes past meeting records, documents, and comments from senior managers, registers the analysis results in a database, searches based on user queries, and displays summaries. This system allows users to quickly and efficiently obtain the information they need.

[1141] System configuration

[1142] The system mainly consists of the following main components:

[1143] 1. Data Collection Components

[1144] Terminal: Serves as an interface for users to upload electronic files such as meeting records, materials, senior management comments, etc. Specifically, the user selects a file using a file selection dialog and clicks the "Upload" button.

[1145] Server: Receives and temporarily stores files sent from the device. For example, it uses a cloud storage service such as Amazon S3.

[1146] 2. Data Analysis Component

[1147] Server: Analyzes the uploaded text data using natural language processing techniques. Specifically, it uses natural language processing libraries such as NLTK and spaCy to perform part-of-speech analysis, keyword extraction, and semantic understanding. For example, it analyzes the sentence "The Project ABC meeting was held on May 3, 2022, and the main topic was budget adjustment."

[1148] Server: Based on the analysis results, generate metadata (meeting date and time, participants, agenda, content of remarks, etc.).

[1149] 3. Database Component

[1150] Server: The parsed text data and generated metadata are registered in a database. Specifically, a database such as MySQL or PostgreSQL is used to execute SQL queries and store the data.

[1151] 4. Query Processing Component

[1152] Terminal: Provides an interface for users to enter queries in natural language. Users enter queries in natural language into a search box and click a "Search" button.

[1153] Server: Analyzes the received query and searches for relevant information from the database. Specifically, it analyzes the query using natural language processing models such as BERT and GPT, and generates and executes the appropriate SQL query.

[1154] 5. Search Results Summary Component

[1155] Server: Using a generative AI model (e.g., GPT-3), the search results are summarized. The summary is then converted into a format that is easy for users to understand. For example, a summary is generated: "The Project ABC meeting took place on May 3, 2022, and the main agenda item was budget adjustment."

[1156] Terminal: Displaying summarized search results to the user. The summarized results are displayed on the terminal interface, allowing the user to easily browse the information.

[1157] Specific examples and prompts

[1158] Data Collection Example

[1159] Terminal: The user uploads "Project ABC Meeting Minutes for May 3, 2022" to the system.

[1160] Server: Receives and temporarily stores files.

[1161] Data analysis example

[1162] Server: The uploaded minutes are analyzed using a natural language processing engine. For example, the sentence "The Project ABC meeting was held on May 3, 2022, and the main topic was budget adjustment" is analyzed.

[1163] Server: From the analysis results, generate metadata such as "Meeting date and time: May 3, 2022," "Participants: List," and "Agenda: Budget adjustment."

[1164] Example queries and search results

[1165] User: In the search box, type "Show me the meeting minutes for project ABC from last May."

[1166] Terminal: Sends input to the server.

[1167] Server: Uses GPT-3 to parse the query and search and summarize "Project ABC meeting on May 3, 2022" from the database.

[1168] Device: Display the summarized search result "Budget adjustment is the main topic."

[1169] Example prompt sentence:

[1170] "What are the main topics for the Project ABC meeting on May 3, 2022?"

[1171] With the above-described procedure and configuration, the present invention provides a system that enables a user to efficiently and quickly obtain the information he or she requires.

[1172] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1173] Program processing steps

[1174] Step 1: Collect data

[1175] Terminal - The user uses the terminal interface to select electronic files such as meeting notes, materials, and senior management comments, and clicks the upload button.

[1176] Input: Electronic file selected by the user

[1177] Output: File upload request from the device to the server

[1178] What happens: The user opens the file selection dialog, selects the appropriate file, and clicks "Upload."

[1179] Step 2: Receiving and temporarily saving the file

[1180] Server - The server receives the file sent from the device and temporarily stores it.

[1181] Input: Electronic file sent from the terminal

[1182] Output: File saved in temporary directory

[1183] Specific operation: The server saves the received file in the specified directory (e.g., / uploads) and records the file metadata (file name, upload time, etc.).

[1184] Step 3: Data analysis using natural language processing

[1185] Server - Analyzes the stored text data using natural language processing techniques.

[1186] Input: A file saved in the temporary directory

[1187] Output: Analysis results (part of speech analysis results, keyword list, semantic understanding results)

[1188] Specific operation: The server uses libraries such as NLTK and spaCy to perform part-of-speech analysis, keyword extraction, and semantic understanding of text data. For example, it parses the sentence "The Project ABC meeting was held on May 3, 2022, and the main topic was budget adjustment."

[1189] Step 4: Generate Metadata

[1190] Server - Generates metadata (meeting date and time, participants, agenda, content of comments, etc.) based on the analysis results.

[1191] Input: Analysis results from natural language processing

[1192] Output: Metadata (meeting date and time, participants, agenda, and comments)

[1193] Specific operation: Extract necessary information from the analysis results and generate metadata such as "Meeting date and time: May 3, 2022," "Participants: List," and "Agenda: Budget adjustment."

[1194] Step 5: Register with the database

[1195] Server - The parsed text data and generated metadata are registered in a database.

[1196] Input: Parsed text data and metadata

[1197] Output: Analysis results and metadata stored in a database

[1198] What happens next: The server runs SQL queries against a database like MySQL or PostgreSQL and inserts the data into the appropriate tables.

[1199] Step 6: Enter and submit your query

[1200] User - The user types a query in natural language and submits it through the terminal interface.

[1201] Input: A query entered by a user (e.g., "I want to see the minutes of the meeting on project ABC from last May")

[1202] Output: Query request from the terminal to the server

[1203] Specific behavior: The user enters a query in the search box and clicks the "Search" button.

[1204] Step 7: Parsing the query and searching

[1205] Server - Parses the incoming query and retrieves the relevant information from the database.

[1206] Input: The query submitted by the user

[1207] Output: Search results (related meeting records and materials)

[1208] What it does: The server uses natural language processing models such as BERT or GPT to analyze the query and generate and execute the appropriate SQL query.

[1209] Step 8: Summarizing search results

[1210] Server - Summarizes search results using a generative AI model.

[1211] Input: Search results from the database

[1212] Output: Summarized search results

[1213] What it does: The server uses a generative AI model (e.g., GPT-3) to summarize the search results and convert them into an understandable format, such as generating a summary like "The Project ABC meeting took place on May 3, 2022, and the main topic was budget adjustment."

[1214] Step 9: View summary results

[1215] Terminal - Display summarized search results to the user.

[1216] Input: Abstracted search results from the server

[1217] Output: Summary results displayed to the user on their screen

[1218] Specific operation: The summary results are displayed on the terminal interface in a format that can be easily viewed by the user.

[1219] Through the above steps, users can efficiently search past meeting records and materials and obtain easy-to-understand summaries.

[1220] (Application example 1)

[1221] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1222] It is difficult to efficiently analyze large amounts of data, such as past meeting minutes, documents, and comments from senior management, and provide an environment where users can easily access important information. Furthermore, there is a need to instantly summarize and present relevant information in response to user queries, which requires the use of advanced natural language processing technology and generative AI models. However, current systems often do not adequately meet these requirements.

[1223] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1224] In this invention, the server includes means for collecting past minutes, documents, and comments from upper management, means for analyzing the collected content using natural language processing technology, means for registering the analysis results in a database, means for receiving and analyzing user queries and searching for appropriate information, means for summarizing the search results and displaying them to the user, and means for generating summaries of relevant minutes and documents using a generative AI model when a user inputs a specific query. This makes it possible to efficiently extract necessary information from large amounts of past data, quickly and appropriately summarize it, and provide it to the user.

[1225] "Past minutes" are documents that record the contents of past meetings and discussions.

[1226] "Materials" refers to documents and digital data used to communicate, store, and analyze information.

[1227] "Comments from upper management" refers to opinions and instructions issued by managers or executives of a company or organization.

[1228] "Natural language processing technology" refers to technology that allows computers to understand, analyze, and generate human language.

[1229] A "generative AI model" is an algorithm that uses artificial intelligence to generate new content from data such as text and images.

[1230] "Metadata format" is additional information for describing the attributes of data, and clearly indicates the type and structure of the data.

[1231] A "query" is a question or request to a database or system to retrieve information.

[1232] A "server" is a computer that provides data over a network, and is responsible for storing, analyzing, and providing data.

[1233] A specific embodiment of the present invention will be described. The present invention relates to a system that collects past minutes, documents, and comments from senior management, analyzes them using natural language processing technology, and registers the results in a database. It has the function of searching for analysis results based on user queries and summarizing and displaying the results using a generative AI model.

[1234] System configuration

[1235] 1. Data Collection Components

[1236] Terminal: Provides an interface for users to upload electronic files such as minutes, documents, and comments from senior management into the system.

[1237] Server: Receives uploaded files and prepares them for analysis.

[1238] 2. Data Analysis Component

[1239] Server: Analyzes the uploaded text data using natural language processing technology. The analysis uses Spacey's natural language processing engine and includes part-of-speech analysis, keyword extraction, and semantic understanding.

[1240] Server: Structures the analysis results and generates metadata such as meeting date and time, participants, agenda, and content of comments.

[1241] 3. Database Component

[1242] Server: The analyzed data and generated metadata are registered in a database built on cloud storage.

[1243] 4. Query Processing Component

[1244] Terminal: Provides an interface for users to enter natural language queries.

[1245] Server: Analyzes the received query and searches for relevant information from a database.

[1246] 5. Search Results Summary Component

[1247] Server: Uses a generative AI model to summarize search results. Leverages Hugging Face's transformers library.

[1248] Terminal: Displays summarized search results to the user.

[1249] Explanation of system behavior

[1250] 1. Data Collection

[1251] Users use their devices to select project minutes, documents, and comments from senior management and upload them to the system. The server receives the uploaded files and temporarily stores their contents.

[1252] 2. Data Analysis

[1253] The server uses Spacey's natural language processing engine to analyze the text of the uploaded minutes, extracting information such as, "The Project ABC meeting took place on May 3, 2022, and the main topic was budget adjustment." It then generates metadata based on the analysis results, including the meeting date and time, participants, and main topic.

[1254] 3. Registering in the database

[1255] The server stores the analyzed text data and metadata in a database, allowing for efficient search and reference later.

[1256] 4. Query Processing

[1257] A user inputs a query into a terminal, such as "I want to know the details of the meeting about strengthening security." The terminal sends the query to the server, which then analyzes the query using a natural language processing engine and searches for relevant minutes from the database.

[1258] 5. Summarizing and displaying search results

[1259] The server uses the generative AI model to summarize the search results, for example, "The Project ABC meeting took place on May 3, 2022, and the main topic was budget adjustment." The device then displays the summarized search results to the user.

[1260] Examples of concrete examples and prompts

[1261] The user inputs a query on their device, such as "Please summarize the contents of the meeting about strengthening security." The server analyzes this query, extracts relevant minutes and materials from the database, and generates a summary using a generative AI model.

[1262] Example prompt sentence:

[1263] Please summarize the meeting on strengthening security.

[1264] In this way, the system enables users to efficiently extract necessary information from large amounts of past data, quickly and appropriately summarize it, and provide it to the users.

[1265] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1266] Step 1:

[1267] Uploading data

[1268] The user uses a terminal to select project minutes, documents, and comments from upper management and upload them to the system. The input is the electronic file selected by the user, which is sent to the server. The server receives the uploaded file and temporarily stores it. This stored data is then analyzed in the next step.

[1269] Step 2:

[1270] Data analysis

[1271] The server uses Spacy, a natural language processing engine, to analyze the uploaded text data. The input includes text data from temporarily saved electronic files. The server performs part-of-speech analysis, keyword extraction, and semantic understanding to extract information such as the meeting date and time, participants, agenda, and content of remarks. The analysis results and metadata are generated as output.

[1272] Step 3:

[1273] Registering in the database

[1274] The server registers the analyzed text data and generated metadata in a database on cloud storage. The analysis results and metadata generated in step 2 are used as input. Once registration in the database is complete, subsequent search processing and reference can be performed efficiently. The output is an update of the database.

[1275] Step 4:

[1276] Entering and submitting a query

[1277] A user uses a terminal to input a query in natural language. For example, a query such as "I want to know the details of the conference on strengthening security" is input. The input is the user's query text, which is sent to the server via the terminal. The output is the query text passed to the server.

[1278] Step 5:

[1279] Query Parsing and Search

[1280] The server uses a natural language processing engine to analyze the received query. The input includes the user's query text. Based on this analysis, it searches the database for relevant information. The output is the text data of the relevant minutes and documents.

[1281] Step 6:

[1282] Generate summary of search results

[1283] The server uses a generative AI model (Hugging Face transformers) to summarize the search results. The input is the text data of the minutes and documents extracted in step 5. The generative AI model summarizes the results and generates a summary text in an easy-to-understand format as output.

[1284] Step 7:

[1285] Viewing summary results

[1286] The terminal displays the summary text received from the server to the user. The input includes the summary text sent from the server. The user can check this summary and obtain the necessary information. The output is displayed visually to the user.

[1287] These steps enable users to efficiently extract necessary information from large amounts of past data and obtain information that is quickly and appropriately summarized.

[1288] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1289] The present invention is a system that collects past meeting minutes, documents, and comments from senior management, analyzes them using natural language processing technology and an emotion engine, and registers the analysis results in a database. It recognizes user queries and their associated emotions, customizes search results, summarizes them, and displays them to the user. A specific embodiment of this system is described below.

[1290] System configuration

[1291] The system consists of the following main components:

[1292] 1. Data Collection Components

[1293] Terminal: Provides an interface for users to upload electronic files such as minutes, documents, and comments from senior management into the system.

[1294] Server: Receives uploaded files and prepares them for analysis.

[1295] 2. Data Analysis Component

[1296] Server: Analyzes the uploaded text data using natural language processing technology. The analysis process involves parts of speech analysis, keyword extraction, and context understanding.

[1297] Server: Based on the analysis results, metadata such as the meeting date and time, participants, agenda, and content of remarks is extracted from the text data. This information is formatted as structured data.

[1298] 3. Database Component

[1299] Server: The analyzed text data and generated metadata are registered in a database, allowing for efficient search and reference later.

[1300] 4. Query Processing Component

[1301] Terminal: Provides an interface for users to enter natural language queries.

[1302] Server: Analyzes the received query and searches for relevant information from a database.

[1303] 5. Emotion Recognition Component

[1304] Terminal: Provides an interface for recognizing the user's emotional state from their facial expressions and voice when entering a query.

[1305] Server: Uses an emotion engine to analyze user emotions and use them as data to customize search results.

[1306] 6. Search Results Summary Component

[1307] Server: Uses a generative AI model to summarize search results and convert them into an easy-to-understand format.

[1308] Terminal: Displays summarized search results to the user.

[1309] 7. Emotional Feedback Component

[1310] Server: Collects user emotional feedback, analyzes and stores the data, and uses it to improve the accuracy of the system.

[1311] System operation example

[1312] 1. Data Collection Example

[1313] Terminal: The user selects project minutes, documents, and comments from upper management and uploads them to the system.

[1314] Server: Receives uploaded files and temporarily stores their contents.

[1315] 2. Data Analysis Example

[1316] Server: Uses a natural language processing engine to analyze the text of the uploaded minutes, extracting information such as "The Project ABC meeting took place on May 3, 2022, and the main topic was budget adjustment."

[1317] Server: Based on the analysis results, generate metadata such as the meeting date and time (May 3, 2022), participants (speaker list), and main agenda item (budget adjustment).

[1318] 3. Example of database registration

[1319] Server: The parsed text data and metadata are stored in a database, allowing for efficient search and reference later.

[1320] 4. Query Processing Example

[1321] User: For example, enter a query such as "I want to see the minutes of the project ABC held in May of last year" into the terminal.

[1322] Device: The emotion engine analyzes the user's facial expressions and voice when entering a query.

[1323] Terminal: Sends queries and emotion data to the server.

[1324] Server: Parses the query using a natural language processing engine and searches the database for relevant minutes.

[1325] 5. Example of customizing, summarizing and displaying search results

[1326] Server: Customize search results based on emotional data, for example, by providing a more concise summary if the user is feeling stressed.

[1327] Server: Uses a generative AI model to summarize the search results, for example, "The Project ABC meeting took place on May 3, 2022, and the main topic was budget adjustment."

[1328] Terminal: Displays summarized search results to the user, allowing the user to quickly view relevant information.

[1329] 6. Collecting Emotional Feedback

[1330] Terminal: Records the user's emotional feedback on search results.

[1331] Server: Collects emotional feedback to help improve the system's accuracy.

[1332] In this way, users can quickly and easily search and refer to past minutes, documents, and comments from upper management, improving work efficiency and speeding up decision-making.In addition, by using an emotion engine, it is possible to provide even more user-friendly and customized services.

[1333] The processing flow will be explained below.

[1334] Step 1:

[1335] Terminal: The user prepares the minutes, materials, and comments from senior management as electronic files. The user uses the system interface to select the files to upload.

[1336] Step 2:

[1337] Terminal: The user sends the selected file to the server. The file is sent to the server through the system's upload function.

[1338] Step 3:

[1339] Server: The received file is stored in a temporary location before analysis, and preparations for analysis are made.

[1340] Step 4:

[1341] Server: The natural language processing engine is launched to analyze the text data in the uploaded file. During the analysis process, parts of speech analysis, keyword extraction, context understanding, etc. are performed.

[1342] Step 5:

[1343] Server: Based on the analysis results, metadata such as the meeting date and time, participants, agenda, and content of remarks is extracted from the text data. This information is formatted as structured data.

[1344] Step 6:

[1345] Server: The analyzed text data and metadata are registered in a database, allowing for efficient search and reference later.

[1346] Step 7:

[1347] User: Enters a query through the system interface to search for specific information, such as "I want to see the minutes of the project ABC meeting held last May."

[1348] Step 8:

[1349] On the device: When a user enters a query, the emotion engine analyzes the user's facial expressions and voice. The emotion engine uses facial recognition and voice analysis technologies to assess the user's emotional state.

[1350] Step 9:

[1351] Terminal: Sends queries and emotion data to the server.

[1352] Step 10:

[1353] Server: Analyzes the received query using natural language processing to understand the meaning of the sentence, allowing it to efficiently search for relevant information in a database.

[1354] Step 11:

[1355] Server: Searches the database for meeting minutes, documents, comments from senior management, etc. that correspond to the query.

[1356] Step 12:

[1357] Server: Customize search results based on emotional data, for example, making summaries more concise if the user is stressed.

[1358] Step 13:

[1359] Server: Using a generative AI model, the search results are summarized. For example, the summary might be something like, "The Project ABC meeting took place on May 3, 2022, and the main topic was budget adjustment."

[1360] Step 14:

[1361] Server: Sends summarized search results to the terminal.

[1362] Step 15:

[1363] Terminal: Displays the search results received from the server to the user, allowing the user to quickly check related information.

[1364] Step 16:

[1365] Users: Optionally, click on the resulting links to view detailed minutes, materials, and comments from senior management.

[1366] Step 17:

[1367] Terminal: Records the user's emotional feedback on the search results. For example, it provides an interface that asks for a simple rating to see if the user is satisfied with the results.

[1368] Step 18:

[1369] Server: Collects emotional feedback to help improve the system's accuracy. The collected data is used to optimize the user experience and improve the emotion engine.

[1370] Example 2

[1371] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1372] Conventional systems that collect and analyze minutes, documents, and comments from senior management face the challenge of efficiently searching and referencing vast amounts of information and providing customized information that reflects the user's emotions. In particular, if users are unable to quickly obtain the information they desire, this can lead to reduced operational efficiency and delays in decision-making. Furthermore, there are few systems that take user emotions into account, making it difficult to improve the user experience.

[1373] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for collecting past minutes, documents, and comments from management, means for analyzing the collected content using natural language processing technology, means for registering the analysis results in a database, means for receiving and analyzing user queries and searching for appropriate information, means for summarizing search results and displaying them to the user, means for analyzing the user's emotions at the time of query input, means for customizing search results based on the user's emotions, and means for collecting emotional feedback and using it to improve the accuracy of the system. This allows users to quickly and appropriately search and refer to past minutes, documents, and comments from upper management, and to obtain information customized according to their emotions.

[1374] "Past minutes" refers to documents and data that record what was discussed at meetings and discussions.

[1375] "Document" means any digital or paper material that records information in textual, graphical or other form.

[1376] "Management comments" refers to documents or data that record opinions or instructions issued by senior managers or executives of a company or organization.

[1377] "Natural language processing technology" refers to technology that enables computers to understand and process human language, and includes parts of speech analysis, keyword extraction, and context understanding.

[1378] "Analysis results" refers to information obtained after analyzing text data using natural language processing technology, and includes metadata and important content.

[1379] "Database" refers to a digital storage system designed to efficiently store, search, and reference information.

[1380] "Query" refers to a natural language question or command that a user enters into a system to search for specific information.

[1381] "Search Results" refers to relevant information extracted from a database based on a query.

[1382] A "generative AI model" is an artificial intelligence model trained using large datasets, and is used specifically for natural language generation and summarization.

[1383] "Emotion analysis" refers to the technology of determining a user's emotional state at any given time from data such as facial expressions and voice.

[1384] "Emotional feedback" refers to the collection of data on users' emotional reactions to search results and information provided by the system.

[1385] System Overview

[1386] This system collects past minutes, documents, and management comments, analyzes them using natural language processing technology and a sentiment analysis engine, and registers the analysis results in a database. It recognizes user queries and their associated sentiment, customizes search results, summarizes them, and displays them to the user.

[1387] Component Configuration

[1388] The system consists of the following main components:

[1389] 1. Data Collection Components

[1390] Terminal: Provides an interface for users to upload electronic files such as minutes, documents, and management comments to the system.

[1391] Server: Receives the uploaded file and stores it in a temporary directory.

[1392] 2. Data Analysis Component

[1393] Server: Specifically, it analyzes the uploaded text data using natural language processing technologies such as SpaCy and NLTK. The analysis process involves part-of-speech analysis, keyword extraction, and context understanding.

[1394] Server: Based on the analysis results, metadata such as the meeting date and time, participants, agenda, and content of remarks is generated from the text data.

[1395] 3. Database Component

[1396] Server: The parsed text data and generated metadata are registered in a database, typically MySQL or PostgreSQL.

[1397] 4. Query Processing Component

[1398] Terminal: Provides an interface for users to enter natural language queries, including a text box and voice input capabilities.

[1399] Server: Analyzes queries using a natural language processing engine and searches for relevant information from a database.

[1400] 5. Emotion Recognition Component

[1401] Terminal: Provides an interface for recognizing the user's emotional state from their facial expressions and voice when entering a query.

[1402] Server: Uses a sentiment analysis engine to analyze user sentiment and use it as data to customize search results.

[1403] 6. Search Results Summary Component

[1404] Server: Uses a generative AI model (such as OpenAI's GPT-3) to summarize and convert search results into an understandable format.

[1405] Terminal: Displays summarized search results to the user.

[1406] 7. Emotional Feedback Component

[1407] Terminal: Records the user's emotional feedback on search results.

[1408] Server: Collects emotional feedback to help improve the system's accuracy.

[1409] Specific operation example

[1410] Data Collection Example

[1411] Terminal: The user selects project minutes, documents, and management comments and uploads them to the system.

[1412] Server: Receives the uploaded file and temporarily stores it in the / tmp / uploads / directory.

[1413] Data analysis example

[1414] Server: Uses a natural language processing engine to analyze the text of the uploaded minutes, extracting information such as "The Project ABC meeting took place on May 3, 2022, and the main topic was budget adjustment."

[1415] Server: Based on the analysis results, generate metadata such as "Date and time: May 3, 2022," "Participants: Tanaka, Sato, Suzuki," and "Agenda: Budget adjustment."

[1416] Database registration example

[1417] Server: The parsed text data and metadata are stored in a MySQL or PostgreSQL database.

[1418] Query Processing Example

[1419] User: Enters query "I want to see the minutes of the project ABC meeting held last May."

[1420] Device: When a user enters a query, the facial recognition camera analyzes the user's facial expression and sends it to the server along with the query.

[1421] Server: Parses the query and retrieves relevant information from a database.

[1422] Customizing search results, summarizing and display examples

[1423] Server: Adjusts search results based on user sentiment data and creates a summary: "The Project ABC meeting took place on May 3, 2022, and the main topic was budget adjustments."

[1424] Terminal: Displays summaries, allowing users to quickly review relevant information.

[1425] Example of collecting emotional feedback

[1426] Device: Collects the user's facial expressions and voice again in response to the displayed search results and records emotional feedback.

[1427] Server: Receives emotional feedback sent from the device and stores it as data to improve the accuracy of the system.

[1428] Prompt Sentence Examples

[1429] "Analyze the minutes data and extract the main topics and participants."

[1430] "Generate a summary of the meeting about project ABC last May"

[1431] This system allows users to quickly and appropriately search and refer to past minutes, documents, and management comments, and has the advantage of being able to obtain customized information based on sentiment.

[1432] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1433] Step 1:

[1434] Terminal: Provides a file upload interface to the user. Allows the user to select a file to upload and receives file input. For example, if the user selects the file "Meeting2022_05_03.docx", the path to this file becomes the input data.

[1435] Step 2:

[1436] User: Clicks the upload button on the terminal to send the selected file to the system. Specifically, the contents of the file are transferred from the terminal to the server as binary data.

[1437] Step 3:

[1438] Server: Receives binary data sent from the terminal and saves the file in a temporary storage directory (e.g. / tmp / uploads / ). The input is the binary data, and the output is the saved file path.

[1439] Step 4:

[1440] Server: The temporarily saved file is passed to a natural language processing engine (e.g., SpaCy or NLTK) to analyze the text. The input is the saved file path, and the output is the analyzed text data. Specifically, the file is read, the character code is converted, and the text is extracted.

[1441] Step 5:

[1442] Server: The extracted text data is passed to a natural language processing engine, where it is analyzed for parts of speech, keyword extraction, and contextual understanding. The input is text data, and the output is metadata (e.g., meeting date and time, participants, agenda, and content of remarks) as the analysis results.

[1443] Step 6:

[1444] Server: The parsed metadata is registered in a database (e.g., MySQL, PostgreSQL). The input is the metadata, and the output is a confirmation of registration in the database. Specifically, the metadata is inserted into the appropriate tables in the database.

[1445] Step 7:

[1446] Terminal: Provides a query input interface to users and accepts query input in natural language. The input is a query entered by the user (e.g., "I want to see the minutes of the project ABC held in May of last year").

[1447] Step 8:

[1448] Terminal: When a user enters a query, the device recognizes their facial expression and voice and passes them to the emotion analysis engine. The input is the user's facial image and voice data, and the output is analyzed emotion data.

[1449] Step 9:

[1450] Terminal: Sends query and emotion data to the server. The input is the query and emotion data, and the output is data transmission to the server.

[1451] Step 10:

[1452] Server: Analyzes the received query using a natural language processing engine and searches for relevant information from a database. The input is the query data, and the output is the search result data.

[1453] Step 11:

[1454] Server: Customize search results using emotional data. The input is search results and emotional data, and the output is customized search results. For example, if the user is feeling stressed, the summary will be made shorter.

[1455] Step 12:

[1456] Server: Summarizes search results using a generative AI model (e.g., OpenAI GPT-3). The input is the customized search results, and the output is the summarized text.

[1457] Step 13:

[1458] Terminal: displays summarized search results to the user. The input is the summarized text and the output is the displayed information.

[1459] Step 14:

[1460] Terminal: After displaying the image, the user's facial expressions and voice are collected again and recorded as emotional feedback. The input is the user's facial expressions and voice, and the output is emotional feedback data.

[1461] Step 15:

[1462] Server: Receives emotional feedback sent from the device and accumulates the data to be used to improve the accuracy of the system. The input is emotional feedback data, and the output is updated system learning data.

[1463] (Application example 2)

[1464] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1465] In recent years, with the increasing automation of customer service, brick-and-mortar stores are also in need of systems that can respond to customer questions effectively and quickly. However, conventional systems have limitations in information search and summarization functions using natural language processing, making it particularly difficult to respond to customer questions while taking emotions into account. Furthermore, they lack a summary function for visually displaying search results concisely, and a customization function based on the user's emotions, which prevents sufficient improvement of customer satisfaction. To solve these issues, a system with emotion recognition and customization functions is needed.

[1466] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1467] In this invention, the server includes means for collecting past minutes, documents, and comments from upper management, means for analyzing the collected content using natural language processing technology, means for registering the analysis results in a database, means for receiving and analyzing user queries and searching for appropriate information, means for analyzing customer voice data and recognizing emotions, means for summarizing search results and displaying them to the user, means for summarizing related information using a generative AI model, and means for collecting user emotional feedback and improving the accuracy of the system. This enables appropriate responses to customer questions that take emotions into consideration.

[1468] Minutes are documents that record the contents of meetings and discussions.

[1469] "Materials" are documents, data files, etc. that contain information about a particular topic or project.

[1470] "Comments from senior management" is a document that compiles opinions and instructions from senior management, such as managers and executives, of a company or organization.

[1471] "Natural language processing" is a technology that allows computers to understand, analyze, and process human language.

[1472] A "database" is a collection of data that makes it easy to search, organize, and manage.

[1473] A "query" is a request or question to a database to retrieve information.

[1474] "Emotion recognition" is a technology that identifies a person's emotional state from voice, text, facial expressions, etc.

[1475] A "generative AI model" is an artificial intelligence model that generates new text based on input data.

[1476] "Emotional feedback" is the process of collecting and analyzing a user's emotional state and reactions.

[1477] This invention is a system that collects past minutes, documents, and comments from upper management, analyzes them using natural language processing technology and an emotion engine, and registers the analysis results in a database. Here, a specific embodiment of the invention will be described.

[1478] System configuration

[1479] The system consists of the following main components:

[1480] Data Collection Component

[1481] Terminal: Provides an interface for users to upload electronic files such as minutes, documents, and comments from senior management into the system.

[1482] Server: Receives uploaded files and prepares them for analysis.

[1483] Data Analysis Components

[1484] Server: Analyzes the uploaded text data using natural language processing technology. The analysis process involves parts of speech analysis, keyword extraction, and context understanding.

[1485] Server: Based on the analysis results, metadata such as the meeting date and time, participants, agenda, and content of remarks is extracted from the text data. This information is formatted as structured data.

[1486] Database Components

[1487] Server: The analyzed text data and generated metadata are registered in a database, allowing for efficient search and reference later.

[1488] Query Processing Component

[1489] Terminal: Provides an interface for users to enter natural language queries.

[1490] Server: Analyzes the received query and searches for relevant information from a database.

[1491] Emotion Recognition Component

[1492] Terminal: Provides an interface for analyzing the customer's voice and text and recognizing their emotional state.

[1493] Server: Uses an emotion engine to analyze user emotions and use them as data to customize search results.

[1494] Search Results Summary Component

[1495] Server: Uses a generative AI model to summarize search results and convert them into an easy-to-understand format.

[1496] Terminal: Displays summarized search results to the user.

[1497] Emotional Feedback Component

[1498] Server: Collects user emotional feedback, analyzes and stores the data, and uses it to improve the accuracy of the system.

[1499] Specific examples

[1500] For example, if a store staff member enters a query into the system such as "Please tell me your recent feedback on customer service at the store," the system operates as follows: First, the server receives the query and analyzes it using natural language processing technology. Next, based on the analysis results, it searches for relevant feedback from the database and summarizes it using a generative AI model. In addition, it optimizes the search results based on the results of sentiment analysis of the voice data acquired by the device. Finally, the summary result and additional information that takes sentiment into consideration are displayed on the device.

[1501] Prompt Sentence Examples

[1502] "What feedback have you received recently about customer service at your store?"

[1503] In this way, store staff can quickly and easily view relevant information and improve the quality of customer service.

[1504] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1505] Step 1:

[1506] Terminal: The user uploads electronic files such as minutes, documents, and comments from upper management to the system. This input data is sent to the server. The specific operation for uploading is to press the file selection button, select the specified file, and click the upload button.

[1507] Input: minutes, materials, comments from upper management

[1508] Output: Collected data

[1509] Step 2:

[1510] Server: Receives and temporarily stores uploaded files, which contain detailed information such as meeting date and time, participants, agenda, and comments.

[1511] Input: Collected data

[1512] Output: Saved data

[1513] Step 3:

[1514] Server: Analyzes the uploaded text data using a natural language processing engine, specifically by analyzing parts of speech, extracting keywords, and understanding context to extract important metadata.

[1515] Input: Saved data

[1516] Output: Analysis data

[1517] Step 4:

[1518] Server: Based on the analysis results, metadata such as the meeting date and time, participants, agenda, and content of remarks is extracted from the text data and formatted as structured data.

[1519] Input: Analysis data

[1520] Output: Metadata

[1521] Step 5:

[1522] Server: The generated metadata is registered in a database, allowing for efficient search and reference later.

[1523] Input: Metadata

[1524] Output: Database update

[1525] Step 6:

[1526] Terminal: The user inputs a query such as a question or request in natural language. For example, "Please tell me the minutes of the latest meeting." The input query is sent to the server.

[1527] Input: User query

[1528] Output: Query data

[1529] Step 7:

[1530] Server: Analyzes the received query and searches the database for relevant information, using natural language processing technology to accurately understand the meaning of the query.

[1531] Input: Query data

[1532] Output: Search results

[1533] Step 8:

[1534] Terminal: Collects customer voice data and analyzes it using emotion recognition technology. Specifically, the device captures the voice file, converts it into text using voice recognition software, and analyzes emotions using an emotion engine.

[1535] Input: Audio data

[1536] Output: Emotion data

[1537] Step 9:

[1538] Server: Customize search results based on emotional data. For example, if a customer is stressed, tailor search results to display them in a concise and easy-to-understand format.

[1539] Input: Emotion data, search results

[1540] Output: Customized search results

[1541] Step 10:

[1542] Server: Uses a generative AI model to summarize search results and present them to the user in a more concise format. Generates summaries using prompts.

[1543] Input: Customized search results

[1544] Output: Summary data

[1545] Step 11:

[1546] Terminal: Display summarized search results to users, allowing them to quickly check relevant information through their terminal.

[1547] Input: Summary data

[1548] Output: Display data

[1549] Step 12:

[1550] Terminal: Record the user's emotional feedback on the summary results. For example, provide a feedback button or a simple survey form to collect the user's satisfaction and additional comments.

[1551] Input: User feedback

[1552] Output: Feedback data

[1553] Step 13:

[1554] Server: Collects emotional feedback and stores it as data to improve the accuracy of the system. Specifically, it analyzes the collected feedback and identifies areas for improvement in the system.

[1555] Input: Feedback data

[1556] Output: Improvement data

[1557] This process makes it possible to provide appropriate information in response to user questions while taking into consideration their emotions, thereby improving the quality of customer service.

[1558] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1559] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1560] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1561] [Fourth embodiment]

[1562] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1563] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1564] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1565] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1566] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1567] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1568] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1569] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1570] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1571] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1572] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1573] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1574] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1575] The present invention is a system that collects past minutes, documents, and comments from senior management, analyzes them using natural language processing technology, and registers the analysis results in a database. The system searches for the analysis results based on queries from users, summarizes the search results, and displays them to the user. A specific embodiment of this system is described below.

[1576] System configuration

[1577] The system consists of the following main components:

[1578] 1. Data Collection Components

[1579] Terminal: Provides an interface for users to upload electronic files such as minutes, documents, and comments from senior management into the system.

[1580] Server: Receives uploaded files and prepares them for analysis.

[1581] 2. Data Analysis Component

[1582] Server: Analyzes the uploaded text data using natural language processing technology, including part-of-speech analysis, keyword extraction, and semantic understanding.

[1583] Server: Structures the analysis results and generates metadata such as meeting date and time, participants, agenda, and content of comments.

[1584] 3. Database Component

[1585] Server: The parsed data and generated metadata are registered in a database.

[1586] 4. Query Processing Component

[1587] Terminal: Provides an interface for users to enter natural language queries.

[1588] Server: Analyzes the received query and searches for relevant information from a database.

[1589] 5. Search Results Summary Component

[1590] Server: Uses a generative AI model to summarize search results and convert them into an easy-to-understand format.

[1591] Terminal: Displays summarized search results to the user.

[1592] System operation example

[1593] 1. Data Collection Example

[1594] Terminal: The user selects project minutes, documents, and comments from upper management and uploads them to the system.

[1595] Server: Receives uploaded files and temporarily stores their contents.

[1596] 2. Data Analysis Example

[1597] Server: Uses a natural language processing engine to analyze the text of the uploaded minutes, extracting information such as "The Project ABC meeting took place on May 3, 2022, and the main topic was budget adjustment."

[1598] Server: Based on the analysis results, generate metadata such as the meeting date and time (May 3, 2022), participants (speaker list), and main agenda item (budget adjustment).

[1599] 3. Example of database registration

[1600] Server: The parsed text data and metadata are stored in a database, allowing for efficient search and reference later.

[1601] 4. Query Processing Example

[1602] User: For example, enter a query such as "I want to see the minutes of the project ABC held in May of last year" into the terminal.

[1603] Terminal: Sends queries to the server.

[1604] Server: Parses the query using a natural language processing engine and searches the database for relevant minutes.

[1605] 5. Example of summary and display of search results

[1606] Server: Uses a generative AI model to summarize the search results, for example, "The Project ABC meeting took place on May 3, 2022, and the main topic was budget adjustment."

[1607] Terminal: Displays summarized search results to the user.

[1608] In this way, users can quickly and easily search and refer to past meeting minutes, documents, and comments from upper management, which improves work efficiency and speeds up decision-making.

[1609] The processing flow will be explained below.

[1610] Step 1:

[1611] Terminal: The user prepares the minutes, materials, and comments from senior management as electronic files. The user uses the system interface to select the files to upload.

[1612] Step 2:

[1613] Terminal: The user sends the selected file to the server. The file is sent to the server through the system's upload function.

[1614] Step 3:

[1615] Server: The received file is stored in a temporary location before analysis, and preparations for analysis are made.

[1616] Step 4:

[1617] Server: The natural language processing engine is launched to analyze the text data in the uploaded file. During the analysis process, parts of speech analysis, keyword extraction, context understanding, etc. are performed.

[1618] Step 5:

[1619] Server: Based on the analysis results, metadata such as the meeting date and time, participants, agenda, and content of remarks is extracted from the text data. This information is formatted as structured data.

[1620] Step 6:

[1621] Server: The analyzed text data and generated metadata are registered in a database, allowing for efficient search and reference later.

[1622] Step 7:

[1623] User: Enters a query through the system interface to search for specific information, such as "I want to see the minutes of the project ABC meeting held last May."

[1624] Step 8:

[1625] Terminal: Sends queries entered by the user to the server.

[1626] Step 9:

[1627] Server: Analyzes the received query using natural language processing to understand the meaning of the sentence, allowing it to efficiently search for relevant information in a database.

[1628] Step 10:

[1629] Server: Searches the database for meeting minutes, documents, comments from senior management, etc. that correspond to the query.

[1630] Step 11:

[1631] Server: Using a generative AI model, the search results are summarized and converted into an easy-to-understand format.

[1632] Step 12:

[1633] Server: Sends summarized search results to the terminal.

[1634] Step 13:

[1635] Terminal: Displays the search results received from the server to the user, allowing the user to quickly check related information.

[1636] Step 14:

[1637] Users: Optionally, click on the resulting links to view detailed minutes, materials, and comments from senior management.

[1638] Example 1

[1639] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1640] Until now, it has been difficult to efficiently collect and analyze past meeting records, documents, and comments from senior management, and to quickly obtain the information users need. In particular, the process of searching for appropriate information from large amounts of data takes time and effort, and it is not easy to summarize the search results in an easy-to-understand format. Therefore, there is a need for a system that can easily search past meeting records and documents and provide easy-to-understand summaries.

[1641] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1642] In this invention, the server includes means for collecting past meeting records, materials, and comments from senior managers, means for analyzing the collected content using natural language processing technology, means for registering the analysis results in a database, means for receiving and analyzing user queries and searching for appropriate information, and means for summarizing the search results and displaying them to the user, thereby enabling the user to quickly and efficiently search past meeting records and materials and obtain easy-to-understand summaries.

[1643] "Past meeting records" refers to minutes and notes taken during meetings and discussions.

[1644] "Materials" refers to documents such as reports, presentations, and explanations used in meetings and discussions.

[1645] "Comments from senior management" refers to opinions or instructions left by senior management of a company or organization in meetings or in writing.

[1646] "Natural language processing technology" refers to technology that allows computers to analyze and understand human language.

[1647] "Analysis results" refers to information and data extracted from text data using natural language processing technology.

[1648] "Means for registering in a database" refers to a method or system for storing analysis results in a database.

[1649] "User query" refers to a search request or question entered by a user into the system.

[1650] "Means for retrieving relevant information" refers to techniques and methods for retrieving relevant information from a database based on a user's query.

[1651] "Means for summarizing search results and displaying them to the user" refers to techniques and methods for summarizing the searched information and providing it to the user in an easy-to-understand format.

[1652] A "generative artificial intelligence model" refers to a technology that uses artificial intelligence learned from large amounts of data to analyze and generate data.

[1653] "Metadata format" refers to a structured data format for describing the content and attributes of data.

[1654] This invention is a system that collects and analyzes past meeting records, documents, and comments from senior managers, registers the analysis results in a database, searches based on user queries, and displays summaries. This system allows users to quickly and efficiently obtain the information they need.

[1655] System configuration

[1656] The system mainly consists of the following main components:

[1657] 1. Data Collection Components

[1658] Terminal: Serves as an interface for users to upload electronic files such as meeting records, materials, senior management comments, etc. Specifically, the user selects a file using a file selection dialog and clicks the "Upload" button.

[1659] Server: Receives and temporarily stores files sent from the device. For example, it uses a cloud storage service such as Amazon S3.

[1660] 2. Data Analysis Component

[1661] Server: Analyzes the uploaded text data using natural language processing techniques. Specifically, it uses natural language processing libraries such as NLTK and spaCy to perform part-of-speech analysis, keyword extraction, and semantic understanding. For example, it analyzes the sentence "The Project ABC meeting was held on May 3, 2022, and the main topic was budget adjustment."

[1662] Server: Based on the analysis results, generate metadata (meeting date and time, participants, agenda, content of remarks, etc.).

[1663] 3. Database Component

[1664] Server: The parsed text data and generated metadata are registered in a database. Specifically, a database such as MySQL or PostgreSQL is used to execute SQL queries and store the data.

[1665] 4. Query Processing Component

[1666] Terminal: Provides an interface for users to enter queries in natural language. Users enter queries in natural language into a search box and click a "Search" button.

[1667] Server: Analyzes the received query and searches for relevant information from the database. Specifically, it analyzes the query using natural language processing models such as BERT and GPT, and generates and executes the appropriate SQL query.

[1668] 5. Search Results Summary Component

[1669] Server: Using a generative AI model (e.g., GPT-3), the search results are summarized. The summary is then converted into a format that is easy for users to understand. For example, a summary is generated: "The Project ABC meeting took place on May 3, 2022, and the main agenda item was budget adjustment."

[1670] Terminal: Displaying summarized search results to the user. The summarized results are displayed on the terminal interface, allowing the user to easily browse the information.

[1671] Specific examples and prompts

[1672] Data Collection Example

[1673] Terminal: The user uploads "Project ABC Meeting Minutes for May 3, 2022" to the system.

[1674] Server: Receives and temporarily stores files.

[1675] Data analysis example

[1676] Server: The uploaded minutes are analyzed using a natural language processing engine. For example, the sentence "The Project ABC meeting was held on May 3, 2022, and the main topic was budget adjustment" is analyzed.

[1677] Server: From the analysis results, generate metadata such as "Meeting date and time: May 3, 2022," "Participants: List," and "Agenda: Budget adjustment."

[1678] Example queries and search results

[1679] User: In the search box, type "Show me the meeting minutes for project ABC from last May."

[1680] Terminal: Sends input to the server.

[1681] Server: Uses GPT-3 to parse the query and search and summarize "Project ABC meeting on May 3, 2022" from the database.

[1682] Device: Display the summarized search result "Budget adjustment is the main topic."

[1683] Example prompt sentence:

[1684] "What are the main topics for the Project ABC meeting on May 3, 2022?"

[1685] With the above-described procedure and configuration, the present invention provides a system that enables a user to efficiently and quickly obtain the information he or she requires.

[1686] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1687] Program processing steps

[1688] Step 1: Collect data

[1689] Terminal - The user uses the terminal interface to select electronic files such as meeting notes, materials, and senior management comments, and clicks the upload button.

[1690] Input: Electronic file selected by the user

[1691] Output: File upload request from the device to the server

[1692] What happens: The user opens the file selection dialog, selects the appropriate file, and clicks "Upload."

[1693] Step 2: Receiving and temporarily saving the file

[1694] Server - The server receives the file sent from the device and temporarily stores it.

[1695] Input: Electronic file sent from the terminal

[1696] Output: File saved in temporary directory

[1697] Specific operation: The server saves the received file in the specified directory (e.g., / uploads) and records the file metadata (file name, upload time, etc.).

[1698] Step 3: Data analysis using natural language processing

[1699] Server - Analyzes the stored text data using natural language processing techniques.

[1700] Input: A file saved in the temporary directory

[1701] Output: Analysis results (part of speech analysis results, keyword list, semantic understanding results)

[1702] Specific operation: The server uses libraries such as NLTK and spaCy to perform part-of-speech analysis, keyword extraction, and semantic understanding of text data. For example, it parses the sentence "The Project ABC meeting was held on May 3, 2022, and the main topic was budget adjustment."

[1703] Step 4: Generate Metadata

[1704] Server - Generates metadata (meeting date and time, participants, agenda, content of comments, etc.) based on the analysis results.

[1705] Input: Analysis results from natural language processing

[1706] Output: Metadata (meeting date and time, participants, agenda, and comments)

[1707] Specific operation: Extract necessary information from the analysis results and generate metadata such as "Meeting date and time: May 3, 2022," "Participants: List," and "Agenda: Budget adjustment."

[1708] Step 5: Register with the database

[1709] Server - The parsed text data and generated metadata are registered in a database.

[1710] Input: Parsed text data and metadata

[1711] Output: Analysis results and metadata stored in a database

[1712] What happens next: The server runs SQL queries against a database like MySQL or PostgreSQL and inserts the data into the appropriate tables.

[1713] Step 6: Enter and submit your query

[1714] User - The user types a query in natural language and submits it through the terminal interface.

[1715] Input: A query entered by a user (e.g., "I want to see the minutes of the meeting on project ABC from last May")

[1716] Output: Query request from the terminal to the server

[1717] Specific behavior: The user enters a query in the search box and clicks the "Search" button.

[1718] Step 7: Parsing the query and searching

[1719] Server - Parses the incoming query and retrieves the relevant information from the database.

[1720] Input: The query submitted by the user

[1721] Output: Search results (related meeting records and materials)

[1722] What it does: The server uses natural language processing models such as BERT or GPT to analyze the query and generate and execute the appropriate SQL query.

[1723] Step 8: Summarizing search results

[1724] Server - Summarizes search results using a generative AI model.

[1725] Input: Search results from the database

[1726] Output: Summarized search results

[1727] What it does: The server uses a generative AI model (e.g., GPT-3) to summarize the search results and convert them into an understandable format, such as generating a summary like "The Project ABC meeting took place on May 3, 2022, and the main topic was budget adjustment."

[1728] Step 9: View summary results

[1729] Terminal - Display summarized search results to the user.

[1730] Input: Abstracted search results from the server

[1731] Output: Summary results displayed to the user on their screen

[1732] Specific operation: The summary results are displayed on the terminal interface in a format that can be easily viewed by the user.

[1733] Through the above steps, users can efficiently search past meeting records and materials and obtain easy-to-understand summaries.

[1734] (Application example 1)

[1735] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1736] It is difficult to efficiently analyze large amounts of data, such as past meeting minutes, documents, and comments from senior management, and provide an environment where users can easily access important information. Furthermore, there is a need to instantly summarize and present relevant information in response to user queries, which requires the use of advanced natural language processing technology and generative AI models. However, current systems often do not adequately meet these requirements.

[1737] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1738] In this invention, the server includes means for collecting past minutes, documents, and comments from upper management, means for analyzing the collected content using natural language processing technology, means for registering the analysis results in a database, means for receiving and analyzing user queries and searching for appropriate information, means for summarizing the search results and displaying them to the user, and means for generating summaries of relevant minutes and documents using a generative AI model when a user inputs a specific query. This makes it possible to efficiently extract necessary information from large amounts of past data, quickly and appropriately summarize it, and provide it to the user.

[1739] "Past minutes" are documents that record the contents of past meetings and discussions.

[1740] "Materials" refers to documents and digital data used to communicate, store, and analyze information.

[1741] "Comments from upper management" refers to opinions and instructions issued by managers or executives of a company or organization.

[1742] "Natural language processing technology" refers to technology that allows computers to understand, analyze, and generate human language.

[1743] A "generative AI model" is an algorithm that uses artificial intelligence to generate new content from data such as text and images.

[1744] "Metadata format" is additional information for describing the attributes of data, and clearly indicates the type and structure of the data.

[1745] A "query" is a question or request to a database or system to retrieve information.

[1746] A "server" is a computer that provides data over a network, and is responsible for storing, analyzing, and providing data.

[1747] A specific embodiment of the present invention will be described. The present invention relates to a system that collects past minutes, documents, and comments from senior management, analyzes them using natural language processing technology, and registers the results in a database. It has the function of searching for analysis results based on user queries and summarizing and displaying the results using a generative AI model.

[1748] System configuration

[1749] 1. Data Collection Components

[1750] Terminal: Provides an interface for users to upload electronic files such as minutes, documents, and comments from senior management into the system.

[1751] Server: Receives uploaded files and prepares them for analysis.

[1752] 2. Data Analysis Component

[1753] Server: Analyzes the uploaded text data using natural language processing technology. The analysis uses Spacey's natural language processing engine and includes part-of-speech analysis, keyword extraction, and semantic understanding.

[1754] Server: Structures the analysis results and generates metadata such as meeting date and time, participants, agenda, and content of comments.

[1755] 3. Database Component

[1756] Server: The analyzed data and generated metadata are registered in a database built on cloud storage.

[1757] 4. Query Processing Component

[1758] Terminal: Provides an interface for users to enter natural language queries.

[1759] Server: Analyzes the received query and searches for relevant information from a database.

[1760] 5. Search Results Summary Component

[1761] Server: Uses a generative AI model to summarize search results. Leverages Hugging Face's transformers library.

[1762] Terminal: Displays summarized search results to the user.

[1763] Explanation of system behavior

[1764] 1. Data Collection

[1765] Users use their devices to select project minutes, documents, and comments from senior management and upload them to the system. The server receives the uploaded files and temporarily stores their contents.

[1766] 2. Data Analysis

[1767] The server uses Spacey's natural language processing engine to analyze the text of the uploaded minutes, extracting information such as, "The Project ABC meeting took place on May 3, 2022, and the main topic was budget adjustment." It then generates metadata based on the analysis results, including the meeting date and time, participants, and main topic.

[1768] 3. Registering in the database

[1769] The server stores the analyzed text data and metadata in a database, allowing for efficient search and reference later.

[1770] 4. Query Processing

[1771] A user inputs a query into a terminal, such as "I want to know the details of the meeting about strengthening security." The terminal sends the query to the server, which then analyzes the query using a natural language processing engine and searches for relevant minutes from the database.

[1772] 5. Summarizing and displaying search results

[1773] The server uses the generative AI model to summarize the search results, for example, "The Project ABC meeting took place on May 3, 2022, and the main topic was budget adjustment." The device then displays the summarized search results to the user.

[1774] Examples of concrete examples and prompts

[1775] The user inputs a query on their device, such as "Please summarize the contents of the meeting about strengthening security." The server analyzes this query, extracts relevant minutes and materials from the database, and generates a summary using a generative AI model.

[1776] Example prompt sentence:

[1777] Please summarize the meeting on strengthening security.

[1778] In this way, the system enables users to efficiently extract necessary information from large amounts of past data, quickly and appropriately summarize it, and provide it to the users.

[1779] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1780] Step 1:

[1781] Uploading data

[1782] The user uses a terminal to select project minutes, documents, and comments from upper management and upload them to the system. The input is the electronic file selected by the user, which is sent to the server. The server receives the uploaded file and temporarily stores it. This stored data is then analyzed in the next step.

[1783] Step 2:

[1784] Data analysis

[1785] The server uses Spacy, a natural language processing engine, to analyze the uploaded text data. The input includes text data from temporarily saved electronic files. The server performs part-of-speech analysis, keyword extraction, and semantic understanding to extract information such as the meeting date and time, participants, agenda, and content of remarks. The analysis results and metadata are generated as output.

[1786] Step 3:

[1787] Registering in the database

[1788] The server registers the analyzed text data and generated metadata in a database on cloud storage. The analysis results and metadata generated in step 2 are used as input. Once registration in the database is complete, subsequent search processing and reference can be performed efficiently. The output is an update of the database.

[1789] Step 4:

[1790] Entering and submitting a query

[1791] A user uses a terminal to input a query in natural language. For example, a query such as "I want to know the details of the conference on strengthening security" is input. The input is the user's query text, which is sent to the server via the terminal. The output is the query text passed to the server.

[1792] Step 5:

[1793] Query Parsing and Search

[1794] The server uses a natural language processing engine to analyze the received query. The input includes the user's query text. Based on this analysis, it searches the database for relevant information. The output is the text data of the relevant minutes and documents.

[1795] Step 6:

[1796] Generate summary of search results

[1797] The server uses a generative AI model (Hugging Face transformers) to summarize the search results. The input is the text data of the minutes and documents extracted in step 5. The generative AI model summarizes the results and generates a summary text in an easy-to-understand format as output.

[1798] Step 7:

[1799] Viewing summary results

[1800] The terminal displays the summary text received from the server to the user. The input includes the summary text sent from the server. The user can check this summary and obtain the necessary information. The output is displayed visually to the user.

[1801] These steps enable users to efficiently extract necessary information from large amounts of past data and obtain information that is quickly and appropriately summarized.

[1802] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1803] The present invention is a system that collects past meeting minutes, documents, and comments from senior management, analyzes them using natural language processing technology and an emotion engine, and registers the analysis results in a database. It recognizes user queries and their associated emotions, customizes search results, summarizes them, and displays them to the user. A specific embodiment of this system is described below.

[1804] System configuration

[1805] The system consists of the following main components:

[1806] 1. Data Collection Components

[1807] Terminal: Provides an interface for users to upload electronic files such as minutes, documents, and comments from senior management into the system.

[1808] Server: Receives uploaded files and prepares them for analysis.

[1809] 2. Data Analysis Component

[1810] Server: Analyzes the uploaded text data using natural language processing technology. The analysis process involves parts of speech analysis, keyword extraction, and context understanding.

[1811] Server: Based on the analysis results, metadata such as the meeting date and time, participants, agenda, and content of remarks is extracted from the text data. This information is formatted as structured data.

[1812] 3. Database Component

[1813] Server: The analyzed text data and generated metadata are registered in a database, allowing for efficient search and reference later.

[1814] 4. Query Processing Component

[1815] Terminal: Provides an interface for users to enter natural language queries.

[1816] Server: Analyzes the received query and searches for relevant information from a database.

[1817] 5. Emotion Recognition Component

[1818] Terminal: Provides an interface for recognizing the user's emotional state from their facial expressions and voice when entering a query.

[1819] Server: Uses an emotion engine to analyze user emotions and use them as data to customize search results.

[1820] 6. Search Results Summary Component

[1821] Server: Uses a generative AI model to summarize search results and convert them into an easy-to-understand format.

[1822] Terminal: Displays summarized search results to the user.

[1823] 7. Emotional Feedback Component

[1824] Server: Collects user emotional feedback, analyzes and stores the data, and uses it to improve the accuracy of the system.

[1825] System operation example

[1826] 1. Data Collection Example

[1827] Terminal: The user selects project minutes, documents, and comments from upper management and uploads them to the system.

[1828] Server: Receives uploaded files and temporarily stores their contents.

[1829] 2. Data Analysis Example

[1830] Server: Uses a natural language processing engine to analyze the text of the uploaded minutes, extracting information such as "The Project ABC meeting took place on May 3, 2022, and the main topic was budget adjustment."

[1831] Server: Based on the analysis results, generate metadata such as the meeting date and time (May 3, 2022), participants (speaker list), and main agenda item (budget adjustment).

[1832] 3. Example of database registration

[1833] Server: The parsed text data and metadata are stored in a database, allowing for efficient search and reference later.

[1834] 4. Query Processing Example

[1835] User: For example, enter a query such as "I want to see the minutes of the project ABC held in May of last year" into the terminal.

[1836] Device: The emotion engine analyzes the user's facial expressions and voice when entering a query.

[1837] Terminal: Sends queries and emotion data to the server.

[1838] Server: Parses the query using a natural language processing engine and searches the database for relevant minutes.

[1839] 5. Example of customizing, summarizing and displaying search results

[1840] Server: Customize search results based on emotional data, for example, by providing a more concise summary if the user is feeling stressed.

[1841] Server: Uses a generative AI model to summarize the search results, for example, "The Project ABC meeting took place on May 3, 2022, and the main topic was budget adjustment."

[1842] Terminal: Displays summarized search results to the user, allowing the user to quickly view relevant information.

[1843] 6. Collecting Emotional Feedback

[1844] Terminal: Records the user's emotional feedback on search results.

[1845] Server: Collects emotional feedback to help improve the system's accuracy.

[1846] In this way, users can quickly and easily search and refer to past minutes, documents, and comments from upper management, improving work efficiency and speeding up decision-making.In addition, by using an emotion engine, it is possible to provide even more user-friendly and customized services.

[1847] The processing flow will be explained below.

[1848] Step 1:

[1849] Terminal: The user prepares the minutes, materials, and comments from senior management as electronic files. The user uses the system interface to select the files to upload.

[1850] Step 2:

[1851] Terminal: The user sends the selected file to the server. The file is sent to the server through the system's upload function.

[1852] Step 3:

[1853] Server: The received file is stored in a temporary location before analysis, and preparations for analysis are made.

[1854] Step 4:

[1855] Server: The natural language processing engine is launched to analyze the text data in the uploaded file. During the analysis process, parts of speech analysis, keyword extraction, context understanding, etc. are performed.

[1856] Step 5:

[1857] Server: Based on the analysis results, metadata such as the meeting date and time, participants, agenda, and content of remarks is extracted from the text data. This information is formatted as structured data.

[1858] Step 6:

[1859] Server: The analyzed text data and metadata are registered in a database, allowing for efficient search and reference later.

[1860] Step 7:

[1861] User: Enters a query through the system interface to search for specific information, such as "I want to see the minutes of the project ABC meeting held last May."

[1862] Step 8:

[1863] On the device: When a user enters a query, the emotion engine analyzes the user's facial expressions and voice. The emotion engine uses facial recognition and voice analysis technologies to assess the user's emotional state.

[1864] Step 9:

[1865] Terminal: Sends queries and emotion data to the server.

[1866] Step 10:

[1867] Server: Analyzes the received query using natural language processing to understand the meaning of the sentence, allowing it to efficiently search for relevant information in a database.

[1868] Step 11:

[1869] Server: Searches the database for meeting minutes, documents, comments from senior management, etc. that correspond to the query.

[1870] Step 12:

[1871] Server: Customize search results based on emotional data, for example, making summaries more concise if the user is stressed.

[1872] Step 13:

[1873] Server: Using a generative AI model, the search results are summarized. For example, the summary might be something like, "The Project ABC meeting took place on May 3, 2022, and the main topic was budget adjustment."

[1874] Step 14:

[1875] Server: Sends summarized search results to the terminal.

[1876] Step 15:

[1877] Terminal: Displays the search results received from the server to the user, allowing the user to quickly check related information.

[1878] Step 16:

[1879] Users: Optionally, click on the resulting links to view detailed minutes, materials, and comments from senior management.

[1880] Step 17:

[1881] Terminal: Records the user's emotional feedback on the search results. For example, it provides an interface that asks for a simple rating to see if the user is satisfied with the results.

[1882] Step 18:

[1883] Server: Collects emotional feedback to help improve the system's accuracy. The collected data is used to optimize the user experience and improve the emotion engine.

[1884] Example 2

[1885] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1886] Conventional systems that collect and analyze minutes, documents, and comments from senior management face the challenge of efficiently searching and referencing vast amounts of information and providing customized information that reflects the user's emotions. In particular, if users are unable to quickly obtain the information they desire, this can lead to reduced operational efficiency and delays in decision-making. Furthermore, there are few systems that take user emotions into account, making it difficult to improve the user experience.

[1887] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for collecting past minutes, documents, and comments from management, means for analyzing the collected content using natural language processing technology, means for registering the analysis results in a database, means for receiving and analyzing user queries and searching for appropriate information, means for summarizing search results and displaying them to the user, means for analyzing the user's emotions at the time of query input, means for customizing search results based on the user's emotions, and means for collecting emotional feedback and using it to improve the accuracy of the system. This allows users to quickly and appropriately search and refer to past minutes, documents, and comments from upper management, and to obtain information customized according to their emotions.

[1888] "Past minutes" refers to documents and data that record what was discussed at meetings and discussions.

[1889] "Document" means any digital or paper material that records information in textual, graphical or other form.

[1890] "Management comments" refers to documents or data that record opinions or instructions issued by senior managers or executives of a company or organization.

[1891] "Natural language processing technology" refers to technology that enables computers to understand and process human language, and includes parts of speech analysis, keyword extraction, and context understanding.

[1892] "Analysis results" refers to information obtained after analyzing text data using natural language processing technology, and includes metadata and important content.

[1893] "Database" refers to a digital storage system designed to efficiently store, search, and reference information.

[1894] "Query" refers to a natural language question or command that a user enters into a system to search for specific information.

[1895] "Search Results" refers to relevant information extracted from a database based on a query.

[1896] A "generative AI model" is an artificial intelligence model trained using large datasets, and is used specifically for natural language generation and summarization.

[1897] "Emotion analysis" refers to the technology of determining a user's emotional state at any given time from data such as facial expressions and voice.

[1898] "Emotional feedback" refers to the collection of data on users' emotional reactions to search results and information provided by the system.

[1899] System Overview

[1900] This system collects past minutes, documents, and management comments, analyzes them using natural language processing technology and a sentiment analysis engine, and registers the analysis results in a database. It recognizes user queries and their associated sentiment, customizes search results, summarizes them, and displays them to the user.

[1901] Component Configuration

[1902] The system consists of the following main components:

[1903] 1. Data Collection Components

[1904] Terminal: Provides an interface for users to upload electronic files such as minutes, documents, and management comments to the system.

[1905] Server: Receives the uploaded file and stores it in a temporary directory.

[1906] 2. Data Analysis Component

[1907] Server: Specifically, it analyzes the uploaded text data using natural language processing technologies such as SpaCy and NLTK. The analysis process involves part-of-speech analysis, keyword extraction, and context understanding.

[1908] Server: Based on the analysis results, metadata such as the meeting date and time, participants, agenda, and content of remarks is generated from the text data.

[1909] 3. Database Component

[1910] Server: The parsed text data and generated metadata are registered in a database, typically MySQL or PostgreSQL.

[1911] 4. Query Processing Component

[1912] Terminal: Provides an interface for users to enter natural language queries, including a text box and voice input capabilities.

[1913] Server: Analyzes queries using a natural language processing engine and searches for relevant information from a database.

[1914] 5. Emotion Recognition Component

[1915] Terminal: Provides an interface for recognizing the user's emotional state from their facial expressions and voice when entering a query.

[1916] Server: Uses a sentiment analysis engine to analyze user sentiment and use it as data to customize search results.

[1917] 6. Search Results Summary Component

[1918] Server: Uses a generative AI model (such as OpenAI's GPT-3) to summarize and convert search results into an understandable format.

[1919] Terminal: Displays summarized search results to the user.

[1920] 7. Emotional Feedback Component

[1921] Terminal: Records the user's emotional feedback on search results.

[1922] Server: Collects emotional feedback to help improve the system's accuracy.

[1923] Specific operation example

[1924] Data Collection Example

[1925] Terminal: The user selects project minutes, documents, and management comments and uploads them to the system.

[1926] Server: Receives the uploaded file and temporarily stores it in the / tmp / uploads / directory.

[1927] Data analysis example

[1928] Server: Uses a natural language processing engine to analyze the text of the uploaded minutes, extracting information such as "The Project ABC meeting took place on May 3, 2022, and the main topic was budget adjustment."

[1929] Server: Based on the analysis results, generate metadata such as "Date and time: May 3, 2022," "Participants: Tanaka, Sato, Suzuki," and "Agenda: Budget adjustment."

[1930] Database registration example

[1931] Server: The parsed text data and metadata are stored in a MySQL or PostgreSQL database.

[1932] Query Processing Example

[1933] User: Enters query "I want to see the minutes of the project ABC meeting held last May."

[1934] Device: When a user enters a query, the facial recognition camera analyzes the user's facial expression and sends it to the server along with the query.

[1935] Server: Parses the query and retrieves relevant information from a database.

[1936] Customizing search results, summarizing and display examples

[1937] Server: Adjusts search results based on user sentiment data and creates a summary: "The Project ABC meeting took place on May 3, 2022, and the main topic was budget adjustments."

[1938] Terminal: Displays summaries, allowing users to quickly review relevant information.

[1939] Example of collecting emotional feedback

[1940] Device: Collects the user's facial expressions and voice again in response to the displayed search results and records emotional feedback.

[1941] Server: Receives emotional feedback sent from the device and stores it as data to improve the accuracy of the system.

[1942] Prompt Sentence Examples

[1943] "Analyze the minutes data and extract the main topics and participants."

[1944] "Generate a summary of the meeting about project ABC last May"

[1945] This system allows users to quickly and appropriately search and refer to past minutes, documents, and management comments, and has the advantage of being able to obtain customized information based on sentiment.

[1946] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1947] Step 1:

[1948] Terminal: Provides a file upload interface to the user. Allows the user to select a file to upload and receives file input. For example, if the user selects the file "Meeting2022_05_03.docx", the path to this file becomes the input data.

[1949] Step 2:

[1950] User: Clicks the upload button on the terminal to send the selected file to the system. Specifically, the contents of the file are transferred from the terminal to the server as binary data.

[1951] Step 3:

[1952] Server: Receives binary data sent from the terminal and saves the file in a temporary storage directory (e.g. / tmp / uploads / ). The input is the binary data, and the output is the saved file path.

[1953] Step 4:

[1954] Server: The temporarily saved file is passed to a natural language processing engine (e.g., SpaCy or NLTK) to analyze the text. The input is the saved file path, and the output is the analyzed text data. Specifically, the file is read, the character code is converted, and the text is extracted.

[1955] Step 5:

[1956] Server: The extracted text data is passed to a natural language processing engine, where it is analyzed for parts of speech, keyword extraction, and contextual understanding. The input is text data, and the output is metadata (e.g., meeting date and time, participants, agenda, and content of remarks) as the analysis results.

[1957] Step 6:

[1958] Server: The parsed metadata is registered in a database (e.g., MySQL, PostgreSQL). The input is the metadata, and the output is a confirmation of registration in the database. Specifically, the metadata is inserted into the appropriate tables in the database.

[1959] Step 7:

[1960] Terminal: Provides a query input interface to users and accepts query input in natural language. The input is a query entered by the user (e.g., "I want to see the minutes of the project ABC held in May of last year").

[1961] Step 8:

[1962] Terminal: When a user enters a query, the device recognizes their facial expression and voice and passes them to the emotion analysis engine. The input is the user's facial image and voice data, and the output is analyzed emotion data.

[1963] Step 9:

[1964] Terminal: Sends query and emotion data to the server. The input is the query and emotion data, and the output is data transmission to the server.

[1965] Step 10:

[1966] Server: Analyzes the received query using a natural language processing engine and searches for relevant information from a database. The input is the query data, and the output is the search result data.

[1967] Step 11:

[1968] Server: Customize search results using emotional data. The input is search results and emotional data, and the output is customized search results. For example, if the user is feeling stressed, the summary will be made shorter.

[1969] Step 12:

[1970] Server: Summarizes search results using a generative AI model (e.g., OpenAI GPT-3). The input is the customized search results, and the output is the summarized text.

[1971] Step 13:

[1972] Terminal: displays summarized search results to the user. The input is the summarized text and the output is the displayed information.

[1973] Step 14:

[1974] Terminal: After displaying the image, the user's facial expressions and voice are collected again and recorded as emotional feedback. The input is the user's facial expressions and voice, and the output is emotional feedback data.

[1975] Step 15:

[1976] Server: Receives emotional feedback sent from the device and accumulates the data to be used to improve the accuracy of the system. The input is emotional feedback data, and the output is updated system learning data.

[1977] (Application example 2)

[1978] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1979] In recent years, with the increasing automation of customer service, brick-and-mortar stores are also in need of systems that can respond to customer questions effectively and quickly. However, conventional systems have limitations in information search and summarization functions using natural language processing, making it particularly difficult to respond to customer questions while taking emotions into account. Furthermore, they lack a summary function for visually displaying search results concisely, and a customization function based on the user's emotions, which prevents sufficient improvement of customer satisfaction. To solve these issues, a system with emotion recognition and customization functions is needed.

[1980] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1981] In this invention, the server includes means for collecting past minutes, documents, and comments from upper management, means for analyzing the collected content using natural language processing technology, means for registering the analysis results in a database, means for receiving and analyzing user queries and searching for appropriate information, means for analyzing customer voice data and recognizing emotions, means for summarizing search results and displaying them to the user, means for summarizing related information using a generative AI model, and means for collecting user emotional feedback and improving the accuracy of the system. This enables appropriate responses to customer questions that take emotions into consideration.

[1982] Minutes are documents that record the contents of meetings and discussions.

[1983] "Materials" are documents, data files, etc. that contain information about a particular topic or project.

[1984] "Comments from senior management" is a document that compiles opinions and instructions from senior management, such as managers and executives, of a company or organization.

[1985] "Natural language processing" is a technology that allows computers to understand, analyze, and process human language.

[1986] A "database" is a collection of data that makes it easy to search, organize, and manage.

[1987] A "query" is a request or question to a database to retrieve information.

[1988] "Emotion recognition" is a technology that identifies a person's emotional state from voice, text, facial expressions, etc.

[1989] A "generative AI model" is an artificial intelligence model that generates new text based on input data.

[1990] "Emotional feedback" is the process of collecting and analyzing a user's emotional state and reactions.

[1991] This invention is a system that collects past minutes, documents, and comments from upper management, analyzes them using natural language processing technology and an emotion engine, and registers the analysis results in a database. Here, a specific embodiment of the invention will be described.

[1992] System configuration

[1993] The system consists of the following main components:

[1994] Data Collection Component

[1995] Terminal: Provides an interface for users to upload electronic files such as minutes, documents, and comments from senior management into the system.

[1996] Server: Receives uploaded files and prepares them for analysis.

[1997] Data Analysis Components

[1998] Server: Analyzes the uploaded text data using natural language processing technology. The analysis process involves parts of speech analysis, keyword extraction, and context understanding.

[1999] Server: Based on the analysis results, metadata such as the meeting date and time, participants, agenda, and content of remarks is extracted from the text data. This information is formatted as structured data.

[2000] Database Components

[2001] Server: The analyzed text data and generated metadata are registered in a database, allowing for efficient search and reference later.

[2002] Query Processing Component

[2003] Terminal: Provides an interface for users to enter natural language queries.

[2004] Server: Analyzes the received query and searches for relevant information from a database.

[2005] Emotion Recognition Component

[2006] Terminal: Provides an interface for analyzing the customer's voice and text and recognizing their emotional state.

[2007] Server: Uses an emotion engine to analyze user emotions and use them as data to customize search results.

[2008] Search Results Summary Component

[2009] Server: Uses a generative AI model to summarize search results and convert them into an easy-to-understand format.

[2010] Terminal: Displays summarized search results to the user.

[2011] Emotional Feedback Component

[2012] Server: Collects user emotional feedback, analyzes and stores the data, and uses it to improve the accuracy of the system.

[2013] Specific examples

[2014] For example, if a store staff member enters a query into the system such as "Please tell me your recent feedback on customer service at the store," the system operates as follows: First, the server receives the query and analyzes it using natural language processing technology. Next, based on the analysis results, it searches for relevant feedback from the database and summarizes it using a generative AI model. In addition, it optimizes the search results based on the results of sentiment analysis of the voice data acquired by the device. Finally, the summary result and additional information that takes sentiment into consideration are displayed on the device.

[2015] Prompt Sentence Examples

[2016] "What feedback have you received recently about customer service at your store?"

[2017] In this way, store staff can quickly and easily view relevant information and improve the quality of customer service.

[2018] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2019] Step 1:

[2020] Terminal: The user uploads electronic files such as minutes, documents, and comments from upper management to the system. This input data is sent to the server. The specific operation for uploading is to press the file selection button, select the specified file, and click the upload button.

[2021] Input: minutes, materials, comments from upper management

[2022] Output: Collected data

[2023] Step 2:

[2024] Server: Receives and temporarily stores uploaded files, which contain detailed information such as meeting date and time, participants, agenda, and comments.

[2025] Input: Collected data

[2026] Output: Saved data

[2027] Step 3:

[2028] Server: Analyzes the uploaded text data using a natural language processing engine, specifically by analyzing parts of speech, extracting keywords, and understanding context to extract important metadata.

[2029] Input: Saved data

[2030] Output: Analysis data

[2031] Step 4:

[2032] Server: Based on the analysis results, metadata such as the meeting date and time, participants, agenda, and content of remarks is extracted from the text data and formatted as structured data.

[2033] Input: Analysis data

[2034] Output: Metadata

[2035] Step 5:

[2036] Server: The generated metadata is registered in a database, allowing for efficient search and reference later.

[2037] Input: Metadata

[2038] Output: Database update

[2039] Step 6:

[2040] Terminal: The user inputs a query such as a question or request in natural language. For example, "Please tell me the minutes of the latest meeting." The input query is sent to the server.

[2041] Input: User query

[2042] Output: Query data

[2043] Step 7:

[2044] Server: Analyzes the received query and searches the database for relevant information, using natural language processing technology to accurately understand the meaning of the query.

[2045] Input: Query data

[2046] Output: Search results

[2047] Step 8:

[2048] Terminal: Collects customer voice data and analyzes it using emotion recognition technology. Specifically, the device captures the voice file, converts it into text using voice recognition software, and analyzes emotions using an emotion engine.

[2049] Input: Audio data

[2050] Output: Emotion data

[2051] Step 9:

[2052] Server: Customize search results based on emotional data. For example, if a customer is stressed, tailor search results to display them in a concise and easy-to-understand format.

[2053] Input: Emotion data, search results

[2054] Output: Customized search results

[2055] Step 10:

[2056] Server: Uses a generative AI model to summarize search results and present them to the user in a more concise format. Generates summaries using prompts.

[2057] Input: Customized search results

[2058] Output: Summary data

[2059] Step 11:

[2060] Terminal: Display summarized search results to users, allowing them to quickly check relevant information through their terminal.

[2061] Input: Summary data

[2062] Output: Display data

[2063] Step 12:

[2064] Terminal: Record the user's emotional feedback on the summary results. For example, provide a feedback button or a simple survey form to collect the user's satisfaction and additional comments.

[2065] Input: User feedback

[2066] Output: Feedback data

[2067] Step 13:

[2068] Server: Collects emotional feedback and stores it as data to improve the accuracy of the system. Specifically, it analyzes the collected feedback and identifies areas for improvement in the system.

[2069] Input: Feedback data

[2070] Output: Improvement data

[2071] This process makes it possible to provide appropriate information in response to user questions while taking into consideration their emotions, thereby improving the quality of customer service.

[2072] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[2073] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2074] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[2075] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[2076] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[2077] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[2078] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[2079] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[2080] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[2081] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[2082] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[2083] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[2084] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[2085] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[2086] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[2087] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[2088] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[2089] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[2090] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[2091] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[2092] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[2093] The following is further disclosed regarding the above embodiment.

[2094] (Claim 1)

[2095] A means of collecting past minutes, documents, and comments from senior management;

[2096] a means for analyzing the collected content using natural language processing technology;

[2097] A means for registering the analysis results in a database;

[2098] means for receiving and analyzing user queries and retrieving appropriate information;

[2099] a means for summarizing and displaying search results to a user;

[2100] A system including:

[2101] (Claim 2)

[2102] 10. The system of claim 1, comprising means for summarizing search results for a query using a generative AI model.

[2103] (Claim 3)

[2104] 10. The system of claim 1, further comprising means for storing the analyzed minutes and / or material content in a database in metadata format.

[2105] "Example 1"

[2106] (Claim 1)

[2107] A means of collecting past meeting notes, documents, and comments from senior management;

[2108] a means for analyzing the collected content using natural language processing technology;

[2109] A means for registering the analysis results in a database;

[2110] means for receiving and analyzing user queries and retrieving appropriate information;

[2111] a means for summarizing and displaying search results to a user;

[2112] A system including:

[2113] (Claim 2)

[2114] 10. The system of claim 1, further comprising means for summarizing search results for a query using a generative artificial intelligence model.

[2115] (Claim 3)

[2116] 10. The system of claim 1, further comprising means for storing the analyzed contents of the meeting records and materials in a database in the form of metadata.

[2117] "Application Example 1"

[2118] (Claim 1)

[2119] A means of collecting past minutes, documents, and comments from senior management;

[2120] a means for analyzing the collected content using natural language processing technology;

[2121] A means for registering the analysis results in a database;

[2122] means for receiving and analyzing user queries and retrieving appropriate information;

[2123] a means for summarizing and displaying search results to a user;

[2124] A means for generating summaries of relevant minutes and materials using a generative AI model when a user inputs a specific query, and

[2125] A system including:

[2126] (Claim 2)

[2127] 10. The system of claim 1, comprising means for summarizing search results for a query using a generative AI model.

[2128] (Claim 3)

[2129] 10. The system of claim 1, further comprising means for storing the analyzed minutes and / or material content in a database in metadata format.

[2130] "Example 2: Combining Emotion Engines"

[2131] (Claim 1)

[2132] A means of collecting past minutes, documents, and management comments;

[2133] a means for analyzing the collected content using natural language processing technology;

[2134] A means for registering the analysis results in a database;

[2135] means for receiving and analyzing user queries and retrieving appropriate information;

[2136] a means for summarizing and displaying search results to a user;

[2137] A means for analyzing a user's sentiment when entering a query;

[2138] A means for customizing search results based on user sentiment;

[2139] A means of collecting emotional feedback and using it to improve the accuracy of the system;

[2140] A system including:

[2141] (Claim 2)

[2142] 10. The system of claim 1, comprising means for summarizing search results for a query using a generative AI model.

[2143] (Claim 3)

[2144] 10. The system of claim 1, further comprising means for storing the analyzed minutes and document contents in a database in the form of metadata.

[2145] "Application example 2 when combining emotion engines"

[2146] (Claim 1)

[2147] A means of collecting past minutes, documents, and comments from senior management;

[2148] a means for analyzing the collected content using natural language processing technology;

[2149] A means for registering the analysis results in a database;

[2150] means for receiving and analyzing user queries and retrieving appropriate information;

[2151] A means for analyzing customer voice data and recognizing emotions;

[2152] a means for summarizing and displaying search results to a user;

[2153] A means of summarizing relevant information using a generative AI model; and

[2154] a means for collecting user emotional feedback to improve the accuracy of the system;

[2155] A system including:

[2156] (Claim 2)

[2157] 10. The system of claim 1, further comprising means for summarizing search results for a query using the generative AI model.

[2158] (Claim 3)

[2159] 2. The system according to claim 1, further comprising means for storing the analyzed contents of the minutes and materials in a database in metadata format. [Explanation of symbols]

[2160] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. A means of collecting past minutes, documents, and comments from senior management; a means for analyzing the collected content using natural language processing technology; A means for registering the analysis results in a database; means for receiving and analyzing user queries and retrieving appropriate information; a means for summarizing and displaying search results to a user; A system including:

2. The system of claim 1 , further comprising means for summarizing search results for a query using a generative AI model.

3. The system according to claim 1, further comprising means for storing the analyzed minutes and contents of the materials in a database in the form of metadata.

Images