System

Abstract

A system is provided.SOLUTION: A system comprising: means for receiving statement data from a user and analyzing the statement content using natural language processing; means for collecting relevant evidence from an internal database or an external information source based on the statement content and evaluating reliability; means for illustrating a flow of discussion and discussion points based on the analyzed statement content and the collected evidence; and means for learning past discussion data and extracting and presenting a new pattern and an effective solution.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] When illogical factors such as emotion, prejudice, and status influence discussions, the quality of decision-making often declines. Furthermore, reaching a logical conclusion can take time, which can lead to problems that do not necessarily produce the best results. This can have a negative impact on the efficiency and performance of the entire organization. This invention aims to provide technology that supports evidence-based, logical, and constructive discussions in real time and promotes fair and transparent communication. [Means for solving the problem]

[0005] The present invention solves the problems by the following means. First, it provides a means for receiving utterance data from users and analyzing the utterance content using natural language processing technology. Next, it provides a means for collecting relevant evidence from an internal database or external information source based on the analyzed utterance content and evaluating the reliability of this evidence. It also provides a means for visually illustrating and displaying the flow of discussion and points of contention based on the analysis results and collected evidence. It also includes a means for learning from past discussion data and extracting and presenting new patterns and effective solutions. This allows users to advance discussions while visually checking evidence-based points of contention, enabling efficient and fair decision-making based on information updated in real time.

[0006] "User" refers to an individual or group who uses the system to hold discussions.

[0007] "Utterance data" refers to the text and voice content that a user inputs to the system.

[0008] "Natural language processing" refers to the technology that allows computers to understand, analyze, and generate human language.

[0009] "Evidence" refers to the evidence or supporting information that supports an argument.

[0010] "Internal Database" refers to the collection of data stored within the System.

[0011] "External information sources" refer to information sources that exist outside the system, such as the Internet or other databases.

[0012] "Reliability" refers to the assessment of whether the evidence obtained is accurate and appropriate as a basis for discussion.

[0013] "Analysis results" refer to the results of understanding and classifying the content of user comments obtained through natural language processing.

[0014] "Illustration" refers to the representation of the flow of discussion and points of contention in a visual format such as a graph or flowchart.

[0015] "Visual display" refers to providing analysis results and evidence to the user in a visible form through a user interface.

[0016] "Past discussion data" refers to records and content of previous discussions.

[0017] "Learning" refers to using machine learning models to extract new patterns and insights from past data.

[0018] A "pattern" refers to a recurring trend or commonality in the data.

[0019] "Solution" refers to a proposal or method for solving the issue or problem in the discussion. [Brief explanation of the drawings]

[0020] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8]FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0021] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0022] First, the terms used in the following description will be explained.

[0023] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0024] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0025] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0026] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0027] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0028] [First embodiment]

[0029] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0030] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0031] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0032] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0033] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0034] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0035] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0036] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0037] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0038] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0039] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0040] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0041] This invention relates to a system that supports discussions in real time in which users participate, and promotes fair and logical discussions. The main components of the system are a server, terminals, and users, each of which performs a specific function.

[0042] Server Roles and Functions

[0043] 1. Receiving and analyzing user speech data

[0044] The server receives user speech data sent from the device in real time. It then analyzes the received speech data using natural language processing (NLP) technology to identify and classify the content and intent of the speech. This allows it to understand the topic of the discussion and lead to the collection of appropriate evidence.

[0045] 2. Evidence collection and reliability assessment

[0046] The server collects relevant evidence from an internal database and external sources based on the analyzed comments, evaluates the reliability of the collected evidence, and selects appropriate evidence to support the discussion. This allows users to advance discussions based on well-founded information.

[0047] 3. Visualization Generation and Presentation

[0048] The server generates data that illustrates the flow of discussion and points of contention based on the analysis results and collected evidence. This includes visual formats such as graphs, flowcharts, and mind maps. The generated illustrations are sent to the terminals so that users can view them in real time.

[0049] 4. Learning from past discussion data and making suggestions

[0050] The server inputs past discussion data into a machine learning model to learn new patterns and effective solutions, and then automatically generates proposals and solutions for the current discussion and sends them to the device.

[0051] Device roles and functions

[0052] 1. Providing a user interface

[0053] The terminal provides a user interface to users. Through this interface, users can input comments and questions and send them to the server. The terminal also displays the visualized data and evidence information sent from the server, allowing users to check the progress of the discussion in real time.

[0054] 2. Viewing Feedback

[0055] The device instantly displays feedback, evidence, and suggestions based on learning results received from the server to the user, allowing users to always advance discussions based on the latest information.

[0056] User Roles and Experience

[0057] 1. Participating in discussions and providing evidence

[0058] Users can participate in discussions through their terminals, input their opinions and questions, and, if necessary, submit their own evidence to the server to use as the basis for the discussion.

[0059] 2. Interpretation and Use of Graphical Information

[0060] Users can refer to the graphical information displayed on their devices to understand the flow of discussions and key points, thereby clarifying issues and problems and providing a foundation for efficient and fair decision-making.

[0061] Specific examples

[0062] For example, consider a discussion about how to promote a new product. If User A types, "Let's start a discussion about how to promote the new product," the server receives this statement and analyzes it using NLP. At the same time, if User B types, "We should use social media for the new product. Past campaigns have been successful," the server collects evidence about the effectiveness of social media and evaluates its reliability. Based on this, illustrated information is displayed on the device, making it easier for users to visually grasp the flow of the discussion.

[0063] Furthermore, by learning from past discussion data, the server can generate suggestions such as "In past successes, social media campaigns have been effective for specific target groups," and display them to the user via their device. This allows users to engage in evidence-based, more logical, and constructive discussions.

[0064] In this way, the present invention supports evidence-based, logical, and constructive discussions in real time, promoting fair and transparent communication.

[0065] The processing flow will be explained below.

[0066] Step 1:

[0067] The user inputs a statement or question through the terminal interface, for example, "Let's start a discussion about how to promote our new product."

[0068] Step 2:

[0069] The terminal receives the user's input and transmits the utterance data to the server, which includes the text information of the utterance.

[0070] Step 3:

[0071] The server analyzes the received speech data in real time, using natural language processing (NLP) techniques to identify the content of the speech and extract themes and keywords.

[0072] Step 4:

[0073] Based on the extracted comments, the server collects relevant evidence from internal databases or external sources, for example, looking for data on the effectiveness of "social media" promotions.

[0074] Step 5:

[0075] The server evaluates the reliability of the collected evidence, and based on the evaluation criteria, selects reliable evidence and uses it as the basis for discussion.

[0076] Step 6:

[0077] Based on the analysis results and collected evidence, the server generates data that illustrates the flow of discussion and the points at issue, including graphs and flowcharts.

[0078] Step 7:

[0079] The server transmits the generated visualization data to the terminal, which displays the received data on the user interface in real time.

[0080] Step 8:

[0081] Users can proceed with the discussion while referring to the visualized data and evidence information displayed on their terminals. If necessary, they can enter further comments or questions into their terminals and send them to the server.

[0082] Step 9:

[0083] The server re-analyzes any additional statements or questions, collects and evaluates new evidence as needed, and updates the visualization data in real time and sends it to the terminal.

[0084] Step 10:

[0085] The server inputs past discussion data into a machine learning model to learn new patterns and effective solutions, and generates suggestions for the current discussion based on the learning results.

[0086] Step 11:

[0087] The server sends the generated proposals and solutions to the terminal, which displays them on the user interface for the user to refer to.

[0088] Step 12:

[0089] The user considers the suggestions and solutions displayed on the terminal and continues the discussion. If necessary, the user can re-enter comments or questions and use the system repeatedly.

[0090] Example 1

[0091] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0092] Conventional discussion support systems lack sufficient analysis and support to support the progress of discussions in real time based on user comments. Furthermore, they lack the ability to collect evidence, evaluate its reliability, and visualize the flow and points of discussion, making it difficult for users to engage in evidence-based, logical, and fair discussions. Furthermore, they lack the ability to effectively extract new solutions and proposals from past discussion data and present them in real time.

[0093] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0094] In this invention, the server includes means for receiving user comment data and analyzing the comment content using natural language processing technology, means for collecting related evidential materials based on the comment content from an internal data management system or external information sources and evaluating the reliability of the collected evidential materials, means for generating data that visually displays the flow of discussion and points of contention based on the analyzed comment content and the collected evidential materials, and means for learning past discussion data using a machine learning model and automatically extracting and presenting new patterns and effective solutions. This enables users to engage in evidence-based, logical, and fair discussion in real time.

[0095] "User" refers to a person who uses the system to participate in discussions and enter their own opinions and questions.

[0096] "Speech data" refers to text data such as opinions and questions entered by users in discussions.

[0097] "Natural language processing technology" refers to technology that uses computers to analyze human language and make it easier to understand its content and intent.

[0098] "Evidence" refers to information or data that forms the basis of an argument, including data collected from internal databases or external sources.

[0099] A "data management system" refers to a system that can efficiently manage and search data stored within it.

[0100] "External information sources" refer to means or systems for obtaining information from external databases or the Internet.

[0101] "Assessing reliability" refers to the process of determining whether collected evidence is accurate.

[0102] "Data that visually displays the flow of discussion and key points" refers to data such as graphs and charts that clearly visualize the progress of the discussion and key points.

[0103] A "machine learning model" refers to an algorithm that learns patterns and regularities based on large amounts of data.

[0104] "New patterns and effective solutions" refers to new trends and problem-solving methods that are derived from past data.

[0105] "Presenting" refers to displaying information or suggestions to the user.

[0106] This invention is a system that supports discussions in which users participate in real time and promotes fair and logical discussions, and its main components are a server, terminals, and users.

[0107] The server receives user speech data and analyzes the content of the speech using natural language processing technology. Specifically, it streams data using Apache Kafka and analyzes it using Google Cloud Natural Language API. By identifying and classifying the content and intent of the speech, the topic of the discussion is understood.

[0108] Based on the analysis results, the server collects relevant evidence from internal data management systems (e.g., MySQL®) and external sources (e.g., academic paper databases). The collected evidence is then evaluated for its reliability using machine learning libraries such as Scikit-learn. This process ensures that only highly reliable information is selected as the basis for discussion.

[0109] Next, the server generates data that visually displays the flow of the discussion and the points at issue based on the analysis results and collected evidence, using D3.js to generate graphs, flowcharts, mind maps, etc. The generated visualization data is then sent to the device via WebSocket.

[0110] The server also uses machine learning models (e.g., TENSORFLOW®) to learn from past discussion data and extract new patterns and effective solutions. Based on this, the server generates proposals and solutions that are presented to users in real time.

[0111] The terminal provides a user interface through which users can input comments and questions. The terminal also displays visualized data and supporting documents sent from the server, allowing users to check the progress of the discussion in real time. Users can participate in the discussion through the terminal and input their own opinions and questions. For example, users can input comments such as "We will start a discussion on how to promote our new product," or specific suggestions such as "We should utilize social media for our new product. Past campaigns have been successful."

[0112] In this system, the server performs analysis and evidence collection, evaluates reliability, generates visualized data, and extracts suggestions, and the terminal displays these to the user, allowing the user to engage in evidence-based, logical, and fair discussion in real time.

[0113] Example prompt: "We're opening a discussion about how to promote a new product. Please provide suggestions based on past successes and reliable evidence."

[0114] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0115] Step 1:

[0116] The user inputs a statement.

[0117] The user enters "Let's start a discussion on how to promote our new product" into the input field of the terminal. The input speech data is in text format, and the terminal sends this data to the server via WebSocket.

[0118] Input: The text of the speech entered by the user into the terminal.

[0119] Output: Speech data sent to the server via WebSocket

[0120] Step 2:

[0121] The server receives the speech data and analyzes it using natural language processing technology.

[0122] The server uses Apache Kafka to receive data sent from the devices in real time. The received speech data is analyzed using the Google Cloud Natural Language API to identify the content and intent of the speech. For example, keywords such as "new product" and "promotion method" are extracted to understand the topic of the discussion.

[0123] Input: Speech data sent from the device

[0124] Output: Parsed keywords and themes

[0125] Step 3:

[0126] The server collects the evidence and evaluates its reliability.

[0127] Based on the analyzed themes, the server collects relevant evidence from internal data management systems and external sources (e.g., academic paper databases). The collected evidence is evaluated for credibility using Scikit-learn, for example, by measuring the number of citations of a paper.

[0128] Input: Parsed keywords and themes

[0129] Output: Evidence with reliability evaluation

[0130] Step 4:

[0131] The server generates data that visualizes the flow of discussion and key points.

[0132] The server uses D3.js to generate visualizations such as graphs and flowcharts based on the analysis results and supporting documentation. For example, a bar graph showing the success rate of past social media campaigns is created. The generated data is sent to the device via WebSocket.

[0133] Input: Analysis results and evidence that has been evaluated for reliability

[0134] Output: The generated visualization data

[0135] Step 5:

[0136] The device displays visualization data and feedback in real time.

[0137] The device then displays the received visualization data on a dashboard, allowing users to see the progress of the discussion in real time, including a bar graph showing the success rate and related supporting documents.

[0138] Input: Visualization data sent from the server

[0139] Output: Visualized data and evidence displayed on the device

[0140] Step 6:

[0141] The server learns from past discussion data and generates new proposals.

[0142] The server uses TensorFlow to learn from past discussion data and extract new patterns and effective solutions. For example, it generates suggestions based on past successes, such as "a social media campaign was effective for a specific target demographic." The generated suggestions are sent to the device.

[0143] Input: Past discussion data

[0144] Output: Extracted new proposals and solutions

[0145] Step 7:

[0146] The device displays new suggestions and feedback to the user.

[0147] The device displays new suggestions sent from the server to the user in real time. For example, specific suggestions such as "Examples of successful social media campaigns" are displayed on the dashboard. Based on this information, the user can make the next statement.

[0148] Input: New suggestions and feedback sent by the server

[0149] Output: suggestions and feedback displayed on the device

[0150] (Application example 1)

[0151] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0152] In today's business environment, discussions and meetings between store staff and customers must be conducted efficiently and logically. However, it is difficult to provide real-time discussion support or immediate evidence, making it difficult for users to quickly resolve the issues they face and make decisions. Furthermore, there is a lack of a way to visually grasp the flow of discussions and key points at issue, which can lead to overlooking key points for deriving optimal solutions. Furthermore, it is difficult to learn new patterns and effective solutions from past discussion data and apply them to the current situation.

[0153] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0154] In this invention, the server includes means for receiving user utterance data and analyzing the utterance content using natural language processing, means for collecting related evidence based on the utterance content from an internal database or external information source and evaluating its reliability, means for illustrating the flow of discussion and points of contention based on the analyzed utterance content and the collected evidence, means for learning from past discussion data and extracting and presenting new patterns and effective solutions, means for converting user utterances into text using speech recognition technology and transmitting the text to the server, and means for providing the analysis results and evidence to a terminal that visually displays them. This allows users to engage in fair and logical discussions in real time and enables efficient decision-making.

[0155] "User utterance data" refers to voice data or text data uttered by a user during a discussion or meeting.

[0156] "Natural language processing" is a set of techniques that allow computers to understand and analyze human language.

[0157] "Evidence" refers to information or data that serves as the basis for an argument.

[0158] An "internal database" is a collection of accumulated data managed within the system.

[0159] "External information sources" refers to information managed outside the system, such as materials on the Internet or external databases.

[0160] "Measures to assess reliability" are processes or techniques used to determine the accuracy or reliability of collected evidence.

[0161] "Visualization" is the visual representation of information or data.

[0162] The "flow of discussion" indicates the process of how the discussion progresses.

[0163] An "issue" is an important topic or focal point in a discussion.

[0164] "Past discussion data" refers to records and information relating to discussions that have taken place in the past.

[0165] "Means of extracting new patterns and effective solutions" refers to techniques and technologies for finding useful patterns and solutions from past data.

[0166] "Converting user speech into text using speech recognition technology" is the process of converting a user's speech into text data.

[0167] A "server" is a computer system that processes and analyzes data and provides services to clients over a network.

[0168] "Means for providing analysis results and evidence to a terminal that visually displays them" refers to the technology or method by which the server sends analysis results and evidence to a terminal and displays them visually.

[0169] "Providing a user interface to a terminal" means displaying a screen or interface on the terminal that allows the user to operate the system.

[0170] "Updating the visualized data in real time" means instantly updating the visualized information as the discussion progresses.

[0171] "Automatically generating proposals based on past data" means that the system automatically creates new proposals based on past data.

[0172] The system embodying this invention has the functions of recognizing and analyzing the comments of users participating in a discussion in real time, collecting and evaluating related evidence, and visualizing the flow of the discussion and the points at issue. The detailed configuration and operation of the system are described below.

[0173] 1. Program Generation

[0174] The server receives speech data from the user and analyzes it using natural language processing (NLP) technology. The user's speech is converted into text using speech recognition technology through the smart glasses and sent to the server. Based on the analyzed data, relevant evidence is then collected from an internal database and external sources and its reliability is evaluated. This operation is achieved by converting speech to text using the Google Speech Recognition API and sending the data to the server using the requests library.

[0175] 2. Processing Description

[0176] When the server receives the user's speech data, it first converts the speech data into text using speech recognition technology, then analyzes the text data using NLP technology to identify the content and intent of the speech, and based on this analysis, begins the process of collecting relevant evidence and assessing its reliability.

[0177] For example, if someone says, "Show me evidence about a new promotion method," the server will collect relevant evidence from internal databases and external information sources and evaluate its reliability, taking into account the source of the collected information, its past usage history, and other factors.

[0178] The flow of the discussion and the points at issue are then visually illustrated based on the analyzed comments and collected evidence. This visualization is done using mind maps, flowcharts, and other tools. This visual data is displayed on the smart glasses, allowing users to understand the progress of the discussion in real time.

[0179] 3. Specific Examples

[0180] Specifically, when a user says, "We'll start discussing how to promote our new product," the server recognizes this speech and converts it into text. Next, when User B says, "We should utilize social media for our new product. Past campaigns have been successful," the server collects evidence about the effectiveness of social media and evaluates its reliability. This allows reliable information to be collected and displayed on the smart glasses.

[0181] In addition, by learning from past discussion data, the server generates suggestions such as "In past success stories, campaigns using social media have been effective for specific target groups," and displays these to the user.

[0182] Examples of prompts include "Show evidence of a new promotional method," "Tell me examples of successful campaigns from the past," and "Diagram the flow of the discussion."

[0183] By implementing this invention, meetings and discussions in physical stores can be conducted more efficiently and logically, enabling fair decision-making.

[0184] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0185] Step 1: Convert user speech into text using speech recognition technology

[0186] When a user speaks through the smart glasses, their voice is collected by the smart glasses' microphone, and this voice data is converted into text using the Google Speech Recognition API.

[0187] Input: User's voice

[0188] Output: Textual speech data

[0189] How it works: When a user says, "We'll start a discussion about new promotional methods," the microphone in the smart glasses collects voice data and converts it into text data using the Google Speech Recognition API.

[0190] Step 2: Send the textual utterance data to the server

[0191] The text data is sent from the smart glasses to a server via the Internet as an HTTP request using the requests library.

[0192] Input: Textual speech data

[0193] Output: Speech data sent to the server

[0194] Specific operation: The user's textual utterance data is sent from the smart glasses to the server. For example, the user's utterance "We will start discussing new promotion methods" is sent to the server as text data.

[0195] Step 3: Analyze the speech data using natural language processing

[0196] The server analyzes the received speech data using natural language processing (NLP) to identify the content and intent of the speech, and understands the topic of the discussion based on the results of this analysis.

[0197] Input: Speech data sent to the server

[0198] Output: Parsed speech and themes

[0199] Specific operation: The server analyzes the statement "We will start a discussion about new promotion methods" using an NLP engine and understands that this statement means to start a discussion about promotion.

[0200] Step 4: Collect relevant evidence and assess its reliability

[0201] Based on the analysis results, the server collects relevant evidence from internal databases and external sources, evaluates the reliability of that evidence, and selects only reliable evidence.

[0202] Input: Parsed speech and themes

[0203] Output: Evidence assessed for reliability

[0204] What it does: The server collects promotion-related statistics and past success stories from internal databases and publicly available internet data, and rates the reliability of each source.

[0205] Step 5: Diagram the flow of discussion and points

[0206] Based on the collected evidence and analysis results, the server generates mind maps and flowcharts to visualize the flow of discussion and key points.

[0207] Input: Evidence and analytical results assessed for reliability

[0208] Output: Diagram of discussion flow and points

[0209] Specific operation: Based on the evidence, the server visualizes the structure of the discussion as a mind map and generates data to be displayed on the smart glasses.

[0210] Step 6: Providing visualized data to smart glasses

[0211] The visualized flow of discussion and points of contention are transmitted from the server to the smart glasses and displayed to the user in real time.

[0212] Input: Diagram of discussion flow and points

[0213] Output: Visual data displayed on smart glasses

[0214] Specific operation: The server sends the generated mind maps and flowcharts to the smart glasses, allowing the user to view them in real time.

[0215] Step 7: Learn from past discussion data and generate new proposals

[0216] The server inputs past discussion data into a machine learning model to learn new patterns and effective solutions, and automatically generates suggestions for the current discussion and provides them to the smart glasses.

[0217] Input: Past discussion data and current discussion status

[0218] Output: Auto-generated suggestions

[0219] Specific operation: By learning data from past successful campaigns, the system generates suggestions for the current discussion, such as "In past successful cases, campaigns using social media were effective," and displays these suggestions on the smart glasses.

[0220] Through these steps, this system efficiently and logically supports discussions and meetings in physical stores, enabling rapid decision-making.

[0221] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0222] This invention relates to a system that supports discussions in real time and promotes fair and logical discussions in which users participate. In particular, the invention aims to further improve the quality of discussions by combining an emotion engine that recognizes users' emotions and supports discussions based on those emotions.

[0223] Server Roles and Functions

[0224] 1. Receiving and analyzing user speech data

[0225] The server receives user speech data sent from the device in real time. It analyzes the received speech data using natural language processing (NLP) and emotion recognition technology to identify the content of the speech and the user's emotions. This allows it to understand the topic of the discussion and the emotions of the user.

[0226] 2. Evidence collection and reliability assessment

[0227] The server collects relevant evidence from an internal database or external sources based on the analyzed content and sentiment, evaluates the reliability of the collected evidence, and selects appropriate evidence as the basis for discussion. It also selects specific evidence based on sentiment.

[0228] 3. Visualization Generation and Presentation

[0229] The server generates data that illustrates the flow of the discussion and the key points at issue based on the analysis results and collected evidence. It also includes the user's emotional information in the illustrated data and displays it visually. The generated illustration is sent to the device, allowing the user to view it in real time.

[0230] 4. Learning from past discussion data and making suggestions

[0231] The server inputs past discussion data into a machine learning model to learn new patterns and effective solutions, and then automatically generates proposals and solutions for the current discussion and sends them to the device.

[0232] Role and Function of the Emotion Engine

[0233] 1. Emotion Recognition from Speech

[0234] The emotion engine analyzes user comment data and recognizes the emotions contained in the comments. This emotion data is used as important information when the server analyzes and supports discussions.

[0235] 2. Emotion-based evidence filtering

[0236] The emotion engine selects appropriate evidence and information based on the recognized emotion. For example, if the user is feeling anxious, it will prioritize collecting and presenting evidence that will alleviate that emotion.

[0237] 3. Visualization of emotional information

[0238] The emotion engine generates data to visually display the recognized emotion information, including the type of emotion (e.g., joy, anger, anxiety, etc.) and intensity. The generated emotion data is displayed on the device along with the flow and points of the discussion.

[0239] Device roles and functions

[0240] 1. Providing a user interface

[0241] The terminal provides a user interface to users. Through this interface, users can input comments and questions and send them to the server. The terminal also displays the visualized data, evidence information, and emotion information sent from the server, allowing users to check the progress of the discussion in real time.

[0242] 2. Viewing Feedback

[0243] The device instantly displays feedback and evidence received from the server, suggestions based on learning results, and emotional information to the user, allowing users to always advance discussions based on the latest information.

[0244] User Roles and Experience

[0245] 1. Participating in discussions and providing emotional information

[0246] Users can participate in discussions through their devices and input their opinions and questions. The emotion engine recognizes the emotions contained in the comments and sends them to the server. This enables the server to support discussions while taking into account the users' emotions.

[0247] 2. Interpretation and Use of Graphical Information

[0248] Users can refer to the graphical information and emotional information displayed on the device to understand the flow of discussion and key points, thereby clarifying issues and problems and providing a foundation for efficient and fair decision-making.

[0249] Specific examples

[0250] For example, consider a discussion about how to promote a new product. If User A types, "Let's start a discussion about how to promote our new product," the server receives this statement and analyzes it using NLP and an emotion engine. At the same time, if User B types, "We should use social media for our new product. Past campaigns have been successful," and expresses some uncertainty, the server will collect evidence about the effectiveness of social media and evaluate its reliability. Furthermore, because the emotion engine recognizes this uncertainty, highly reliable evidence is prioritized.

[0251] Based on this, illustrated information is displayed on the device, making it easier for users to visually grasp the flow of the discussion. In addition, emotional information is also displayed, so other participants can proceed with the discussion while taking into consideration User B's emotions.

[0252] Furthermore, by learning from past discussion data, the server can generate suggestions such as "In past successes, social media campaigns have been effective for specific target groups," and display them to the user via their device. This allows users to engage in evidence-based, more logical, and constructive discussions.

[0253] In this way, the present invention is a system that supports evidence-based, logical, and constructive discussions in real time, and furthermore, enables discussions that take into account the user's emotions using an emotion engine, thereby promoting fair and transparent communication.

[0254] The processing flow will be explained below.

[0255] Step 1:

[0256] The user inputs a statement or question through the terminal interface, for example, "Let's start a discussion about how to promote our new product."

[0257] Step 2:

[0258] The terminal receives the user's input and transmits the utterance data to the server, which includes the text information of the utterance.

[0259] Step 3:

[0260] The server analyzes the received speech data in real time, using natural language processing (NLP) techniques to identify the content of the speech and extract themes and keywords.

[0261] Step 4:

[0262] The server recognizes emotions from user utterances using an emotion engine, which identifies emotions (e.g., joy, anger, anxiety, etc.) contained in the utterances.

[0263] Step 5:

[0264] Based on the analyzed content and perceived sentiment, the server collects relevant evidence from internal databases or external sources, for example, looking for data on the effectiveness of "social media" promotions.

[0265] Step 6:

[0266] The server evaluates the reliability of the collected evidence. Based on the evaluation criteria, it selects reliable evidence and uses it as the basis for discussion. Based on emotional information, it prioritizes and selects appropriate evidence for a specific emotion.

[0267] Step 7:

[0268] Based on the analysis results, recognized emotions, and collected evidence, the server generates data that illustrates the flow of discussion and points of contention, including graphs, flowcharts, and emotional information.

[0269] Step 8:

[0270] The server transmits the generated visualization data to the terminal, which displays the received data on the user interface in real time.

[0271] Step 9:

[0272] Users can proceed with the discussion while referring to the graphical data, evidence information, and emotion information displayed on their terminals. If necessary, they can input further comments or questions into their terminals and send them to the server.

[0273] Step 10:

[0274] The server re-analyzes any additional statements or questions, collects and evaluates new evidence as needed, and sends updated graphical data and emotional information to the device in real time.

[0275] Step 11:

[0276] The server inputs past discussion data into a machine learning model to learn new patterns and effective solutions, and generates suggestions for the current discussion based on the learning results.

[0277] Step 12:

[0278] The server sends the generated proposals and solutions to the terminal, which displays them on the user interface for the user to refer to.

[0279] Step 13:

[0280] The user considers the suggestions and solutions displayed on the terminal and continues the discussion. If necessary, the user can re-enter comments or questions and use the system repeatedly.

[0281] Step 14:

[0282] As the discussion progresses, the server continuously recognizes the user's emotions using an emotion engine and adjusts the discussion support accordingly based on that information. This helps to ensure an efficient discussion while maintaining a balance between emotions and evidence.

[0283] Example 2

[0284] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0285] Conventional discussion support systems only analyze the content of user comments and do not take into account user emotions, making it difficult to hold fair and logical discussions. Furthermore, they often lack sufficient visualization to visually grasp the progress of the discussion, and do not properly evaluate the reliability of evidence. Furthermore, past discussion data is not effectively utilized, limiting the ability to extract new patterns and solutions. There is a need for a system that can solve these problems and improve the quality of discussions.

[0286] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0287] In this invention, the server includes means for receiving user utterance data and analyzing the utterance content and emotions using natural language processing, means for collecting related evidence from an internal database or external information sources based on the analyzed utterance content and emotions and evaluating their reliability, means for illustrating the flow of the discussion, issues, and emotional information based on the analysis results and the collected evidence, and means for learning from past discussion data and extracting and presenting new patterns and effective solutions. This makes it possible to support fair and logical discussions that take user emotions into consideration, and by visually understanding the progress of the discussion, it is possible to improve the quality of the discussion.

[0288] "User comment data" refers to text data of discussions and questions that users input into their terminals and send to the server.

[0289] "Natural language processing" refers to techniques and methods that allow computers to understand, interpret, and generate human language.

[0290] "Emotion analysis" refers to the process of identifying emotions (e.g., joy, anger, anxiety, etc.) contained in user speech data.

[0291] "Evidence" refers to the data, examples, and information used as the basis for an argument.

[0292] An "internal database" is a type of data storage for saving data used within the system.

[0293] "External information source" is a general term for a location that can be accessed from outside the system to obtain information.

[0294] "Assessing reliability" is the process of determining the accuracy and validity of the evidence collected.

[0295] "Graphing" is the process of presenting information in a visual form, such as a graph or chart.

[0296] "Past discussion data" refers to the accumulation of text data and analysis results from previous discussions.

[0297] "Extracting new patterns and effective solutions" refers to the process of finding new ways and strategies to advance discussions efficiently and effectively through the analysis of past data.

[0298] This invention is a system that supports discussions in which users participate in real time and promotes fair and logical discussions. In particular, it aims to further improve the quality of discussions by combining it with an emotion engine that recognizes users' emotions and supports discussions based on those emotions. The detailed configuration and operation of this system are described below.

[0299] Server Roles and Functions

[0300] Receiving and analyzing user speech data

[0301] The server receives user speech data sent from the device in real time. Specifically, data communication is performed using WebSocket or HTTP protocols. The received speech data is analyzed using natural language processing (NLP) technology (e.g., Google Cloud Natural Language API) and emotion recognition technology (e.g., IBM Watson (registered trademark) Tone Analyzer). This allows the server to understand the topic of the discussion and the emotions of the users.

[0302] Evidence collection and reliability assessment

[0303] The server collects relevant evidence from an internal database or external sources (e.g., Google Scholar, Web of Science) based on the analyzed content and sentiment of comments. The collected evidence is evaluated for reliability using machine learning algorithms (e.g., Random Forest, Support Vector Machine), allowing appropriate evidence to be selected as the basis for discussion.

[0304] Visualization generation and presentation

[0305] The server generates data that illustrates the flow of discussion and key points based on the analysis results and collected evidence. Specifically, it uses D3.js and Chart.js to create visual diagrams. The diagrams also include the user's emotional information and send it to the device, allowing the user to view it in real time.

[0306] Learning from past discussion data and making suggestions

[0307] The server uses TensorFlow and PyTorch to input past discussion data into machine learning models to learn new patterns and effective solutions, and then automatically generates proposals and solutions for the current discussion and sends them to the device.

[0308] Role and Function of the Emotion Engine

[0309] Emotion recognition from speech

[0310] The emotion engine analyzes user speech data and recognizes the emotions contained in the speech. This emotion data is used as important information when the server analyzes and supports discussions. The technology used is an NLP toolkit (e.g., nltk, spacy) or an emotion analysis API (e.g., Microsoft® Azure® Text Analytics).

[0311] Emotion-based evidence selection

[0312] The emotion engine selects appropriate evidence and information based on the recognized emotion. For example, if the user is feeling anxious, it will prioritize collecting and presenting evidence that will alleviate that emotion.

[0313] Visualization of emotional information

[0314] The emotion engine generates data to visually display the recognized emotion information, including the type of emotion (e.g., joy, anger, anxiety, etc.) and intensity. This data is displayed on the device along with the visualization data generated by the server.

[0315] Device roles and functions

[0316] Providing a user interface

[0317] The terminal provides a user interface to users. Through this interface, users can input comments and questions and send them to the server. The terminal also displays the visualized data, evidence information, and emotion information sent from the server, allowing users to check the progress of the discussion in real time. Technologies used include React.js and Vue.js.

[0318] View Feedback

[0319] The device instantly displays feedback, evidence, suggestions based on learning results, and emotional information received from the server to the user, allowing users to always advance discussions based on the latest information.

[0320] User Roles and Experience

[0321] Participating in discussions and providing emotional information

[0322] Users can participate in discussions through their devices and input their opinions and questions. The emotion engine recognizes the emotions contained in the comments and sends them to the server. This enables the server to support discussions while taking into account the users' emotions.

[0323] Interpreting and using graphical information

[0324] Users can refer to the graphical information and emotional information displayed on the device to understand the flow of discussion and key points, thereby clarifying issues and problems and providing a foundation for efficient and fair decision-making.

[0325] Examples of concrete examples and prompts

[0326] For example, consider a discussion about how to promote a new product. If User A types, "Let's start a discussion about how to promote our new product," the server receives this statement and analyzes it using NLP and an emotion engine. At the same time, if User B types, "We should use social media for our new product. Past campaigns have been successful," and expresses some uncertainty, the server will collect evidence about the effectiveness of social media and evaluate its reliability. Furthermore, because the emotion engine recognizes this uncertainty, highly reliable evidence is prioritized.

[0327] Based on this, illustrated information is displayed on the device, making it easier for users to visually grasp the flow of the discussion. Emotional information is also displayed, allowing other participants to proceed with the discussion while taking User B's emotions into consideration. Furthermore, by learning from past discussion data, the server generates suggestions such as "In past success stories, social media campaigns have been effective for specific target groups," and displays these to the user via their device. This allows users to engage in evidence-based, more logical, and constructive discussions.

[0328] Prompt Sentence Examples

[0329] "Start a discussion about how to promote the new product. Gather evidence of past successes and effective methods, and consider user sentiment as you proceed with the discussion."

[0330] In this way, the present invention is a system that supports evidence-based, logical, and constructive discussions in real time, and furthermore, enables discussions that take into account the user's emotions using an emotion engine, thereby promoting fair and transparent communication.

[0331] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0332] Step 1:

[0333] Users input comments or questions through the device's user interface. For example, the input can be text data such as "Let's start a discussion about how to promote a new product." The device captures this comment data and sends it to the server using WebSocket or HTTP protocol. The server receives this comment data.

[0334] Step 2:

[0335] The server analyzes the received speech data. The input is the user's speech data, which is sent to natural language processing (NLP) technology (e.g., Google Cloud Natural Language API) to analyze the speech content. At the same time, emotion is analyzed using emotion recognition technology (e.g., IBM Watson Tone Analyzer). The output is the speech theme, keywords, and emotional information.

[0336] Step 3:

[0337] The server collects relevant evidence based on the analyzed speech content and sentiment. The input is the analysis results, and based on this, data is collected from internal databases and external sources (e.g., Google Scholar, Web of Science). The collected evidence is evaluated for reliability using a machine learning algorithm (e.g., Random Forest). Highly reliable evidence is selected as the output.

[0338] Step 4:

[0339] The server generates data that visualizes the flow of discussion, key points, and sentiment information based on the analysis results and collected evidence. The input is reliable evidence and analysis results. Visualization tools such as D3.js and Chart.js are used for visualization. Visual visualization data is generated as output and sent to the device.

[0340] Step 5:

[0341] The server inputs past discussion data into a machine learning model (e.g., TensorFlow, PyTorch) to learn new patterns and effective solutions. The input is past discussion data, and learning is based on this. As an output, new patterns and proposals are generated and sent to the device.

[0342] Step 6:

[0343] The terminal receives graphical data, evidence, suggestions, and emotion information from the server and displays them in real time on the user interface. The input is data sent from the server. This allows users to visually grasp the progress of the discussion and evidence, and receive feedback in real time.

[0344] Step 7:

[0345] The user then looks at the device's interface and types in a new comment or question. This input includes new comments or questions based on suggestions from the server. The device captures this input and sends it back to the server, repeating the cycle and moving the discussion forward.

[0346] (Application example 2)

[0347] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0348] Real-time customer service and customer support in virtual stores requires advanced technology. It is particularly difficult to accurately recognize customer emotions and instantly provide suggestions and evidence based on those emotions. Therefore, systems are needed to improve the quality of customer service. Furthermore, there is a lack of interfaces to facilitate smooth communication between customers and staff, and this needs to be resolved.

[0349] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for receiving utterance data from users and analyzing the utterance content using natural language processing; means for collecting related evidence from an internal database or external information source based on the utterance content and evaluating its reliability; means for illustrating the flow of discussion and points of contention based on the analyzed utterance content and the collected evidence; means for learning from past discussion data and extracting and presenting new patterns and effective solutions; means for recognizing emotions contained in utterances and selecting and displaying appropriate evidence based on the emotions; and means for making customer service suggestions based on emotions and supporting customer service in real time. This enables real-time customer service support that takes customer emotions into consideration in a virtual store.

[0350] A "user" is an entity that provides and receives information through the system, such as a customer who uses a service or a customer service staff member.

[0351] "Utterance data" refers to text information and voice data provided by user input, which is used by the system for analysis.

[0352] "Natural language processing" is a technology that allows computers to understand and analyze human language, and is used to analyze the content of user utterances.

[0353] "Evidence" refers to relevant information and supporting documents provided based on the content and intent of a user's statements, and provides a basis for supporting the discussion.

[0354] An "internal database" is a collection of information held within the system and is a database used to collect evidence related to the content of statements.

[0355] "External sources" are sources for collecting information from outside the system, such as the Internet or other external databases.

[0356] "Credibility assessment" is the process of evaluating the accuracy and veracity of collected evidence, and is an evaluation carried out to select highly reliable information.

[0357] "Visualization" is the process of visually displaying analyzed data and evidence, presenting information to users in an easy-to-understand manner.

[0358] "Past discussion data" refers to discussions that have taken place in the past and their records, and is data that can be learned to derive new patterns and solutions.

[0359] "Emotion recognition" is a technology that analyzes and recognizes the emotions contained in a user's speech, and is used to understand the user's psychological state.

[0360] "Customer service suggestions" is the process of presenting customer service methods and advice based on recognized emotions and statements.

[0361] "Real-time" refers to the process of receiving, analyzing, and presenting data almost simultaneously, providing an immediate response without delay.

[0362] The present invention relates to a real-time customer support system for a virtual store. This system receives utterance data entered by users (customers or wait staff) and analyzes it using natural language processing. It also collects relevant evidence based on the content of the utterances and evaluates its reliability. Furthermore, by combining this with emotion recognition technology, it provides responses and suggestions that take the user's emotions into consideration. Below, an embodiment of the present invention will be described in detail.

[0363] Program Overview

[0364] Server Roles and Functions

[0365] The server performs the following functions:

[0366] 1. Receiving and analyzing user speech data

[0367] Speech data sent by users is received in real time and analyzed using natural language processing (NLP) technology. Software used for this includes NLP libraries such as SpaCy. Keywords are extracted from the analyzed speech content to identify the content of the speech.

[0368] 2. Recognizing emotions in speech

[0369] Using an emotion recognition library such as TextBlob, we analyze the emotions contained in the user's speech, which allows us to identify the emotions the user is feeling (e.g., joy, anger, anxiety, etc.).

[0370] 3. Evidence collection and reliability assessment

[0371] Based on the extracted keywords and recognized sentiment, relevant evidence is collected from internal databases or external sources, and the reliability of the evidence is evaluated to select reliable information.

[0372] 4. Visualization Generation and Presentation

[0373] Based on the analysis results and collected evidence, the flow of discussion and points of contention are visualized. NetworkX is used to visually display the relationships and flow of data, and the server sends the generated visualized data to the terminal.

[0374] Device roles and functions

[0375] The terminal performs the following functions:

[0376] 1. Providing a user interface

[0377] The terminal provides a user interface, allowing users to input comments and questions, and displays the visualized data and evidence information sent from the server in real time, allowing users to check the progress of the discussion.

[0378] 2. Viewing Feedback

[0379] The device instantly displays feedback, evidence, suggestions based on learning results, and emotional information received from the server to the user, allowing users to always advance discussions based on the latest information.

[0380] User Roles and Experience

[0381] Users can use the system as follows:

[0382] 1. Participating in discussions and providing emotional information

[0383] Users can participate in discussions through their devices and input their opinions and questions. The emotion engine recognizes the emotions contained in the comments and sends them to the server. This allows the server to support discussions while taking into account the users' emotions.

[0384] 2. Interpretation and Use of Graphical Information

[0385] Users can refer to the graphical information and emotional information displayed on their devices to understand the flow of the discussion and key points at issue, which allows them to handle customer inquiries efficiently and fairly.

[0386] Specific examples

[0387] For example, if a customer types, "Tell me about recent campaigns," the system analyzes this in real time and provides relevant campaign information and evidence. If emotion recognition indicates that the customer has a positive emotion, the system will provide additional information or suggestions that will further enhance that emotion.

[0388] Example prompt sentence:

[0389] Customer message: "Tell me about your latest promotions."

[0390] Analyzed Keywords: Campaign

[0391] Emotion: Positive

[0392] Recommended evidence: "Our latest 20% discount offer is running now. Plus, there are additional perks for new users."

[0393] In this way, the system can respond immediately to customer demands and emotions and provide high-quality service.

[0394] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0395] Step 1:

[0396] The server receives user utterance data. The input is the user's utterance (text or audio data). The output is the received utterance data. This data is then analyzed in the next processing step.

[0397] Step 2:

[0398] The server analyzes the received utterance data. Specifically, it uses natural language processing (NLP) technology to analyze the content of the utterance. The software used is an NLP library such as SpaCy. It receives the utterance data as input and extracts keywords and the grammatical structure of the sentences. The analyzed keywords and structural data of the utterance content are obtained as output.

[0399] Step 3:

[0400] The server uses emotion recognition technology to recognize emotions from the analyzed utterance data. The emotion recognition library used is TextBlob. As input, it receives the analyzed utterance data and identifies the user's emotion (e.g., positive, negative, neutral). As output, it obtains the recognized emotion data.

[0401] Step 4:

[0402] The server collects relevant evidence based on the analyzed utterance data and recognized emotions from an internal database or external sources, using the internet or external APIs as needed. It receives keywords and emotion data as input, and searches for and collects relevant evidence. The collected evidence data is obtained as output.

[0403] Step 5:

[0404] The server evaluates the reliability of the collected evidence. Specifically, it uses the source of the evidence, past performance, etc. as evaluation criteria. It receives the collected evidence data as input and scores or ranks its reliability. The output is the evidence data that has undergone reliability evaluation.

[0405] Step 6:

[0406] The server visualizes the flow of discussion and points of contention based on the analysis results and collected evidence. The software used is NetworkX. It receives analysis data and evidence data as input and generates graphical data to display them visually. The graphical data is obtained as output.

[0407] Step 7:

[0408] The server uses machine learning models to learn from past discussion data and extract new patterns and effective solutions. It receives past discussion data as input and extracts new patterns for the current discussion and future. The output is proposal data.

[0409] Step 8:

[0410] The terminal provides a user interface, allowing users to input comments in real time and displaying graphical data and evidence information sent from the server. As input, the terminal receives graphical data and evidence information from the server and displays them to the user. As output, the terminal provides an interface that allows users to input new comments and questions.

[0411] Step 9:

[0412] The terminal immediately displays the feedback, evidence, and emotion information received from the server to the user. As input, it receives feedback data, evidence, and emotion data from the server and displays them on the user interface. As output, it displays a display that allows users to advance the discussion based on the latest information.

[0413] Step 10:

[0414] Users participate in discussions through their terminals and input their opinions and questions. As input, speech data is provided to the terminal and sent to the server. As output, emotions are analyzed by the emotion engine and sent to the server.

[0415] Step 11:

[0416] Users refer to the graphical data and emotional information displayed on their devices to understand the flow of the discussion and key points. As input, they receive the graphical data and emotional information displayed on their devices and advance the discussion. As output, more logical and constructive discussions based on evidence become possible.

[0417] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0418] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0419] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0420] [Second embodiment]

[0421] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0422] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0423] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0424] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0425] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0426] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0427] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0428] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0429] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0430] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0431] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0432] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0433] This invention relates to a system that supports discussions in real time in which users participate, and promotes fair and logical discussions. The main components of the system are a server, terminals, and users, each of which performs a specific function.

[0434] Server Roles and Functions

[0435] 1. Receiving and analyzing user speech data

[0436] The server receives user speech data sent from the device in real time. It then analyzes the received speech data using natural language processing (NLP) technology to identify and classify the content and intent of the speech. This allows it to understand the topic of the discussion and lead to the collection of appropriate evidence.

[0437] 2. Evidence collection and reliability assessment

[0438] The server collects relevant evidence from an internal database and external sources based on the analyzed comments, evaluates the reliability of the collected evidence, and selects appropriate evidence to support the discussion. This allows users to advance discussions based on well-founded information.

[0439] 3. Visualization Generation and Presentation

[0440] The server generates data that illustrates the flow of discussion and points of contention based on the analysis results and collected evidence. This includes visual formats such as graphs, flowcharts, and mind maps. The generated illustrations are sent to the terminals so that users can view them in real time.

[0441] 4. Learning from past discussion data and making suggestions

[0442] The server inputs past discussion data into a machine learning model to learn new patterns and effective solutions, and then automatically generates proposals and solutions for the current discussion and sends them to the device.

[0443] Device roles and functions

[0444] 1. Providing a user interface

[0445] The terminal provides a user interface to users. Through this interface, users can input comments and questions and send them to the server. The terminal also displays the visualized data and evidence information sent from the server, allowing users to check the progress of the discussion in real time.

[0446] 2. Viewing Feedback

[0447] The device instantly displays feedback, evidence, and suggestions based on learning results received from the server to the user, allowing users to always advance discussions based on the latest information.

[0448] User Roles and Experience

[0449] 1. Participating in discussions and providing evidence

[0450] Users can participate in discussions through their terminals, input their opinions and questions, and, if necessary, submit their own evidence to the server to use as the basis for the discussion.

[0451] 2. Interpretation and Use of Graphical Information

[0452] Users can refer to the graphical information displayed on their devices to understand the flow of discussions and key points, thereby clarifying issues and problems and providing a foundation for efficient and fair decision-making.

[0453] Specific examples

[0454] For example, consider a discussion about how to promote a new product. If User A types, "Let's start a discussion about how to promote the new product," the server receives this statement and analyzes it using NLP. At the same time, if User B types, "We should use social media for the new product. Past campaigns have been successful," the server collects evidence about the effectiveness of social media and evaluates its reliability. Based on this, illustrated information is displayed on the device, making it easier for users to visually grasp the flow of the discussion.

[0455] Furthermore, by learning from past discussion data, the server can generate suggestions such as "In past successes, social media campaigns have been effective for specific target groups," and display them to the user via their device. This allows users to engage in evidence-based, more logical, and constructive discussions.

[0456] In this way, the present invention supports evidence-based, logical, and constructive discussions in real time, promoting fair and transparent communication.

[0457] The processing flow will be explained below.

[0458] Step 1:

[0459] The user inputs a statement or question through the terminal interface, for example, "Let's start a discussion about how to promote our new product."

[0460] Step 2:

[0461] The terminal receives the user's input and transmits the utterance data to the server, which includes the text information of the utterance.

[0462] Step 3:

[0463] The server analyzes the received speech data in real time, using natural language processing (NLP) techniques to identify the content of the speech and extract themes and keywords.

[0464] Step 4:

[0465] Based on the extracted comments, the server collects relevant evidence from internal databases or external sources, for example, looking for data on the effectiveness of "social media" promotions.

[0466] Step 5:

[0467] The server evaluates the reliability of the collected evidence, and based on the evaluation criteria, selects reliable evidence and uses it as the basis for discussion.

[0468] Step 6:

[0469] Based on the analysis results and collected evidence, the server generates data that illustrates the flow of discussion and the points at issue, including graphs and flowcharts.

[0470] Step 7:

[0471] The server transmits the generated visualization data to the terminal, which displays the received data on the user interface in real time.

[0472] Step 8:

[0473] Users can proceed with the discussion while referring to the visualized data and evidence information displayed on their terminals. If necessary, they can enter further comments or questions into their terminals and send them to the server.

[0474] Step 9:

[0475] The server re-analyzes any additional statements or questions, collects and evaluates new evidence as needed, and updates the visualization data in real time and sends it to the terminal.

[0476] Step 10:

[0477] The server inputs past discussion data into a machine learning model to learn new patterns and effective solutions, and generates suggestions for the current discussion based on the learning results.

[0478] Step 11:

[0479] The server sends the generated proposals and solutions to the terminal, which displays them on the user interface for the user to refer to.

[0480] Step 12:

[0481] The user considers the suggestions and solutions displayed on the terminal and continues the discussion. If necessary, the user can re-enter comments or questions and use the system repeatedly.

[0482] Example 1

[0483] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0484] Conventional discussion support systems lack sufficient analysis and support to support the progress of discussions in real time based on user comments. Furthermore, they lack the ability to collect evidence, evaluate its reliability, and visualize the flow and points of discussion, making it difficult for users to engage in evidence-based, logical, and fair discussions. Furthermore, they lack the ability to effectively extract new solutions and proposals from past discussion data and present them in real time.

[0485] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0486] In this invention, the server includes means for receiving user comment data and analyzing the comment content using natural language processing technology, means for collecting related evidential materials based on the comment content from an internal data management system or external information sources and evaluating the reliability of the collected evidential materials, means for generating data that visually displays the flow of discussion and points of contention based on the analyzed comment content and the collected evidential materials, and means for learning past discussion data using a machine learning model and automatically extracting and presenting new patterns and effective solutions. This enables users to engage in evidence-based, logical, and fair discussion in real time.

[0487] "User" refers to a person who uses the system to participate in discussions and enter their own opinions and questions.

[0488] "Speech data" refers to text data such as opinions and questions entered by users in discussions.

[0489] "Natural language processing technology" refers to technology that uses computers to analyze human language and make it easier to understand its content and intent.

[0490] "Evidence" refers to information or data that forms the basis of an argument, including data collected from internal databases or external sources.

[0491] A "data management system" refers to a system that can efficiently manage and search data stored within it.

[0492] "External information sources" refer to means or systems for obtaining information from external databases or the Internet.

[0493] "Assessing reliability" refers to the process of determining whether collected evidence is accurate.

[0494] "Data that visually displays the flow of discussion and key points" refers to data such as graphs and charts that clearly visualize the progress of the discussion and key points.

[0495] A "machine learning model" refers to an algorithm that learns patterns and regularities based on large amounts of data.

[0496] "New patterns and effective solutions" refers to new trends and problem-solving methods that are derived from past data.

[0497] "Presenting" refers to displaying information or suggestions to the user.

[0498] This invention is a system that supports discussions in which users participate in real time and promotes fair and logical discussions, and its main components are a server, terminals, and users.

[0499] The server receives user speech data and analyzes the content of the speech using natural language processing technology. Specifically, it streams the data using Apache Kafka and analyzes it using Google Cloud Natural Language API. By identifying and classifying the content and intent of the speech, it understands the topic of the discussion.

[0500] Based on the analysis results, the server collects relevant evidence from internal data management systems (e.g., MySQL) and external sources (e.g., academic paper databases). The collected evidence is then evaluated for its reliability using machine learning libraries such as Scikit-learn. This process ensures that only reliable information is selected as the basis for discussion.

[0501] Next, the server generates data that visually displays the flow of the discussion and the points at issue based on the analysis results and collected evidence, using D3.js to generate graphs, flowcharts, mind maps, etc. The generated visualization data is then sent to the device via WebSocket.

[0502] The server then uses machine learning models (e.g., TensorFlow) to learn from past discussion data and extract new patterns and effective solutions. Based on this, the server generates proposals and solutions that are presented to users in real time.

[0503] The terminal provides a user interface through which users can input comments and questions. The terminal also displays visualized data and supporting documents sent from the server, allowing users to check the progress of the discussion in real time. Users can participate in the discussion through the terminal and input their own opinions and questions. For example, users can input comments such as "We will start a discussion on how to promote our new product," or specific suggestions such as "We should utilize social media for our new product. Past campaigns have been successful."

[0504] In this system, the server performs analysis and evidence collection, evaluates reliability, generates visualized data, and extracts suggestions, and the terminal displays these to the user, allowing the user to engage in evidence-based, logical, and fair discussion in real time.

[0505] Example prompt: "We're opening a discussion about how to promote a new product. Please provide suggestions based on past successes and reliable evidence."

[0506] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0507] Step 1:

[0508] The user inputs a statement.

[0509] The user enters "Let's start a discussion on how to promote our new product" into the input field of the terminal. The input speech data is in text format, and the terminal sends this data to the server via WebSocket.

[0510] Input: The text of the speech entered by the user into the terminal.

[0511] Output: Speech data sent to the server via WebSocket

[0512] Step 2:

[0513] The server receives the speech data and analyzes it using natural language processing technology.

[0514] The server uses Apache Kafka to receive data sent from the devices in real time. The received speech data is analyzed using the Google Cloud Natural Language API to identify the content and intent of the speech. For example, keywords such as "new product" and "promotion method" are extracted to understand the topic of the discussion.

[0515] Input: Speech data sent from the device

[0516] Output: Parsed keywords and themes

[0517] Step 3:

[0518] The server collects the evidence and evaluates its reliability.

[0519] Based on the analyzed themes, the server collects relevant evidence from internal data management systems and external sources (e.g., academic paper databases). The collected evidence is evaluated for credibility using Scikit-learn, for example, by measuring the number of citations of a paper.

[0520] Input: Parsed keywords and themes

[0521] Output: Evidence with reliability evaluation

[0522] Step 4:

[0523] The server generates data that visualizes the flow of discussion and key points.

[0524] The server uses D3.js to generate visualizations such as graphs and flowcharts based on the analysis results and supporting documentation. For example, a bar graph showing the success rate of past social media campaigns is created. The generated data is sent to the device via WebSocket.

[0525] Input: Analysis results and evidence that has been evaluated for reliability

[0526] Output: The generated visualization data

[0527] Step 5:

[0528] The device displays visualization data and feedback in real time.

[0529] The device then displays the received visualization data on a dashboard, allowing users to see the progress of the discussion in real time, including a bar graph showing the success rate and related supporting documents.

[0530] Input: Visualization data sent from the server

[0531] Output: Visualized data and evidence displayed on the device

[0532] Step 6:

[0533] The server learns from past discussion data and generates new proposals.

[0534] The server uses TensorFlow to learn from past discussion data and extract new patterns and effective solutions. For example, it generates suggestions based on past successes, such as "a social media campaign was effective for a specific target demographic." The generated suggestions are sent to the device.

[0535] Input: Past discussion data

[0536] Output: Extracted new proposals and solutions

[0537] Step 7:

[0538] The device displays new suggestions and feedback to the user.

[0539] The device displays new suggestions sent from the server to the user in real time. For example, specific suggestions such as "Examples of successful social media campaigns" are displayed on the dashboard. Based on this information, the user can make the next statement.

[0540] Input: New suggestions and feedback sent by the server

[0541] Output: suggestions and feedback displayed on the device

[0542] (Application example 1)

[0543] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0544] In today's business environment, discussions and meetings between store staff and customers must be conducted efficiently and logically. However, it is difficult to provide real-time discussion support or immediate evidence, making it difficult for users to quickly resolve the issues they face and make decisions. Furthermore, there is a lack of a way to visually grasp the flow of discussions and key points at issue, which can lead to overlooking key points for deriving optimal solutions. Furthermore, it is difficult to learn new patterns and effective solutions from past discussion data and apply them to the current situation.

[0545] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0546] In this invention, the server includes means for receiving user utterance data and analyzing the utterance content using natural language processing, means for collecting related evidence based on the utterance content from an internal database or external information source and evaluating its reliability, means for illustrating the flow of discussion and points of contention based on the analyzed utterance content and the collected evidence, means for learning from past discussion data and extracting and presenting new patterns and effective solutions, means for converting user utterances into text using speech recognition technology and transmitting the text to the server, and means for providing the analysis results and evidence to a terminal that visually displays them. This allows users to engage in fair and logical discussions in real time and enables efficient decision-making.

[0547] "User utterance data" refers to voice data or text data uttered by a user during a discussion or meeting.

[0548] "Natural language processing" is a set of techniques that allow computers to understand and analyze human language.

[0549] "Evidence" refers to information or data that serves as the basis for an argument.

[0550] An "internal database" is a collection of accumulated data managed within the system.

[0551] "External information sources" refers to information managed outside the system, such as materials on the Internet or external databases.

[0552] "Measures to assess reliability" are processes or techniques used to determine the accuracy or reliability of collected evidence.

[0553] "Visualization" is the visual representation of information or data.

[0554] The "flow of discussion" indicates the process of how the discussion progresses.

[0555] An "issue" is an important topic or focal point in a discussion.

[0556] "Past discussion data" refers to records and information relating to discussions that have taken place in the past.

[0557] "Means of extracting new patterns and effective solutions" refers to techniques and technologies for finding useful patterns and solutions from past data.

[0558] "Converting user speech into text using speech recognition technology" is the process of converting a user's speech into text data.

[0559] A "server" is a computer system that processes and analyzes data and provides services to clients over a network.

[0560] "Means for providing analysis results and evidence to a terminal that visually displays them" refers to the technology or method by which the server sends analysis results and evidence to a terminal and displays them visually.

[0561] "Providing a user interface to a terminal" means displaying a screen or interface on the terminal that allows the user to operate the system.

[0562] "Updating the visualized data in real time" means instantly updating the visualized information as the discussion progresses.

[0563] "Automatically generating proposals based on past data" means that the system automatically creates new proposals based on past data.

[0564] The system embodying this invention has the functions of recognizing and analyzing the comments of users participating in a discussion in real time, collecting and evaluating related evidence, and visualizing the flow of the discussion and the points at issue. The detailed configuration and operation of the system are described below.

[0565] 1. Program Generation

[0566] The server receives speech data from the user and analyzes it using natural language processing (NLP) technology. The user's speech is converted into text using speech recognition technology through the smart glasses and sent to the server. Based on the analyzed data, relevant evidence is then collected from an internal database and external sources and its reliability is evaluated. This operation is achieved by converting speech to text using the Google Speech Recognition API and sending the data to the server using the requests library.

[0567] 2. Processing Description

[0568] When the server receives the user's speech data, it first converts the speech data into text using speech recognition technology, then analyzes the text data using NLP technology to identify the content and intent of the speech, and based on this analysis, begins the process of collecting relevant evidence and assessing its reliability.

[0569] For example, if someone says, "Show me evidence about a new promotion method," the server will collect relevant evidence from internal databases and external information sources and evaluate its reliability, taking into account the source of the collected information, its past usage history, and other factors.

[0570] The flow of the discussion and the points at issue are then visually illustrated based on the analyzed comments and collected evidence. This visualization is done using mind maps, flowcharts, and other tools. This visual data is displayed on the smart glasses, allowing users to understand the progress of the discussion in real time.

[0571] 3. Specific Examples

[0572] Specifically, when a user says, "We'll start discussing how to promote our new product," the server recognizes this speech and converts it into text. Next, when User B says, "We should utilize social media for our new product. Past campaigns have been successful," the server collects evidence about the effectiveness of social media and evaluates its reliability. This allows reliable information to be collected and displayed on the smart glasses.

[0573] In addition, by learning from past discussion data, the server generates suggestions such as "In past success stories, campaigns using social media have been effective for specific target groups," and displays these to the user.

[0574] Examples of prompts include "Show evidence of a new promotional method," "Tell me examples of successful campaigns from the past," and "Diagram the flow of the discussion."

[0575] By implementing this invention, meetings and discussions in physical stores can be conducted more efficiently and logically, enabling fair decision-making.

[0576] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0577] Step 1: Convert user speech into text using speech recognition technology

[0578] When a user speaks through the smart glasses, their voice is collected by the smart glasses' microphone, and this voice data is converted into text using the Google Speech Recognition API.

[0579] Input: User's voice

[0580] Output: Textual speech data

[0581] How it works: When a user says, "We'll start a discussion about new promotional methods," the microphone in the smart glasses collects voice data and converts it into text data using the Google Speech Recognition API.

[0582] Step 2: Send the textual utterance data to the server

[0583] The text data is sent from the smart glasses to a server via the Internet as an HTTP request using the requests library.

[0584] Input: Textual speech data

[0585] Output: Speech data sent to the server

[0586] Specific operation: The user's textual utterance data is sent from the smart glasses to the server. For example, the user's utterance "We will start discussing new promotion methods" is sent to the server as text data.

[0587] Step 3: Analyze the speech data using natural language processing

[0588] The server analyzes the received speech data using natural language processing (NLP) to identify the content and intent of the speech, and understands the topic of the discussion based on the results of this analysis.

[0589] Input: Speech data sent to the server

[0590] Output: Parsed speech and themes

[0591] Specific operation: The server analyzes the statement "We will start a discussion about new promotion methods" using an NLP engine and understands that this statement means to start a discussion about promotion.

[0592] Step 4: Collect relevant evidence and assess its reliability

[0593] Based on the analysis results, the server collects relevant evidence from internal databases and external sources, evaluates the reliability of that evidence, and selects only reliable evidence.

[0594] Input: Parsed speech and themes

[0595] Output: Evidence assessed for reliability

[0596] What it does: The server collects promotion-related statistics and past success stories from internal databases and publicly available internet data, and rates the reliability of each source.

[0597] Step 5: Diagram the flow of discussion and points

[0598] Based on the collected evidence and analysis results, the server generates mind maps and flowcharts to visualize the flow of discussion and key points.

[0599] Input: Evidence and analytical results assessed for reliability

[0600] Output: Diagram of discussion flow and points

[0601] Specific operation: Based on the evidence, the server visualizes the structure of the discussion as a mind map and generates data to be displayed on the smart glasses.

[0602] Step 6: Providing visualized data to smart glasses

[0603] The visualized flow of discussion and points of contention are transmitted from the server to the smart glasses and displayed to the user in real time.

[0604] Input: Diagram of discussion flow and points

[0605] Output: Visual data displayed on smart glasses

[0606] Specific operation: The server sends the generated mind maps and flowcharts to the smart glasses, allowing the user to view them in real time.

[0607] Step 7: Learn from past discussion data and generate new proposals

[0608] The server inputs past discussion data into a machine learning model to learn new patterns and effective solutions, and automatically generates suggestions for the current discussion and provides them to the smart glasses.

[0609] Input: Past discussion data and current discussion status

[0610] Output: Auto-generated suggestions

[0611] Specific operation: By learning data from past successful campaigns, the system generates suggestions for the current discussion, such as "In past successful cases, campaigns using social media were effective," and displays these suggestions on the smart glasses.

[0612] Through these steps, this system efficiently and logically supports discussions and meetings in physical stores, enabling rapid decision-making.

[0613] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0614] This invention relates to a system that supports discussions in real time and promotes fair and logical discussions in which users participate. In particular, the invention aims to further improve the quality of discussions by combining an emotion engine that recognizes users' emotions and supports discussions based on those emotions.

[0615] Server Roles and Functions

[0616] 1. Receiving and analyzing user speech data

[0617] The server receives user speech data sent from the device in real time. It analyzes the received speech data using natural language processing (NLP) and emotion recognition technology to identify the content of the speech and the user's emotions. This allows it to understand the topic of the discussion and the emotions of the user.

[0618] 2. Evidence collection and reliability assessment

[0619] The server collects relevant evidence from an internal database or external sources based on the analyzed content and sentiment, evaluates the reliability of the collected evidence, and selects appropriate evidence as the basis for discussion. It also selects specific evidence based on sentiment.

[0620] 3. Visualization Generation and Presentation

[0621] The server generates data that illustrates the flow of the discussion and the key points at issue based on the analysis results and collected evidence. It also includes the user's emotional information in the illustrated data and displays it visually. The generated illustration is sent to the device, allowing the user to view it in real time.

[0622] 4. Learning from past discussion data and making suggestions

[0623] The server inputs past discussion data into a machine learning model to learn new patterns and effective solutions, and then automatically generates proposals and solutions for the current discussion and sends them to the device.

[0624] Role and Function of the Emotion Engine

[0625] 1. Emotion Recognition from Speech

[0626] The emotion engine analyzes user comment data and recognizes the emotions contained in the comments. This emotion data is used as important information when the server analyzes and supports discussions.

[0627] 2. Emotion-based evidence filtering

[0628] The emotion engine selects appropriate evidence and information based on the recognized emotion. For example, if the user is feeling anxious, it will prioritize collecting and presenting evidence that will alleviate that emotion.

[0629] 3. Visualization of emotional information

[0630] The emotion engine generates data to visually display the recognized emotion information, including the type of emotion (e.g., joy, anger, anxiety, etc.) and intensity. The generated emotion data is displayed on the device along with the flow and points of the discussion.

[0631] Device roles and functions

[0632] 1. Providing a user interface

[0633] The terminal provides a user interface to users. Through this interface, users can input comments and questions and send them to the server. The terminal also displays the visualized data, evidence information, and emotion information sent from the server, allowing users to check the progress of the discussion in real time.

[0634] 2. Viewing Feedback

[0635] The device instantly displays feedback and evidence received from the server, suggestions based on learning results, and emotional information to the user, allowing users to always advance discussions based on the latest information.

[0636] User Roles and Experience

[0637] 1. Participating in discussions and providing emotional information

[0638] Users can participate in discussions through their devices and input their opinions and questions. The emotion engine recognizes the emotions contained in the comments and sends them to the server. This enables the server to support discussions while taking into account the users' emotions.

[0639] 2. Interpretation and Use of Graphical Information

[0640] Users can refer to the graphical information and emotional information displayed on the device to understand the flow of discussion and key points, thereby clarifying issues and problems and providing a foundation for efficient and fair decision-making.

[0641] Specific examples

[0642] For example, consider a discussion about how to promote a new product. If User A types, "Let's start a discussion about how to promote our new product," the server receives this statement and analyzes it using NLP and an emotion engine. At the same time, if User B types, "We should use social media for our new product. Past campaigns have been successful," and expresses some uncertainty, the server will collect evidence about the effectiveness of social media and evaluate its reliability. Furthermore, because the emotion engine recognizes this uncertainty, highly reliable evidence is prioritized.

[0643] Based on this, illustrated information is displayed on the device, making it easier for users to visually grasp the flow of the discussion. In addition, emotional information is also displayed, so other participants can proceed with the discussion while taking into consideration User B's emotions.

[0644] Furthermore, by learning from past discussion data, the server can generate suggestions such as "In past successes, social media campaigns have been effective for specific target groups," and display them to the user via their device. This allows users to engage in evidence-based, more logical, and constructive discussions.

[0645] In this way, the present invention is a system that supports evidence-based, logical, and constructive discussions in real time, and furthermore, enables discussions that take into account the user's emotions using an emotion engine, thereby promoting fair and transparent communication.

[0646] The processing flow will be explained below.

[0647] Step 1:

[0648] The user inputs a statement or question through the terminal interface, for example, "Let's start a discussion about how to promote our new product."

[0649] Step 2:

[0650] The terminal receives the user's input and transmits the utterance data to the server, which includes the text information of the utterance.

[0651] Step 3:

[0652] The server analyzes the received speech data in real time, using natural language processing (NLP) techniques to identify the content of the speech and extract themes and keywords.

[0653] Step 4:

[0654] The server recognizes emotions from user utterances using an emotion engine, which identifies emotions (e.g., joy, anger, anxiety, etc.) contained in the utterances.

[0655] Step 5:

[0656] Based on the analyzed content and perceived sentiment, the server collects relevant evidence from internal databases or external sources, for example, looking for data on the effectiveness of "social media" promotions.

[0657] Step 6:

[0658] The server evaluates the reliability of the collected evidence. Based on the evaluation criteria, it selects reliable evidence and uses it as the basis for discussion. Based on emotional information, it prioritizes and selects appropriate evidence for a specific emotion.

[0659] Step 7:

[0660] Based on the analysis results, recognized emotions, and collected evidence, the server generates data that illustrates the flow of discussion and points of contention, including graphs, flowcharts, and emotional information.

[0661] Step 8:

[0662] The server transmits the generated visualization data to the terminal, which displays the received data on the user interface in real time.

[0663] Step 9:

[0664] Users can proceed with the discussion while referring to the graphical data, evidence information, and emotion information displayed on their terminals. If necessary, they can input further comments or questions into their terminals and send them to the server.

[0665] Step 10:

[0666] The server re-analyzes any additional statements or questions, collects and evaluates new evidence as needed, and sends updated graphical data and emotional information to the device in real time.

[0667] Step 11:

[0668] The server inputs past discussion data into a machine learning model to learn new patterns and effective solutions, and generates suggestions for the current discussion based on the learning results.

[0669] Step 12:

[0670] The server sends the generated proposals and solutions to the terminal, which displays them on the user interface for the user to refer to.

[0671] Step 13:

[0672] The user considers the suggestions and solutions displayed on the terminal and continues the discussion. If necessary, the user can re-enter comments or questions and use the system repeatedly.

[0673] Step 14:

[0674] As the discussion progresses, the server continuously recognizes the user's emotions using an emotion engine and adjusts the discussion support accordingly based on that information. This helps to ensure an efficient discussion while maintaining a balance between emotions and evidence.

[0675] Example 2

[0676] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0677] Conventional discussion support systems only analyze the content of user comments and do not take into account user emotions, making it difficult to hold fair and logical discussions. Furthermore, they often lack sufficient visualization to visually grasp the progress of the discussion, and do not properly evaluate the reliability of evidence. Furthermore, past discussion data is not effectively utilized, limiting the ability to extract new patterns and solutions. There is a need for a system that can solve these problems and improve the quality of discussions.

[0678] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0679] In this invention, the server includes means for receiving user utterance data and analyzing the utterance content and emotions using natural language processing, means for collecting related evidence from an internal database or external information sources based on the analyzed utterance content and emotions and evaluating their reliability, means for illustrating the flow of the discussion, issues, and emotional information based on the analysis results and the collected evidence, and means for learning from past discussion data and extracting and presenting new patterns and effective solutions. This makes it possible to support fair and logical discussions that take user emotions into consideration, and by visually understanding the progress of the discussion, it is possible to improve the quality of the discussion.

[0680] "User comment data" refers to text data of discussions and questions that users input into their terminals and send to the server.

[0681] "Natural language processing" refers to techniques and methods that allow computers to understand, interpret, and generate human language.

[0682] "Emotion analysis" refers to the process of identifying emotions (e.g., joy, anger, anxiety, etc.) contained in user speech data.

[0683] "Evidence" refers to the data, examples, and information used as the basis for an argument.

[0684] An "internal database" is a type of data storage for saving data used within the system.

[0685] "External information source" is a general term for a location that can be accessed from outside the system to obtain information.

[0686] "Assessing reliability" is the process of determining the accuracy and validity of the evidence collected.

[0687] "Graphing" is the process of presenting information in a visual form, such as a graph or chart.

[0688] "Past discussion data" refers to the accumulation of text data and analysis results from previous discussions.

[0689] "Extracting new patterns and effective solutions" refers to the process of finding new ways and strategies to advance discussions efficiently and effectively through the analysis of past data.

[0690] This invention is a system that supports discussions in which users participate in real time and promotes fair and logical discussions. In particular, it aims to further improve the quality of discussions by combining it with an emotion engine that recognizes users' emotions and supports discussions based on those emotions. The detailed configuration and operation of this system are described below.

[0691] Server Roles and Functions

[0692] Receiving and analyzing user speech data

[0693] The server receives user speech data sent from the device in real time. Specifically, data communication is carried out using WebSocket or HTTP protocols. The received speech data is analyzed using natural language processing (NLP) technology (e.g., Google Cloud Natural Language API) and emotion recognition technology (e.g., IBM Watson Tone Analyzer). This allows the server to understand the topic of the discussion and the emotions of the users.

[0694] Evidence collection and reliability assessment

[0695] The server collects relevant evidence from an internal database or external sources (e.g., Google Scholar, Web of Science) based on the analyzed content and sentiment of comments. The collected evidence is evaluated for reliability using machine learning algorithms (e.g., Random Forest, Support Vector Machine), allowing appropriate evidence to be selected as the basis for discussion.

[0696] Visualization generation and presentation

[0697] The server generates data that illustrates the flow of discussion and key points based on the analysis results and collected evidence. Specifically, it uses D3.js and Chart.js to create visual diagrams. The diagrams also include the user's emotional information and send it to the device, allowing the user to view it in real time.

[0698] Learning from past discussion data and making suggestions

[0699] The server uses TensorFlow and PyTorch to input past discussion data into machine learning models to learn new patterns and effective solutions, and then automatically generates proposals and solutions for the current discussion and sends them to the device.

[0700] Role and Function of the Emotion Engine

[0701] Emotion recognition from speech

[0702] The emotion engine analyzes user speech data and recognizes the emotions contained in the speech. This emotion data is used as important information when the server analyzes and supports discussions. The technology used is an NLP toolkit (e.g., nltk, spacy) or an emotion analysis API (e.g., Microsoft Azure Text Analytics).

[0703] Emotion-based evidence selection

[0704] The emotion engine selects appropriate evidence and information based on the recognized emotion. For example, if the user is feeling anxious, it will prioritize collecting and presenting evidence that will alleviate that emotion.

[0705] Visualization of emotional information

[0706] The emotion engine generates data to visually display the recognized emotion information, including the type of emotion (e.g., joy, anger, anxiety, etc.) and intensity. This data is displayed on the device along with the visualization data generated by the server.

[0707] Device roles and functions

[0708] Providing a user interface

[0709] The terminal provides a user interface to users. Through this interface, users can input comments and questions and send them to the server. The terminal also displays the visualized data, evidence information, and emotion information sent from the server, allowing users to check the progress of the discussion in real time. Technologies used include React.js and Vue.js.

[0710] View Feedback

[0711] The device instantly displays feedback, evidence, suggestions based on learning results, and emotional information received from the server to the user, allowing users to always advance discussions based on the latest information.

[0712] User Roles and Experience

[0713] Participating in discussions and providing emotional information

[0714] Users can participate in discussions through their devices and input their opinions and questions. The emotion engine recognizes the emotions contained in the comments and sends them to the server. This enables the server to support discussions while taking into account the users' emotions.

[0715] Interpreting and using graphical information

[0716] Users can refer to the graphical information and emotional information displayed on the device to understand the flow of discussion and key points, thereby clarifying issues and problems and providing a foundation for efficient and fair decision-making.

[0717] Examples of concrete examples and prompts

[0718] For example, consider a discussion about how to promote a new product. If User A types, "Let's start a discussion about how to promote our new product," the server receives this statement and analyzes it using NLP and an emotion engine. At the same time, if User B types, "We should use social media for our new product. Past campaigns have been successful," and expresses some uncertainty, the server will collect evidence about the effectiveness of social media and evaluate its reliability. Furthermore, because the emotion engine recognizes this uncertainty, highly reliable evidence is prioritized.

[0719] Based on this, illustrated information is displayed on the device, making it easier for users to visually grasp the flow of the discussion. Emotional information is also displayed, allowing other participants to proceed with the discussion while taking User B's emotions into consideration. Furthermore, by learning from past discussion data, the server generates suggestions such as "In past success stories, social media campaigns have been effective for specific target groups," and displays these to the user via their device. This allows users to engage in evidence-based, more logical, and constructive discussions.

[0720] Prompt Sentence Examples

[0721] "Start a discussion about how to promote the new product. Gather evidence of past successes and effective methods, and consider user sentiment as you proceed with the discussion."

[0722] In this way, the present invention is a system that supports evidence-based, logical, and constructive discussions in real time, and furthermore, enables discussions that take into account the user's emotions using an emotion engine, thereby promoting fair and transparent communication.

[0723] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0724] Step 1:

[0725] Users input comments or questions through the device's user interface. For example, the input can be text data such as "Let's start a discussion about how to promote a new product." The device captures this comment data and sends it to the server using WebSocket or HTTP protocol. The server receives this comment data.

[0726] Step 2:

[0727] The server analyzes the received speech data. The input is the user's speech data, which is sent to natural language processing (NLP) technology (e.g., Google Cloud Natural Language API) to analyze the speech content. At the same time, emotion is analyzed using emotion recognition technology (e.g., IBM Watson Tone Analyzer). The output is the speech theme, keywords, and emotional information.

[0728] Step 3:

[0729] The server collects relevant evidence based on the analyzed speech content and sentiment. The input is the analysis results, and based on this, data is collected from internal databases and external sources (e.g., Google Scholar, Web of Science). The collected evidence is evaluated for reliability using a machine learning algorithm (e.g., Random Forest). Highly reliable evidence is selected as the output.

[0730] Step 4:

[0731] The server generates data that visualizes the flow of discussion, key points, and sentiment information based on the analysis results and collected evidence. The input is reliable evidence and analysis results. Visualization tools such as D3.js and Chart.js are used for visualization. Visual visualization data is generated as output and sent to the device.

[0732] Step 5:

[0733] The server inputs past discussion data into a machine learning model (e.g., TensorFlow, PyTorch) to learn new patterns and effective solutions. The input is past discussion data, and learning is based on this. As an output, new patterns and proposals are generated and sent to the device.

[0734] Step 6:

[0735] The terminal receives graphical data, evidence, suggestions, and emotion information from the server and displays them in real time on the user interface. The input is data sent from the server. This allows users to visually grasp the progress of the discussion and evidence, and receive feedback in real time.

[0736] Step 7:

[0737] The user then looks at the device's interface and types in a new comment or question. This input includes new comments or questions based on suggestions from the server. The device captures this input and sends it back to the server, repeating the cycle and moving the discussion forward.

[0738] (Application example 2)

[0739] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0740] Real-time customer service and customer support in virtual stores requires advanced technology. It is particularly difficult to accurately recognize customer emotions and instantly provide suggestions and evidence based on those emotions. Therefore, systems are needed to improve the quality of customer service. Furthermore, there is a lack of interfaces to facilitate smooth communication between customers and staff, and this needs to be resolved.

[0741] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for receiving utterance data from users and analyzing the utterance content using natural language processing; means for collecting related evidence from an internal database or external information source based on the utterance content and evaluating its reliability; means for illustrating the flow of discussion and points of contention based on the analyzed utterance content and the collected evidence; means for learning from past discussion data and extracting and presenting new patterns and effective solutions; means for recognizing emotions contained in utterances and selecting and displaying appropriate evidence based on the emotions; and means for making customer service suggestions based on emotions and supporting customer service in real time. This enables real-time customer service support that takes customer emotions into consideration in a virtual store.

[0742] A "user" is an entity that provides and receives information through the system, such as a customer who uses a service or a customer service staff member.

[0743] "Utterance data" refers to text information and voice data provided by user input, which is used by the system for analysis.

[0744] "Natural language processing" is a technology that allows computers to understand and analyze human language, and is used to analyze the content of user utterances.

[0745] "Evidence" refers to relevant information and supporting documents provided based on the content and intent of a user's statements, and provides a basis for supporting the discussion.

[0746] An "internal database" is a collection of information held within the system and is a database used to collect evidence related to the content of statements.

[0747] "External sources" are sources for collecting information from outside the system, such as the Internet or other external databases.

[0748] "Credibility assessment" is the process of evaluating the accuracy and veracity of collected evidence, and is an evaluation carried out to select highly reliable information.

[0749] "Visualization" is the process of visually displaying analyzed data and evidence, presenting information to users in an easy-to-understand manner.

[0750] "Past discussion data" refers to discussions that have taken place in the past and their records, and is data that can be learned to derive new patterns and solutions.

[0751] "Emotion recognition" is a technology that analyzes and recognizes the emotions contained in a user's speech, and is used to understand the user's psychological state.

[0752] "Customer service suggestions" is the process of presenting customer service methods and advice based on recognized emotions and statements.

[0753] "Real-time" refers to the process of receiving, analyzing, and presenting data almost simultaneously, providing an immediate response without delay.

[0754] The present invention relates to a real-time customer support system for a virtual store. This system receives utterance data entered by users (customers or wait staff) and analyzes it using natural language processing. It also collects relevant evidence based on the content of the utterances and evaluates its reliability. Furthermore, by combining this with emotion recognition technology, it provides responses and suggestions that take the user's emotions into consideration. Below, an embodiment of the present invention will be described in detail.

[0755] Program Overview

[0756] Server Roles and Functions

[0757] The server performs the following functions:

[0758] 1. Receiving and analyzing user speech data

[0759] Speech data sent by users is received in real time and analyzed using natural language processing (NLP) technology. Software used for this includes NLP libraries such as SpaCy. Keywords are extracted from the analyzed speech content to identify the content of the speech.

[0760] 2. Recognizing emotions in speech

[0761] Using an emotion recognition library such as TextBlob, we analyze the emotions contained in the user's speech, which allows us to identify the emotions the user is feeling (e.g., joy, anger, anxiety, etc.).

[0762] 3. Evidence collection and reliability assessment

[0763] Based on the extracted keywords and recognized sentiment, relevant evidence is collected from internal databases or external sources, and the reliability of the evidence is evaluated to select reliable information.

[0764] 4. Visualization Generation and Presentation

[0765] Based on the analysis results and collected evidence, the flow of discussion and points of contention are visualized. NetworkX is used to visually display the relationships and flow of data, and the server sends the generated visualized data to the terminal.

[0766] Device roles and functions

[0767] The terminal performs the following functions:

[0768] 1. Providing a user interface

[0769] The terminal provides a user interface, allowing users to input comments and questions, and displays the visualized data and evidence information sent from the server in real time, allowing users to check the progress of the discussion.

[0770] 2. Viewing Feedback

[0771] The device instantly displays feedback, evidence, suggestions based on learning results, and emotional information received from the server to the user, allowing users to always advance discussions based on the latest information.

[0772] User Roles and Experience

[0773] Users can use the system as follows:

[0774] 1. Participating in discussions and providing emotional information

[0775] Users can participate in discussions through their devices and input their opinions and questions. The emotion engine recognizes the emotions contained in the comments and sends them to the server. This allows the server to support discussions while taking into account the users' emotions.

[0776] 2. Interpretation and Use of Graphical Information

[0777] Users can refer to the graphical information and emotional information displayed on their devices to understand the flow of the discussion and key points at issue, which allows them to handle customer inquiries efficiently and fairly.

[0778] Specific examples

[0779] For example, if a customer types, "Tell me about recent campaigns," the system analyzes this in real time and provides relevant campaign information and evidence. If emotion recognition indicates that the customer has a positive emotion, the system will provide additional information or suggestions that will further enhance that emotion.

[0780] Example prompt sentence:

[0781] Customer message: "Tell me about your latest promotions."

[0782] Analyzed Keywords: Campaign

[0783] Emotion: Positive

[0784] Recommended evidence: "Our latest 20% discount offer is running now. Plus, there are additional perks for new users."

[0785] In this way, the system can respond immediately to customer demands and emotions and provide high-quality service.

[0786] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0787] Step 1:

[0788] The server receives user utterance data. The input is the user's utterance (text or audio data). The output is the received utterance data. This data is then analyzed in the next processing step.

[0789] Step 2:

[0790] The server analyzes the received utterance data. Specifically, it uses natural language processing (NLP) technology to analyze the content of the utterance. The software used is an NLP library such as SpaCy. It receives the utterance data as input and extracts keywords and the grammatical structure of the sentences. The analyzed keywords and structural data of the utterance content are obtained as output.

[0791] Step 3:

[0792] The server uses emotion recognition technology to recognize emotions from the analyzed utterance data. The emotion recognition library used is TextBlob. As input, it receives the analyzed utterance data and identifies the user's emotion (e.g., positive, negative, neutral). As output, it obtains the recognized emotion data.

[0793] Step 4:

[0794] The server collects relevant evidence based on the analyzed utterance data and recognized emotions from an internal database or external sources, using the internet or external APIs as needed. It receives keywords and emotion data as input, and searches for and collects relevant evidence. The collected evidence data is obtained as output.

[0795] Step 5:

[0796] The server evaluates the reliability of the collected evidence. Specifically, it uses the source of the evidence, past performance, etc. as evaluation criteria. It receives the collected evidence data as input and scores or ranks its reliability. The output is the evidence data that has undergone reliability evaluation.

[0797] Step 6:

[0798] The server visualizes the flow of discussion and points of contention based on the analysis results and collected evidence. The software used is NetworkX. It receives analysis data and evidence data as input and generates graphical data to display them visually. The graphical data is obtained as output.

[0799] Step 7:

[0800] The server uses machine learning models to learn from past discussion data and extract new patterns and effective solutions. It receives past discussion data as input and extracts new patterns for the current discussion and future. The output is proposal data.

[0801] Step 8:

[0802] The terminal provides a user interface, allowing users to input comments in real time and displaying graphical data and evidence information sent from the server. As input, the terminal receives graphical data and evidence information from the server and displays them to the user. As output, the terminal provides an interface that allows users to input new comments and questions.

[0803] Step 9:

[0804] The terminal immediately displays the feedback, evidence, and emotion information received from the server to the user. As input, it receives feedback data, evidence, and emotion data from the server and displays them on the user interface. As output, it displays a display that allows users to advance the discussion based on the latest information.

[0805] Step 10:

[0806] Users participate in discussions through their terminals and input their opinions and questions. As input, speech data is provided to the terminal and sent to the server. As output, emotions are analyzed by the emotion engine and sent to the server.

[0807] Step 11:

[0808] Users refer to the graphical data and emotional information displayed on their devices to understand the flow of the discussion and key points. As input, they receive the graphical data and emotional information displayed on their devices and advance the discussion. As output, more logical and constructive discussions based on evidence become possible.

[0809] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0810] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0811] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0812] [Third embodiment]

[0813] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0814] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0815] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0816] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0817] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0818] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0819] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0820] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0821] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0822] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0823] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0824] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0825] This invention relates to a system that supports discussions in real time in which users participate, and promotes fair and logical discussions. The main components of the system are a server, terminals, and users, each of which performs a specific function.

[0826] Server Roles and Functions

[0827] 1. Receiving and analyzing user speech data

[0828] The server receives user speech data sent from the device in real time. It then analyzes the received speech data using natural language processing (NLP) technology to identify and classify the content and intent of the speech. This allows it to understand the topic of the discussion and lead to the collection of appropriate evidence.

[0829] 2. Evidence collection and reliability assessment

[0830] The server collects relevant evidence from an internal database and external sources based on the analyzed comments, evaluates the reliability of the collected evidence, and selects appropriate evidence to support the discussion. This allows users to advance discussions based on well-founded information.

[0831] 3. Visualization Generation and Presentation

[0832] The server generates data that illustrates the flow of discussion and points of contention based on the analysis results and collected evidence. This includes visual formats such as graphs, flowcharts, and mind maps. The generated illustrations are sent to the terminals so that users can view them in real time.

[0833] 4. Learning from past discussion data and making suggestions

[0834] The server inputs past discussion data into a machine learning model to learn new patterns and effective solutions, and then automatically generates proposals and solutions for the current discussion and sends them to the device.

[0835] Device roles and functions

[0836] 1. Providing a user interface

[0837] The terminal provides a user interface to users. Through this interface, users can input comments and questions and send them to the server. The terminal also displays the visualized data and evidence information sent from the server, allowing users to check the progress of the discussion in real time.

[0838] 2. Viewing Feedback

[0839] The device instantly displays feedback, evidence, and suggestions based on learning results received from the server to the user, allowing users to always advance discussions based on the latest information.

[0840] User Roles and Experience

[0841] 1. Participating in discussions and providing evidence

[0842] Users can participate in discussions through their terminals, input their opinions and questions, and, if necessary, submit their own evidence to the server to use as the basis for the discussion.

[0843] 2. Interpretation and Use of Graphical Information

[0844] Users can refer to the graphical information displayed on their devices to understand the flow of discussions and key points, thereby clarifying issues and problems and providing a foundation for efficient and fair decision-making.

[0845] Specific examples

[0846] For example, consider a discussion about how to promote a new product. If User A types, "Let's start a discussion about how to promote the new product," the server receives this statement and analyzes it using NLP. At the same time, if User B types, "We should use social media for the new product. Past campaigns have been successful," the server collects evidence about the effectiveness of social media and evaluates its reliability. Based on this, illustrated information is displayed on the device, making it easier for users to visually grasp the flow of the discussion.

[0847] Furthermore, by learning from past discussion data, the server can generate suggestions such as "In past successes, social media campaigns have been effective for specific target groups," and display them to the user via their device. This allows users to engage in evidence-based, more logical, and constructive discussions.

[0848] In this way, the present invention supports evidence-based, logical, and constructive discussions in real time, promoting fair and transparent communication.

[0849] The processing flow will be explained below.

[0850] Step 1:

[0851] The user inputs a statement or question through the terminal interface, for example, "Let's start a discussion about how to promote our new product."

[0852] Step 2:

[0853] The terminal receives the user's input and transmits the utterance data to the server, which includes the text information of the utterance.

[0854] Step 3:

[0855] The server analyzes the received speech data in real time, using natural language processing (NLP) techniques to identify the content of the speech and extract themes and keywords.

[0856] Step 4:

[0857] Based on the extracted comments, the server collects relevant evidence from internal databases or external sources, for example, looking for data on the effectiveness of "social media" promotions.

[0858] Step 5:

[0859] The server evaluates the reliability of the collected evidence, and based on the evaluation criteria, selects reliable evidence and uses it as the basis for discussion.

[0860] Step 6:

[0861] Based on the analysis results and collected evidence, the server generates data that illustrates the flow of discussion and the points at issue, including graphs and flowcharts.

[0862] Step 7:

[0863] The server transmits the generated visualization data to the terminal, which displays the received data on the user interface in real time.

[0864] Step 8:

[0865] Users can proceed with the discussion while referring to the visualized data and evidence information displayed on their terminals. If necessary, they can enter further comments or questions into their terminals and send them to the server.

[0866] Step 9:

[0867] The server re-analyzes any additional statements or questions, collects and evaluates new evidence as needed, and updates the visualization data in real time and sends it to the terminal.

[0868] Step 10:

[0869] The server inputs past discussion data into a machine learning model to learn new patterns and effective solutions, and generates suggestions for the current discussion based on the learning results.

[0870] Step 11:

[0871] The server sends the generated proposals and solutions to the terminal, which displays them on the user interface for the user to refer to.

[0872] Step 12:

[0873] The user considers the suggestions and solutions displayed on the terminal and continues the discussion. If necessary, the user can re-enter comments or questions and use the system repeatedly.

[0874] Example 1

[0875] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0876] Conventional discussion support systems lack sufficient analysis and support to support the progress of discussions in real time based on user comments. Furthermore, they lack the ability to collect evidence, evaluate its reliability, and visualize the flow and points of discussion, making it difficult for users to engage in evidence-based, logical, and fair discussions. Furthermore, they lack the ability to effectively extract new solutions and proposals from past discussion data and present them in real time.

[0877] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0878] In this invention, the server includes means for receiving user comment data and analyzing the comment content using natural language processing technology, means for collecting related evidential materials based on the comment content from an internal data management system or external information sources and evaluating the reliability of the collected evidential materials, means for generating data that visually displays the flow of discussion and points of contention based on the analyzed comment content and the collected evidential materials, and means for learning past discussion data using a machine learning model and automatically extracting and presenting new patterns and effective solutions. This enables users to engage in evidence-based, logical, and fair discussion in real time.

[0879] "User" refers to a person who uses the system to participate in discussions and enter their own opinions and questions.

[0880] "Speech data" refers to text data such as opinions and questions entered by users in discussions.

[0881] "Natural language processing technology" refers to technology that uses computers to analyze human language and make it easier to understand its content and intent.

[0882] "Evidence" refers to information or data that forms the basis of an argument, including data collected from internal databases or external sources.

[0883] A "data management system" refers to a system that can efficiently manage and search data stored within it.

[0884] "External information sources" refer to means or systems for obtaining information from external databases or the Internet.

[0885] "Assessing reliability" refers to the process of determining whether collected evidence is accurate.

[0886] "Data that visually displays the flow of discussion and key points" refers to data such as graphs and charts that clearly visualize the progress of the discussion and key points.

[0887] A "machine learning model" refers to an algorithm that learns patterns and regularities based on large amounts of data.

[0888] "New patterns and effective solutions" refers to new trends and problem-solving methods that are derived from past data.

[0889] "Presenting" refers to displaying information or suggestions to the user.

[0890] This invention is a system that supports discussions in which users participate in real time and promotes fair and logical discussions, and its main components are a server, terminals, and users.

[0891] The server receives user speech data and analyzes the content of the speech using natural language processing technology. Specifically, it streams the data using Apache Kafka and analyzes it using Google Cloud Natural Language API. By identifying and classifying the content and intent of the speech, it understands the topic of the discussion.

[0892] Based on the analysis results, the server collects relevant evidence from internal data management systems (e.g., MySQL) and external sources (e.g., academic paper databases). The collected evidence is then evaluated for its reliability using machine learning libraries such as Scikit-learn. This process ensures that only reliable information is selected as the basis for discussion.

[0893] Next, the server generates data that visually displays the flow of the discussion and the points at issue based on the analysis results and collected evidence, using D3.js to generate graphs, flowcharts, mind maps, etc. The generated visualization data is then sent to the device via WebSocket.

[0894] The server then uses machine learning models (e.g., TensorFlow) to learn from past discussion data and extract new patterns and effective solutions. Based on this, the server generates proposals and solutions that are presented to users in real time.

[0895] The terminal provides a user interface through which users can input comments and questions. The terminal also displays visualized data and supporting documents sent from the server, allowing users to check the progress of the discussion in real time. Users can participate in the discussion through the terminal and input their own opinions and questions. For example, users can input comments such as "We will start a discussion on how to promote our new product," or specific suggestions such as "We should utilize social media for our new product. Past campaigns have been successful."

[0896] In this system, the server performs analysis and evidence collection, evaluates reliability, generates visualized data, and extracts suggestions, and the terminal displays these to the user, allowing the user to engage in evidence-based, logical, and fair discussion in real time.

[0897] Example prompt: "We're opening a discussion about how to promote a new product. Please provide suggestions based on past successes and reliable evidence."

[0898] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0899] Step 1:

[0900] The user inputs a statement.

[0901] The user enters "Let's start a discussion on how to promote our new product" into the input field of the terminal. The input speech data is in text format, and the terminal sends this data to the server via WebSocket.

[0902] Input: The text of the speech entered by the user into the terminal.

[0903] Output: Speech data sent to the server via WebSocket

[0904] Step 2:

[0905] The server receives the speech data and analyzes it using natural language processing technology.

[0906] The server uses Apache Kafka to receive data sent from the devices in real time. The received speech data is analyzed using the Google Cloud Natural Language API to identify the content and intent of the speech. For example, keywords such as "new product" and "promotion method" are extracted to understand the topic of the discussion.

[0907] Input: Speech data sent from the device

[0908] Output: Parsed keywords and themes

[0909] Step 3:

[0910] The server collects the evidence and evaluates its reliability.

[0911] Based on the analyzed themes, the server collects relevant evidence from internal data management systems and external sources (e.g., academic paper databases). The collected evidence is evaluated for credibility using Scikit-learn, for example, by measuring the number of citations of a paper.

[0912] Input: Parsed keywords and themes

[0913] Output: Evidence with reliability evaluation

[0914] Step 4:

[0915] The server generates data that visualizes the flow of discussion and key points.

[0916] The server uses D3.js to generate visualizations such as graphs and flowcharts based on the analysis results and supporting documentation. For example, a bar graph showing the success rate of past social media campaigns is created. The generated data is sent to the device via WebSocket.

[0917] Input: Analysis results and evidence that has been evaluated for reliability

[0918] Output: The generated visualization data

[0919] Step 5:

[0920] The device displays visualization data and feedback in real time.

[0921] The device then displays the received visualization data on a dashboard, allowing users to see the progress of the discussion in real time, including a bar graph showing the success rate and related supporting documents.

[0922] Input: Visualization data sent from the server

[0923] Output: Visualized data and evidence displayed on the device

[0924] Step 6:

[0925] The server learns from past discussion data and generates new proposals.

[0926] The server uses TensorFlow to learn from past discussion data and extract new patterns and effective solutions. For example, it generates suggestions based on past successes, such as "a social media campaign was effective for a specific target demographic." The generated suggestions are sent to the device.

[0927] Input: Past discussion data

[0928] Output: Extracted new proposals and solutions

[0929] Step 7:

[0930] The device displays new suggestions and feedback to the user.

[0931] The device displays new suggestions sent from the server to the user in real time. For example, specific suggestions such as "Examples of successful social media campaigns" are displayed on the dashboard. Based on this information, the user can make the next statement.

[0932] Input: New suggestions and feedback sent by the server

[0933] Output: suggestions and feedback displayed on the device

[0934] (Application example 1)

[0935] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0936] In today's business environment, discussions and meetings between store staff and customers must be conducted efficiently and logically. However, it is difficult to provide real-time discussion support or immediate evidence, making it difficult for users to quickly resolve the issues they face and make decisions. Furthermore, there is a lack of a way to visually grasp the flow of discussions and key points at issue, which can lead to overlooking key points for deriving optimal solutions. Furthermore, it is difficult to learn new patterns and effective solutions from past discussion data and apply them to the current situation.

[0937] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0938] In this invention, the server includes means for receiving user utterance data and analyzing the utterance content using natural language processing, means for collecting related evidence based on the utterance content from an internal database or external information source and evaluating its reliability, means for illustrating the flow of discussion and points of contention based on the analyzed utterance content and the collected evidence, means for learning from past discussion data and extracting and presenting new patterns and effective solutions, means for converting user utterances into text using speech recognition technology and transmitting the text to the server, and means for providing the analysis results and evidence to a terminal that visually displays them. This allows users to engage in fair and logical discussions in real time and enables efficient decision-making.

[0939] "User utterance data" refers to voice data or text data uttered by a user during a discussion or meeting.

[0940] "Natural language processing" is a set of techniques that allow computers to understand and analyze human language.

[0941] "Evidence" refers to information or data that serves as the basis for an argument.

[0942] An "internal database" is a collection of accumulated data managed within the system.

[0943] "External information sources" refers to information managed outside the system, such as materials on the Internet or external databases.

[0944] "Measures to assess reliability" are processes or techniques used to determine the accuracy or reliability of collected evidence.

[0945] "Visualization" is the visual representation of information or data.

[0946] The "flow of discussion" indicates the process of how the discussion progresses.

[0947] An "issue" is an important topic or focal point in a discussion.

[0948] "Past discussion data" refers to records and information relating to discussions that have taken place in the past.

[0949] "Means of extracting new patterns and effective solutions" refers to techniques and technologies for finding useful patterns and solutions from past data.

[0950] "Converting user speech into text using speech recognition technology" is the process of converting a user's speech into text data.

[0951] A "server" is a computer system that processes and analyzes data and provides services to clients over a network.

[0952] "Means for providing analysis results and evidence to a terminal that visually displays them" refers to the technology or method by which the server sends analysis results and evidence to a terminal and displays them visually.

[0953] "Providing a user interface to a terminal" means displaying a screen or interface on the terminal that allows the user to operate the system.

[0954] "Updating the visualized data in real time" means instantly updating the visualized information as the discussion progresses.

[0955] "Automatically generating proposals based on past data" means that the system automatically creates new proposals based on past data.

[0956] The system embodying this invention has the functions of recognizing and analyzing the comments of users participating in a discussion in real time, collecting and evaluating related evidence, and visualizing the flow of the discussion and the points at issue. The detailed configuration and operation of the system are described below.

[0957] 1. Program Generation

[0958] The server receives speech data from the user and analyzes it using natural language processing (NLP) technology. The user's speech is converted into text using speech recognition technology through the smart glasses and sent to the server. Based on the analyzed data, relevant evidence is then collected from an internal database and external sources and its reliability is evaluated. This operation is achieved by converting speech to text using the Google Speech Recognition API and sending the data to the server using the requests library.

[0959] 2. Processing Description

[0960] When the server receives the user's speech data, it first converts the speech data into text using speech recognition technology, then analyzes the text data using NLP technology to identify the content and intent of the speech, and based on this analysis, begins the process of collecting relevant evidence and assessing its reliability.

[0961] For example, if someone says, "Show me evidence about a new promotion method," the server will collect relevant evidence from internal databases and external information sources and evaluate its reliability, taking into account the source of the collected information, its past usage history, and other factors.

[0962] The flow of the discussion and the points at issue are then visually illustrated based on the analyzed comments and collected evidence. This visualization is done using mind maps, flowcharts, and other tools. This visual data is displayed on the smart glasses, allowing users to understand the progress of the discussion in real time.

[0963] 3. Specific Examples

[0964] Specifically, when a user says, "We'll start discussing how to promote our new product," the server recognizes this speech and converts it into text. Next, when User B says, "We should utilize social media for our new product. Past campaigns have been successful," the server collects evidence about the effectiveness of social media and evaluates its reliability. This allows reliable information to be collected and displayed on the smart glasses.

[0965] In addition, by learning from past discussion data, the server generates suggestions such as "In past success stories, campaigns using social media have been effective for specific target groups," and displays these to the user.

[0966] Examples of prompts include "Show evidence of a new promotional method," "Tell me examples of successful campaigns from the past," and "Diagram the flow of the discussion."

[0967] By implementing this invention, meetings and discussions in physical stores can be conducted more efficiently and logically, enabling fair decision-making.

[0968] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0969] Step 1: Convert user speech into text using speech recognition technology

[0970] When a user speaks through the smart glasses, their voice is collected by the smart glasses' microphone, and this voice data is converted into text using the Google Speech Recognition API.

[0971] Input: User's voice

[0972] Output: Textual speech data

[0973] How it works: When a user says, "We'll start a discussion about new promotional methods," the microphone in the smart glasses collects voice data and converts it into text data using the Google Speech Recognition API.

[0974] Step 2: Send the textual utterance data to the server

[0975] The text data is sent from the smart glasses to a server via the Internet as an HTTP request using the requests library.

[0976] Input: Textual speech data

[0977] Output: Speech data sent to the server

[0978] Specific operation: The user's textual utterance data is sent from the smart glasses to the server. For example, the user's utterance "We will start discussing new promotion methods" is sent to the server as text data.

[0979] Step 3: Analyze the speech data using natural language processing

[0980] The server analyzes the received speech data using natural language processing (NLP) to identify the content and intent of the speech, and understands the topic of the discussion based on the results of this analysis.

[0981] Input: Speech data sent to the server

[0982] Output: Parsed speech and themes

[0983] Specific operation: The server analyzes the statement "We will start a discussion about new promotion methods" using an NLP engine and understands that this statement means to start a discussion about promotion.

[0984] Step 4: Collect relevant evidence and assess its reliability

[0985] Based on the analysis results, the server collects relevant evidence from internal databases and external sources, evaluates the reliability of that evidence, and selects only reliable evidence.

[0986] Input: Parsed speech and themes

[0987] Output: Evidence assessed for reliability

[0988] What it does: The server collects promotion-related statistics and past success stories from internal databases and publicly available internet data, and rates the reliability of each source.

[0989] Step 5: Diagram the flow of discussion and points

[0990] Based on the collected evidence and analysis results, the server generates mind maps and flowcharts to visualize the flow of discussion and key points.

[0991] Input: Evidence and analytical results assessed for reliability

[0992] Output: Diagram of discussion flow and points

[0993] Specific operation: Based on the evidence, the server visualizes the structure of the discussion as a mind map and generates data to be displayed on the smart glasses.

[0994] Step 6: Providing visualized data to smart glasses

[0995] The visualized flow of discussion and points of contention are transmitted from the server to the smart glasses and displayed to the user in real time.

[0996] Input: Diagram of discussion flow and points

[0997] Output: Visual data displayed on smart glasses

[0998] Specific operation: The server sends the generated mind maps and flowcharts to the smart glasses, allowing the user to view them in real time.

[0999] Step 7: Learn from past discussion data and generate new proposals

[1000] The server inputs past discussion data into a machine learning model to learn new patterns and effective solutions, and automatically generates suggestions for the current discussion and provides them to the smart glasses.

[1001] Input: Past discussion data and current discussion status

[1002] Output: Auto-generated suggestions

[1003] Specific operation: By learning data from past successful campaigns, the system generates suggestions for the current discussion, such as "In past successful cases, campaigns using social media were effective," and displays these suggestions on the smart glasses.

[1004] Through these steps, this system efficiently and logically supports discussions and meetings in physical stores, enabling rapid decision-making.

[1005] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1006] This invention relates to a system that supports discussions in real time and promotes fair and logical discussions in which users participate. In particular, the invention aims to further improve the quality of discussions by combining an emotion engine that recognizes users' emotions and supports discussions based on those emotions.

[1007] Server Roles and Functions

[1008] 1. Receiving and analyzing user speech data

[1009] The server receives user speech data sent from the device in real time. It analyzes the received speech data using natural language processing (NLP) and emotion recognition technology to identify the content of the speech and the user's emotions. This allows it to understand the topic of the discussion and the emotions of the user.

[1010] 2. Evidence collection and reliability assessment

[1011] The server collects relevant evidence from an internal database or external sources based on the analyzed content and sentiment, evaluates the reliability of the collected evidence, and selects appropriate evidence as the basis for discussion. It also selects specific evidence based on sentiment.

[1012] 3. Visualization Generation and Presentation

[1013] The server generates data that illustrates the flow of the discussion and the key points at issue based on the analysis results and collected evidence. It also includes the user's emotional information in the illustrated data and displays it visually. The generated illustration is sent to the device, allowing the user to view it in real time.

[1014] 4. Learning from past discussion data and making suggestions

[1015] The server inputs past discussion data into a machine learning model to learn new patterns and effective solutions, and then automatically generates proposals and solutions for the current discussion and sends them to the device.

[1016] Role and Function of the Emotion Engine

[1017] 1. Emotion Recognition from Speech

[1018] The emotion engine analyzes user comment data and recognizes the emotions contained in the comments. This emotion data is used as important information when the server analyzes and supports discussions.

[1019] 2. Emotion-based evidence filtering

[1020] The emotion engine selects appropriate evidence and information based on the recognized emotion. For example, if the user is feeling anxious, it will prioritize collecting and presenting evidence that will alleviate that emotion.

[1021] 3. Visualization of emotional information

[1022] The emotion engine generates data to visually display the recognized emotion information, including the type of emotion (e.g., joy, anger, anxiety, etc.) and intensity. The generated emotion data is displayed on the device along with the flow and points of the discussion.

[1023] Device roles and functions

[1024] 1. Providing a user interface

[1025] The terminal provides a user interface to users. Through this interface, users can input comments and questions and send them to the server. The terminal also displays the visualized data, evidence information, and emotion information sent from the server, allowing users to check the progress of the discussion in real time.

[1026] 2. Viewing Feedback

[1027] The device instantly displays feedback and evidence received from the server, suggestions based on learning results, and emotional information to the user, allowing users to always advance discussions based on the latest information.

[1028] User Roles and Experience

[1029] 1. Participating in discussions and providing emotional information

[1030] Users can participate in discussions through their devices and input their opinions and questions. The emotion engine recognizes the emotions contained in the comments and sends them to the server. This enables the server to support discussions while taking into account the users' emotions.

[1031] 2. Interpretation and Use of Graphical Information

[1032] Users can refer to the graphical information and emotional information displayed on the device to understand the flow of discussion and key points, thereby clarifying issues and problems and providing a foundation for efficient and fair decision-making.

[1033] Specific examples

[1034] For example, consider a discussion about how to promote a new product. If User A types, "Let's start a discussion about how to promote our new product," the server receives this statement and analyzes it using NLP and an emotion engine. At the same time, if User B types, "We should use social media for our new product. Past campaigns have been successful," and expresses some uncertainty, the server will collect evidence about the effectiveness of social media and evaluate its reliability. Furthermore, because the emotion engine recognizes this uncertainty, highly reliable evidence is prioritized.

[1035] Based on this, illustrated information is displayed on the device, making it easier for users to visually grasp the flow of the discussion. In addition, emotional information is also displayed, so other participants can proceed with the discussion while taking into consideration User B's emotions.

[1036] Furthermore, by learning from past discussion data, the server can generate suggestions such as "In past successes, social media campaigns have been effective for specific target groups," and display them to the user via their device. This allows users to engage in evidence-based, more logical, and constructive discussions.

[1037] In this way, the present invention is a system that supports evidence-based, logical, and constructive discussions in real time, and furthermore, enables discussions that take into account the user's emotions using an emotion engine, thereby promoting fair and transparent communication.

[1038] The processing flow will be explained below.

[1039] Step 1:

[1040] The user inputs a statement or question through the terminal interface, for example, "Let's start a discussion about how to promote our new product."

[1041] Step 2:

[1042] The terminal receives the user's input and transmits the utterance data to the server, which includes the text information of the utterance.

[1043] Step 3:

[1044] The server analyzes the received speech data in real time, using natural language processing (NLP) techniques to identify the content of the speech and extract themes and keywords.

[1045] Step 4:

[1046] The server recognizes emotions from user utterances using an emotion engine, which identifies emotions (e.g., joy, anger, anxiety, etc.) contained in the utterances.

[1047] Step 5:

[1048] Based on the analyzed content and perceived sentiment, the server collects relevant evidence from internal databases or external sources, for example, looking for data on the effectiveness of "social media" promotions.

[1049] Step 6:

[1050] The server evaluates the reliability of the collected evidence. Based on the evaluation criteria, it selects reliable evidence and uses it as the basis for discussion. Based on emotional information, it prioritizes and selects appropriate evidence for a specific emotion.

[1051] Step 7:

[1052] Based on the analysis results, recognized emotions, and collected evidence, the server generates data that illustrates the flow of discussion and points of contention, including graphs, flowcharts, and emotional information.

[1053] Step 8:

[1054] The server transmits the generated visualization data to the terminal, which displays the received data on the user interface in real time.

[1055] Step 9:

[1056] Users can proceed with the discussion while referring to the graphical data, evidence information, and emotion information displayed on their terminals. If necessary, they can input further comments or questions into their terminals and send them to the server.

[1057] Step 10:

[1058] The server re-analyzes any additional statements or questions, collects and evaluates new evidence as needed, and sends updated graphical data and emotional information to the device in real time.

[1059] Step 11:

[1060] The server inputs past discussion data into a machine learning model to learn new patterns and effective solutions, and generates suggestions for the current discussion based on the learning results.

[1061] Step 12:

[1062] The server sends the generated proposals and solutions to the terminal, which displays them on the user interface for the user to refer to.

[1063] Step 13:

[1064] The user considers the suggestions and solutions displayed on the terminal and continues the discussion. If necessary, the user can re-enter comments or questions and use the system repeatedly.

[1065] Step 14:

[1066] As the discussion progresses, the server continuously recognizes the user's emotions using an emotion engine and adjusts the discussion support accordingly based on that information. This helps to ensure an efficient discussion while maintaining a balance between emotions and evidence.

[1067] Example 2

[1068] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1069] Conventional discussion support systems only analyze the content of user comments and do not take into account user emotions, making it difficult to hold fair and logical discussions. Furthermore, they often lack sufficient visualization to visually grasp the progress of the discussion, and do not properly evaluate the reliability of evidence. Furthermore, past discussion data is not effectively utilized, limiting the ability to extract new patterns and solutions. There is a need for a system that can solve these problems and improve the quality of discussions.

[1070] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1071] In this invention, the server includes means for receiving user utterance data and analyzing the utterance content and emotions using natural language processing, means for collecting related evidence from an internal database or external information sources based on the analyzed utterance content and emotions and evaluating their reliability, means for illustrating the flow of the discussion, issues, and emotional information based on the analysis results and the collected evidence, and means for learning from past discussion data and extracting and presenting new patterns and effective solutions. This makes it possible to support fair and logical discussions that take user emotions into consideration, and by visually understanding the progress of the discussion, it is possible to improve the quality of the discussion.

[1072] "User comment data" refers to text data of discussions and questions that users input into their terminals and send to the server.

[1073] "Natural language processing" refers to techniques and methods that allow computers to understand, interpret, and generate human language.

[1074] "Emotion analysis" refers to the process of identifying emotions (e.g., joy, anger, anxiety, etc.) contained in user speech data.

[1075] "Evidence" refers to the data, examples, and information used as the basis for an argument.

[1076] An "internal database" is a type of data storage for saving data used within the system.

[1077] "External information source" is a general term for a location that can be accessed from outside the system to obtain information.

[1078] "Assessing reliability" is the process of determining the accuracy and validity of the evidence collected.

[1079] "Graphing" is the process of presenting information in a visual form, such as a graph or chart.

[1080] "Past discussion data" refers to the accumulation of text data and analysis results from previous discussions.

[1081] "Extracting new patterns and effective solutions" refers to the process of finding new ways and strategies to advance discussions efficiently and effectively through the analysis of past data.

[1082] This invention is a system that supports discussions in which users participate in real time and promotes fair and logical discussions. In particular, it aims to further improve the quality of discussions by combining it with an emotion engine that recognizes users' emotions and supports discussions based on those emotions. The detailed configuration and operation of this system are described below.

[1083] Server Roles and Functions

[1084] Receiving and analyzing user speech data

[1085] The server receives user speech data sent from the device in real time. Specifically, data communication is carried out using WebSocket or HTTP protocols. The received speech data is analyzed using natural language processing (NLP) technology (e.g., Google Cloud Natural Language API) and emotion recognition technology (e.g., IBM Watson Tone Analyzer). This allows the server to understand the topic of the discussion and the emotions of the users.

[1086] Evidence collection and reliability assessment

[1087] The server collects relevant evidence from an internal database or external sources (e.g., Google Scholar, Web of Science) based on the analyzed content and sentiment of comments. The collected evidence is evaluated for reliability using machine learning algorithms (e.g., Random Forest, Support Vector Machine), allowing appropriate evidence to be selected as the basis for discussion.

[1088] Visualization generation and presentation

[1089] The server generates data that illustrates the flow of discussion and key points based on the analysis results and collected evidence. Specifically, it uses D3.js and Chart.js to create visual diagrams. The diagrams also include the user's emotional information and send it to the device, allowing the user to view it in real time.

[1090] Learning from past discussion data and making suggestions

[1091] The server uses TensorFlow and PyTorch to input past discussion data into machine learning models to learn new patterns and effective solutions, and then automatically generates proposals and solutions for the current discussion and sends them to the device.

[1092] Role and Function of the Emotion Engine

[1093] Emotion recognition from speech

[1094] The emotion engine analyzes user speech data and recognizes the emotions contained in the speech. This emotion data is used as important information when the server analyzes and supports discussions. The technology used is an NLP toolkit (e.g., nltk, spacy) or an emotion analysis API (e.g., Microsoft Azure Text Analytics).

[1095] Emotion-based evidence selection

[1096] The emotion engine selects appropriate evidence and information based on the recognized emotion. For example, if the user is feeling anxious, it will prioritize collecting and presenting evidence that will alleviate that emotion.

[1097] Visualization of emotional information

[1098] The emotion engine generates data to visually display the recognized emotion information, including the type of emotion (e.g., joy, anger, anxiety, etc.) and intensity. This data is displayed on the device along with the visualization data generated by the server.

[1099] Device roles and functions

[1100] Providing a user interface

[1101] The terminal provides a user interface to users. Through this interface, users can input comments and questions and send them to the server. The terminal also displays the visualized data, evidence information, and emotion information sent from the server, allowing users to check the progress of the discussion in real time. Technologies used include React.js and Vue.js.

[1102] View Feedback

[1103] The device instantly displays feedback, evidence, suggestions based on learning results, and emotional information received from the server to the user, allowing users to always advance discussions based on the latest information.

[1104] User Roles and Experience

[1105] Participating in discussions and providing emotional information

[1106] Users can participate in discussions through their devices and input their opinions and questions. The emotion engine recognizes the emotions contained in the comments and sends them to the server. This enables the server to support discussions while taking into account the users' emotions.

[1107] Interpreting and using graphical information

[1108] Users can refer to the graphical information and emotional information displayed on the device to understand the flow of discussion and key points, thereby clarifying issues and problems and providing a foundation for efficient and fair decision-making.

[1109] Examples of concrete examples and prompts

[1110] For example, consider a discussion about how to promote a new product. If User A types, "Let's start a discussion about how to promote our new product," the server receives this statement and analyzes it using NLP and an emotion engine. At the same time, if User B types, "We should use social media for our new product. Past campaigns have been successful," and expresses some uncertainty, the server will collect evidence about the effectiveness of social media and evaluate its reliability. Furthermore, because the emotion engine recognizes this uncertainty, highly reliable evidence is prioritized.

[1111] Based on this, illustrated information is displayed on the device, making it easier for users to visually grasp the flow of the discussion. Emotional information is also displayed, allowing other participants to proceed with the discussion while taking User B's emotions into consideration. Furthermore, by learning from past discussion data, the server generates suggestions such as "In past success stories, social media campaigns have been effective for specific target groups," and displays these to the user via their device. This allows users to engage in evidence-based, more logical, and constructive discussions.

[1112] Prompt Sentence Examples

[1113] "Start a discussion about how to promote the new product. Gather evidence of past successes and effective methods, and consider user sentiment as you proceed with the discussion."

[1114] In this way, the present invention is a system that supports evidence-based, logical, and constructive discussions in real time, and furthermore, enables discussions that take into account the user's emotions using an emotion engine, thereby promoting fair and transparent communication.

[1115] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1116] Step 1:

[1117] Users input comments or questions through the device's user interface. For example, the input can be text data such as "Let's start a discussion about how to promote a new product." The device captures this comment data and sends it to the server using WebSocket or HTTP protocol. The server receives this comment data.

[1118] Step 2:

[1119] The server analyzes the received speech data. The input is the user's speech data, which is sent to natural language processing (NLP) technology (e.g., Google Cloud Natural Language API) to analyze the speech content. At the same time, emotion is analyzed using emotion recognition technology (e.g., IBM Watson Tone Analyzer). The output is the speech theme, keywords, and emotional information.

[1120] Step 3:

[1121] The server collects relevant evidence based on the analyzed speech content and sentiment. The input is the analysis results, and based on this, data is collected from internal databases and external sources (e.g., Google Scholar, Web of Science). The collected evidence is evaluated for reliability using a machine learning algorithm (e.g., Random Forest). Highly reliable evidence is selected as the output.

[1122] Step 4:

[1123] The server generates data that visualizes the flow of discussion, key points, and sentiment information based on the analysis results and collected evidence. The input is reliable evidence and analysis results. Visualization tools such as D3.js and Chart.js are used for visualization. Visual visualization data is generated as output and sent to the device.

[1124] Step 5:

[1125] The server inputs past discussion data into a machine learning model (e.g., TensorFlow, PyTorch) to learn new patterns and effective solutions. The input is past discussion data, and learning is based on this. As an output, new patterns and proposals are generated and sent to the device.

[1126] Step 6:

[1127] The terminal receives graphical data, evidence, suggestions, and emotion information from the server and displays them in real time on the user interface. The input is data sent from the server. This allows users to visually grasp the progress of the discussion and evidence, and receive feedback in real time.

[1128] Step 7:

[1129] The user then looks at the device's interface and types in a new comment or question. This input includes new comments or questions based on suggestions from the server. The device captures this input and sends it back to the server, repeating the cycle and moving the discussion forward.

[1130] (Application example 2)

[1131] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1132] Real-time customer service and customer support in virtual stores requires advanced technology. It is particularly difficult to accurately recognize customer emotions and instantly provide suggestions and evidence based on those emotions. Therefore, systems are needed to improve the quality of customer service. Furthermore, there is a lack of interfaces to facilitate smooth communication between customers and staff, and this needs to be resolved.

[1133] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for receiving utterance data from users and analyzing the utterance content using natural language processing; means for collecting related evidence from an internal database or external information source based on the utterance content and evaluating its reliability; means for illustrating the flow of discussion and points of contention based on the analyzed utterance content and the collected evidence; means for learning from past discussion data and extracting and presenting new patterns and effective solutions; means for recognizing emotions contained in utterances and selecting and displaying appropriate evidence based on the emotions; and means for making customer service suggestions based on emotions and supporting customer service in real time. This enables real-time customer service support that takes customer emotions into consideration in a virtual store.

[1134] A "user" is an entity that provides and receives information through the system, such as a customer who uses a service or a customer service staff member.

[1135] "Utterance data" refers to text information and voice data provided by user input, which is used by the system for analysis.

[1136] "Natural language processing" is a technology that allows computers to understand and analyze human language, and is used to analyze the content of user utterances.

[1137] "Evidence" refers to relevant information and supporting documents provided based on the content and intent of a user's statements, and provides a basis for supporting the discussion.

[1138] An "internal database" is a collection of information held within the system and is a database used to collect evidence related to the content of statements.

[1139] "External sources" are sources for collecting information from outside the system, such as the Internet or other external databases.

[1140] "Credibility assessment" is the process of evaluating the accuracy and veracity of collected evidence, and is an evaluation carried out to select highly reliable information.

[1141] "Visualization" is the process of visually displaying analyzed data and evidence, presenting information to users in an easy-to-understand manner.

[1142] "Past discussion data" refers to discussions that have taken place in the past and their records, and is data that can be learned to derive new patterns and solutions.

[1143] "Emotion recognition" is a technology that analyzes and recognizes the emotions contained in a user's speech, and is used to understand the user's psychological state.

[1144] "Customer service suggestions" is the process of presenting customer service methods and advice based on recognized emotions and statements.

[1145] "Real-time" refers to the process of receiving, analyzing, and presenting data almost simultaneously, providing an immediate response without delay.

[1146] The present invention relates to a real-time customer support system for a virtual store. This system receives utterance data entered by users (customers or wait staff) and analyzes it using natural language processing. It also collects relevant evidence based on the content of the utterances and evaluates its reliability. Furthermore, by combining this with emotion recognition technology, it provides responses and suggestions that take the user's emotions into consideration. Below, an embodiment of the present invention will be described in detail.

[1147] Program Overview

[1148] Server Roles and Functions

[1149] The server performs the following functions:

[1150] 1. Receiving and analyzing user speech data

[1151] Speech data sent by users is received in real time and analyzed using natural language processing (NLP) technology. Software used for this includes NLP libraries such as SpaCy. Keywords are extracted from the analyzed speech content to identify the content of the speech.

[1152] 2. Recognizing emotions in speech

[1153] Using an emotion recognition library such as TextBlob, we analyze the emotions contained in the user's speech, which allows us to identify the emotions the user is feeling (e.g., joy, anger, anxiety, etc.).

[1154] 3. Evidence collection and reliability assessment

[1155] Based on the extracted keywords and recognized sentiment, relevant evidence is collected from internal databases or external sources, and the reliability of the evidence is evaluated to select reliable information.

[1156] 4. Visualization Generation and Presentation

[1157] Based on the analysis results and collected evidence, the flow of discussion and points of contention are visualized. NetworkX is used to visually display the relationships and flow of data, and the server sends the generated visualized data to the terminal.

[1158] Device roles and functions

[1159] The terminal performs the following functions:

[1160] 1. Providing a user interface

[1161] The terminal provides a user interface, allowing users to input comments and questions, and displays the visualized data and evidence information sent from the server in real time, allowing users to check the progress of the discussion.

[1162] 2. Viewing Feedback

[1163] The device instantly displays feedback, evidence, suggestions based on learning results, and emotional information received from the server to the user, allowing users to always advance discussions based on the latest information.

[1164] User Roles and Experience

[1165] Users can use the system as follows:

[1166] 1. Participating in discussions and providing emotional information

[1167] Users can participate in discussions through their devices and input their opinions and questions. The emotion engine recognizes the emotions contained in the comments and sends them to the server. This allows the server to support discussions while taking into account the users' emotions.

[1168] 2. Interpretation and Use of Graphical Information

[1169] Users can refer to the graphical information and emotional information displayed on their devices to understand the flow of the discussion and key points at issue, which allows them to handle customer inquiries efficiently and fairly.

[1170] Specific examples

[1171] For example, if a customer types, "Tell me about recent campaigns," the system analyzes this in real time and provides relevant campaign information and evidence. If emotion recognition indicates that the customer has a positive emotion, the system will provide additional information or suggestions that will further enhance that emotion.

[1172] Example prompt sentence:

[1173] Customer message: "Tell me about your latest promotions."

[1174] Analyzed Keywords: Campaign

[1175] Emotion: Positive

[1176] Recommended evidence: "Our latest 20% discount offer is running now. Plus, there are additional perks for new users."

[1177] In this way, the system can respond immediately to customer demands and emotions and provide high-quality service.

[1178] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1179] Step 1:

[1180] The server receives user utterance data. The input is the user's utterance (text or audio data). The output is the received utterance data. This data is then analyzed in the next processing step.

[1181] Step 2:

[1182] The server analyzes the received utterance data. Specifically, it uses natural language processing (NLP) technology to analyze the content of the utterance. The software used is an NLP library such as SpaCy. It receives the utterance data as input and extracts keywords and the grammatical structure of the sentences. The analyzed keywords and structural data of the utterance content are obtained as output.

[1183] Step 3:

[1184] The server uses emotion recognition technology to recognize emotions from the analyzed utterance data. The emotion recognition library used is TextBlob. As input, it receives the analyzed utterance data and identifies the user's emotion (e.g., positive, negative, neutral). As output, it obtains the recognized emotion data.

[1185] Step 4:

[1186] The server collects relevant evidence based on the analyzed utterance data and recognized emotions from an internal database or external sources, using the internet or external APIs as needed. It receives keywords and emotion data as input, and searches for and collects relevant evidence. The collected evidence data is obtained as output.

[1187] Step 5:

[1188] The server evaluates the reliability of the collected evidence. Specifically, it uses the source of the evidence, past performance, etc. as evaluation criteria. It receives the collected evidence data as input and scores or ranks its reliability. The output is the evidence data that has undergone reliability evaluation.

[1189] Step 6:

[1190] The server visualizes the flow of discussion and points of contention based on the analysis results and collected evidence. The software used is NetworkX. It receives analysis data and evidence data as input and generates graphical data to display them visually. The graphical data is obtained as output.

[1191] Step 7:

[1192] The server uses machine learning models to learn from past discussion data and extract new patterns and effective solutions. It receives past discussion data as input and extracts new patterns for the current discussion and future. The output is proposal data.

[1193] Step 8:

[1194] The terminal provides a user interface, allowing users to input comments in real time and displaying graphical data and evidence information sent from the server. As input, the terminal receives graphical data and evidence information from the server and displays them to the user. As output, the terminal provides an interface that allows users to input new comments and questions.

[1195] Step 9:

[1196] The terminal immediately displays the feedback, evidence, and emotion information received from the server to the user. As input, it receives feedback data, evidence, and emotion data from the server and displays them on the user interface. As output, it displays a display that allows users to advance the discussion based on the latest information.

[1197] Step 10:

[1198] Users participate in discussions through their terminals and input their opinions and questions. As input, speech data is provided to the terminal and sent to the server. As output, emotions are analyzed by the emotion engine and sent to the server.

[1199] Step 11:

[1200] Users refer to the graphical data and emotional information displayed on their devices to understand the flow of the discussion and key points. As input, they receive the graphical data and emotional information displayed on their devices and advance the discussion. As output, more logical and constructive discussions based on evidence become possible.

[1201] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1202] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1203] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1204] [Fourth embodiment]

[1205] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1206] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1207] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1208] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1209] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1210] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1211] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1212] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1213] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1214] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1215] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1216] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1217] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1218] This invention relates to a system that supports discussions in real time in which users participate, and promotes fair and logical discussions. The main components of the system are a server, terminals, and users, each of which performs a specific function.

[1219] Server Roles and Functions

[1220] 1. Receiving and analyzing user speech data

[1221] The server receives user speech data sent from the device in real time. It then analyzes the received speech data using natural language processing (NLP) technology to identify and classify the content and intent of the speech. This allows it to understand the topic of the discussion and lead to the collection of appropriate evidence.

[1222] 2. Evidence collection and reliability assessment

[1223] The server collects relevant evidence from an internal database and external sources based on the analyzed comments, evaluates the reliability of the collected evidence, and selects appropriate evidence to support the discussion. This allows users to advance discussions based on well-founded information.

[1224] 3. Visualization Generation and Presentation

[1225] The server generates data that illustrates the flow of discussion and points of contention based on the analysis results and collected evidence. This includes visual formats such as graphs, flowcharts, and mind maps. The generated illustrations are sent to the terminals so that users can view them in real time.

[1226] 4. Learning from past discussion data and making suggestions

[1227] The server inputs past discussion data into a machine learning model to learn new patterns and effective solutions, and then automatically generates proposals and solutions for the current discussion and sends them to the device.

[1228] Device roles and functions

[1229] 1. Providing a user interface

[1230] The terminal provides a user interface to users. Through this interface, users can input comments and questions and send them to the server. The terminal also displays the visualized data and evidence information sent from the server, allowing users to check the progress of the discussion in real time.

[1231] 2. Viewing Feedback

[1232] The device instantly displays feedback, evidence, and suggestions based on learning results received from the server to the user, allowing users to always advance discussions based on the latest information.

[1233] User Roles and Experience

[1234] 1. Participating in discussions and providing evidence

[1235] Users can participate in discussions through their terminals, input their opinions and questions, and, if necessary, submit their own evidence to the server to use as the basis for the discussion.

[1236] 2. Interpretation and Use of Graphical Information

[1237] Users can refer to the graphical information displayed on their devices to understand the flow of discussions and key points, thereby clarifying issues and problems and providing a foundation for efficient and fair decision-making.

[1238] Specific examples

[1239] For example, consider a discussion about how to promote a new product. If User A types, "Let's start a discussion about how to promote the new product," the server receives this statement and analyzes it using NLP. At the same time, if User B types, "We should use social media for the new product. Past campaigns have been successful," the server collects evidence about the effectiveness of social media and evaluates its reliability. Based on this, illustrated information is displayed on the device, making it easier for users to visually grasp the flow of the discussion.

[1240] Furthermore, by learning from past discussion data, the server can generate suggestions such as "In past successes, social media campaigns have been effective for specific target groups," and display them to the user via their device. This allows users to engage in evidence-based, more logical, and constructive discussions.

[1241] In this way, the present invention supports evidence-based, logical, and constructive discussions in real time, promoting fair and transparent communication.

[1242] The processing flow will be explained below.

[1243] Step 1:

[1244] The user inputs a statement or question through the terminal interface, for example, "Let's start a discussion about how to promote our new product."

[1245] Step 2:

[1246] The terminal receives the user's input and transmits the utterance data to the server, which includes the text information of the utterance.

[1247] Step 3:

[1248] The server analyzes the received speech data in real time, using natural language processing (NLP) techniques to identify the content of the speech and extract themes and keywords.

[1249] Step 4:

[1250] Based on the extracted comments, the server collects relevant evidence from internal databases or external sources, for example, looking for data on the effectiveness of "social media" promotions.

[1251] Step 5:

[1252] The server evaluates the reliability of the collected evidence, and based on the evaluation criteria, selects reliable evidence and uses it as the basis for discussion.

[1253] Step 6:

[1254] Based on the analysis results and collected evidence, the server generates data that illustrates the flow of discussion and the points at issue, including graphs and flowcharts.

[1255] Step 7:

[1256] The server transmits the generated visualization data to the terminal, which displays the received data on the user interface in real time.

[1257] Step 8:

[1258] Users can proceed with the discussion while referring to the visualized data and evidence information displayed on their terminals. If necessary, they can enter further comments or questions into their terminals and send them to the server.

[1259] Step 9:

[1260] The server re-analyzes any additional statements or questions, collects and evaluates new evidence as needed, and updates the visualization data in real time and sends it to the terminal.

[1261] Step 10:

[1262] The server inputs past discussion data into a machine learning model to learn new patterns and effective solutions, and generates suggestions for the current discussion based on the learning results.

[1263] Step 11:

[1264] The server sends the generated proposals and solutions to the terminal, which displays them on the user interface for the user to refer to.

[1265] Step 12:

[1266] The user considers the suggestions and solutions displayed on the terminal and continues the discussion. If necessary, the user can re-enter comments or questions and use the system repeatedly.

[1267] Example 1

[1268] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1269] Conventional discussion support systems lack sufficient analysis and support to support the progress of discussions in real time based on user comments. Furthermore, they lack the ability to collect evidence, evaluate its reliability, and visualize the flow and points of discussion, making it difficult for users to engage in evidence-based, logical, and fair discussions. Furthermore, they lack the ability to effectively extract new solutions and proposals from past discussion data and present them in real time.

[1270] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1271] In this invention, the server includes means for receiving user comment data and analyzing the comment content using natural language processing technology, means for collecting related evidential materials based on the comment content from an internal data management system or external information sources and evaluating the reliability of the collected evidential materials, means for generating data that visually displays the flow of discussion and points of contention based on the analyzed comment content and the collected evidential materials, and means for learning past discussion data using a machine learning model and automatically extracting and presenting new patterns and effective solutions. This enables users to engage in evidence-based, logical, and fair discussion in real time.

[1272] "User" refers to a person who uses the system to participate in discussions and enter their own opinions and questions.

[1273] "Speech data" refers to text data such as opinions and questions entered by users in discussions.

[1274] "Natural language processing technology" refers to technology that uses computers to analyze human language and make it easier to understand its content and intent.

[1275] "Evidence" refers to information or data that forms the basis of an argument, including data collected from internal databases or external sources.

[1276] A "data management system" refers to a system that can efficiently manage and search data stored within it.

[1277] "External information sources" refer to means or systems for obtaining information from external databases or the Internet.

[1278] "Assessing reliability" refers to the process of determining whether collected evidence is accurate.

[1279] "Data that visually displays the flow of discussion and key points" refers to data such as graphs and charts that clearly visualize the progress of the discussion and key points.

[1280] A "machine learning model" refers to an algorithm that learns patterns and regularities based on large amounts of data.

[1281] "New patterns and effective solutions" refers to new trends and problem-solving methods that are derived from past data.

[1282] "Presenting" refers to displaying information or suggestions to the user.

[1283] This invention is a system that supports discussions in which users participate in real time and promotes fair and logical discussions, and its main components are a server, terminals, and users.

[1284] The server receives user speech data and analyzes the content of the speech using natural language processing technology. Specifically, it streams the data using Apache Kafka and analyzes it using Google Cloud Natural Language API. By identifying and classifying the content and intent of the speech, it understands the topic of the discussion.

[1285] Based on the analysis results, the server collects relevant evidence from internal data management systems (e.g., MySQL) and external sources (e.g., academic paper databases). The collected evidence is then evaluated for its reliability using machine learning libraries such as Scikit-learn. This process ensures that only reliable information is selected as the basis for discussion.

[1286] Next, the server generates data that visually displays the flow of the discussion and the points at issue based on the analysis results and collected evidence, using D3.js to generate graphs, flowcharts, mind maps, etc. The generated visualization data is then sent to the device via WebSocket.

[1287] The server then uses machine learning models (e.g., TensorFlow) to learn from past discussion data and extract new patterns and effective solutions. Based on this, the server generates proposals and solutions that are presented to users in real time.

[1288] The terminal provides a user interface through which users can input comments and questions. The terminal also displays visualized data and supporting documents sent from the server, allowing users to check the progress of the discussion in real time. Users can participate in the discussion through the terminal and input their own opinions and questions. For example, users can input comments such as "We will start a discussion on how to promote our new product," or specific suggestions such as "We should utilize social media for our new product. Past campaigns have been successful."

[1289] In this system, the server performs analysis and evidence collection, evaluates reliability, generates visualized data, and extracts suggestions, and the terminal displays these to the user, allowing the user to engage in evidence-based, logical, and fair discussion in real time.

[1290] Example prompt: "We're opening a discussion about how to promote a new product. Please provide suggestions based on past successes and reliable evidence."

[1291] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1292] Step 1:

[1293] The user inputs a statement.

[1294] The user enters "Let's start a discussion on how to promote our new product" into the input field of the terminal. The input speech data is in text format, and the terminal sends this data to the server via WebSocket.

[1295] Input: The text of the speech entered by the user into the terminal.

[1296] Output: Speech data sent to the server via WebSocket

[1297] Step 2:

[1298] The server receives the speech data and analyzes it using natural language processing technology.

[1299] The server uses Apache Kafka to receive data sent from the devices in real time. The received speech data is analyzed using the Google Cloud Natural Language API to identify the content and intent of the speech. For example, keywords such as "new product" and "promotion method" are extracted to understand the topic of the discussion.

[1300] Input: Speech data sent from the device

[1301] Output: Parsed keywords and themes

[1302] Step 3:

[1303] The server collects the evidence and evaluates its reliability.

[1304] Based on the analyzed themes, the server collects relevant evidence from internal data management systems and external sources (e.g., academic paper databases). The collected evidence is evaluated for credibility using Scikit-learn, for example, by measuring the number of citations of a paper.

[1305] Input: Parsed keywords and themes

[1306] Output: Evidence with reliability evaluation

[1307] Step 4:

[1308] The server generates data that visualizes the flow of discussion and key points.

[1309] The server uses D3.js to generate visualizations such as graphs and flowcharts based on the analysis results and supporting documentation. For example, a bar graph showing the success rate of past social media campaigns is created. The generated data is sent to the device via WebSocket.

[1310] Input: Analysis results and evidence that has been evaluated for reliability

[1311] Output: The generated visualization data

[1312] Step 5:

[1313] The device displays visualization data and feedback in real time.

[1314] The device then displays the received visualization data on a dashboard, allowing users to see the progress of the discussion in real time, including a bar graph showing the success rate and related supporting documents.

[1315] Input: Visualization data sent from the server

[1316] Output: Visualized data and evidence displayed on the device

[1317] Step 6:

[1318] The server learns from past discussion data and generates new proposals.

[1319] The server uses TensorFlow to learn from past discussion data and extract new patterns and effective solutions. For example, it generates suggestions based on past successes, such as "a social media campaign was effective for a specific target demographic." The generated suggestions are sent to the device.

[1320] Input: Past discussion data

[1321] Output: Extracted new proposals and solutions

[1322] Step 7:

[1323] The device displays new suggestions and feedback to the user.

[1324] The device displays new suggestions sent from the server to the user in real time. For example, specific suggestions such as "Examples of successful social media campaigns" are displayed on the dashboard. Based on this information, the user can make the next statement.

[1325] Input: New suggestions and feedback sent by the server

[1326] Output: suggestions and feedback displayed on the device

[1327] (Application example 1)

[1328] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1329] In today's business environment, discussions and meetings between store staff and customers must be conducted efficiently and logically. However, it is difficult to provide real-time discussion support or immediate evidence, making it difficult for users to quickly resolve the issues they face and make decisions. Furthermore, there is a lack of a way to visually grasp the flow of discussions and key points at issue, which can lead to overlooking key points for deriving optimal solutions. Furthermore, it is difficult to learn new patterns and effective solutions from past discussion data and apply them to the current situation.

[1330] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1331] In this invention, the server includes means for receiving user utterance data and analyzing the utterance content using natural language processing, means for collecting related evidence based on the utterance content from an internal database or external information source and evaluating its reliability, means for illustrating the flow of discussion and points of contention based on the analyzed utterance content and the collected evidence, means for learning from past discussion data and extracting and presenting new patterns and effective solutions, means for converting user utterances into text using speech recognition technology and transmitting the text to the server, and means for providing the analysis results and evidence to a terminal that visually displays them. This allows users to engage in fair and logical discussions in real time and enables efficient decision-making.

[1332] "User utterance data" refers to voice data or text data uttered by a user during a discussion or meeting.

[1333] "Natural language processing" is a set of techniques that allow computers to understand and analyze human language.

[1334] "Evidence" refers to information or data that serves as the basis for an argument.

[1335] An "internal database" is a collection of accumulated data managed within the system.

[1336] "External information sources" refers to information managed outside the system, such as materials on the Internet or external databases.

[1337] "Measures to assess reliability" are processes or techniques used to determine the accuracy or reliability of collected evidence.

[1338] "Visualization" is the visual representation of information or data.

[1339] The "flow of discussion" indicates the process of how the discussion progresses.

[1340] An "issue" is an important topic or focal point in a discussion.

[1341] "Past discussion data" refers to records and information relating to discussions that have taken place in the past.

[1342] "Means of extracting new patterns and effective solutions" refers to techniques and technologies for finding useful patterns and solutions from past data.

[1343] "Converting user speech into text using speech recognition technology" is the process of converting a user's speech into text data.

[1344] A "server" is a computer system that processes and analyzes data and provides services to clients over a network.

[1345] "Means for providing analysis results and evidence to a terminal that visually displays them" refers to the technology or method by which the server sends analysis results and evidence to a terminal and displays them visually.

[1346] "Providing a user interface to a terminal" means displaying a screen or interface on the terminal that allows the user to operate the system.

[1347] "Updating the visualized data in real time" means instantly updating the visualized information as the discussion progresses.

[1348] "Automatically generating proposals based on past data" means that the system automatically creates new proposals based on past data.

[1349] The system embodying this invention has the functions of recognizing and analyzing the comments of users participating in a discussion in real time, collecting and evaluating related evidence, and visualizing the flow of the discussion and the points at issue. The detailed configuration and operation of the system are described below.

[1350] 1. Program Generation

[1351] The server receives speech data from the user and analyzes it using natural language processing (NLP) technology. The user's speech is converted into text using speech recognition technology through the smart glasses and sent to the server. Based on the analyzed data, relevant evidence is then collected from an internal database and external sources and its reliability is evaluated. This operation is achieved by converting speech to text using the Google Speech Recognition API and sending the data to the server using the requests library.

[1352] 2. Processing Description

[1353] When the server receives the user's speech data, it first converts the speech data into text using speech recognition technology, then analyzes the text data using NLP technology to identify the content and intent of the speech, and based on this analysis, begins the process of collecting relevant evidence and assessing its reliability.

[1354] For example, if someone says, "Show me evidence about a new promotion method," the server will collect relevant evidence from internal databases and external information sources and evaluate its reliability, taking into account the source of the collected information, its past usage history, and other factors.

[1355] The flow of the discussion and the points at issue are then visually illustrated based on the analyzed comments and collected evidence. This visualization is done using mind maps, flowcharts, and other tools. This visual data is displayed on the smart glasses, allowing users to understand the progress of the discussion in real time.

[1356] 3. Specific Examples

[1357] Specifically, when a user says, "We'll start discussing how to promote our new product," the server recognizes this speech and converts it into text. Next, when User B says, "We should utilize social media for our new product. Past campaigns have been successful," the server collects evidence about the effectiveness of social media and evaluates its reliability. This allows reliable information to be collected and displayed on the smart glasses.

[1358] In addition, by learning from past discussion data, the server generates suggestions such as "In past success stories, campaigns using social media have been effective for specific target groups," and displays these to the user.

[1359] Examples of prompts include "Show evidence of a new promotional method," "Tell me examples of successful campaigns from the past," and "Diagram the flow of the discussion."

[1360] By implementing this invention, meetings and discussions in physical stores can be conducted more efficiently and logically, enabling fair decision-making.

[1361] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1362] Step 1: Convert user speech into text using speech recognition technology

[1363] When a user speaks through the smart glasses, their voice is collected by the smart glasses' microphone, and this voice data is converted into text using the Google Speech Recognition API.

[1364] Input: User's voice

[1365] Output: Textual speech data

[1366] How it works: When a user says, "We'll start a discussion about new promotional methods," the microphone in the smart glasses collects voice data and converts it into text data using the Google Speech Recognition API.

[1367] Step 2: Send the textual utterance data to the server

[1368] The text data is sent from the smart glasses to a server via the Internet as an HTTP request using the requests library.

[1369] Input: Textual speech data

[1370] Output: Speech data sent to the server

[1371] Specific operation: The user's textual utterance data is sent from the smart glasses to the server. For example, the user's utterance "We will start discussing new promotion methods" is sent to the server as text data.

[1372] Step 3: Analyze the speech data using natural language processing

[1373] The server analyzes the received speech data using natural language processing (NLP) to identify the content and intent of the speech, and understands the topic of the discussion based on the results of this analysis.

[1374] Input: Speech data sent to the server

[1375] Output: Parsed speech and themes

[1376] Specific operation: The server analyzes the statement "We will start a discussion about new promotion methods" using an NLP engine and understands that this statement means to start a discussion about promotion.

[1377] Step 4: Collect relevant evidence and assess its reliability

[1378] Based on the analysis results, the server collects relevant evidence from internal databases and external sources, evaluates the reliability of that evidence, and selects only reliable evidence.

[1379] Input: Parsed speech and themes

[1380] Output: Evidence assessed for reliability

[1381] What it does: The server collects promotion-related statistics and past success stories from internal databases and publicly available internet data, and rates the reliability of each source.

[1382] Step 5: Diagram the flow of discussion and points

[1383] Based on the collected evidence and analysis results, the server generates mind maps and flowcharts to visualize the flow of discussion and key points.

[1384] Input: Evidence and analytical results assessed for reliability

[1385] Output: Diagram of discussion flow and points

[1386] Specific operation: Based on the evidence, the server visualizes the structure of the discussion as a mind map and generates data to be displayed on the smart glasses.

[1387] Step 6: Providing visualized data to smart glasses

[1388] The visualized flow of discussion and points of contention are transmitted from the server to the smart glasses and displayed to the user in real time.

[1389] Input: Diagram of discussion flow and points

[1390] Output: Visual data displayed on smart glasses

[1391] Specific operation: The server sends the generated mind maps and flowcharts to the smart glasses, allowing the user to view them in real time.

[1392] Step 7: Learn from past discussion data and generate new proposals

[1393] The server inputs past discussion data into a machine learning model to learn new patterns and effective solutions, and automatically generates suggestions for the current discussion and provides them to the smart glasses.

[1394] Input: Past discussion data and current discussion status

[1395] Output: Auto-generated suggestions

[1396] Specific operation: By learning data from past successful campaigns, the system generates suggestions for the current discussion, such as "In past successful cases, campaigns using social media were effective," and displays these suggestions on the smart glasses.

[1397] Through these steps, this system efficiently and logically supports discussions and meetings in physical stores, enabling rapid decision-making.

[1398] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1399] This invention relates to a system that supports discussions in real time and promotes fair and logical discussions in which users participate. In particular, the invention aims to further improve the quality of discussions by combining an emotion engine that recognizes users' emotions and supports discussions based on those emotions.

[1400] Server Roles and Functions

[1401] 1. Receiving and analyzing user speech data

[1402] The server receives user speech data sent from the device in real time. It analyzes the received speech data using natural language processing (NLP) and emotion recognition technology to identify the content of the speech and the user's emotions. This allows it to understand the topic of the discussion and the emotions of the user.

[1403] 2. Evidence collection and reliability assessment

[1404] The server collects relevant evidence from an internal database or external sources based on the analyzed content and sentiment, evaluates the reliability of the collected evidence, and selects appropriate evidence as the basis for discussion. It also selects specific evidence based on sentiment.

[1405] 3. Visualization Generation and Presentation

[1406] The server generates data that illustrates the flow of the discussion and the key points at issue based on the analysis results and collected evidence. It also includes the user's emotional information in the illustrated data and displays it visually. The generated illustration is sent to the device, allowing the user to view it in real time.

[1407] 4. Learning from past discussion data and making suggestions

[1408] The server inputs past discussion data into a machine learning model to learn new patterns and effective solutions, and then automatically generates proposals and solutions for the current discussion and sends them to the device.

[1409] Role and Function of the Emotion Engine

[1410] 1. Emotion Recognition from Speech

[1411] The emotion engine analyzes user comment data and recognizes the emotions contained in the comments. This emotion data is used as important information when the server analyzes and supports discussions.

[1412] 2. Emotion-based evidence filtering

[1413] The emotion engine selects appropriate evidence and information based on the recognized emotion. For example, if the user is feeling anxious, it will prioritize collecting and presenting evidence that will alleviate that emotion.

[1414] 3. Visualization of emotional information

[1415] The emotion engine generates data to visually display the recognized emotion information, including the type of emotion (e.g., joy, anger, anxiety, etc.) and intensity. The generated emotion data is displayed on the device along with the flow and points of the discussion.

[1416] Device roles and functions

[1417] 1. Providing a user interface

[1418] The terminal provides a user interface to users. Through this interface, users can input comments and questions and send them to the server. The terminal also displays the visualized data, evidence information, and emotion information sent from the server, allowing users to check the progress of the discussion in real time.

[1419] 2. Viewing Feedback

[1420] The device instantly displays feedback and evidence received from the server, suggestions based on learning results, and emotional information to the user, allowing users to always advance discussions based on the latest information.

[1421] User Roles and Experience

[1422] 1. Participating in discussions and providing emotional information

[1423] Users can participate in discussions through their devices and input their opinions and questions. The emotion engine recognizes the emotions contained in the comments and sends them to the server. This enables the server to support discussions while taking into account the users' emotions.

[1424] 2. Interpretation and Use of Graphical Information

[1425] Users can refer to the graphical information and emotional information displayed on the device to understand the flow of discussion and key points, thereby clarifying issues and problems and providing a foundation for efficient and fair decision-making.

[1426] Specific examples

[1427] For example, consider a discussion about how to promote a new product. If User A types, "Let's start a discussion about how to promote our new product," the server receives this statement and analyzes it using NLP and an emotion engine. At the same time, if User B types, "We should use social media for our new product. Past campaigns have been successful," and expresses some uncertainty, the server will collect evidence about the effectiveness of social media and evaluate its reliability. Furthermore, because the emotion engine recognizes this uncertainty, highly reliable evidence is prioritized.

[1428] Based on this, illustrated information is displayed on the device, making it easier for users to visually grasp the flow of the discussion. In addition, emotional information is also displayed, so other participants can proceed with the discussion while taking into consideration User B's emotions.

[1429] Furthermore, by learning from past discussion data, the server can generate suggestions such as "In past successes, social media campaigns have been effective for specific target groups," and display them to the user via their device. This allows users to engage in evidence-based, more logical, and constructive discussions.

[1430] In this way, the present invention is a system that supports evidence-based, logical, and constructive discussions in real time, and furthermore, enables discussions that take into account the user's emotions using an emotion engine, thereby promoting fair and transparent communication.

[1431] The processing flow will be explained below.

[1432] Step 1:

[1433] The user inputs a statement or question through the terminal interface, for example, "Let's start a discussion about how to promote our new product."

[1434] Step 2:

[1435] The terminal receives the user's input and transmits the utterance data to the server, which includes the text information of the utterance.

[1436] Step 3:

[1437] The server analyzes the received speech data in real time, using natural language processing (NLP) techniques to identify the content of the speech and extract themes and keywords.

[1438] Step 4:

[1439] The server recognizes emotions from user utterances using an emotion engine, which identifies emotions (e.g., joy, anger, anxiety, etc.) contained in the utterances.

[1440] Step 5:

[1441] Based on the analyzed content and perceived sentiment, the server collects relevant evidence from internal databases or external sources, for example, looking for data on the effectiveness of "social media" promotions.

[1442] Step 6:

[1443] The server evaluates the reliability of the collected evidence. Based on the evaluation criteria, it selects reliable evidence and uses it as the basis for discussion. Based on emotional information, it prioritizes and selects appropriate evidence for a specific emotion.

[1444] Step 7:

[1445] Based on the analysis results, recognized emotions, and collected evidence, the server generates data that illustrates the flow of discussion and points of contention, including graphs, flowcharts, and emotional information.

[1446] Step 8:

[1447] The server transmits the generated visualization data to the terminal, which displays the received data on the user interface in real time.

[1448] Step 9:

[1449] Users can proceed with the discussion while referring to the graphical data, evidence information, and emotion information displayed on their terminals. If necessary, they can input further comments or questions into their terminals and send them to the server.

[1450] Step 10:

[1451] The server re-analyzes any additional statements or questions, collects and evaluates new evidence as needed, and sends updated graphical data and emotional information to the device in real time.

[1452] Step 11:

[1453] The server inputs past discussion data into a machine learning model to learn new patterns and effective solutions, and generates suggestions for the current discussion based on the learning results.

[1454] Step 12:

[1455] The server sends the generated proposals and solutions to the terminal, which displays them on the user interface for the user to refer to.

[1456] Step 13:

[1457] The user considers the suggestions and solutions displayed on the terminal and continues the discussion. If necessary, the user can re-enter comments or questions and use the system repeatedly.

[1458] Step 14:

[1459] As the discussion progresses, the server continuously recognizes the user's emotions using an emotion engine and adjusts the discussion support accordingly based on that information. This helps to ensure an efficient discussion while maintaining a balance between emotions and evidence.

[1460] Example 2

[1461] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1462] Conventional discussion support systems only analyze the content of user comments and do not take into account user emotions, making it difficult to hold fair and logical discussions. Furthermore, they often lack sufficient visualization to visually grasp the progress of the discussion, and do not properly evaluate the reliability of evidence. Furthermore, past discussion data is not effectively utilized, limiting the ability to extract new patterns and solutions. There is a need for a system that can solve these problems and improve the quality of discussions.

[1463] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1464] In this invention, the server includes means for receiving user utterance data and analyzing the utterance content and emotions using natural language processing, means for collecting related evidence from an internal database or external information sources based on the analyzed utterance content and emotions and evaluating their reliability, means for illustrating the flow of the discussion, issues, and emotional information based on the analysis results and the collected evidence, and means for learning from past discussion data and extracting and presenting new patterns and effective solutions. This makes it possible to support fair and logical discussions that take user emotions into consideration, and by visually understanding the progress of the discussion, it is possible to improve the quality of the discussion.

[1465] "User comment data" refers to text data of discussions and questions that users input into their terminals and send to the server.

[1466] "Natural language processing" refers to techniques and methods that allow computers to understand, interpret, and generate human language.

[1467] "Emotion analysis" refers to the process of identifying emotions (e.g., joy, anger, anxiety, etc.) contained in user speech data.

[1468] "Evidence" refers to the data, examples, and information used as the basis for an argument.

[1469] An "internal database" is a type of data storage for saving data used within the system.

[1470] "External information source" is a general term for a location that can be accessed from outside the system to obtain information.

[1471] "Assessing reliability" is the process of determining the accuracy and validity of the evidence collected.

[1472] "Graphing" is the process of presenting information in a visual form, such as a graph or chart.

[1473] "Past discussion data" refers to the accumulation of text data and analysis results from previous discussions.

[1474] "Extracting new patterns and effective solutions" refers to the process of finding new ways and strategies to advance discussions efficiently and effectively through the analysis of past data.

[1475] This invention is a system that supports discussions in which users participate in real time and promotes fair and logical discussions. In particular, it aims to further improve the quality of discussions by combining it with an emotion engine that recognizes users' emotions and supports discussions based on those emotions. The detailed configuration and operation of this system are described below.

[1476] Server Roles and Functions

[1477] Receiving and analyzing user speech data

[1478] The server receives user speech data sent from the device in real time. Specifically, data communication is carried out using WebSocket or HTTP protocols. The received speech data is analyzed using natural language processing (NLP) technology (e.g., Google Cloud Natural Language API) and emotion recognition technology (e.g., IBM Watson Tone Analyzer). This allows the server to understand the topic of the discussion and the emotions of the users.

[1479] Evidence collection and reliability assessment

[1480] The server collects relevant evidence from an internal database or external sources (e.g., Google Scholar, Web of Science) based on the analyzed content and sentiment of comments. The collected evidence is evaluated for reliability using machine learning algorithms (e.g., Random Forest, Support Vector Machine), allowing appropriate evidence to be selected as the basis for discussion.

[1481] Visualization generation and presentation

[1482] The server generates data that illustrates the flow of discussion and key points based on the analysis results and collected evidence. Specifically, it uses D3.js and Chart.js to create visual diagrams. The diagrams also include the user's emotional information and send it to the device, allowing the user to view it in real time.

[1483] Learning from past discussion data and making suggestions

[1484] The server uses TensorFlow and PyTorch to input past discussion data into machine learning models to learn new patterns and effective solutions, and then automatically generates proposals and solutions for the current discussion and sends them to the device.

[1485] Role and Function of the Emotion Engine

[1486] Emotion recognition from speech

[1487] The emotion engine analyzes user speech data and recognizes the emotions contained in the speech. This emotion data is used as important information when the server analyzes and supports discussions. The technology used is an NLP toolkit (e.g., nltk, spacy) or an emotion analysis API (e.g., Microsoft Azure Text Analytics).

[1488] Emotion-based evidence selection

[1489] The emotion engine selects appropriate evidence and information based on the recognized emotion. For example, if the user is feeling anxious, it will prioritize collecting and presenting evidence that will alleviate that emotion.

[1490] Visualization of emotional information

[1491] The emotion engine generates data to visually display the recognized emotion information, including the type of emotion (e.g., joy, anger, anxiety, etc.) and intensity. This data is displayed on the device along with the visualization data generated by the server.

[1492] Device roles and functions

[1493] Providing a user interface

[1494] The terminal provides a user interface to users. Through this interface, users can input comments and questions and send them to the server. The terminal also displays the visualized data, evidence information, and emotion information sent from the server, allowing users to check the progress of the discussion in real time. Technologies used include React.js and Vue.js.

[1495] View Feedback

[1496] The device instantly displays feedback, evidence, suggestions based on learning results, and emotional information received from the server to the user, allowing users to always advance discussions based on the latest information.

[1497] User Roles and Experience

[1498] Participating in discussions and providing emotional information

[1499] Users can participate in discussions through their devices and input their opinions and questions. The emotion engine recognizes the emotions contained in the comments and sends them to the server. This enables the server to support discussions while taking into account the users' emotions.

[1500] Interpreting and using graphical information

[1501] Users can refer to the graphical information and emotional information displayed on the device to understand the flow of discussion and key points, thereby clarifying issues and problems and providing a foundation for efficient and fair decision-making.

[1502] Examples of concrete examples and prompts

[1503] For example, consider a discussion about how to promote a new product. If User A types, "Let's start a discussion about how to promote our new product," the server receives this statement and analyzes it using NLP and an emotion engine. At the same time, if User B types, "We should use social media for our new product. Past campaigns have been successful," and expresses some uncertainty, the server will collect evidence about the effectiveness of social media and evaluate its reliability. Furthermore, because the emotion engine recognizes this uncertainty, highly reliable evidence is prioritized.

[1504] Based on this, illustrated information is displayed on the device, making it easier for users to visually grasp the flow of the discussion. Emotional information is also displayed, allowing other participants to proceed with the discussion while taking User B's emotions into consideration. Furthermore, by learning from past discussion data, the server generates suggestions such as "In past success stories, social media campaigns have been effective for specific target groups," and displays these to the user via their device. This allows users to engage in evidence-based, more logical, and constructive discussions.

[1505] Prompt Sentence Examples

[1506] "Start a discussion about how to promote the new product. Gather evidence of past successes and effective methods, and consider user sentiment as you proceed with the discussion."

[1507] In this way, the present invention is a system that supports evidence-based, logical, and constructive discussions in real time, and furthermore, enables discussions that take into account the user's emotions using an emotion engine, thereby promoting fair and transparent communication.

[1508] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1509] Step 1:

[1510] Users input comments or questions through the device's user interface. For example, the input can be text data such as "Let's start a discussion about how to promote a new product." The device captures this comment data and sends it to the server using WebSocket or HTTP protocol. The server receives this comment data.

[1511] Step 2:

[1512] The server analyzes the received speech data. The input is the user's speech data, which is sent to natural language processing (NLP) technology (e.g., Google Cloud Natural Language API) to analyze the speech content. At the same time, emotion is analyzed using emotion recognition technology (e.g., IBM Watson Tone Analyzer). The output is the speech theme, keywords, and emotional information.

[1513] Step 3:

[1514] The server collects relevant evidence based on the analyzed speech content and sentiment. The input is the analysis results, and based on this, data is collected from internal databases and external sources (e.g., Google Scholar, Web of Science). The collected evidence is evaluated for reliability using a machine learning algorithm (e.g., Random Forest). Highly reliable evidence is selected as the output.

[1515] Step 4:

[1516] The server generates data that visualizes the flow of discussion, key points, and sentiment information based on the analysis results and collected evidence. The input is reliable evidence and analysis results. Visualization tools such as D3.js and Chart.js are used for visualization. Visual visualization data is generated as output and sent to the device.

[1517] Step 5:

[1518] The server inputs past discussion data into a machine learning model (e.g., TensorFlow, PyTorch) to learn new patterns and effective solutions. The input is past discussion data, and learning is based on this. As an output, new patterns and proposals are generated and sent to the device.

[1519] Step 6:

[1520] The terminal receives graphical data, evidence, suggestions, and emotion information from the server and displays them in real time on the user interface. The input is data sent from the server. This allows users to visually grasp the progress of the discussion and evidence, and receive feedback in real time.

[1521] Step 7:

[1522] The user then looks at the device's interface and types in a new comment or question. This input includes new comments or questions based on suggestions from the server. The device captures this input and sends it back to the server, repeating the cycle and moving the discussion forward.

[1523] (Application example 2)

[1524] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1525] Real-time customer service and customer support in virtual stores requires advanced technology. It is particularly difficult to accurately recognize customer emotions and instantly provide suggestions and evidence based on those emotions. Therefore, systems are needed to improve the quality of customer service. Furthermore, there is a lack of interfaces to facilitate smooth communication between customers and staff, and this needs to be resolved.

[1526] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for receiving utterance data from users and analyzing the utterance content using natural language processing; means for collecting related evidence from an internal database or external information source based on the utterance content and evaluating its reliability; means for illustrating the flow of discussion and points of contention based on the analyzed utterance content and the collected evidence; means for learning from past discussion data and extracting and presenting new patterns and effective solutions; means for recognizing emotions contained in utterances and selecting and displaying appropriate evidence based on the emotions; and means for making customer service suggestions based on emotions and supporting customer service in real time. This enables real-time customer service support that takes customer emotions into consideration in a virtual store.

[1527] A "user" is an entity that provides and receives information through the system, such as a customer who uses a service or a customer service staff member.

[1528] "Utterance data" refers to text information and voice data provided by user input, which is used by the system for analysis.

[1529] "Natural language processing" is a technology that allows computers to understand and analyze human language, and is used to analyze the content of user utterances.

[1530] "Evidence" refers to relevant information and supporting documents provided based on the content and intent of a user's statements, and provides a basis for supporting the discussion.

[1531] An "internal database" is a collection of information held within the system and is a database used to collect evidence related to the content of statements.

[1532] "External sources" are sources for collecting information from outside the system, such as the Internet or other external databases.

[1533] "Credibility assessment" is the process of evaluating the accuracy and veracity of collected evidence, and is an evaluation carried out to select highly reliable information.

[1534] "Visualization" is the process of visually displaying analyzed data and evidence, presenting information to users in an easy-to-understand manner.

[1535] "Past discussion data" refers to discussions that have taken place in the past and their records, and is data that can be learned to derive new patterns and solutions.

[1536] "Emotion recognition" is a technology that analyzes and recognizes the emotions contained in a user's speech, and is used to understand the user's psychological state.

[1537] "Customer service suggestions" is the process of presenting customer service methods and advice based on recognized emotions and statements.

[1538] "Real-time" refers to the process of receiving, analyzing, and presenting data almost simultaneously, providing an immediate response without delay.

[1539] The present invention relates to a real-time customer support system for a virtual store. This system receives utterance data entered by users (customers or wait staff) and analyzes it using natural language processing. It also collects relevant evidence based on the content of the utterances and evaluates its reliability. Furthermore, by combining this with emotion recognition technology, it provides responses and suggestions that take the user's emotions into consideration. Below, an embodiment of the present invention will be described in detail.

[1540] Program Overview

[1541] Server Roles and Functions

[1542] The server performs the following functions:

[1543] 1. Receiving and analyzing user speech data

[1544] Speech data sent by users is received in real time and analyzed using natural language processing (NLP) technology. Software used for this includes NLP libraries such as SpaCy. Keywords are extracted from the analyzed speech content to identify the content of the speech.

[1545] 2. Recognizing emotions in speech

[1546] Using an emotion recognition library such as TextBlob, we analyze the emotions contained in the user's speech, which allows us to identify the emotions the user is feeling (e.g., joy, anger, anxiety, etc.).

[1547] 3. Evidence collection and reliability assessment

[1548] Based on the extracted keywords and recognized sentiment, relevant evidence is collected from internal databases or external sources, and the reliability of the evidence is evaluated to select reliable information.

[1549] 4. Visualization Generation and Presentation

[1550] Based on the analysis results and collected evidence, the flow of discussion and points of contention are visualized. NetworkX is used to visually display the relationships and flow of data, and the server sends the generated visualized data to the terminal.

[1551] Device roles and functions

[1552] The terminal performs the following functions:

[1553] 1. Providing a user interface

[1554] The terminal provides a user interface, allowing users to input comments and questions, and displays the visualized data and evidence information sent from the server in real time, allowing users to check the progress of the discussion.

[1555] 2. Viewing Feedback

[1556] The device instantly displays feedback, evidence, suggestions based on learning results, and emotional information received from the server to the user, allowing users to always advance discussions based on the latest information.

[1557] User Roles and Experience

[1558] Users can use the system as follows:

[1559] 1. Participating in discussions and providing emotional information

[1560] Users can participate in discussions through their devices and input their opinions and questions. The emotion engine recognizes the emotions contained in the comments and sends them to the server. This allows the server to support discussions while taking into account the users' emotions.

[1561] 2. Interpretation and Use of Graphical Information

[1562] Users can refer to the graphical information and emotional information displayed on their devices to understand the flow of the discussion and key points at issue, which allows them to handle customer inquiries efficiently and fairly.

[1563] Specific examples

[1564] For example, if a customer types, "Tell me about recent campaigns," the system analyzes this in real time and provides relevant campaign information and evidence. If emotion recognition indicates that the customer has a positive emotion, the system will provide additional information or suggestions that will further enhance that emotion.

[1565] Example prompt sentence:

[1566] Customer message: "Tell me about your latest promotions."

[1567] Analyzed Keywords: Campaign

[1568] Emotion: Positive

[1569] Recommended evidence: "Our latest 20% discount offer is running now. Plus, there are additional perks for new users."

[1570] In this way, the system can respond immediately to customer demands and emotions and provide high-quality service.

[1571] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1572] Step 1:

[1573] The server receives user utterance data. The input is the user's utterance (text or audio data). The output is the received utterance data. This data is then analyzed in the next processing step.

[1574] Step 2:

[1575] The server analyzes the received utterance data. Specifically, it uses natural language processing (NLP) technology to analyze the content of the utterance. The software used is an NLP library such as SpaCy. It receives the utterance data as input and extracts keywords and the grammatical structure of the sentences. The analyzed keywords and structural data of the utterance content are obtained as output.

[1576] Step 3:

[1577] The server uses emotion recognition technology to recognize emotions from the analyzed utterance data. The emotion recognition library used is TextBlob. As input, it receives the analyzed utterance data and identifies the user's emotion (e.g., positive, negative, neutral). As output, it obtains the recognized emotion data.

[1578] Step 4:

[1579] The server collects relevant evidence based on the analyzed utterance data and recognized emotions from an internal database or external sources, using the internet or external APIs as needed. It receives keywords and emotion data as input, and searches for and collects relevant evidence. The collected evidence data is obtained as output.

[1580] Step 5:

[1581] The server evaluates the reliability of the collected evidence. Specifically, it uses the source of the evidence, past performance, etc. as evaluation criteria. It receives the collected evidence data as input and scores or ranks its reliability. The output is the evidence data that has undergone reliability evaluation.

[1582] Step 6:

[1583] The server visualizes the flow of discussion and points of contention based on the analysis results and collected evidence. The software used is NetworkX. It receives analysis data and evidence data as input and generates graphical data to display them visually. The graphical data is obtained as output.

[1584] Step 7:

[1585] The server uses machine learning models to learn from past discussion data and extract new patterns and effective solutions. It receives past discussion data as input and extracts new patterns for the current discussion and future. The output is proposal data.

[1586] Step 8:

[1587] The terminal provides a user interface, allowing users to input comments in real time and displaying graphical data and evidence information sent from the server. As input, the terminal receives graphical data and evidence information from the server and displays them to the user. As output, the terminal provides an interface that allows users to input new comments and questions.

[1588] Step 9:

[1589] The terminal immediately displays the feedback, evidence, and emotion information received from the server to the user. As input, it receives feedback data, evidence, and emotion data from the server and displays them on the user interface. As output, it displays a display that allows users to advance the discussion based on the latest information.

[1590] Step 10:

[1591] Users participate in discussions through their terminals and input their opinions and questions. As input, speech data is provided to the terminal and sent to the server. As output, emotions are analyzed by the emotion engine and sent to the server.

[1592] Step 11:

[1593] Users refer to the graphical data and emotional information displayed on their devices to understand the flow of the discussion and key points. As input, they receive the graphical data and emotional information displayed on their devices and advance the discussion. As output, more logical and constructive discussions based on evidence become possible.

[1594] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1595] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1596] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1597] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1598] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1599] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1600] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1601] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1602] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1603] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1604] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1605] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1606] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1607] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1608] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1609] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1610] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1611] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1612] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1613] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1614] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1615] The following is further disclosed regarding the above embodiment.

[1616] (Claim 1)

[1617] means for receiving utterance data from a user and analyzing the utterance content using natural language processing;

[1618] A means of gathering relevant evidence based on what is said from internal databases or external sources and assessing its reliability;

[1619] A method for illustrating the flow of discussion and points of contention based on the analyzed statements and collected evidence, and

[1620] A means of learning from past discussion data, extracting and presenting new patterns and effective solutions, and

[1621] A system including:

[1622] (Claim 2)

[1623] 10. The system of claim 1, further receiving additional comments or questions from the user and updating the visualization data in real time.

[1624] (Claim 3)

[1625] The system of claim 1, wherein a user interface is provided on the terminal, and the progress and evidence of the illustrated discussion are displayed to the user in real time.

[1626] "Example 1"

[1627] (Claim 1)

[1628] A means for receiving speech data from a user and analyzing the speech content using natural language processing technology;

[1629] A means of collecting relevant evidence based on the statements from internal data management systems or external sources and assessing their reliability;

[1630] A means for generating data that visually displays the flow of discussion and points of contention based on the analyzed statements and collected evidence;

[1631] A method to learn from past discussion data using a machine learning model and automatically extract and present new patterns and effective solutions.

[1632] A system including:

[1633] (Claim 2)

[1634] 10. The system of claim 1, further comprising: receiving additional comments or questions from the user and updating the visual display data in real time.

[1635] (Claim 3)

[1636] 2. The system according to claim 1, wherein a user interface is provided on the terminal, and the progress of the discussion and supporting documents are displayed to the user in real time.

[1637] "Application Example 1"

[1638] (Claim 1)

[1639] means for receiving utterance data from a user and analyzing the utterance content using natural language processing;

[1640] A means of gathering relevant evidence based on what is said from internal databases or external sources and assessing its reliability;

[1641] A method for illustrating the flow of discussion and points of contention based on the analyzed statements and collected evidence, and

[1642] A means of learning from past discussion data, extracting and presenting new patterns and effective solutions, and

[1643] A means for converting user speech into text using speech recognition technology and transmitting the text to a server;

[1644] A means of providing analytical results and evidence to a device that displays them visually;

[1645] A system including:

[1646] (Claim 2)

[1647] 10. The system of claim 1, wherein the system collects user voice through a user interface and updates visualization data in real time.

[1648] (Claim 3)

[1649] 2. The system of claim 1, wherein suggestions based on past data are automatically generated and displayed on the terminal.

[1650] "Example 2: Combining Emotion Engines"

[1651] (Claim 1)

[1652] means for receiving speech data from a user and analyzing the speech content and emotions using natural language processing;

[1653] A means for collecting relevant evidence from an internal database or external sources based on the analyzed statements and sentiment, and assessing their reliability;

[1654] A method for visualizing the flow of discussion, points of contention, and emotional information based on the analysis results and collected evidence;

[1655] A means of learning from past discussion data, extracting and presenting new patterns and effective solutions, and

[1656] A system including:

[1657] (Claim 2)

[1658] The system of claim 1 further receives additional comments or questions from the user and updates the visualization data and emotion information in real time.

[1659] (Claim 3)

[1660] The system of claim 1, which provides a user interface on a terminal and displays graphical discussion progress, evidence, and sentiment information to a user in real time.

[1661] "Application example 2 when combining emotion engines"

[1662] (Claim 1)

[1663] means for receiving utterance data from a user and analyzing the utterance content using natural language processing;

[1664] A means of gathering relevant evidence based on what is said from internal databases or external sources and assessing its reliability;

[1665] A method for illustrating the flow of discussion and points of contention based on the analyzed statements and collected evidence, and

[1666] A means of learning from past discussion data, extracting and presenting new patterns and effective solutions, and

[1667] A means of recognizing the sentiment contained in a statement and filtering and displaying appropriate evidence based on that sentiment;

[1668] A means to provide customer service suggestions based on emotions and support customer service in real time,

[1669] A system including:

[1670] (Claim 2)

[1671] 10. The system of claim 1, further receiving additional comments or questions from the user and updating the visualization data and sentiment-based suggestion data in real time.

[1672] (Claim 3)

[1673] The system of claim 1, which provides a user interface on the terminal and displays graphical discussion progress, evidence, and sentiment information to the user in real time. [Explanation of symbols]

[1674] ...

Claims

1. means for receiving utterance data from a user and analyzing the utterance content using natural language processing; A means of gathering relevant evidence based on what is said from internal databases or external sources and assessing its reliability; A method for illustrating the flow of discussion and points of contention based on the analyzed statements and collected evidence, and A means of learning from past discussion data, extracting and presenting new patterns and effective solutions, and A system including:

2. 10. The system of claim 1, further comprising: a means for receiving additional comments or questions from the user and updating the visualization data in real time.

3. The system according to claim 1, wherein a user interface is provided on a terminal, and the progress and evidence of a graphical discussion are displayed to a user in real time.

Images