system

Abstract

Provide a system. 【Solution means】 Means for collecting and storing past case data, Means for executing a generation model that performs learning based on the accumulated data, Means for analyzing data in real time and proposing an optimal solution, Means for accepting user feedback and updating the analysis results, Means for protecting data privacy, Means for analyzing speech information and non-verbal data collected in real time and inferring the intentions and emotions of participants, Means for processing video data and recognizing non-verbal information, Means for providing an optimized solution and reflecting feedback in model learning, A system including the above.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a character of the chatbot, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In modern society, disputes and consensus formation regarding contracts occurring between companies or individuals often take a long time and tend to be costly. In particular, during the negotiation process, there is a need for rapid data analysis to accurately understand the intentions and emotions of the participants and effectively reflect them in a solution, as well as the presentation of appropriate solutions. However, conventional methods often cannot fully utilize past case studies and are often inefficient processes. As a result, there is a problem that consensus formation is delayed and there is a waste of time and resources in dispute resolution.

Means for Solving the Problems

[0005] This invention provides a means for collecting and accumulating past dispute resolution cases and learning from a generative model based on them. This model has the ability to analyze negotiation data in real time and propose the optimal solution. Furthermore, by incorporating user feedback and updating the analysis results, it can derive more accurate solutions in real time. In addition, by thoroughly protecting data privacy, it is possible to handle highly confidential information securely. This makes it possible to realize a system that supports rapid and efficient consensus building.

[0006] "Past dispute resolution cases" refer to data and records that show how disputes that occurred in the past were resolved among the parties involved.

[0007] A "generative model" is an algorithm or structure that uses machine learning techniques to learn patterns from data and generate results for new data.

[0008] "Real-time analysis" is a process that processes ongoing events and information immediately and provides analysis results quickly.

[0009] An "optimal solution" is the solution that is judged to best satisfy the interests of both parties under the given conditions and circumstances.

[0010] "User feedback" refers to user opinions, evaluations, or input for improvement regarding a solution.

[0011] "Privacy protection" refers to legal and technical measures taken to prevent third parties from unauthorized access to or use of personal information and highly confidential data. [Brief explanation of the drawing]

[0012] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2]This is a conceptual diagram showing an example of the essential functions of the data processing device and smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] This is a sequence diagram showing the processing flow of the data processing system in Example 2, when an emotion engine is combined. [Figure 14] This is a sequence diagram showing the processing flow of the data processing system in Application Example 2, which combines an emotion engine. [Modes for carrying out the invention]

[0013] Hereinafter, an example of an embodiment of the system relating to the technology of this disclosure will be described with reference to the attached drawings.

[0014] First, the terms used in the following description will be explained.

[0015] In the following embodiments, the numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0016] In the following embodiments, the numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0017] In the following embodiments, the numbered storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, etc.

[0018] In the following embodiments, the numbered communication I / F (Interface) is an interface including a communication processor and an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark), etc.

[0019] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0020] [First Embodiment]

[0021] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0022] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0023] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0024] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0025] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0026] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0027] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0028] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0029] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0030] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0031] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0032] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0033] This invention provides specific methods for effectively performing data collection, model training, real-time analysis, solution prediction, and user feedback reception in a dispute resolution support system. This enables rapid and efficient consensus building.

[0034] First, the server collects historical case data related to dispute resolution from specialized databases and digital archives. This includes court records, negotiation histories, and legal documents. The collected data is converted into a reusable format and prepared for model training.

[0035] Subsequently, this data is used to train a generative AI model. The server extracts meaningful patterns using an automated process and learns results and solutions from similar cases. This learning process enables the model to propose solutions immediately and appropriately in new negotiations.

[0036] Next, once negotiations begin, the terminals collect real-time data (such as voice utterances, text input, and facial expressions and movements analyzed via video). Using this information, the server performs data analysis to infer the participants' intentions, emotions, and positions. Based on this, the optimal solution is generated, taking into account past learning results.

[0037] The server then provides the proposed solutions to the user's terminal. The solutions are presented as multiple options based on their feasibility and likelihood of success. The user evaluates these options through the interface and provides feedback as needed.

[0038] As a concrete example, consider the use of this system in a contract dispute between companies. The user first inputs data on past contract disputes into the system. The server analyzes this information, searches its database for similar cases, and utilizes a learning model. As a result, the model quickly proposes the optimal agreement during negotiations, allowing the user to proceed with agreement formation on the spot. Through this process, agreement formation is made more efficient, contributing to the early resolution of disputes.

[0039] The following describes the processing flow.

[0040] Step 1:

[0041] The server calls a specialized database API to collect historical case data related to dispute resolution, retrieving court records, negotiation details, and case law data. The retrieved data is cleaned for duplicates and errors and stored as a clean dataset.

[0042] Step 2:

[0043] The server trains a generative AI model using a clean dataset. The data input to the model is classified, and key information necessary for learning is highlighted through feature extraction. After training, the model learns patterns from past cases and acquires knowledge applicable to new conflict resolution situations.

[0044] Step 3:

[0045] The device uses voice and video sensors during negotiations to capture real-time statements, facial expressions, and the other party's reactions, and transmits this data to a server. This data includes indicators of the participants' intentions and emotions.

[0046] Step 4:

[0047] The server analyzes the received real-time data using natural language processing and image analysis algorithms. Based on the analysis results, it infers the intentions and emotions of the other party in negotiations and generates the optimal solution. The generated solution reflects past learning achievements.

[0048] Step 5:

[0049] The server prioritizes solutions based on the analysis and presents them to the terminal as multiple options. Each option includes detailed information about its effectiveness and expected impact.

[0050] Step 6:

[0051] Users select the most appropriate option from those presented via their device and provide feedback as needed. This feedback includes not only manual evaluations but also the provision of new information.

[0052] Step 7:

[0053] The device sends user feedback to the server, which then uses that feedback to fine-tune the AI ​​model. This refines the solution and improves the quality of proposals in future negotiations.

[0054] Step 8:

[0055] The server implements encryption and access control during data transmission, reception, and storage to ensure thorough privacy protection. This guarantees that data handled during the dispute resolution process will not be leaked externally and will be managed securely.

[0056] (Example 1)

[0057] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0058] In conflict resolution, there is a need to provide swift and appropriate solutions based on past cases. However, conventional systems have shortcomings in data collection and analysis, making it difficult to grasp participants' intentions and emotions in real time and to provide flexible solutions.

[0059] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0060] In this invention, the server includes a device for collecting and storing past problem-solving cases, a device for executing generative artificial intelligence that learns based on the accumulated information, and a device for analyzing information during negotiations in real time and proposing the optimal solution. This makes it possible to infer the intentions and emotions of the participants, quickly generate and present a variety of solutions, and flexibly improve them based on user evaluations.

[0061] "Past problem-solving examples" refer to specific cases and approaches to various problems that have been solved in the past.

[0062] A "storage device" refers to a system equipped with the functionality to efficiently store collected data and make it accessible as needed.

[0063] "Generative artificial intelligence" refers to algorithms and models that learn from collected data and generate new information.

[0064] "Information being negotiated in real time" refers to dynamic data such as voice, actions, and facial expressions generated during the negotiation process.

[0065] "Inferring intentions and emotions" refers to the process of analyzing collected data and forming hypotheses about participants' goals and emotions.

[0066] "Proposing a solution" refers to outlining specific steps or strategies for solving a problem based on the analysis results.

[0067] "Diverse solutions" refers to multiple solutions offered from different perspectives and approaches.

[0068] "User evaluation" refers to opinions and feedback from users regarding the solutions provided by the system.

[0069] "Flexible improvement" refers to the ability to adjust the system's suggested solutions and processes based on user feedback.

[0070] To implement this invention, the system will function using the following hardware and software.

[0071] First, the server accesses specialized databases and digital archives to collect and store past problem-solving cases. Specifically, it uses Python scripts to retrieve data via APIs and saves it to a MySQL® database. The collected data is preprocessed using natural language processing libraries such as NLTK and spaCy, and prepared as a training dataset for generative AI models.

[0072] Next, the server builds a generative AI model using machine learning frameworks such as TENSORFLOW® or PyTorch based on the pre-processed data. The model has the ability to learn from past cases, extract patterns, and predict new solutions.

[0073] The terminal collects real-time data using sensors and cameras while negotiations are taking place. For example, audio data is collected via a microphone and converted into text using the Google® Speech-to-Text API. OpenCV is used for video analysis to capture participants' facial expressions and movements.

[0074] Based on the collected real-time data, the server performs intent and sentiment analysis, and generates the optimal solution by comparing it with past learning results. Multiple solutions are then presented to the user's device, and the user provides evaluation and feedback through the interface.

[0075] As a concrete example, consider a scenario in which this system is used in contract negotiations between companies. The user inputs data on past contract disputes into the server. The server analyzes this information and utilizes a generation AI model that references similar cases in the database. As a result, the optimal agreement is quickly presented during negotiations. For example, by giving instructions such as "Propose the best solution for the current contract negotiation," the system will quickly provide a solution based on that instruction. This enables rapid agreement formation.

[0076] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0077] Step 1:

[0078] The server collects past problem-solving cases from specialized databases and digital archives. It receives data via API as input and stores this data in a MySQL database. Data collection is performed using a Python script, and the collected data is saved for subsequent analysis and model training.

[0079] Step 2:

[0080] The server preprocesses the collected data. The input is the raw data obtained in step 1. This data is processed using natural language processing libraries such as NLTK and spaCy, including tokenization, stop word removal, and syntactic analysis. The output is clean text data, which is used to train the generative AI model.

[0081] Step 3:

[0082] The server trains a generative AI model using pre-processed data. It receives processed text data as input and trains a machine learning model using TensorFlow or PyTorch. This process improves the ability to extract patterns and predict solutions through supervised and unsupervised learning. The output is the trained AI model.

[0083] Step 4:

[0084] The device collects information about ongoing negotiations in real time. It receives audio and video data from sensors as input. Audio data is captured using a microphone and converted to text using the Google Speech-to-Text API. Video data is captured by a camera and analyzed using OpenCV to infer participants' intentions and emotions from their facial expressions and movements. The output is the analyzed real-time data.

[0085] Step 5:

[0086] The server analyzes real-time data sent from the terminal and generates the optimal solution. It receives real-time analysis data as input, compares it with past learning results, and generates a solution. The generated solution is presented to the user as output.

[0087] Step 6:

[0088] The server receives feedback from users. Input consists of user ratings and comments on the options submitted through the evaluation screen. This feedback is used to improve the AI ​​model and update the data to help suggest more appropriate solutions. Output consists of an improved AI model and the addition of new data.

[0089] (Application Example 1)

[0090] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0091] In modern society, conflicts and troubles require swift and safe resolution. However, current systems struggle with real-time data analysis and the presentation of appropriate solutions, particularly inferring intentions and emotions from nonverbal information in collective settings. This hinders effective consensus building among participants. A solution is needed to address these challenges and achieve smooth problem resolution while maintaining security.

[0092] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0093] In this invention, the server includes means for collecting and storing past case data, means for executing a generative model that learns based on the accumulated data, and means for analyzing the data in real time and proposing the optimal solution. This enables accurate analysis of participants' intentions and emotions, and efficient consensus building.

[0094] "Past case data" refers to records of similar conflicts and problems that have occurred in the past, and serves as learning material for conflict resolution.

[0095] "Storing" refers to the act of systematically saving collected data within a system such as a database so that it can be used effectively.

[0096] A "generative model" is an AI model built to derive and propose patterns and solutions from data using machine learning algorithms.

[0097] "Analyzing data in real time" is the process of immediately processing ongoing situations and information to obtain analysis results.

[0098] "Nonverbal information" refers to information such as body movements and facial expressions that are used as clues to infer a person's emotions and intentions.

[0099] "Processing video data" refers to the process of analyzing visual data acquired from devices such as cameras and extracting specific information.

[0100] "Incorporating feedback into model learning" means making adjustments to improve the performance of an AI model based on evaluations and responses obtained from users.

[0101] In order to implement this invention, it is important to properly configure the server, user terminal, and necessary software environment.

[0102] The server collects past case data and stores it in a database. Specifically, it efficiently retrieves relevant data from various public databases and internal documents, and prepares it as a training dataset for machine learning. This data is used to train a generative AI model and forms the basis for proposing solutions to new conflicts and problems. Python and TensorFlow can be used to build the generative AI model.

[0103] The user terminal is designed as a smartphone or smart glasses and is equipped with the ability to collect audio and video in real time. This allows for the acquisition of nonverbal information such as voice, facial expressions, and body language in negotiations and disputes, and this data is sent to a server for immediate analysis. The analysis uses tools such as OpenCV to process and interpret emotions and intentions from the video data in real time.

[0104] The user receives solutions provided by the server through an interface, evaluates these options, and provides feedback. This feedback is used to improve the accuracy of the generated AI model on the server and to enhance the accuracy of future suggestions. This allows the system to continuously improve itself.

[0105] As a concrete example, in a work environment where misunderstandings may occur during communication, the system intervenes immediately when an incident occurs to facilitate rapid problem resolution. An example of a prompt message would be, "Which solution is best to mitigate the incident caused by past misunderstandings?"

[0106] This configuration allows the system to respond promptly and accurately, and is expected to facilitate the rapid resolution of complex conflicts and problems.

[0107] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0108] Step 1:

[0109] The server collects historical case data. It gathers relevant information from databases and public documents and converts it into a format usable by machine learning. At this stage, the input is unorganized raw data, and the output is a structured dataset. Specifically, natural language processing techniques are used to analyze legal documents and negotiation records.

[0110] Step 2:

[0111] The server trains the generative AI model. Using the structured data obtained in Step 1, the AI ​​model is trained using TensorFlow. The case dataset is used as input, and the output is a generalized solution pattern. The model automatically learns the optimal solution using a pattern recognition algorithm.

[0112] Step 3:

[0113] The device collects data in real time. During negotiations or when problems occur, it uses smartphones or smart glasses to acquire audio and video data. The input data consists of audio files and video frames, which are transferred to the server as compressed digital data. Non-verbal information is also acquired simultaneously through the device's sensors.

[0114] Step 4:

[0115] The server performs real-time analysis. It receives data sent from the terminal, analyzes facial expressions and movements from video using OpenCV, and converts audio data into text using natural language processing technology. Based on the analyzed intent and emotion information, a generative AI model proposes the optimal solution. The input is real-time data, and the output is the inferred intent and emotion, as well as the proposed solution.

[0116] Step 5:

[0117] The user evaluates the proposed solutions. The server receives the solutions presented to the terminal and allows the user to review each option through the interface. The input consists of the presented solutions, and the user selects one and returns feedback to the server. The evaluation results are collected and used to improve the model.

[0118] Step 6:

[0119] The server utilizes feedback to optimize the generated AI model. It analyzes feedback data collected from users and updates the model's parameters. The input is user feedback, and the output is the improved AI model. Improved model accuracy leads to higher quality suggestions in subsequent iterations.

[0120] In this series of processes, the roles of the server, terminal, and user work together in coordination, enabling the rapid and effective resolution of problems.

[0121] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0122] This invention describes a method for combining an emotion engine with a conflict resolution support system to grasp the emotional state of participants in real time and utilize this information to propose solutions. This makes it possible to guide consensus building in a more precise and amicable manner.

[0123] The server utilizes an emotion engine to analyze real-time video and audio data collected during negotiations. This analysis detects emotions from participants' facial expressions, tone of voice, and word choice, and then quantitatively evaluates their emotional state based on this data. The emotion engine identifies basic emotions such as joy, anger, sadness, surprise, and fear, providing insights into the dynamics of negotiations.

[0124] The terminal presents the user with emotional data processed by the server, making it easier for the user to understand the emotional state of the other party during negotiations. This information can also serve as an important indicator when selecting a proposed solution.

[0125] Furthermore, the server uses data obtained from the emotion engine to influence the output of the generative AI model. Specifically, it adjusts the proposed solutions by taking into account the other party's emotional state. For example, if the negotiating partner is stressed, a gentler and less burdensome solution can be prioritized. In this way, the accuracy and effectiveness of the proposals are improved.

[0126] As a concrete example, in the case of negotiations for a business partnership, the user activates the dispute resolution support system via their terminal. As the negotiations progress, the emotion engine analyzes the emotions of the participants, and the server reflects the analysis results in real time. By selecting the solution that best suits the other party's emotions from among several proposed solutions, the user can reach a smoother agreement.

[0127] This invention contributes to increasing the transparency and reliability of negotiations and innovates the process of reaching agreements in dispute resolution.

[0128] The following describes the processing flow.

[0129] Step 1:

[0130] The server collects past dispute resolution cases from the database, performs data cleaning, and converts and stores them in the format required for the generated AI model. This process prepares the model so that it can learn properly and handle new cases.

[0131] Step 2:

[0132] The server trains a generative AI model using cleaned data. The model learns patterns from the cases and accumulates insights that will be useful for suggesting future solutions. This training forms the foundation for the model to accurately predict new cases.

[0133] Step 3:

[0134] Once negotiations begin, the terminal collects audio and video data in real time and sends it to the server. This data includes what is said, facial expressions, and tone of voice. This real-time data is used for instant analysis.

[0135] Step 4:

[0136] The server analyzes the received real-time data using an emotion engine. This analysis quantifies the participants' emotional states, generating an emotional profile. This profile represents basic emotional states such as joy, anger, sadness, and happiness.

[0137] Step 5:

[0138] Based on the analyzed emotional profiles, the server utilizes a generative AI model to propose the optimal solution. Here, adjustments are made to match the participants' emotions, ensuring flexibility in the solution.

[0139] Step 6:

[0140] The device presents the generated solutions to the user. Multiple options are displayed, each with a description of its characteristics and expected effects, taken into account from the emotional profile.

[0141] Step 7:

[0142] Users evaluate the presented solutions, select the option they believe to be the best, and enter feedback into their device. This feedback includes information about whether they adopted the suggestion or made minor adjustments.

[0143] Step 8:

[0144] The terminal sends user feedback to the server, which adjusts the model parameters based on the feedback to improve the accuracy of future suggestions. This feedback loop strengthens the overall learning ability of the system.

[0145] Step 9:

[0146] The server encrypts and stores data, ensuring security to protect information privacy. This allows the system to operate securely and prevents data leaks.

[0147] (Example 2)

[0148] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0149] Traditional dispute resolution support systems have faced the challenge of difficulty in proposing optimal solutions while taking into account the emotional states of the participants. There is a need for a mechanism that understands the emotional dynamics of participants during negotiations and facilitates consensus building based on that understanding.

[0150] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0151] In this invention, the server includes means for collecting and storing past cases, means for executing a generative model that learns based on the accumulated information, and means for analyzing video and audio information obtained from negotiations in real time and quantitatively evaluating the emotions of the participants. This makes it possible to grasp the emotional state of the participants in real time and propose the optimal solution based on that.

[0152] A "case study" refers to a specific event or related incident that occurred in the past, and contains information that can be useful in resolving disputes.

[0153] "Information" is a general term for the data and knowledge collected, and in particular includes data related to video and audio acquired in real time.

[0154] A "generative model" refers to an algorithm or method that learns patterns and trends from data and generates new predictions or suggestions based on them.

[0155] "Analysis" is a general term for processes and analyses performed on data and information to understand their structure and elements, and in particular includes the process of quantifying emotional states.

[0156] "Emotion" refers to a person's psychological state, and in negotiations, it refers to the psychological tendencies and changes of participants.

[0157] "Solutions" refer to a general term for the measures and means proposed to resolve conflicts and problems, and include specific means adjusted by generative AI models.

[0158] "Privacy protection" refers to a set of measures and policies to prevent the improper disclosure of personal information and data.

[0159] This invention is a system for understanding the emotional state of participants in dispute resolution in real time and proposing the optimal solution based on that understanding.

[0160] The server collects video and audio information obtained from the negotiation in real time and processes this data using an emotion analysis engine. Specifically, it uses image recognition software to analyze facial expressions in the video data and an audio analysis algorithm to analyze the tone and speed of voices in the audio data. As a result, the emotional state of the participants is quantified and stored in a database.

[0161] The terminal displays emotional data sent from the server on the user interface, allowing users to visualize the progress of negotiations from an emotional perspective. For example, the degree of emotion is shown with colors or graphs, allowing users to intuitively understand the other party's state.

[0162] The server utilizes machine learning algorithms as a generative AI model to generate optimal solutions based on accumulated emotional data. In this process, prompt statements are input to the generative AI model, and the system proposes the most suitable solution that reflects the emotional data. For example, a prompt statement such as "Gently adjust the solution when participants are experiencing stress" can be used.

[0163] Users can review the proposed solutions through their device and select the most suitable option from multiple choices. This dynamic presentation of options allows users to navigate a smooth process towards agreement while being considerate of the other party's feelings.

[0164] As a concrete example, in business partnership negotiations between companies, activating this system would enable flexible proposals that respond to changes in the other party's emotions, leading to smoother negotiations. This system would improve the transparency and reliability of agreement building and promote an innovative negotiation process.

[0165] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0166] Step 1:

[0167] The server collects video and audio data in real time from the negotiation site as input. External input devices such as cameras and microphones are used for this purpose. This data becomes the basic information for sentiment analysis in the next processing step.

[0168] Step 2:

[0169] The server analyzes the input video data using image recognition software, quantifying emotions from the participants' facial expressions. For example, it identifies emotions such as joy and anger from eye and mouth movements and outputs this as quantified data. In parallel, audio data is processed through a voice analysis algorithm, recognizing emotions from voice tone and pitch, and this too is output as quantified data.

[0170] Step 3:

[0171] The server aggregates the quantified emotion data obtained in Step 2 and inputs it into the emotion analysis engine. This engine uses deep learning to extract complex patterns and evaluate overall emotional trends. This process comprehensively analyzes the participants' current emotional state and outputs the results to a database.

[0172] Step 4:

[0173] The terminal visually displays emotional state data received from the server using a GUI. Graphs and color coding are used to allow the user to intuitively understand the emotional changes of the negotiating partner. For example, heightened emotions are displayed in red, and calmer emotions in blue.

[0174] Step 5:

[0175] The server inputs emotional state data as prompt statements into an AI model, which then generates optimal solution suggestions. Since these prompt statements reflect the participant's subjective emotional state, the suggested solutions take that state into account. The generated solutions are output as data.

[0176] Step 6:

[0177] The user reviews multiple proposed solutions on their device. Each solution is accompanied by an explanation based on the other party's emotional state, allowing the user to choose the option best suited to the other party's changing emotions. This enables the user to conduct negotiations more smoothly.

[0178] (Application Example 2)

[0179] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0180] In customer service, the challenge lies in understanding customers' emotional states in real time, improving the quality of service provided, and increasing customer satisfaction. Furthermore, it is necessary to improve the ability to quickly identify customers' potential dissatisfactions and needs and provide immediate, personalized support.

[0181] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0182] In this invention, the server includes a device for collecting and storing past solution cases, a device for executing a generative model that learns based on the accumulated data, a device for analyzing information in real time and proposing the optimal solution, a device for receiving user feedback and updating the analysis results, a device for protecting the privacy of information, a device for analyzing the video and audio of participants and evaluating their emotional state in real time, and a device for adjusting the proposed solution based on the evaluated emotional state. This makes it possible to accurately grasp the emotional state of customers, provide appropriate services, and increase customer satisfaction.

[0183] "Past case studies" refer to real-world case information that has been collected and stored in the system to date.

[0184] A "generative model" is an algorithm or program that learns from past data and proposes new solutions.

[0185] "Real-time analysis" is a process that instantly processes participants' video and audio data in real time and extracts the necessary information.

[0186] "Emotional state" refers to the mental condition or emotions inferred from nonverbal information such as a participant's facial expressions and tone of voice.

[0187] "Evaluation" is the process of measuring emotional states and the suitability of solutions based on collected data, and then making a judgment.

[0188] "Adjusting solutions" refers to the action of optimizing the proposed solutions based on the emotional information collected.

[0189] "Feedback" refers to the response information provided by users, which is used to improve and adjust the system.

[0190] The system for implementing this invention has a structure that connects a smart device and a server to understand the customer's emotional state and provide appropriate feedback. Specifically, the server receives video and audio data and processes them using an emotion analysis algorithm. This makes it possible to evaluate the customer's emotional state in real time from their facial expressions and tone of voice.

[0191] The hardware used includes smart glasses and high-performance microphones. These capture customer video and audio and transmit the data to a server via Bluetooth or Wi-Fi. Meanwhile, the server processes the data using emotion analysis SDKs such as Affectiva and IBM Watson® to quantitatively evaluate the customer's emotional state.

[0192] Based on this, the server uses a generative AI model to suggest the optimal customer service approach. For example, if a customer is feeling stressed, it might suggest a calmer tone when explaining the product. This allows users to gain guidance on how to improve customer satisfaction.

[0193] As a concrete example, if a customer shows signs of anxiety during customer service at a physical store, this system will suggest "explaining the product's benefits and customer testimonials in detail to reassure them." Furthermore, by inputting prompts like the following into the AI ​​model, further improvement suggestions can be obtained.

[0194] Example of a prompt:

[0195] "The customer appeared to be feeling uneasy based on their expression during the product explanation. Please suggest appropriate countermeasures for this situation."

[0196] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0197] Step 1:

[0198] The terminal captures the customer's video and audio using its built-in camera and microphone. This process takes video and audio data as input and outputs recorded data in digital format. The terminal then preprocesses this data, performing noise reduction and resolution adjustments.

[0199] Step 2:

[0200] The terminal transmits captured data to the server via Bluetooth or Wi-Fi. The input is the video and audio data acquired by the terminal, and the output is the digital data transferred to the server. The terminal checks the stability and speed of data transfer and retries if a transfer error occurs.

[0201] Step 3:

[0202] The server feeds the received data into an emotion analysis algorithm. The input is the video and audio data transferred in the previous step, and the output is the analysis result indicating the emotional state. The server uses Affectiva or IBM Watson SDK to quantify basic emotions such as joy, anger, sadness, and happiness.

[0203] Step 4:

[0204] The server uses a generative AI model to generate optimal customer service suggestions based on sentiment analysis results. In this step, sentiment analysis results are used as input, and specific customer service suggestions and prompts are generated as output. The server then performs an analysis process to present the generated information to the user in real time.

[0205] Step 5:

[0206] The user reviews customer service suggestions provided by the server via their terminal and adjusts their actions as needed. The input is the customer service suggestion from the server, and the output is the user's improved customer service behavior. The user selects the optimal response according to the customer service situation and provides service that meets the customer's needs.

[0207] Step 6:

[0208] The server collects feedback on user behavior and uses it to improve the accuracy of the generative model. The input is user feedback, and the output is an updated analysis algorithm and generative AI model. The server periodically updates the database to prepare for more accurate analysis in future applications.

[0209] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0210] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0211] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0212] [Second Embodiment]

[0213] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0214] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0215] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0216] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0217] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0218] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0219] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0220] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0221] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0222] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0223] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0224] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0225] This invention provides specific methods for effectively performing data collection, model training, real-time analysis, solution prediction, and user feedback reception in a dispute resolution support system. This enables rapid and efficient consensus building.

[0226] First, the server collects historical case data related to dispute resolution from specialized databases and digital archives. This includes court records, negotiation histories, and legal documents. The collected data is converted into a reusable format and prepared for model training.

[0227] Subsequently, this data is used to train a generative AI model. The server extracts meaningful patterns using an automated process and learns results and solutions from similar cases. This learning process enables the model to propose solutions immediately and appropriately in new negotiations.

[0228] Next, once negotiations begin, the terminals collect real-time data (such as voice utterances, text input, and facial expressions and movements analyzed via video). Using this information, the server performs data analysis to infer the participants' intentions, emotions, and positions. Based on this, the optimal solution is generated, taking into account past learning results.

[0229] The server then provides the proposed solutions to the user's terminal. The solutions are presented as multiple options based on their feasibility and likelihood of success. The user evaluates these options through the interface and provides feedback as needed.

[0230] As a concrete example, consider the use of this system in a contract dispute between companies. The user first inputs data on past contract disputes into the system. The server analyzes this information, searches its database for similar cases, and utilizes a learning model. As a result, the model quickly proposes the optimal agreement during negotiations, allowing the user to proceed with agreement formation on the spot. Through this process, agreement formation is made more efficient, contributing to the early resolution of disputes.

[0231] The following describes the processing flow.

[0232] Step 1:

[0233] The server calls a specialized database API to collect historical case data related to dispute resolution, retrieving court records, negotiation details, and case law data. The retrieved data is cleaned for duplicates and errors and stored as a clean dataset.

[0234] Step 2:

[0235] The server trains a generative AI model using a clean dataset. The data input to the model is classified, and key information necessary for learning is highlighted through feature extraction. After training, the model learns patterns from past cases and acquires knowledge applicable to new conflict resolution situations.

[0236] Step 3:

[0237] The device uses voice and video sensors during negotiations to capture real-time statements, facial expressions, and the other party's reactions, and transmits this data to a server. This data includes indicators of the participants' intentions and emotions.

[0238] Step 4:

[0239] The server analyzes the received real-time data using natural language processing and image analysis algorithms. Based on the analysis results, it infers the intentions and emotions of the other party in negotiations and generates the optimal solution. The generated solution reflects past learning achievements.

[0240] Step 5:

[0241] The server prioritizes solutions based on the analysis and presents them to the terminal as multiple options. Each option includes detailed information about its effectiveness and expected impact.

[0242] Step 6:

[0243] Users select the most appropriate option from those presented via their device and provide feedback as needed. This feedback includes not only manual evaluations but also the provision of new information.

[0244] Step 7:

[0245] The device sends user feedback to the server, which then uses that feedback to fine-tune the AI ​​model. This refines the solution and improves the quality of proposals in future negotiations.

[0246] Step 8:

[0247] The server implements encryption and access control during data transmission, reception, and storage to ensure thorough privacy protection. This guarantees that data handled during the dispute resolution process will not be leaked externally and will be managed securely.

[0248] (Example 1)

[0249] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0250] In conflict resolution, there is a need to provide swift and appropriate solutions based on past cases. However, conventional systems have shortcomings in data collection and analysis, making it difficult to grasp participants' intentions and emotions in real time and to provide flexible solutions.

[0251] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0252] In this invention, the server includes a device for collecting and storing past problem-solving cases, a device for executing generative artificial intelligence that learns based on the accumulated information, and a device for analyzing information during negotiations in real time and proposing the optimal solution. This makes it possible to infer the intentions and emotions of the participants, quickly generate and present a variety of solutions, and flexibly improve them based on user evaluations.

[0253] "Past problem-solving examples" refer to specific cases and approaches to various problems that have been solved in the past.

[0254] A "storage device" refers to a system equipped with the functionality to efficiently store collected data and make it accessible as needed.

[0255] "Generative artificial intelligence" refers to algorithms and models that learn from collected data and generate new information.

[0256] "Information being negotiated in real time" refers to dynamic data such as voice, actions, and facial expressions generated during the negotiation process.

[0257] "Inferring intentions and emotions" refers to the process of analyzing collected data and forming hypotheses about participants' goals and emotions.

[0258] "Proposing a solution" refers to outlining specific steps or strategies for solving a problem based on the analysis results.

[0259] "Diverse solutions" refers to multiple solutions offered from different perspectives and approaches.

[0260] "User evaluation" refers to opinions and feedback from users regarding the solutions provided by the system.

[0261] "Flexible improvement" refers to the ability to adjust the system's suggested solutions and processes based on user feedback.

[0262] To implement this invention, the system will function using the following hardware and software.

[0263] First, the server accesses specialized databases and digital archives to collect and store past problem-solving examples. Specifically, it uses Python scripts to retrieve data via APIs and saves it to a MySQL database. The collected data is preprocessed using natural language processing libraries such as NLTK and spaCy, and prepared as a training dataset for generative AI models.

[0264] Next, the server builds a generative AI model using machine learning frameworks such as TensorFlow or PyTorch based on the pre-processed data. The model has the ability to learn from past examples, extract patterns, and predict new solutions.

[0265] The device collects real-time data using sensors and cameras while negotiations are taking place. For example, audio data is collected via a microphone and converted into text using the Google Speech-to-Text API. OpenCV is used for video analysis to capture participants' facial expressions and movements.

[0266] Based on the collected real-time data, the server performs intent and sentiment analysis, and generates the optimal solution by comparing it with past learning results. Multiple solutions are then presented to the user's device, and the user provides evaluation and feedback through the interface.

[0267] As a concrete example, consider a scenario in which this system is used in contract negotiations between companies. The user inputs data on past contract disputes into the server. The server analyzes this information and utilizes a generation AI model that references similar cases in the database. As a result, the optimal agreement is quickly presented during negotiations. For example, by giving instructions such as "Propose the best solution for the current contract negotiation," the system will quickly provide a solution based on that instruction. This enables rapid agreement formation.

[0268] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0269] Step 1:

[0270] The server collects past problem-solving cases from specialized databases and digital archives. It receives data via API as input and stores this data in a MySQL database. Data collection is performed using a Python script, and the collected data is saved for subsequent analysis and model training.

[0271] Step 2:

[0272] The server preprocesses the collected data. The input is the raw data obtained in step 1. This data is processed using natural language processing libraries such as NLTK and spaCy, including tokenization, stop word removal, and syntactic analysis. The output is clean text data, which is used to train the generative AI model.

[0273] Step 3:

[0274] The server trains a generative AI model using pre-processed data. It receives processed text data as input and trains a machine learning model using TensorFlow or PyTorch. This process improves the ability to extract patterns and predict solutions through supervised and unsupervised learning. The output is the trained AI model.

[0275] Step 4:

[0276] The device collects information about ongoing negotiations in real time. It receives audio and video data from sensors as input. Audio data is captured using a microphone and converted to text using the Google Speech-to-Text API. Video data is captured by a camera and analyzed using OpenCV to infer participants' intentions and emotions from their facial expressions and movements. The output is the analyzed real-time data.

[0277] Step 5:

[0278] The server analyzes the real-time data transmitted from the terminal and generates an optimal solution. It receives the real-time analysis data as input, compares it with the past learning results, and generates a solution. The generated solution is presented to the user as output.

[0279] Step 6:

[0280] The server receives the feedback provided by the user. The input is the evaluation and comments on the options sent by the user through the evaluation screen. This feedback is used to improve the AI model and update the data to help propose more appropriate solutions. The output is the improved AI model or the addition of new data.

[0281] (Application Example 1)

[0282] Next, Application Example 1 will be described. In the following description, the data processing device 12 is referred to as the "server", and the smart glasses 214 are referred to as the "terminal".

[0283] In modern society, conflicts and troubles that occur are required to be resolved quickly and safely. However, in the current system, it is difficult to analyze data in real time and present appropriate solutions, especially to infer intentions and emotions from non-verbal information in a collective environment. Therefore, it hinders participants from effectively forming consensus. Means are required to solve these problems and achieve smooth problem-solving with security considerations.

[0284] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0285] In this invention, the server includes means for collecting and storing past case data, means for executing a generation model for learning based on the accumulated data, and means for analyzing data in real time and proposing an optimal solution. Thereby, it is possible to accurately analyze the intentions and emotions of participants and enable efficient consensus formation.

[0286] "Past case data" refers to recorded information on similar conflicts and problems that occurred in the past, and serves as learning material for dispute resolution.

[0287] "To accumulate" means to systematically store the collected data within a system such as a database so that it can be effectively utilized.

[0288] "Generative model" refers to an AI model constructed using machine learning algorithms to derive patterns and solutions from data and propose them.

[0289] "Analyzing data in real time" refers to the process of immediately processing the current situation and information to obtain analysis results.

[0290] "Non-verbal information" refers to information such as body movements and expressions that are used as clues for inferring people's emotions and intentions.

[0291] "Processing video data" refers to the task of analyzing visual data obtained from devices such as cameras and extracting specific information.

[0292] "Reflecting feedback in model learning" means making adjustments to improve the performance of an AI model based on evaluations and reactions obtained from users.

[0293] To implement this invention, it is important to appropriately configure a server, user terminals, and the necessary software environment.

[0294] The server collects past case data and stores it in a database. Specifically, it efficiently retrieves relevant data from various public databases and internal documents, and prepares it as a training dataset for machine learning. This data is used to train a generative AI model and forms the basis for proposing solutions to new conflicts and problems. Python and TensorFlow can be used to build the generative AI model.

[0295] The user terminal is designed as a smartphone or smart glasses and is equipped with the ability to collect audio and video in real time. This allows for the acquisition of nonverbal information such as voice, facial expressions, and body language in negotiations and disputes, and this data is sent to a server for immediate analysis. The analysis uses tools such as OpenCV to process and interpret emotions and intentions from the video data in real time.

[0296] The user receives solutions provided by the server through an interface, evaluates these options, and provides feedback. This feedback is used to improve the accuracy of the generated AI model on the server and to enhance the accuracy of future suggestions. This allows the system to continuously improve itself.

[0297] As a concrete example, in a work environment where misunderstandings may occur during communication, the system intervenes immediately when an incident occurs to facilitate rapid problem resolution. An example of a prompt message would be, "Which solution is best to mitigate the incident caused by past misunderstandings?"

[0298] This configuration allows the system to respond promptly and accurately, and is expected to facilitate the rapid resolution of complex conflicts and problems.

[0299] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0300] Step 1:

[0301] The server collects past case data. It collects relevant information from databases and public materials and converts this into a form that can be used in machine learning. The input at this stage is unorganized raw data, which is output as a structured dataset. Specifically, natural language processing technology is used to analyze legal documents and negotiation records.

[0302] Step 2:

[0303] The server trains a generative AI model. Using the structured data obtained in Step 1, it trains the AI model using TensorFlow. The input is a dataset of cases, and the output is a generalized solution pattern. The model automatically learns the optimal solution through a pattern recognition algorithm.

[0304] Step 3:

[0305] The terminal collects data in real time. During negotiations or when problems occur, it uses smartphones or smart glasses to obtain audio and video data. The input data is audio files and video frames, which are transferred to the server as compressed digital data. Non-verbal information from the device's sensors is also obtained simultaneously.

[0306] Step 4:

[0307] The server performs real-time analysis. It receives the data sent from the terminal, uses OpenCV to analyze expressions and movements from videos, and converts audio data into text using natural language processing technology. Based on the analyzed intent and emotion information, the generative AI model proposes an optimal solution. The input is real-time data, and the output is the inferred intent and emotion and the proposed solution.

[0308] [[ID=2​​​The user evaluates the proposed solutions. The server receives the solutions presented to the terminal and allows the user to review each option through the interface. The input consists of the presented solutions, and the user selects one and returns feedback to the server. The evaluation results are collected and used to improve the model.

[0310] Step 6:

[0311] The server utilizes feedback to optimize the generated AI model. It analyzes feedback data collected from users and updates the model's parameters. The input is user feedback, and the output is the improved AI model. Improved model accuracy leads to higher quality suggestions in subsequent iterations.

[0312] In this series of processes, the roles of the server, terminal, and user work together in coordination, enabling the rapid and effective resolution of problems.

[0313] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0314] This invention describes a method for combining an emotion engine with a conflict resolution support system to grasp the emotional state of participants in real time and utilize this information to propose solutions. This makes it possible to guide consensus building in a more precise and amicable manner.

[0315] The server utilizes an emotion engine to analyze real-time video and audio data collected during negotiations. This analysis detects emotions from participants' facial expressions, tone of voice, and word choice, and then quantitatively evaluates their emotional state based on this data. The emotion engine identifies basic emotions such as joy, anger, sadness, surprise, and fear, providing insights into the dynamics of negotiations.

[0316] The terminal presents the user with emotional data processed by the server, making it easier for the user to understand the emotional state of the other party during negotiations. This information can also serve as an important indicator when selecting a proposed solution.

[0317] Furthermore, the server uses data obtained from the emotion engine to influence the output of the generative AI model. Specifically, it adjusts the proposed solutions by taking into account the other party's emotional state. For example, if the negotiating partner is stressed, a gentler and less burdensome solution can be prioritized. In this way, the accuracy and effectiveness of the proposals are improved.

[0318] As a concrete example, in the case of negotiations for a business partnership, the user activates the dispute resolution support system via their terminal. As the negotiations progress, the emotion engine analyzes the emotions of the participants, and the server reflects the analysis results in real time. By selecting the solution that best suits the other party's emotions from among several proposed solutions, the user can reach a smoother agreement.

[0319] This invention contributes to increasing the transparency and reliability of negotiations and innovates the process of reaching agreements in dispute resolution.

[0320] The following describes the processing flow.

[0321] Step 1:

[0322] The server collects past dispute resolution cases from the database, performs data cleaning, and converts and saves them in the format required for the generated AI model. This process is necessary to prepare the model so that it can learn properly and handle new cases.

[0323] Step 2:

[0324] The server trains a generative AI model using cleaned data. The model learns patterns from the cases and accumulates insights that will be useful for suggesting future solutions. This training forms the foundation for the model to accurately predict new cases.

[0325] Step 3:

[0326] Once negotiations begin, the terminal collects audio and video data in real time and sends it to the server. This data includes what is said, facial expressions, and tone of voice. This real-time data is used for instant analysis.

[0327] Step 4:

[0328] The server analyzes the received real-time data using an emotion engine. This analysis quantifies the participants' emotional states, generating an emotional profile. This profile represents basic emotional states such as joy, anger, sadness, and happiness.

[0329] Step 5:

[0330] Based on the analyzed emotional profiles, the server utilizes a generative AI model to propose the optimal solution. Here, adjustments are made to match the participants' emotions, ensuring flexibility in the solution.

[0331] Step 6:

[0332] The device presents the generated solutions to the user. Multiple options are displayed, each with a description of its characteristics and expected effects, taken into account from the emotional profile.

[0333] Step 7:

[0334] Users evaluate the presented solutions, select the option they believe to be the best, and enter feedback into their device. This feedback includes information about whether they adopted the suggestion or made minor adjustments.

[0335] Step 8:

[0336] The terminal sends user feedback to the server, which adjusts the model parameters based on the feedback to improve the accuracy of future suggestions. This feedback loop strengthens the overall learning ability of the system.

[0337] Step 9:

[0338] The server encrypts and stores data, ensuring security to protect information privacy. This allows the system to operate securely and prevents data leaks.

[0339] (Example 2)

[0340] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0341] Traditional dispute resolution support systems have faced the challenge of difficulty in proposing optimal solutions while taking into account the emotional states of the participants. There is a need for a mechanism that understands the emotional dynamics of participants during negotiations and facilitates consensus building based on that understanding.

[0342] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0343] In this invention, the server includes means for collecting and storing past cases, means for executing a generative model that learns based on the accumulated information, and means for analyzing video and audio information obtained from negotiations in real time and quantitatively evaluating the emotions of the participants. This makes it possible to grasp the emotional state of the participants in real time and propose the optimal solution based on that.

[0344] A "case study" refers to a specific event or related incident that occurred in the past, and contains information that can be useful in resolving disputes.

[0345] "Information" is a general term for the data and knowledge collected, and in particular includes data related to video and audio acquired in real time.

[0346] A "generative model" refers to an algorithm or method that learns patterns and trends from data and generates new predictions or suggestions based on them.

[0347] "Analysis" is a general term for processes and analyses performed on data and information in order to understand their structure and elements, and in particular includes the process of quantifying emotional states.

[0348] "Emotion" refers to a person's psychological state, and in negotiations, it refers to the psychological tendencies and changes of participants.

[0349] "Solutions" refer to a general term for the measures and means proposed to resolve conflicts and problems, and include specific means adjusted by generative AI models.

[0350] "Privacy protection" refers to a set of measures and policies to prevent the improper disclosure of personal information and data.

[0351] This invention is a system for understanding the emotional state of participants in dispute resolution in real time and proposing the optimal solution based on that understanding.

[0352] The server collects video and audio information obtained from the negotiation in real time and processes this data using an emotion analysis engine. Specifically, it uses image recognition software to analyze facial expressions in the video data and an audio analysis algorithm to analyze the tone and speed of voices in the audio data. As a result, the emotional state of the participants is quantified and stored in a database.

[0353] The terminal displays emotional data sent from the server on the user interface, allowing users to visualize the progress of negotiations from an emotional perspective. For example, the degree of emotion is shown with colors or graphs, allowing users to intuitively understand the other party's state.

[0354] The server utilizes machine learning algorithms as a generative AI model to generate optimal solutions based on accumulated emotional data. In this process, prompt statements are input to the generative AI model, and the system proposes the most suitable solution that reflects the emotional data. For example, a prompt statement such as "Gently adjust the solution when participants are experiencing stress" can be used.

[0355] Users can review the proposed solutions through their device and select the most suitable option from multiple choices. This dynamic presentation of options allows users to navigate a smooth process towards agreement while being considerate of the other party's feelings.

[0356] As a concrete example, in business partnership negotiations between companies, activating this system would enable flexible proposals that respond to changes in the other party's emotions, leading to smoother negotiations. This system would improve the transparency and reliability of agreement building and promote an innovative negotiation process.

[0357] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0358] Step 1:

[0359] The server collects video and audio data in real time from the negotiation site as input. External input devices such as cameras and microphones are used for this purpose. This data becomes the basic information for sentiment analysis in the next processing step.

[0360] Step 2:

[0361] The server analyzes the input video data using image recognition software, quantifying emotions from the participants' facial expressions. For example, it identifies emotions such as joy and anger from eye and mouth movements and outputs this as quantified data. In parallel, audio data is processed through a voice analysis algorithm, recognizing emotions from voice tone and pitch, and this too is output as quantified data.

[0362] Step 3:

[0363] The server aggregates the quantified emotion data obtained in Step 2 and inputs it into the emotion analysis engine. This engine uses deep learning to extract complex patterns and evaluate overall emotional trends. This process comprehensively analyzes the participants' current emotional state and outputs the results to a database.

[0364] Step 4:

[0365] The terminal visually displays emotional state data received from the server using a GUI. Graphs and color coding are used to allow the user to intuitively understand the emotional changes of the negotiating partner. For example, heightened emotions are displayed in red, and calmer emotions in blue.

[0366] Step 5:

[0367] The server inputs emotional state data as prompt statements into an AI model, which then generates optimal solution suggestions. Since these prompt statements reflect the participant's subjective emotional state, the suggested solutions take that state into account. The generated solutions are output as data.

[0368] Step 6:

[0369] The user reviews multiple proposed solutions on their device. Each solution is accompanied by an explanation based on the other party's emotional state, allowing the user to choose the option best suited to the other party's changing emotions. This enables the user to conduct negotiations more smoothly.

[0370] (Application Example 2)

[0371] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart glasses 214 as the "terminal".

[0372] In customer service, the challenge lies in understanding customers' emotional states in real time, improving the quality of service provided, and increasing customer satisfaction. Furthermore, it is necessary to improve the ability to quickly identify customers' potential dissatisfactions and needs and provide immediate, personalized support.

[0373] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0374] In this invention, the server includes a device for collecting and storing past solution cases, a device for executing a generative model that learns based on the accumulated data, a device for analyzing information in real time and proposing the optimal solution, a device for receiving user feedback and updating the analysis results, a device for protecting the privacy of information, a device for analyzing the video and audio of participants and evaluating their emotional state in real time, and a device for adjusting the proposed solution based on the evaluated emotional state. This makes it possible to accurately grasp the emotional state of customers, provide appropriate services, and increase customer satisfaction.

[0375] "Past case studies" refer to real-world case information that has been collected and stored in the system to date.

[0376] A "generative model" is an algorithm or program that learns from past data and proposes new solutions.

[0377] "Real-time analysis" is a process that instantly processes participants' video and audio data in real time and extracts the necessary information.

[0378] "Emotional state" refers to the mental condition or emotions inferred from nonverbal information such as a participant's facial expressions and tone of voice.

[0379] "Evaluation" is the process of measuring emotional states and the suitability of solutions based on collected data, and then making a judgment.

[0380] "Adjusting solutions" refers to the action of optimizing the proposed solutions based on the emotional information collected.

[0381] "Feedback" refers to the response information provided by users, which is used to improve and adjust the system.

[0382] The system for implementing this invention has a structure that connects a smart device and a server to understand the customer's emotional state and provide appropriate feedback. Specifically, the server receives video and audio data and processes them using an emotion analysis algorithm. This makes it possible to evaluate the customer's emotional state in real time from their facial expressions and tone of voice.

[0383] The hardware used includes smart glasses and high-performance microphones. These capture customer video and audio and transmit the data to a server via Bluetooth or Wi-Fi. Meanwhile, the server processes the data using emotion analysis SDKs such as Affectiva and IBM Watson to quantitatively evaluate the customer's emotional state.

[0384] Based on this, the server uses a generative AI model to suggest the optimal customer service approach. For example, if a customer is feeling stressed, it might suggest a calmer tone when explaining the product. This allows users to gain guidance on how to improve customer satisfaction.

[0385] As a concrete example, if a customer shows signs of anxiety during customer service at a physical store, this system will suggest "explaining the product's benefits and customer testimonials in detail to reassure them." Furthermore, by inputting prompts like the following into the AI ​​model, further improvement suggestions can be obtained.

[0386] Example of a prompt:

[0387] "The customer appeared to be feeling uneasy based on their expression during the product explanation. Please suggest appropriate countermeasures for this situation."

[0388] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0389] Step 1:

[0390] The terminal captures the customer's video and audio using its built-in camera and microphone. This process takes video and audio data as input and outputs recorded data in digital format. The terminal then preprocesses this data, performing noise reduction and resolution adjustments.

[0391] Step 2:

[0392] The terminal transmits captured data to the server via Bluetooth or Wi-Fi. The input is the video and audio data acquired by the terminal, and the output is the digital data transferred to the server. The terminal checks the stability and speed of data transfer and retries if a transfer error occurs.

[0393] Step 3:

[0394] The server feeds the received data into an emotion analysis algorithm. The input is the video and audio data transferred in the previous step, and the output is the analysis result indicating the emotional state. The server uses Affectiva or IBM Watson SDK to quantify basic emotions such as joy, anger, sadness, and happiness.

[0395] Step 4:

[0396] The server uses a generative AI model to generate optimal customer service suggestions based on sentiment analysis results. In this step, sentiment analysis results are used as input, and specific customer service suggestions and prompts are generated as output. The server then performs an analysis process to present the generated information to the user in real time.

[0397] Step 5:

[0398] The user reviews customer service suggestions provided by the server via their terminal and adjusts their actions as needed. The input is the customer service suggestion from the server, and the output is the user's improved customer service behavior. The user selects the optimal response according to the customer service situation and provides service that meets the customer's needs.

[0399] Step 6:

[0400] The server collects feedback on user behavior and uses it to improve the accuracy of the generative model. The input is user feedback, and the output is an updated analysis algorithm and generative AI model. The server periodically updates the database to prepare for more accurate analysis in future applications.

[0401] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0402] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0403] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0404] [Third Embodiment]

[0405] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0406] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0407] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0408] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0409] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0410] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0411] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0412] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0413] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0414] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0415] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0416] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0417] This invention provides specific methods for effectively performing data collection, model training, real-time analysis, solution prediction, and user feedback reception in a dispute resolution support system. This enables rapid and efficient consensus building.

[0418] First, the server collects historical case data related to dispute resolution from specialized databases and digital archives. This includes court records, negotiation histories, and legal documents. The collected data is converted into a reusable format and prepared for model training.

[0419] Subsequently, this data is used to train a generative AI model. The server extracts meaningful patterns using an automated process and learns results and solutions from similar cases. This learning process enables the model to propose solutions immediately and appropriately in new negotiations.

[0420] Next, once negotiations begin, the terminals collect real-time data (such as voice utterances, text input, and facial expressions and movements analyzed via video). Using this information, the server performs data analysis to infer the participants' intentions, emotions, and positions. Based on this, the optimal solution is generated, taking into account past learning results.

[0421] The server then provides the proposed solutions to the user's terminal. The solutions are presented as multiple options based on their feasibility and likelihood of success. The user evaluates these options through the interface and provides feedback as needed.

[0422] As a concrete example, consider the use of this system in a contract dispute between companies. The user first inputs data on past contract disputes into the system. The server analyzes this information, searches its database for similar cases, and utilizes a learning model. As a result, the model quickly proposes the optimal agreement during negotiations, allowing the user to proceed with agreement formation on the spot. Through this process, agreement formation is made more efficient, contributing to the early resolution of disputes.

[0423] The following describes the processing flow.

[0424] Step 1:

[0425] The server calls a specialized database API to collect historical case data related to dispute resolution, retrieving court records, negotiation details, and case law data. The retrieved data is cleaned for duplicates and errors and stored as a clean dataset.

[0426] Step 2:

[0427] The server trains a generative AI model using a clean dataset. The data input to the model is classified, and key information necessary for learning is highlighted through feature extraction. After training, the model learns patterns from past cases and acquires knowledge applicable to new conflict resolution situations.

[0428] Step 3:

[0429] The device uses voice and video sensors during negotiations to capture real-time statements, facial expressions, and the other party's reactions, and transmits this data to a server. This data includes indicators of the participants' intentions and emotions.

[0430] Step 4:

[0431] The server analyzes the received real-time data using natural language processing and image analysis algorithms. Based on the analysis results, it infers the intentions and emotions of the other party in negotiations and generates the optimal solution. The generated solution reflects past learning achievements.

[0432] Step 5:

[0433] The server prioritizes solutions based on the analysis and presents them to the terminal as multiple options. Each option includes detailed information about its effectiveness and expected impact.

[0434] Step 6:

[0435] Users select the most appropriate option from those presented via their device and provide feedback as needed. This feedback includes not only manual evaluations but also the provision of new information.

[0436] Step 7:

[0437] The device sends user feedback to the server, which then uses that feedback to fine-tune the AI ​​model. This refines the solution and improves the quality of proposals in future negotiations.

[0438] Step 8:

[0439] The server implements encryption and access control during data transmission, reception, and storage to ensure thorough privacy protection. This guarantees that data handled during the dispute resolution process will not be leaked externally and will be managed securely.

[0440] (Example 1)

[0441] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0442] In conflict resolution, there is a need to provide swift and appropriate solutions based on past cases. However, conventional systems have shortcomings in data collection and analysis, making it difficult to grasp participants' intentions and emotions in real time and to provide flexible solutions.

[0443] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0444] In this invention, the server includes a device for collecting and storing past problem-solving cases, a device for executing generative artificial intelligence that learns based on the accumulated information, and a device for analyzing information during negotiations in real time and proposing the optimal solution. This makes it possible to infer the intentions and emotions of the participants, quickly generate and present a variety of solutions, and flexibly improve them based on user evaluations.

[0445] "Past problem-solving examples" refer to specific cases and approaches to various problems that have been solved in the past.

[0446] A "storage device" refers to a system equipped with the functionality to efficiently store collected data and make it accessible as needed.

[0447] "Generative artificial intelligence" refers to algorithms and models that learn from collected data and generate new information.

[0448] "Information being negotiated in real time" refers to dynamic data such as voice, actions, and facial expressions generated during the negotiation process.

[0449] "Inferring intentions and emotions" refers to the process of analyzing collected data and forming hypotheses about participants' goals and emotions.

[0450] "Proposing a solution" refers to outlining specific steps or strategies for solving a problem based on the analysis results.

[0451] "Diverse solutions" refers to multiple solutions offered from different perspectives and approaches.

[0452] "User evaluation" refers to opinions and feedback from users regarding the solutions provided by the system.

[0453] "Flexible improvement" refers to the ability to adjust the system's suggested solutions and processes based on user feedback.

[0454] To implement this invention, the system will function using the following hardware and software.

[0455] First, the server accesses specialized databases and digital archives to collect and store past problem-solving examples. Specifically, it uses Python scripts to retrieve data via APIs and saves it to a MySQL database. The collected data is preprocessed using natural language processing libraries such as NLTK and spaCy, and prepared as a training dataset for generative AI models.

[0456] Next, the server builds a generative AI model using machine learning frameworks such as TensorFlow or PyTorch based on the pre-processed data. The model has the ability to learn from past examples, extract patterns, and predict new solutions.

[0457] The device collects real-time data using sensors and cameras while negotiations are taking place. For example, audio data is collected via a microphone and converted into text using the Google Speech-to-Text API. OpenCV is used for video analysis to capture participants' facial expressions and movements.

[0458] Based on the collected real-time data, the server performs intent and sentiment analysis, and generates the optimal solution by comparing it with past learning results. Multiple solutions are then presented to the user's device, and the user provides evaluation and feedback through the interface.

[0459] As a concrete example, consider a scenario in which this system is used in contract negotiations between companies. The user inputs data on past contract disputes into the server. The server analyzes this information and utilizes a generation AI model that references similar cases in the database. As a result, the optimal agreement is quickly presented during negotiations. For example, by giving instructions such as "Propose the best solution for the current contract negotiation," the system will quickly provide a solution based on that instruction. This enables rapid agreement formation.

[0460] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0461] Step 1:

[0462] The server collects past problem-solving cases from specialized databases and digital archives. It receives data via API as input and stores this data in a MySQL database. Data collection is performed using a Python script, and the collected data is saved for subsequent analysis and model training.

[0463] Step 2:

[0464] The server preprocesses the collected data. The input is the raw data obtained in step 1. This data is processed using natural language processing libraries such as NLTK and spaCy, including tokenization, stop word removal, and syntactic analysis. The output is clean text data, which is used to train the generative AI model.

[0465] Step 3:

[0466] The server trains a generative AI model using pre-processed data. It receives processed text data as input and trains a machine learning model using TensorFlow or PyTorch. This process improves the ability to extract patterns and predict solutions through supervised and unsupervised learning. The output is the trained AI model.

[0467] Step 4:

[0468] The device collects information about ongoing negotiations in real time. It receives audio and video data from sensors as input. Audio data is captured using a microphone and converted to text using the Google Speech-to-Text API. Video data is captured by a camera and analyzed using OpenCV to infer participants' intentions and emotions from their facial expressions and movements. The output is the analyzed real-time data.

[0469] Step 5:

[0470] The server analyzes real-time data sent from the terminal and generates the optimal solution. It receives real-time analysis data as input, compares it with past learning results, and generates a solution. The generated solution is presented to the user as output.

[0471] Step 6:

[0472] The server receives feedback from users. Input consists of user ratings and comments on the options submitted through the evaluation screen. This feedback is used to improve the AI ​​model and update the data to help suggest more appropriate solutions. Output consists of an improved AI model and the addition of new data.

[0473] (Application Example 1)

[0474] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0475] In modern society, conflicts and troubles require swift and safe resolution. However, current systems struggle with real-time data analysis and the presentation of appropriate solutions, particularly inferring intentions and emotions from nonverbal information in collective settings. This hinders effective consensus building among participants. A solution is needed to address these challenges and achieve smooth problem resolution while maintaining security.

[0476] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0477] In this invention, the server includes means for collecting and storing past case data, means for executing a generative model that learns based on the accumulated data, and means for analyzing the data in real time and proposing the optimal solution. This enables accurate analysis of participants' intentions and emotions, and efficient consensus building.

[0478] "Past case data" refers to records of similar conflicts and problems that have occurred in the past, and serves as learning material for conflict resolution.

[0479] "Storing" refers to the act of systematically saving collected data within a system such as a database so that it can be used effectively.

[0480] A "generative model" is an AI model built to derive and propose patterns and solutions from data using machine learning algorithms.

[0481] "Analyzing data in real time" is the process of immediately processing ongoing situations and information to obtain analysis results.

[0482] "Nonverbal information" refers to information such as body movements and facial expressions that are used as clues to infer a person's emotions and intentions.

[0483] "Processing video data" refers to the process of analyzing visual data acquired from devices such as cameras and extracting specific information.

[0484] "Incorporating feedback into model learning" means making adjustments to improve the performance of an AI model based on evaluations and responses obtained from users.

[0485] In order to implement this invention, it is important to properly configure the server, user terminal, and necessary software environment.

[0486] The server collects past case data and stores it in a database. Specifically, it efficiently retrieves relevant data from various public databases and internal documents, and prepares it as a training dataset for machine learning. This data is used to train a generative AI model and forms the basis for proposing solutions to new conflicts and problems. Python and TensorFlow can be used to build the generative AI model.

[0487] The user terminal is designed as a smartphone or smart glasses and is equipped with the ability to collect audio and video in real time. This allows for the acquisition of nonverbal information such as voice, facial expressions, and body language in negotiations and disputes, and this data is sent to a server for immediate analysis. The analysis uses tools such as OpenCV to process and interpret emotions and intentions from the video data in real time.

[0488] The user receives solutions provided by the server through an interface, evaluates these options, and provides feedback. This feedback is used to improve the accuracy of the generated AI model on the server and to enhance the accuracy of future suggestions. This allows the system to continuously improve itself.

[0489] As a concrete example, in a work environment where misunderstandings may occur during communication, the system intervenes immediately when an incident occurs to facilitate rapid problem resolution. An example of a prompt message would be, "Which solution is best to mitigate the incident caused by past misunderstandings?"

[0490] This configuration allows the system to respond promptly and accurately, and is expected to facilitate the rapid resolution of complex conflicts and problems.

[0491] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0492] Step 1:

[0493] The server collects historical case data. It gathers relevant information from databases and public documents and converts it into a format usable by machine learning. At this stage, the input is unorganized raw data, and the output is a structured dataset. Specifically, natural language processing techniques are used to analyze legal documents and negotiation records.

[0494] Step 2:

[0495] The server trains the generative AI model. Using the structured data obtained in Step 1, the AI ​​model is trained using TensorFlow. The case dataset is used as input, and the output is a generalized solution pattern. The model automatically learns the optimal solution using a pattern recognition algorithm.

[0496] Step 3:

[0497] The device collects data in real time. During negotiations or when problems occur, it uses smartphones or smart glasses to acquire audio and video data. The input data consists of audio files and video frames, which are transferred to the server as compressed digital data. Non-verbal information is also acquired simultaneously through the device's sensors.

[0498] Step 4:

[0499] The server performs real-time analysis. It receives data sent from the terminal, analyzes facial expressions and movements from video using OpenCV, and converts audio data into text using natural language processing technology. Based on the analyzed intent and emotion information, a generative AI model proposes the optimal solution. The input is real-time data, and the output is the inferred intent and emotion, as well as the proposed solution.

[0500] Step 5:

[0501] The user evaluates the proposed solutions. The server receives the solutions presented to the terminal and allows the user to review each option through the interface. The input consists of the presented solutions, and the user selects one and returns feedback to the server. The evaluation results are collected and used to improve the model.

[0502] Step 6:

[0503] The server utilizes feedback to optimize the generated AI model. It analyzes feedback data collected from users and updates the model's parameters. The input is user feedback, and the output is the improved AI model. Improved model accuracy leads to higher quality suggestions in subsequent iterations.

[0504] In this series of processes, the roles of the server, terminal, and user work together in coordination, enabling the rapid and effective resolution of problems.

[0505] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0506] This invention describes a method for combining an emotion engine with a conflict resolution support system to grasp the emotional state of participants in real time and utilize this information to propose solutions. This makes it possible to guide consensus building in a more precise and amicable manner.

[0507] The server utilizes an emotion engine to analyze real-time video and audio data collected during negotiations. This analysis detects emotions from participants' facial expressions, tone of voice, and word choice, and then quantitatively evaluates their emotional state based on this data. The emotion engine identifies basic emotions such as joy, anger, sadness, surprise, and fear, providing insights into the dynamics of negotiations.

[0508] The terminal presents the user with emotional data processed by the server, making it easier for the user to understand the emotional state of the other party during negotiations. This information can also serve as an important indicator when selecting a proposed solution.

[0509] Furthermore, the server uses data obtained from the emotion engine to influence the output of the generative AI model. Specifically, it adjusts the proposed solutions by taking into account the other party's emotional state. For example, if the negotiating partner is stressed, a gentler and less burdensome solution can be prioritized. In this way, the accuracy and effectiveness of the proposals are improved.

[0510] As a concrete example, in the case of negotiations for a business partnership, the user activates the dispute resolution support system via their terminal. As the negotiations progress, the emotion engine analyzes the emotions of the participants, and the server reflects the analysis results in real time. By selecting the solution that best suits the other party's emotions from among several proposed solutions, the user can reach a smoother agreement.

[0511] This invention contributes to increasing the transparency and reliability of negotiations and innovates the process of reaching agreements in dispute resolution.

[0512] The following describes the processing flow.

[0513] Step 1:

[0514] The server collects past dispute resolution cases from the database, performs data cleaning, and converts and saves them in the format required for the generated AI model. This process is necessary to prepare the model so that it can learn properly and handle new cases.

[0515] Step 2:

[0516] The server trains a generative AI model using cleaned data. The model learns patterns from the cases and accumulates insights that will be useful for suggesting future solutions. This training forms the foundation for the model to accurately predict new cases.

[0517] Step 3:

[0518] Once negotiations begin, the terminal collects audio and video data in real time and sends it to the server. This data includes what is said, facial expressions, and tone of voice. This real-time data is used for instant analysis.

[0519] Step 4:

[0520] The server analyzes the received real-time data using an emotion engine. This analysis quantifies the participants' emotional states, generating an emotional profile. This profile represents basic emotional states such as joy, anger, sadness, and happiness.

[0521] Step 5:

[0522] Based on the analyzed emotional profiles, the server utilizes a generative AI model to propose the optimal solution. Here, adjustments are made to match the participants' emotions, ensuring flexibility in the solution.

[0523] Step 6:

[0524] The device presents the generated solutions to the user. Multiple options are displayed, each with a description of its characteristics and expected effects, taken into account from the emotional profile.

[0525] Step 7:

[0526] Users evaluate the presented solutions, select the option they believe to be the best, and enter feedback into their device. This feedback includes information about whether they adopted the suggestion or made minor adjustments.

[0527] Step 8:

[0528] The terminal sends user feedback to the server, which adjusts the model parameters based on the feedback to improve the accuracy of future suggestions. This feedback loop strengthens the overall learning ability of the system.

[0529] Step 9:

[0530] The server encrypts and stores data, ensuring security to protect information privacy. This allows the system to operate securely and prevents data leaks.

[0531] (Example 2)

[0532] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0533] Traditional dispute resolution support systems have faced the challenge of difficulty in proposing optimal solutions while taking into account the emotional states of the participants. There is a need for a mechanism that understands the emotional dynamics of participants during negotiations and facilitates consensus building based on that understanding.

[0534] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0535] In this invention, the server includes means for collecting and storing past cases, means for executing a generative model that learns based on the accumulated information, and means for analyzing video and audio information obtained from negotiations in real time and quantitatively evaluating the emotions of the participants. This makes it possible to grasp the emotional state of the participants in real time and propose the optimal solution based on that.

[0536] A "case study" refers to a specific event or related incident that occurred in the past, and contains information that can be useful in resolving disputes.

[0537] "Information" is a general term for the data and knowledge collected, and in particular includes data related to video and audio acquired in real time.

[0538] A "generative model" refers to an algorithm or method that learns patterns and trends from data and generates new predictions or suggestions based on them.

[0539] "Analysis" is a general term for processes and analyses performed on data and information in order to understand their structure and elements, and in particular includes the process of quantifying emotional states.

[0540] "Emotion" refers to a person's psychological state, and in negotiations, it refers to the psychological tendencies and changes of participants.

[0541] "Solutions" refer to a general term for the measures and means proposed to resolve conflicts and problems, and include specific means adjusted by generative AI models.

[0542] "Privacy protection" refers to a set of measures and policies to prevent the improper disclosure of personal information and data.

[0543] This invention is a system for understanding the emotional state of participants in dispute resolution in real time and proposing the optimal solution based on that understanding.

[0544] The server collects video and audio information obtained from the negotiation in real time and processes this data using an emotion analysis engine. Specifically, it uses image recognition software to analyze facial expressions in the video data and an audio analysis algorithm to analyze the tone and speed of voices in the audio data. As a result, the emotional state of the participants is quantified and stored in a database.

[0545] The terminal displays emotional data sent from the server on the user interface, allowing users to visualize the progress of negotiations from an emotional perspective. For example, the degree of emotion is shown with colors or graphs, allowing users to intuitively understand the other party's state.

[0546] The server utilizes machine learning algorithms as a generative AI model to generate optimal solutions based on accumulated emotional data. In this process, prompt statements are input to the generative AI model, and the system proposes the most suitable solution that reflects the emotional data. For example, a prompt statement such as "Gently adjust the solution when participants are experiencing stress" can be used.

[0547] Users can review the proposed solutions through their device and select the most suitable option from multiple choices. This dynamic presentation of options allows users to navigate a smooth process towards agreement while being considerate of the other party's feelings.

[0548] As a concrete example, in business partnership negotiations between companies, activating this system would enable flexible proposals that respond to changes in the other party's emotions, leading to smoother negotiations. This system would improve the transparency and reliability of agreement building and promote an innovative negotiation process.

[0549] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0550] Step 1:

[0551] The server collects video and audio data in real time from the negotiation site as input. External input devices such as cameras and microphones are used for this purpose. This data becomes the basic information for sentiment analysis in the next processing step.

[0552] Step 2:

[0553] The server analyzes the input video data using image recognition software, quantifying emotions from the participants' facial expressions. For example, it identifies emotions such as joy and anger from eye and mouth movements and outputs this as quantified data. In parallel, audio data is processed through a voice analysis algorithm, recognizing emotions from voice tone and pitch, and this too is output as quantified data.

[0554] Step 3:

[0555] The server aggregates the quantified emotion data obtained in Step 2 and inputs it into the emotion analysis engine. This engine uses deep learning to extract complex patterns and evaluate overall emotional trends. This process comprehensively analyzes the participants' current emotional state and outputs the results to a database.

[0556] Step 4:

[0557] The terminal visually displays emotional state data received from the server using a GUI. Graphs and color coding are used to allow the user to intuitively understand the emotional changes of the negotiating partner. For example, heightened emotions are displayed in red, and calmer emotions in blue.

[0558] Step 5:

[0559] The server inputs emotional state data as prompt statements into an AI model, which then generates optimal solution suggestions. Since these prompt statements reflect the participant's subjective emotional state, the suggested solutions take that state into account. The generated solutions are output as data.

[0560] Step 6:

[0561] The user reviews multiple proposed solutions on their device. Each solution is accompanied by an explanation based on the other party's emotional state, allowing the user to choose the option best suited to the other party's changing emotions. This enables the user to conduct negotiations more smoothly.

[0562] (Application Example 2)

[0563] Next, we will explain Application Example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0564] In customer service, the challenge lies in understanding customers' emotional states in real time, improving the quality of service provided, and increasing customer satisfaction. Furthermore, it is necessary to improve the ability to quickly identify customers' potential dissatisfactions and needs and provide immediate, personalized support.

[0565] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0566] In this invention, the server includes a device for collecting and storing past solution cases, a device for executing a generative model that learns based on the accumulated data, a device for analyzing information in real time and proposing the optimal solution, a device for receiving user feedback and updating the analysis results, a device for protecting the privacy of information, a device for analyzing the video and audio of participants and evaluating their emotional state in real time, and a device for adjusting the proposed solution based on the evaluated emotional state. This makes it possible to accurately grasp the emotional state of customers, provide appropriate services, and increase customer satisfaction.

[0567] "Past case studies" refer to real-world case information that has been collected and stored in the system to date.

[0568] A "generative model" is an algorithm or program that learns from past data and proposes new solutions.

[0569] "Real-time analysis" is a process that instantly processes participants' video and audio data in real time and extracts the necessary information.

[0570] "Emotional state" refers to the mental condition or emotions inferred from nonverbal information such as a participant's facial expressions and tone of voice.

[0571] "Evaluation" is the process of measuring emotional states and the suitability of solutions based on collected data, and then making a judgment.

[0572] "Adjusting solutions" refers to the action of optimizing the proposed solutions based on the emotional information collected.

[0573] "Feedback" refers to the response information provided by users, which is used to improve and adjust the system.

[0574] The system for implementing this invention has a structure that connects a smart device and a server to understand the customer's emotional state and provide appropriate feedback. Specifically, the server receives video and audio data and processes them using an emotion analysis algorithm. This makes it possible to evaluate the customer's emotional state in real time from their facial expressions and tone of voice.

[0575] The hardware used includes smart glasses and high-performance microphones. These capture customer video and audio and transmit the data to a server via Bluetooth or Wi-Fi. Meanwhile, the server processes the data using emotion analysis SDKs such as Affectiva and IBM Watson to quantitatively evaluate the customer's emotional state.

[0576] Based on this, the server uses a generative AI model to suggest the optimal customer service approach. For example, if a customer is feeling stressed, it might suggest a calmer tone when explaining the product. This allows users to gain guidance on how to improve customer satisfaction.

[0577] As a concrete example, if a customer shows signs of anxiety during customer service at a physical store, this system will suggest "explaining the product's benefits and customer testimonials in detail to reassure them." Furthermore, by inputting prompts like the following into the AI ​​model, further improvement suggestions can be obtained.

[0578] Example of a prompt:

[0579] "The customer appeared to be feeling uneasy based on their expression during the product explanation. Please suggest appropriate countermeasures for this situation."

[0580] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0581] Step 1:

[0582] The terminal captures the customer's video and audio using its built-in camera and microphone. This process takes video and audio data as input and outputs recorded data in digital format. The terminal then preprocesses this data, performing noise reduction and resolution adjustments.

[0583] Step 2:

[0584] The terminal transmits captured data to the server via Bluetooth or Wi-Fi. The input is the video and audio data acquired by the terminal, and the output is the digital data transferred to the server. The terminal checks the stability and speed of data transfer and retries if a transfer error occurs.

[0585] Step 3:

[0586] The server feeds the received data into an emotion analysis algorithm. The input is the video and audio data transferred in the previous step, and the output is the analysis result indicating the emotional state. The server uses Affectiva or IBM Watson SDK to quantify basic emotions such as joy, anger, sadness, and happiness.

[0587] Step 4:

[0588] The server uses a generative AI model to generate optimal customer service suggestions based on sentiment analysis results. In this step, sentiment analysis results are used as input, and specific customer service suggestions and prompts are generated as output. The server then performs an analysis process to present the generated information to the user in real time.

[0589] Step 5:

[0590] The user reviews customer service suggestions provided by the server via their terminal and adjusts their actions as needed. The input is the customer service suggestion from the server, and the output is the user's improved customer service behavior. The user selects the optimal response according to the customer service situation and provides service that meets the customer's needs.

[0591] Step 6:

[0592] The server collects feedback on user behavior and uses it to improve the accuracy of the generative model. The input is user feedback, and the output is an updated analysis algorithm and generative AI model. The server periodically updates the database to prepare for more accurate analysis in future applications.

[0593] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0594] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0595] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0596] [Fourth Embodiment]

[0597] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0598] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0599] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0600] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0601] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0602] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0603] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0604] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0605] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0606] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0607] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0608] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0609] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0610] This invention provides specific methods for effectively performing data collection, model training, real-time analysis, solution prediction, and user feedback reception in a dispute resolution support system. This enables rapid and efficient consensus building.

[0611] First, the server collects historical case data related to dispute resolution from specialized databases and digital archives. This includes court records, negotiation histories, and legal documents. The collected data is converted into a reusable format and prepared for model training.

[0612] Subsequently, this data is used to train a generative AI model. The server extracts meaningful patterns using an automated process and learns results and solutions from similar cases. This learning process enables the model to propose solutions immediately and appropriately in new negotiations.

[0613] Next, once negotiations begin, the terminals collect real-time data (such as voice utterances, text input, and facial expressions and movements analyzed via video). Using this information, the server performs data analysis to infer the participants' intentions, emotions, and positions. Based on this, the optimal solution is generated, taking into account past learning results.

[0614] The server then provides the proposed solutions to the user's terminal. The solutions are presented as multiple options based on their feasibility and likelihood of success. The user evaluates these options through the interface and provides feedback as needed.

[0615] As a concrete example, consider the use of this system in a contract dispute between companies. The user first inputs data on past contract disputes into the system. The server analyzes this information, searches its database for similar cases, and utilizes a learning model. As a result, the model quickly proposes the optimal agreement during negotiations, allowing the user to proceed with agreement formation on the spot. Through this process, agreement formation is made more efficient, contributing to the early resolution of disputes.

[0616] The following describes the processing flow.

[0617] Step 1:

[0618] The server calls a specialized database API to collect historical case data related to dispute resolution, retrieving court records, negotiation details, and case law data. The retrieved data is cleaned for duplicates and errors and stored as a clean dataset.

[0619] Step 2:

[0620] The server trains a generative AI model using a clean dataset. The data input to the model is classified, and key information necessary for learning is highlighted through feature extraction. After training, the model learns patterns from past cases and acquires knowledge applicable to new conflict resolution situations.

[0621] Step 3:

[0622] The device uses voice and video sensors during negotiations to capture real-time statements, facial expressions, and the other party's reactions, and transmits this data to a server. This data includes indicators of the participants' intentions and emotions.

[0623] Step 4:

[0624] The server analyzes the received real-time data using natural language processing and image analysis algorithms. Based on the analysis results, it infers the intentions and emotions of the other party in negotiations and generates the optimal solution. The generated solution reflects past learning achievements.

[0625] Step 5:

[0626] The server prioritizes solutions based on the analysis and presents them to the terminal as multiple options. Each option includes detailed information about its effectiveness and expected impact.

[0627] Step 6:

[0628] Users select the most appropriate option from those presented via their device and provide feedback as needed. This feedback includes not only manual evaluations but also the provision of new information.

[0629] Step 7:

[0630] The device sends user feedback to the server, which then uses that feedback to fine-tune the AI ​​model. This refines the solution and improves the quality of proposals in future negotiations.

[0631] Step 8:

[0632] The server implements encryption and access control during data transmission, reception, and storage to ensure thorough privacy protection. This guarantees that data handled during the dispute resolution process will not be leaked externally and will be managed securely.

[0633] (Example 1)

[0634] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0635] In conflict resolution, there is a need to provide swift and appropriate solutions based on past cases. However, conventional systems have shortcomings in data collection and analysis, making it difficult to grasp participants' intentions and emotions in real time and to provide flexible solutions.

[0636] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0637] In this invention, the server includes a device for collecting and storing past problem-solving cases, a device for executing generative artificial intelligence that learns based on the accumulated information, and a device for analyzing information during negotiations in real time and proposing the optimal solution. This makes it possible to infer the intentions and emotions of the participants, quickly generate and present a variety of solutions, and flexibly improve them based on user evaluations.

[0638] "Past problem-solving examples" refer to specific cases and approaches to various problems that have been solved in the past.

[0639] A "storage device" refers to a system equipped with the functionality to efficiently store collected data and make it accessible as needed.

[0640] "Generative artificial intelligence" refers to algorithms and models that learn from collected data and generate new information.

[0641] "Information being negotiated in real time" refers to dynamic data such as voice, actions, and facial expressions generated during the negotiation process.

[0642] "Inferring intentions and emotions" refers to the process of analyzing collected data and forming hypotheses about participants' goals and emotions.

[0643] "Proposing a solution" refers to outlining specific steps or strategies for solving a problem based on the analysis results.

[0644] "Diverse solutions" refers to multiple solutions offered from different perspectives and approaches.

[0645] "User evaluation" refers to opinions and feedback from users regarding the solutions provided by the system.

[0646] "Flexible improvement" refers to the ability to adjust the system's suggested solutions and processes based on user feedback.

[0647] To implement this invention, the system will function using the following hardware and software.

[0648] First, the server accesses specialized databases and digital archives to collect and store past problem-solving examples. Specifically, it uses Python scripts to retrieve data via APIs and saves it to a MySQL database. The collected data is preprocessed using natural language processing libraries such as NLTK and spaCy, and prepared as a training dataset for generative AI models.

[0649] Next, the server builds a generative AI model using machine learning frameworks such as TensorFlow or PyTorch based on the pre-processed data. The model has the ability to learn from past examples, extract patterns, and predict new solutions.

[0650] The device collects real-time data using sensors and cameras while negotiations are taking place. For example, audio data is collected via a microphone and converted into text using the Google Speech-to-Text API. OpenCV is used for video analysis to capture participants' facial expressions and movements.

[0651] Based on the collected real-time data, the server performs intent and sentiment analysis, and generates the optimal solution by comparing it with past learning results. Multiple solutions are then presented to the user's device, and the user provides evaluation and feedback through the interface.

[0652] As a concrete example, consider a scenario in which this system is used in contract negotiations between companies. The user inputs data on past contract disputes into the server. The server analyzes this information and utilizes a generation AI model that references similar cases in the database. As a result, the optimal agreement is quickly presented during negotiations. For example, by giving instructions such as "Propose the best solution for the current contract negotiation," the system will quickly provide a solution based on that instruction. This enables rapid agreement formation.

[0653] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0654] Step 1:

[0655] The server collects past problem-solving cases from specialized databases and digital archives. It receives data via API as input and stores this data in a MySQL database. Data collection is performed using a Python script, and the collected data is saved for subsequent analysis and model training.

[0656] Step 2:

[0657] The server preprocesses the collected data. The input is the raw data obtained in step 1. This data is processed using natural language processing libraries such as NLTK and spaCy, including tokenization, stop word removal, and syntactic analysis. The output is clean text data, which is used to train the generative AI model.

[0658] Step 3:

[0659] The server trains a generative AI model using pre-processed data. It receives processed text data as input and trains a machine learning model using TensorFlow or PyTorch. This process improves the ability to extract patterns and predict solutions through supervised and unsupervised learning. The output is the trained AI model.

[0660] Step 4:

[0661] The device collects information about ongoing negotiations in real time. It receives audio and video data from sensors as input. Audio data is captured using a microphone and converted to text using the Google Speech-to-Text API. Video data is captured by a camera and analyzed using OpenCV to infer participants' intentions and emotions from their facial expressions and movements. The output is the analyzed real-time data.

[0662] Step 5:

[0663] The server analyzes real-time data sent from the terminal and generates the optimal solution. It receives real-time analysis data as input, compares it with past learning results, and generates a solution. The generated solution is presented to the user as output.

[0664] Step 6:

[0665] The server receives feedback from users. Input consists of user ratings and comments on the options submitted through the evaluation screen. This feedback is used to improve the AI ​​model and update the data to help suggest more appropriate solutions. Output consists of an improved AI model and the addition of new data.

[0666] (Application Example 1)

[0667] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0668] In modern society, conflicts and troubles require swift and safe resolution. However, current systems struggle with real-time data analysis and the presentation of appropriate solutions, particularly inferring intentions and emotions from nonverbal information in collective settings. This hinders effective consensus building among participants. A solution is needed to address these challenges and achieve smooth problem resolution while maintaining security.

[0669] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0670] In this invention, the server includes means for collecting and storing past case data, means for executing a generative model that learns based on the accumulated data, and means for analyzing the data in real time and proposing the optimal solution. This enables accurate analysis of participants' intentions and emotions, and efficient consensus building.

[0671] "Past case data" refers to records of similar conflicts and problems that have occurred in the past, and serves as learning material for conflict resolution.

[0672] "Storing" refers to the act of systematically saving collected data within a system such as a database so that it can be used effectively.

[0673] A "generative model" is an AI model built to derive and propose patterns and solutions from data using machine learning algorithms.

[0674] "Analyzing data in real time" is the process of immediately processing ongoing situations and information to obtain analysis results.

[0675] "Nonverbal information" refers to information such as body movements and facial expressions that are used as clues to infer a person's emotions and intentions.

[0676] "Processing video data" refers to the process of analyzing visual data acquired from devices such as cameras and extracting specific information.

[0677] "Incorporating feedback into model learning" means making adjustments to improve the performance of an AI model based on evaluations and responses obtained from users.

[0678] In order to implement this invention, it is important to properly configure the server, user terminal, and necessary software environment.

[0679] The server collects past case data and stores it in a database. Specifically, it efficiently retrieves relevant data from various public databases and internal documents, and prepares it as a training dataset for machine learning. This data is used to train a generative AI model and forms the basis for proposing solutions to new conflicts and problems. Python and TensorFlow can be used to build the generative AI model.

[0680] The user terminal is designed as a smartphone or smart glasses and is equipped with the ability to collect audio and video in real time. This allows for the acquisition of nonverbal information such as voice, facial expressions, and body language in negotiations and disputes, and this data is sent to a server for immediate analysis. The analysis uses tools such as OpenCV to process and interpret emotions and intentions from the video data in real time.

[0681] The user receives solutions provided by the server through an interface, evaluates these options, and provides feedback. This feedback is used to improve the accuracy of the generated AI model on the server and to enhance the accuracy of future suggestions. This allows the system to continuously improve itself.

[0682] As a concrete example, in a work environment where misunderstandings may occur during communication, the system intervenes immediately when an incident occurs to facilitate rapid problem resolution. An example of a prompt message would be, "Which solution is best to mitigate the incident caused by past misunderstandings?"

[0683] This configuration allows the system to respond promptly and accurately, and is expected to facilitate the rapid resolution of complex conflicts and problems.

[0684] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0685] Step 1:

[0686] The server collects historical case data. It gathers relevant information from databases and public documents and converts it into a format usable by machine learning. At this stage, the input is unorganized raw data, and the output is a structured dataset. Specifically, natural language processing techniques are used to analyze legal documents and negotiation records.

[0687] Step 2:

[0688] The server trains the generative AI model. Using the structured data obtained in Step 1, the AI ​​model is trained using TensorFlow. The case dataset is used as input, and the output is a generalized solution pattern. The model automatically learns the optimal solution using a pattern recognition algorithm.

[0689] Step 3:

[0690] The device collects data in real time. During negotiations or when problems occur, it uses smartphones or smart glasses to acquire audio and video data. The input data consists of audio files and video frames, which are transferred to the server as compressed digital data. Non-verbal information is also acquired simultaneously through the device's sensors.

[0691] Step 4:

[0692] The server performs real-time analysis. It receives data sent from the terminal, analyzes facial expressions and movements from video using OpenCV, and converts audio data into text using natural language processing technology. Based on the analyzed intent and emotion information, a generative AI model proposes the optimal solution. The input is real-time data, and the output is the inferred intent and emotion, as well as the proposed solution.

[0693] Step 5:

[0694] The user evaluates the proposed solutions. The server receives the solutions presented to the terminal and allows the user to review each option through the interface. The input consists of the presented solutions, and the user selects one and returns feedback to the server. The evaluation results are collected and used to improve the model.

[0695] Step 6:

[0696] The server utilizes feedback to optimize the generated AI model. It analyzes feedback data collected from users and updates the model's parameters. The input is user feedback, and the output is the improved AI model. Improved model accuracy leads to higher quality suggestions in subsequent iterations.

[0697] In this series of processes, the roles of the server, terminal, and user work together in coordination, enabling the rapid and effective resolution of problems.

[0698] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0699] This invention describes a method for combining an emotion engine with a conflict resolution support system to grasp the emotional state of participants in real time and utilize this information to propose solutions. This makes it possible to guide consensus building in a more precise and amicable manner.

[0700] The server utilizes an emotion engine to analyze real-time video and audio data collected during negotiations. This analysis detects emotions from participants' facial expressions, tone of voice, and word choice, and then quantitatively evaluates their emotional state based on this data. The emotion engine identifies basic emotions such as joy, anger, sadness, surprise, and fear, providing insights into the dynamics of negotiations.

[0701] The terminal presents the user with emotional data processed by the server, making it easier for the user to understand the emotional state of the other party during negotiations. This information can also serve as an important indicator when selecting a proposed solution.

[0702] Furthermore, the server uses data obtained from the emotion engine to influence the output of the generative AI model. Specifically, it adjusts the proposed solutions by taking into account the other party's emotional state. For example, if the negotiating partner is stressed, a gentler and less burdensome solution can be prioritized. In this way, the accuracy and effectiveness of the proposals are improved.

[0703] As a concrete example, in the case of negotiations for a business partnership, the user activates the dispute resolution support system via their terminal. As the negotiations progress, the emotion engine analyzes the emotions of the participants, and the server reflects the analysis results in real time. By selecting the solution that best suits the other party's emotions from among several proposed solutions, the user can reach a smoother agreement.

[0704] This invention contributes to increasing the transparency and reliability of negotiations and innovates the process of reaching agreements in dispute resolution.

[0705] The following describes the processing flow.

[0706] Step 1:

[0707] The server collects past dispute resolution cases from the database, performs data cleaning, and converts and saves them in the format required for the generated AI model. This process is necessary to prepare the model so that it can learn properly and handle new cases.

[0708] Step 2:

[0709] The server trains a generative AI model using cleaned data. The model learns patterns from the cases and accumulates insights that will be useful for suggesting future solutions. This training forms the foundation for the model to accurately predict new cases.

[0710] Step 3:

[0711] Once negotiations begin, the terminal collects audio and video data in real time and sends it to the server. This data includes what is said, facial expressions, and tone of voice. This real-time data is used for instant analysis.

[0712] Step 4:

[0713] The server analyzes the received real-time data using an emotion engine. This analysis quantifies the participants' emotional states, generating an emotional profile. This profile represents basic emotional states such as joy, anger, sadness, and happiness.

[0714] Step 5:

[0715] Based on the analyzed emotional profiles, the server utilizes a generative AI model to propose the optimal solution. Here, adjustments are made to match the participants' emotions, ensuring flexibility in the solution.

[0716] Step 6:

[0717] The device presents the generated solutions to the user. Multiple options are displayed, each with a description of its characteristics and expected effects, taken into account from the emotional profile.

[0718] Step 7:

[0719] Users evaluate the presented solutions, select the option they believe to be the best, and enter feedback into their device. This feedback includes information about whether they adopted the suggestion or made minor adjustments.

[0720] Step 8:

[0721] The terminal sends user feedback to the server, which adjusts the model parameters based on the feedback to improve the accuracy of future suggestions. This feedback loop strengthens the overall learning ability of the system.

[0722] Step 9:

[0723] The server encrypts and stores data, ensuring security to protect information privacy. This allows the system to operate securely and prevents data leaks.

[0724] (Example 2)

[0725] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0726] Traditional dispute resolution support systems have faced the challenge of difficulty in proposing optimal solutions while taking into account the emotional states of the participants. There is a need for a mechanism that understands the emotional dynamics of participants during negotiations and facilitates consensus building based on that understanding.

[0727] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0728] In this invention, the server includes means for collecting and storing past cases, means for executing a generative model that learns based on the accumulated information, and means for analyzing video and audio information obtained from negotiations in real time and quantitatively evaluating the emotions of the participants. This makes it possible to grasp the emotional state of the participants in real time and propose the optimal solution based on that.

[0729] A "case study" refers to a specific event or related incident that occurred in the past, and contains information that can be useful in resolving disputes.

[0730] "Information" is a general term for the data and knowledge collected, and in particular includes data related to video and audio acquired in real time.

[0731] A "generative model" refers to an algorithm or method that learns patterns and trends from data and generates new predictions or suggestions based on them.

[0732] "Analysis" is a general term for processes and analyses performed on data and information in order to understand their structure and elements, and in particular includes the process of quantifying emotional states.

[0733] "Emotion" refers to a person's psychological state, and in negotiations, it refers to the psychological tendencies and changes of participants.

[0734] "Solutions" refer to a general term for the measures and means proposed to resolve conflicts and problems, and include specific means adjusted by generative AI models.

[0735] "Privacy protection" refers to a set of measures and policies to prevent the improper disclosure of personal information and data.

[0736] This invention is a system for understanding the emotional state of participants in dispute resolution in real time and proposing the optimal solution based on that understanding.

[0737] The server collects video and audio information obtained from the negotiation in real time and processes this data using an emotion analysis engine. Specifically, it uses image recognition software to analyze facial expressions in the video data and an audio analysis algorithm to analyze the tone and speed of voices in the audio data. As a result, the emotional state of the participants is quantified and stored in a database.

[0738] The terminal displays emotional data sent from the server on the user interface, allowing users to visualize the progress of negotiations from an emotional perspective. For example, the degree of emotion is shown with colors or graphs, allowing users to intuitively understand the other party's state.

[0739] The server utilizes machine learning algorithms as a generative AI model to generate optimal solutions based on accumulated emotional data. In this process, prompt statements are input to the generative AI model, and the system proposes the most suitable solution that reflects the emotional data. For example, a prompt statement such as "Gently adjust the solution when participants are experiencing stress" can be used.

[0740] Users can review the proposed solutions through their device and select the most suitable option from multiple choices. This dynamic presentation of options allows users to navigate a smooth process towards agreement while being considerate of the other party's feelings.

[0741] As a concrete example, in business partnership negotiations between companies, activating this system would enable flexible proposals that respond to changes in the other party's emotions, leading to smoother negotiations. This system would improve the transparency and reliability of agreement building and promote an innovative negotiation process.

[0742] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0743] Step 1:

[0744] The server collects video and audio data in real time from the negotiation site as input. External input devices such as cameras and microphones are used for this purpose. This data becomes the basic information for sentiment analysis in the next processing step.

[0745] Step 2:

[0746] The server analyzes the input video data using image recognition software, quantifying emotions from the participants' facial expressions. For example, it identifies emotions such as joy and anger from eye and mouth movements and outputs this as quantified data. In parallel, audio data is processed through a voice analysis algorithm, recognizing emotions from voice tone and pitch, and this too is output as quantified data.

[0747] Step 3:

[0748] The server aggregates the quantified emotion data obtained in Step 2 and inputs it into the emotion analysis engine. This engine uses deep learning to extract complex patterns and evaluate overall emotional trends. This process comprehensively analyzes the participants' current emotional state and outputs the results to a database.

[0749] Step 4:

[0750] The terminal visually displays emotional state data received from the server using a GUI. Graphs and color coding are used to allow the user to intuitively understand the emotional changes of the negotiating partner. For example, heightened emotions are displayed in red, and calmer emotions in blue.

[0751] Step 5:

[0752] The server inputs emotional state data as prompt statements into an AI model, which then generates optimal solution suggestions. Since these prompt statements reflect the participant's subjective emotional state, the suggested solutions take that state into account. The generated solutions are output as data.

[0753] Step 6:

[0754] The user reviews multiple proposed solutions on their device. Each solution is accompanied by an explanation based on the other party's emotional state, allowing the user to choose the option best suited to the other party's changing emotions. This enables the user to conduct negotiations more smoothly.

[0755] (Application Example 2)

[0756] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0757] In customer service, the challenge lies in understanding customers' emotional states in real time, improving the quality of service provided, and increasing customer satisfaction. Furthermore, it is necessary to improve the ability to quickly identify customers' potential dissatisfactions and needs and provide immediate, personalized support.

[0758] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0759] In this invention, the server includes a device for collecting and storing past solution cases, a device for executing a generative model that learns based on the accumulated data, a device for analyzing information in real time and proposing the optimal solution, a device for receiving user feedback and updating the analysis results, a device for protecting the privacy of information, a device for analyzing the video and audio of participants and evaluating their emotional state in real time, and a device for adjusting the proposed solution based on the evaluated emotional state. This makes it possible to accurately grasp the emotional state of customers, provide appropriate services, and increase customer satisfaction.

[0760] "Past case studies" refer to real-world case information that has been collected and stored in the system to date.

[0761] A "generative model" is an algorithm or program that learns from past data and proposes new solutions.

[0762] "Real-time analysis" is a process that instantly processes participants' video and audio data in real time and extracts the necessary information.

[0763] "Emotional state" refers to the mental condition or emotions inferred from nonverbal information such as a participant's facial expressions and tone of voice.

[0764] "Evaluation" is the process of measuring emotional states and the suitability of solutions based on collected data, and then making a judgment.

[0765] "Adjusting solutions" refers to the action of optimizing the proposed solutions based on the emotional information collected.

[0766] "Feedback" refers to the response information provided by users, which is used to improve and adjust the system.

[0767] The system for implementing this invention has a structure that connects a smart device and a server to understand the customer's emotional state and provide appropriate feedback. Specifically, the server receives video and audio data and processes them using an emotion analysis algorithm. This makes it possible to evaluate the customer's emotional state in real time from their facial expressions and tone of voice.

[0768] The hardware used includes smart glasses and high-performance microphones. These capture customer video and audio and transmit the data to a server via Bluetooth or Wi-Fi. Meanwhile, the server processes the data using emotion analysis SDKs such as Affectiva and IBM Watson to quantitatively evaluate the customer's emotional state.

[0769] Based on this, the server uses a generative AI model to suggest the optimal customer service approach. For example, if a customer is feeling stressed, it might suggest a calmer tone when explaining the product. This allows users to gain guidance on how to improve customer satisfaction.

[0770] As a concrete example, if a customer shows signs of anxiety during customer service at a physical store, this system will suggest "explaining the product's benefits and customer testimonials in detail to reassure them." Furthermore, by inputting prompts like the following into the AI ​​model, further improvement suggestions can be obtained.

[0771] Example of a prompt:

[0772] "The customer appeared to be feeling uneasy based on their expression during the product explanation. Please suggest appropriate countermeasures for this situation."

[0773] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0774] Step 1:

[0775] The terminal captures the customer's video and audio using its built-in camera and microphone. This process takes video and audio data as input and outputs recorded data in digital format. The terminal then preprocesses this data, performing noise reduction and resolution adjustments.

[0776] Step 2:

[0777] The terminal transmits captured data to the server via Bluetooth or Wi-Fi. The input is the video and audio data acquired by the terminal, and the output is the digital data transferred to the server. The terminal checks the stability and speed of data transfer and retries if a transfer error occurs.

[0778] Step 3:

[0779] The server feeds the received data into an emotion analysis algorithm. The input is the video and audio data transferred in the previous step, and the output is the analysis result indicating the emotional state. The server uses Affectiva or IBM Watson SDK to quantify basic emotions such as joy, anger, sadness, and happiness.

[0780] Step 4:

[0781] The server uses a generative AI model to generate optimal customer service suggestions based on sentiment analysis results. In this step, sentiment analysis results are used as input, and specific customer service suggestions and prompts are generated as output. The server then performs an analysis process to present the generated information to the user in real time.

[0782] Step 5:

[0783] The user reviews customer service suggestions provided by the server via their terminal and adjusts their actions as needed. The input is the customer service suggestion from the server, and the output is the user's improved customer service behavior. The user selects the optimal response according to the customer service situation and provides service that meets the customer's needs.

[0784] Step 6:

[0785] The server collects feedback on user behavior and uses it to improve the accuracy of the generative model. The input is user feedback, and the output is an updated analysis algorithm and generative AI model. The server periodically updates the database to prepare for more accurate analysis in future applications.

[0786] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0787] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0788] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0789] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0790] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0791] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0792] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0793] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0794] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0795] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0796] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0797] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0798] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0799] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0800] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0801] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0802] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0803] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0804] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0805] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0806] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0807] The following is further disclosed regarding the embodiments described above.

[0808] (Claim 1)

[0809] A means of collecting and accumulating past dispute resolution cases,

[0810] A means of executing a generative model that learns based on accumulated data,

[0811] A means of analyzing negotiation data in real time and proposing the optimal solution,

[0812] A means of receiving user feedback and updating analysis results,

[0813] Means of protecting data privacy,

[0814] A dispute resolution support system that includes this.

[0815] (Claim 2)

[0816] The system according to claim 1, further comprising means for analyzing spoken data and nonverbal information collected in real time to infer the intentions and emotions of participants.

[0817] (Claim 3)

[0818] The system according to claim 1, further comprising means for displaying multiple options for a proposed solution and their analysis results, and allowing the user to select or modify them.

[0819] "Example 1"

[0820] (Claim 1)

[0821] A device for collecting and accumulating past problem-solving cases,

[0822] A device that performs generative artificial intelligence that learns based on accumulated information,

[0823] A device that analyzes information during negotiations in real time and proposes the optimal solution,

[0824] A device that accepts user evaluations and updates the analysis results,

[0825] A device for protecting information,

[0826] A device that analyzes voice input and video data from a terminal to infer the intentions and emotions of participants,

[0827] A device that presents solutions obtained through dynamic analysis as multiple options,

[0828] A system that includes this.

[0829] (Claim 2)

[0830] The system according to claim 1, further comprising a device that displays multiple options for a proposed solution and their analysis results, allowing the user to select or modify them.

[0831] (Claim 3)

[0832] The system according to claim 1, further comprising a device for performing dynamic analysis using a generative artificial intelligence model that makes optimal suggestions by comparing them with past learning results.

[0833] "Application Example 1"

[0834] (Claim 1)

[0835] A means of collecting and accumulating past case data,

[0836] A means of executing a generative model that learns based on accumulated data,

[0837] A means of analyzing data in real time and proposing the optimal solution,

[0838] A means of receiving user feedback and updating analysis results,

[0839] Means of protecting data privacy,

[0840] A means of analyzing spoken and nonverbal data collected in real time to infer participants' intentions and emotions,

[0841] A means of processing video data and recognizing nonverbal information,

[0842] A means to provide optimized solutions and incorporate feedback into model learning,

[0843] A system that includes this.

[0844] (Claim 2)

[0845] The system according to claim 1, which generates analysis results from data collected in real time and presents the optimal solution.

[0846] (Claim 3)

[0847] The system according to claim 1, which displays multiple options and their analysis results, and allows the user to select or modify them.

[0848] "Example 2 of combining an emotion engine"

[0849] (Claim 1)

[0850] Means for collecting and accumulating past cases,

[0851] A means of executing a generative model that learns based on accumulated information,

[0852] A means of quantitatively evaluating the emotions of participants by analyzing video and audio information obtained from negotiations in real time,

[0853] A means of proposing the optimal solution based on analyzed emotional information and reflecting it in a generative AI model,

[0854] A means for receiving user input and updating analysis results and suggestions,

[0855] Means for protecting the privacy of information,

[0856] A system that includes this.

[0857] (Claim 2)

[0858] The system according to claim 1, further comprising means for processing speech and nonverbal information collected in real time to identify the emotional state of a participant.

[0859] (Claim 3)

[0860] The system according to claim 1, further comprising means for displaying multiple options for a proposed solution and their sentiment analysis results, and allowing the user to select or modify them.

[0861] "Application example 2 when combining with an emotional engine"

[0862] (Claim 1)

[0863] A device for collecting and accumulating past case studies,

[0864] A device that runs a generative model that learns based on accumulated data,

[0865] A device that analyzes information in real time and proposes the optimal solution,

[0866] A device that receives user feedback and updates the analysis results,

[0867] A device that protects the privacy of information,

[0868] A device that analyzes participants' video and audio to evaluate their emotional state in real time,

[0869] A device that adjusts proposed solutions based on the assessed emotional state,

[0870] A system that includes this.

[0871] (Claim 2)

[0872] The system according to claim 1, further comprising a device for analyzing spoken and nonverbal information collected in real time to infer the intentions and emotions of participants.

[0873] (Claim 3)

[0874] The system according to claim 1, further comprising a device that displays multiple options for a proposed solution and their analysis results, allowing a user to select or modify them. [Explanation of Symbols]

[0875] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means of collecting and accumulating past case data, A means of executing a generative model that learns based on accumulated data, A means of analyzing data in real time and proposing the optimal solution, A means of receiving user feedback and updating analysis results, Means of protecting data privacy, A means of analyzing spoken and nonverbal data collected in real time to infer participants' intentions and emotions, A means of processing video data and recognizing nonverbal information, A means to provide optimized solutions and incorporate feedback into model learning, A system that includes this.

2. The system according to claim 1, which generates analysis results from data collected in real time and presents the optimal solution.

3. The system according to claim 1, which displays multiple options and their analysis results, and allows the user to select or modify them.

Images