system

Abstract

We provide the system. [Solution] A means of acquiring a user's biometric information in real time, A means for analyzing the user's emotional state based on the aforementioned biometric information, A means for proposing the optimal action based on the aforementioned analysis results, A method for compiling information after the work is completed and analyzing the factors for success, A means for visualizing the aforementioned analysis results and providing feedback to the user, Means for detecting the surrounding environment and potential threats, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance as a response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In business activities, there are problems such as a decrease in the conclusion rate of business negotiations due to lack of experience and difficulty in grasping customer psychology. There is also a problem that a large amount of time is required for business negotiation preparation and post-processing, and efficient business activities cannot be carried out. In particular, young sales representatives find it difficult to appropriately understand the reactions of customers during business negotiations and take appropriate actions, and as a result, the success rate of business negotiations is significantly affected. It is required to solve such problems, provide real-time decision-making support during business negotiations, and improve the conclusion rate of business negotiations and shorten the business negotiation time.

Means for Solving the Problems

[0005] This invention provides a means for acquiring a user's biometric data in real time and a means for analyzing the user's emotional state based on that biometric data. It also combines this with a means for proposing optimal actions during business negotiations based on the analysis results, and provides a means for aggregating data after the negotiation and analyzing the factors for success. Furthermore, by visualizing the analysis results and providing feedback to the user, the invention aims to improve the overall efficiency of business negotiations and increase the closing rate. In addition, it provides a means for providing real-time voice notifications based on the analysis results, and uses user negotiation data to perform machine learning and optimize future proposals, thereby realizing a comprehensive system for improving the quality of sales activities.

[0006] "Means for acquiring user biometric data" refers to devices or methods for collecting a user's brainwaves, voice, facial expressions, etc., in real time using sensors and microphones during business negotiations.

[0007] "Means for analyzing emotional states" refers to algorithms or processes that identify a user's psychological state based on acquired biometric data and evaluate emotions such as excitement, attention, and stress.

[0008] "Means of suggesting optimal actions" refers to a method or system that provides instructions or advice generated to guide the user's next statements or actions during a business negotiation, based on their analyzed emotional state.

[0009] "Means for aggregating data and analyzing success factors" refers to a process or device for aggregating all data collected during a business negotiation and identifying elements that contributed to the successful conclusion of the deal and areas for improvement.

[0010] "Means of visualizing analysis results and providing feedback to users" refers to methods or systems that display data analysis results after a business negotiation in a visual format such as a dashboard, and present users with areas for improvement and success factors.

[0011] "Means of providing real-time voice notifications" refers to a program or device that immediately communicates the results of the emotional state analyzed during a business negotiation to the user via voice.

[0012] "Methods for optimizing future proposals using machine learning" refers to methods or systems that use machine learning algorithms based on past sales negotiation data to improve the accuracy of advice given in future sales negotiations. [Brief explanation of the drawing]

[0013] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Embodiment 2 when the emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when the emotion engine is combined.

Mode for Carrying Out the Invention

[0014] Hereinafter, an example of an embodiment of the system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0015] First, the terms used in the following description will be explained.

[0016] In the following embodiments, the numbered processor (hereinafter simply referred to as "processor") may be one arithmetic unit or a combination of a plurality of arithmetic units. Also, the processor may be one type of arithmetic unit or a combination of a plurality of types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0017] In the following embodiments, the numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0018] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0019] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0020] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0021] [First Embodiment]

[0022] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0023] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0024] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0025] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0026] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0027] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0028] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0029] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0030] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0031] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0032] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0033] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0034] This invention is a system that optimizes user behavior during business negotiations, primarily characterized by real-time acquisition and analysis of biometric data, and action suggestions based on the results. This system operates via a terminal worn by the user and is centrally controlled by a server.

[0035] Data acquisition and sentiment analysis:

[0036] First, the device collects biometric data using sensors such as the user's brainwave sensor, microphone, and camera. This data is used to capture in real time the user's state of mind, such as their level of concentration, stress, and emotions, during business negotiations. The collected data is immediately encrypted and sent to the server.

[0037] Analysis and proposals for supporting business negotiations:

[0038] The server uses advanced algorithms based on biometric data to analyze emotional states. This identifies how interested the customer is in the deal, whether they are experiencing stress, and so on. Based on the analysis, it determines the optimal course of action and generates a proposal. This proposal includes an appropriate talk script, negotiation strategy, and closing timing.

[0039] Real-time notifications and feedback:

[0040] The proposal is sent from the server to the terminal and notified to the user in real time via audio through earphones. For example, the moment a customer shows interest in price negotiation, the optimal negotiation phrase is notified, allowing the user to change the topic at the appropriate time. After the negotiation is concluded, the server uses the aggregated data to analyze the factors contributing to the success of the negotiation and visualizes the results on a dashboard.

[0041] Machine learning for learning and improvement:

[0042] Sales negotiation data is continuously analyzed by machine learning algorithms. Based on this analysis, the server optimizes the support provided for subsequent sales negotiations, thereby improving the accuracy of proposals. This allows users to continuously improve themselves over time and enhance the quality of their sales activities.

[0043] Specific example:

[0044] For example, if a terminal acquires customer facial expression data during a business negotiation, and the server analyzes this data to determine that the customer is bored with the negotiation, it will notify the user in real time to introduce a new topic to change the flow of the negotiation. Also, if the customer shows interest in price, the server will provide voice guidance on the appropriate timing for closing the deal, helping to increase the closing rate. In this way, the present invention has a structure that optimizes sales activities based on data and supports the success of business negotiations.

[0045] The following describes the processing flow.

[0046] Step 1:

[0047] The device uses the user's brainwave sensor, microphone, and camera to acquire biometric data in real time during business negotiations. This includes measuring concentration levels using brainwaves, analyzing emotions from voice, and recognizing facial expressions using the camera.

[0048] Step 2:

[0049] The device encrypts the collected biometric data and transmits it to the server in real time. Buffering technology may be used during this process to ensure smooth data transmission.

[0050] Step 3:

[0051] The server analyzes the biometric data it receives. Here, a machine learning model is used to estimate the user's emotional state and score their level of interest, stress level, and attention focus.

[0052] Step 4:

[0053] The server generates suggestions for the user based on the results of sentiment analysis. This includes refining the talk script while considering the progress of the sales negotiation, and identifying the timing of negotiations and closing points.

[0054] Step 5:

[0055] The server sends the generated proposal to the terminal, which then provides real-time audio notifications to the user via earphones. The user is then provided with specific instructions to improve the flow of the sales negotiation.

[0056] Step 6:

[0057] Once a business negotiation is complete, the server analyzes the collected data to identify the factors that contributed to the negotiation's success and areas for improvement. These analysis results are then visualized on a dashboard.

[0058] Step 7:

[0059] The server uses machine learning based on past sales data to improve the accuracy of proposals for future sales meetings. This establishes a continuous improvement cycle.

[0060] Step 8:

[0061] Users develop self-improvement plans based on feedback from the dashboard and prepare for new business opportunities. This feedback helps improve users' sales skills.

[0062] (Example 1)

[0063] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0064] To ensure the success of business negotiations, it is crucial to accurately understand the customer's emotional state in real time and make actionable suggestions based on that understanding. However, traditional methods have made it difficult to grasp the situation in real time during negotiations and make appropriate suggestions, thus failing to maximize the outcome of negotiations.

[0065] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0066] In this invention, the server includes means for encrypting biometric information and transmitting it to a central device, means for analyzing the biometric information and evaluating the emotional state at the central device, and means for utilizing a generative model to propose the optimal action based on the evaluation results. This makes it possible to accurately grasp the emotional state of a customer in real time during a business negotiation and improve the outcome of the negotiation by proposing the optimal action.

[0067] "Biometric information" refers to data that indicates the physical state of an organism, and includes information such as brain waves, heart rate, facial expressions, and voice tone.

[0068] A "device" refers to an instrument or combination designed to perform a specific function, and in this context, it refers to a sensor device.

[0069] A "central system" refers to a core computer system used for data aggregation, processing, and analysis.

[0070] "Encryption" is a security technology that transforms information according to specific rules to prevent unauthorized access.

[0071] "Action suggestions" are guidelines and advice that instruct users on what they believe to be the most optimal actions based on the results of biometric data analysis.

[0072] A "generative model" is an artificial intelligence algorithm used to generate appropriate responses or outputs for a given input.

[0073] "Machine learning" is a data analysis technique aimed at enabling computers to automatically improve their performance through experience.

[0074] "Voice guidance" refers to audio messages used to convey information or instructions to users aurally.

[0075] This invention provides a system that supports business negotiations in real time through a terminal worn by the user. The terminal functions as a device that collects biometric information and provides feedback to the user as needed. Specifically, it is equipped with an electroencephalogram (EEG) sensor, a microphone, and a camera, which enable the acquisition of the user's brainwaves, voice, and facial expression data.

[0076] The device protects the collected biometric information using AES encryption technology and transmits it to the server via a secure HTTP protocol. After receiving this data, the server analyzes it using a generative AI model to evaluate the user's emotional state and level of interest. Based on the evaluation, prompts are used to generate optimal action suggestions for the progress of the negotiation. For example, a prompt such as, "Analyze the customer's level of interest in the price offered in real time and suggest the best way to proceed with the negotiation," might be used.

[0077] The proposed solutions are provided to the user in real time as voice guidance, enabling timely responses during sales negotiations. For example, if a customer shows interest in a particular product, the suggested closing phrase is notified to the user through their earphones. After the negotiation ends, the server analyzes the results, visualizes them, and provides feedback to the user. This allows the user to continuously make data-driven improvements in future negotiations.

[0078] The overall effect of this system is to improve user performance during sales negotiations and contribute to increasing the success rate of those negotiations.

[0079] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0080] Step 1:

[0081] The user puts on the device and begins the business negotiation. The device uses an electroencephalogram (EEG) sensor, microphone, and camera to collect the user's biometric information in real time. This information includes brainwaves, voice tone, and facial expression data. The collected biometric information is used as input, and the data is temporarily stored on the device.

[0082] Step 2:

[0083] The device encrypts the collected biometric information using AES encryption technology. The encrypted data is then sent to the server using the HTTPS protocol. This process is crucial to ensure the confidentiality and security of the data.

[0084] Step 3:

[0085] The server receives and decrypts encrypted data sent from the terminal. Using the decrypted biometric information as input data, it utilizes a generative AI model to analyze emotional states and interest levels. This analysis checks whether brainwave patterns and voice tone meet specific criteria to evaluate the user's state.

[0086] Step 4:

[0087] Based on the analysis results, the server generates optimal action suggestions. Using a generation AI model, it generates prompts tailored to the progress of the sales negotiation. For example, by generating a prompt such as, "Suggest the best phrase when the customer shows strong interest in price," appropriate action suggestions are output.

[0088] Step 5:

[0089] The server sends the generated suggestions to the terminal, which then provides an audio notification to the user via earphones. This notification is in real time and supports the user's decision-making during the business negotiation. For example, if the customer's interest begins to wane, the server suggests when to introduce a new topic.

[0090] Step 6:

[0091] After a sales negotiation concludes, the server aggregates and analyzes the data collected during the negotiation. Using the analysis results, it identifies success factors and areas for improvement, and visualizes them on a dashboard. This information helps users improve their sales activities.

[0092] Step 7:

[0093] The server applies machine learning algorithms using accumulated biometric data and analysis results. This learning process contributes to optimizing proposals for future business negotiations. A feedback loop is established to improve the accuracy of proposals based on the input data.

[0094] (Application Example 1)

[0095] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0096] Traditional security systems make it difficult for on-site security guards to instantly grasp the surrounding situation and potential threats, highlighting the need for improved security quality. Furthermore, if biometric information can be used to analyze the emotional state of security guards and encourage appropriate actions, more efficient security operations will be possible.

[0097] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0098] In this invention, the server includes means for acquiring the user's biometric information in real time, means for analyzing the user's emotional state based on the biometric information, and means for detecting the surrounding environment and potential threats. This enables real-time support for security guards and early detection of potential threats.

[0099] "User biometric information" refers to data that indicates the user's physical state and is acquired and analyzed in real time, including brain waves, heart rate, and skin potential.

[0100] "Means for analyzing emotional states" refers to processes and devices for determining a user's emotions and psychological state based on the user's biometric information.

[0101] "Means of suggesting optimal actions" refers to a system that, based on analyzed emotional state and environmental information, presents the user with the most appropriate behavioral patterns and choices.

[0102] "Methods for aggregating information and analyzing success factors" refers to the process of compiling collected data and extracting and analyzing factors that lead to specific results.

[0103] "Means for visualizing analysis results and providing feedback" refers to functions and devices that display analyzed information in a format that is easy for users to understand and that provide advice and suggestions for improvement based on that information.

[0104] "Means for detecting the surrounding environment and potential threats" refers to technologies and devices that collect and analyze information about the external environment to detect potential threats at an early stage.

[0105] This invention is a system that uses a smart device worn by the user to collect and analyze biometric information in real time, and proposes optimal actions in combination with information about the surrounding environment. This system operates via a terminal such as smart glasses worn by security guards and communicates with a server via a network.

[0106] The server acquires biometric data such as brainwaves and heart rate transmitted from the smart glasses and analyzes the user's emotional state using advanced algorithms. The analysis results indicate the user's stress level and concentration level, and based on this, it suggests appropriate actions in real time. It also collects environmental information through the camera and voice input, which is used to detect potential threats. By capturing changes in the surroundings, it enables immediate alerts and warnings to security personnel.

[0107] The analysis process includes sentiment analysis software and threat detection applications, which are executed using the server's processing power. Feedback to the user is provided in real time via audio devices based on the analysis results. For example, if a security guard determines that the surroundings are suspicious during a night patrol, a warning such as "Alert: Please check your surroundings" will be issued.

[0108] Next, the analyzed data is stored on a server and used to analyze the factors for success. Machine learning algorithms learn from this data and refine their suggestions for future security activities. In this way, the usefulness and accuracy of the system improve over time.

[0109] As a concrete example, we can support security guards' actions by using generative AI models to generate effective prompts using a large amount of data obtained from both the user and the environment. For example, consider the following prompt: "You are an AI assisting security guards. Based on your analysis of the surrounding environment, please suggest alerts in real time. Explain how to detect potential threats from EEG data and visual information of the environment."

[0110] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0111] Step 1:

[0112] The device uses sensors to acquire brainwaves, heart rate, and ambient audio and video data in real time. This biometric and environmental data is encrypted and transmitted to a server. The input for this step is the biometric and environmental data from the sensors, and the output is the encrypted data sent to the server.

[0113] Step 2:

[0114] The server analyzes the received data and applies an emotion analysis algorithm to identify the user's emotional state. The analysis process performs statistical calculations based on the data to determine stress levels, concentration levels, and other parameters. The input for this step is encrypted biometric data, and the output is an evaluation of the user's emotional state.

[0115] Step 3:

[0116] The server then combines the analysis results and environmental data to execute an algorithm for detecting potential threats. Image processing techniques are used to detect abnormal activity in the environment, and audio analysis is used to identify unusual sounds in the surroundings. The input for this step is the analysis results and environmental data, and the output is information about the detected potential threats.

[0117] Step 4:

[0118] The server generates optimal action suggestions for the user based on emotional state and threat information. This process uses a generative AI model to generate appropriate instructions and warnings as text, which are then prepared for voice notification. The input for this step is emotional state and threat information, and the output is text data as action suggestions.

[0119] Step 5:

[0120] The terminal notifies the user via voice of action suggestions received from the server. The user receives the instructions through earphones or a device, prompting immediate action. The input for this step is action suggestion data from the server, and the output is the voice notification provided to the user.

[0121] Step 6:

[0122] The user modifies or adjusts their actions based on the notifications they receive and performs their security duties. The input for this step is the voice notification, and the output is the user's actual actions.

[0123] Step 7:

[0124] The server aggregates all data and results and feeds them into a learning algorithm to improve the accuracy of future suggestions. The input for this step is the past data and records of user behavior, and the output is a learning model of optimized action suggestions.

[0125] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0126] This invention relates to a sales negotiation support system that combines an emotion engine that analyzes emotions based on the user's biometric data. This system analyzes the user's emotional state in real time and uses the analysis results to propose optimal actions during sales negotiations. After the negotiation is completed, the data is compiled, success factors are analyzed, and feedback can be provided. A specific embodiment of the system is shown below.

[0127] Data acquisition and analysis using an emotion engine:

[0128] First, the device acquires the user's biometric data—specifically, brainwaves, voice, and facial expression data—in real time. This data is a crucial source for accurately understanding the user's emotional state. The acquired biometric data is securely encrypted and transmitted to the server.

[0129] Analysis of emotional states using an emotion engine:

[0130] The server is equipped with an emotion engine that analyzes the transmitted biometric data. The emotion engine identifies the user's emotional state—for example, excitement, reassurance, stress, etc.—and evaluates these states in real time. The analysis results serve as indicators to optimize the user's behavior during business negotiations.

[0131] Business negotiation support and proposals:

[0132] Based on the analysis results provided by the emotion engine, the server generates optimal action suggestions during a business negotiation. These suggestions include timing for topic changes, closing the deal, and communication methods tailored to the customer's emotions. The optimized suggestions are sent to the device and delivered to the user as an audio notification through the earphones.

[0133] Post-sales analysis and feedback:

[0134] Once a business negotiation is complete, the server aggregates all the data and analyzes the factors that contributed to the negotiation's success and areas for improvement. This feedback is provided to the user via a dashboard, serving as valuable material for self-improvement.

[0135] Optimization using machine learning:

[0136] Sales negotiation data is fed into a machine learning model, improving the accuracy of proposals in subsequent negotiations. This process continuously enhances the user's sales skills.

[0137] Specific example:

[0138] For example, if the terminal collects the user's brainwave data and it indicates a high stress level, the emotion engine analyzes it, and the server suggests a topic change using a relaxing tone for the user. Also, if the customer's facial expression data indicates excitement, the server notifies the user of the appropriate time to close the deal, thereby supporting the effective progress of the business negotiation. In this way, the present invention provides an advanced support system for leading business negotiations to success.

[0139] The following describes the processing flow.

[0140] Step 1:

[0141] Since the device acquires the user's biometric data, it uses an electroencephalogram (EEG) sensor to collect brainwave data during business negotiations. In addition, it collects audio and facial expression data by recording voice with a microphone and capturing the user's facial expressions with a camera.

[0142] Step 2:

[0143] The device encrypts the biometric data it collects and sends it to the server in real time. The latest encryption technology is used to ensure data security.

[0144] Step 3:

[0145] The server utilizes an emotion engine to analyze the received biometric data. This identifies the user's emotional state, clarifying, for example, how excited, stressed, or anxious the user is during a business negotiation.

[0146] Step 4:

[0147] The server analyzes the results and suggests the optimal course of action during the sales negotiation. This suggestion includes automatically generated talk scripts and metrics for determining the flow of the negotiation. This information changes depending on the situation and is updated according to the progress of the negotiation.

[0148] Step 5:

[0149] The server sends the proposal to the terminal, and the terminal provides real-time audio notifications to the user via earphones. For example, if the customer's facial expression indicates excitement, the user is given instructions to move on to closing the deal.

[0150] Step 6:

[0151] After a business negotiation concludes, the server analyzes all collected data to identify success factors and areas for improvement. The analysis results are presented to the user as a dashboard, providing material for self-improvement.

[0152] Step 7:

[0153] The server uses sales negotiation data to perform machine learning and updates the model to optimize proposals for future sales negotiations. This improves the overall accuracy of the system and the quality of user support.

[0154] Step 8:

[0155] Based on feedback from the dashboard, users can refine their plans for future business meetings and prepare more effectively. This feedback contributes to continuous skill improvement, ultimately increasing the success rate of business meetings.

[0156] (Example 2)

[0157] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0158] In business negotiations, understanding a user's emotional state in real time and proposing the optimal course of action based on that understanding is a challenging task. Furthermore, there is a need for methods to analyze the success or failure of a negotiation after its conclusion and use that knowledge to improve future negotiations. Conventional methods have been unable to efficiently acquire and analyze biosignals in real time, provide voice notifications, and analyze emotional information; therefore, these challenges need to be addressed.

[0159] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0160] In this invention, the server includes means for acquiring the user's biometric signals in real time, means for analyzing the user's emotions based on the biometric signals, means for generating optimal actions during business negotiations based on the analysis results, means for aggregating information after the business negotiations are completed and analyzing the factors for success, and means for encrypting biometric signals to ensure data privacy. This enables real-time support and efficient feedback tailored to the user's emotional state during business negotiations.

[0161] "Biosignals" refer to data such as brain waves, voice, and facial expressions that can be obtained from the user's body.

[0162] "Emotional analysis" is the process of evaluating a user's emotional state based on biosignals and identifying mental states such as excitement, relief, and stress.

[0163] "Behavior generation" refers to the procedure for proposing the optimal response and actions during a business negotiation based on the analyzed emotional state.

[0164] "Information aggregation" is the process of compiling data recorded after a business negotiation and analyzing the factors contributing to its success and areas for improvement.

[0165] "Ensuring data privacy" refers to the technology of protecting the security and confidentiality of personal information by encrypting biometric signals.

[0166] This system is characterized by its ability to acquire and analyze biometric signals in real time to support users' business negotiations. Specifically, the terminal uses an electroencephalogram (EEG) sensor, built-in microphone, and camera to acquire the user's biometric signals (brainwaves, voice, and facial expression data) in real time. This data is encrypted using AES encryption technology and securely transmitted to the server.

[0167] The server is equipped with an emotion analysis engine built using a variety of machine learning algorithms. The server evaluates the biosignals received by this analysis engine to identify the user's emotional state. Based on this analysis, the server generates the optimal actions to take during a business negotiation. In this action generation process, a generative AI model is used, and by inputting prompt messages, the server outputs specific proposal content.

[0168] For example, the server generates optimal actions during a business negotiation by inputting a prompt message such as, "The user's brainwave data indicates high stress. Please generate suggestions to promote relaxation." The suggestions generated by the server are provided to the user as voice notifications through the terminal. The user can then proceed with the negotiation while listening to these suggestions through earphones.

[0169] After a sales negotiation concludes, the server aggregates and analyzes all data to identify success factors and areas for improvement. This information is then fed back to the user via a dashboard. Furthermore, the negotiation data is fed into a machine learning model, which is used to improve the accuracy of future proposals. This process enables users to continuously improve their sales skills.

[0170] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0171] Step 1:

[0172] The device acquires the user's biometric signals—brainwaves, voice, and facial expression data—in real time. Specifically, it acquires brainwave data using an EEG sensor, collects voice data through a built-in microphone, and acquires facial expression data using a camera. This input data serves as fundamental information for analyzing the user's emotional state.

[0173] Step 2:

[0174] The biometric signals acquired by the device are encrypted using AES encryption technology to ensure security. The encrypted data is then transmitted to the server via the internet. This process protects the privacy of the data.

[0175] Step 3:

[0176] The server decrypts the received encrypted data and inputs it into the emotion analysis engine. The analysis engine uses machine learning algorithms to identify the user's emotional state from biosignals. For example, it analyzes brainwave patterns to assess stress levels and analyzes voice tone to determine emotions. The output of the analysis is the user's current emotional state.

[0177] Step 4:

[0178] The server generates optimal actions during a business negotiation based on the analysis results. This process utilizes a generation AI model, taking prompt text as input to output specific proposals. These proposals can include elements such as timing for topic changes and adjusting the tone of speech. The generated proposals serve as a guide for users to effectively respond during business negotiations.

[0179] Step 5:

[0180] Action suggestions generated by the server are sent to the terminal and delivered as voice notifications to the user via earphones. Users receive these notifications in real time, enabling them to take the most appropriate action during the sales negotiation. This feedback is crucial support that directly impacts the success or failure of the negotiation.

[0181] Step 6:

[0182] Once a business negotiation concludes, the server aggregates all data and uses machine learning models to analyze success factors and areas for improvement. This clarifies the negotiation results and provides data to improve the accuracy of proposals in future negotiations. Users receive feedback through a dashboard, supporting continuous skill development.

[0183] (Application Example 2)

[0184] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0185] In current business negotiations and customer service situations, it is difficult to grasp customers' emotions and interests in real time and respond appropriately. This leads to a decline in the quality of communication with customers and a lack of improvement in customer satisfaction. Furthermore, there is a lack of means to use the results of negotiations and customer service to improve future interactions.

[0186] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0187] In this invention, the server includes means for acquiring the user's biosignals in real time, means for analyzing the user's emotional state based on those biosignals, means for proposing the optimal action during the conversation based on the analysis results, means for aggregating data after the conversation and analyzing the factors for success, and means for presenting optimal product information based on the customer's state during the conversation with the user. This enables accurate understanding of the customer's emotions during business negotiations and customer service, the proposal and execution of the optimal response, and post-conversation feedback for improvement in the future.

[0188] "Biosignals" refer to information obtained from the user's body, such as brainwaves, voice, and facial expressions, and are used to understand the user's emotional state in real time.

[0189] "Emotional state" refers to the user's psychological condition, such as excitement, reassurance, or stress, and is identified by analyzing biosignals.

[0190] "Optimal behavior during dialogue" refers to effective communication methods and guidelines for actions suggested to users during dialogues and business negotiations, based on the customer's emotional state.

[0191] "Success factors" are the elements that contributed to the outcome of business negotiations or discussions, analyzed and identified, and used for improvement in future events.

[0192] The "emotion engine" is a core technological tool for a system that analyzes biosignals to identify and evaluate a user's emotional state in real time.

[0193] "Product information" refers to the specific details, features, and benefits of a product or service that are presented to the user during a conversation.

[0194] "Feedback" refers to evaluations and suggestions for improvement based on analysis results provided to the user after a business negotiation or conversation has concluded, and is intended to contribute to improving the user's performance in future interactions.

[0195] To implement this invention, it is first necessary to build a system that acquires biosignals and analyzes the user's emotional state based on that information. Specifically, a wearable device such as smart glasses is used to acquire biosignals such as the user's brainwaves, voice, and facial expressions in real time. The acquired data is securely encrypted and transmitted to a server.

[0196] The server is equipped with an emotion engine that analyzes transmitted biosignals to identify the user's emotional state—for example, excitement, relief, or stress. Based on the analysis results, the server generates indicators to suggest optimal actions and product information during the interaction, and notifies the user of these indicators via smart glasses or earphones.

[0197] The software used to implement this system will be Python, combined with libraries such as OpenCV, TENSORFLOW®, and PyTorch for emotion recognition. The hardware will utilize smart glasses such as Google® Glass® to efficiently collect and analyze a wide variety of biosignals.

[0198] For example, if a user detects an excited expression on a customer's face during a sales negotiation, the server analyzes the results and suggests a suitable time to close the deal, notifying the user. Conversely, if the customer shows signs of anxiety, the server suggests ways to explain the product's benefits in a reassuring tone.

[0199] By utilizing generative AI models, the accuracy of the proposed content gradually improves through machine learning.

[0200] Furthermore, the following example prompt messages can be used as a concrete example.

[0201] "Please tell me how to determine a customer's level of interest in a new product based on their facial expressions and voice data, and then notify staff in real time of the most appropriate product description."

[0202] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0203] Step 1:

[0204] The device acquires the user's biometric signals (brainwaves, voice, facial expressions) using sensors. This data is acquired in real time and encrypted for security. The input is biometric signal data, and the output is encrypted biometric signal data.

[0205] Step 2:

[0206] The terminal sends encrypted biometric data to the server. Here, data is transferred over a network, with encrypted biometric data as input and secure data transfer to the server as output.

[0207] Step 3:

[0208] The server decrypts the received biosignal data, and the emotion engine analyzes the emotional state. The input is the decrypted biosignal data, and the output is the analyzed emotional state (excitement, relief, stress). In this step, the data analysis algorithm is in operation.

[0209] Step 4:

[0210] The server uses the analysis results to generate optimal actions or product information for the interaction. This generation process utilizes emotional states and leverages a generative AI model. The input is the analyzed emotional state, and the output is action suggestions or product information.

[0211] Step 5:

[0212] The server sends suggested actions and product information to the terminal. The input is the suggested information on the server, and the output is the suggested information received by the terminal. This step involves processing related to data transmission and display.

[0213] Step 6:

[0214] The device notifies the user of the suggested information it has received. The user receives the information via smart glasses or earphones. The input is the suggested information received by the device, and the output is the information notification to the user.

[0215] Step 7:

[0216] After a business meeting or customer interaction concludes, the collected data is gathered again on the server, and the success factors are analyzed. In this step, the server processes the data using statistical methods, with the input being the business meeting data and the output being feedback information.

[0217] Step 8:

[0218] The server supplies feedback information to a machine learning model to improve the accuracy of future suggestions. The input consists of user sales data and feedback information, and the output is an optimized suggestion model. This process involves generative AI models and data calculations.

[0219] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0220] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0221] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0222] [Second Embodiment]

[0223] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0224] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0225] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0226] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0227] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0228] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0229] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0230] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0231] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0232] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0233] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0234] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0235] This invention is a system that optimizes user behavior during business negotiations, primarily characterized by real-time acquisition and analysis of biometric data, and action suggestions based on the results. This system operates via a terminal worn by the user and is centrally controlled by a server.

[0236] Data acquisition and sentiment analysis:

[0237] First, the device collects biometric data using sensors such as the user's brainwave sensor, microphone, and camera. This data is used to capture in real time the user's state of mind, such as their level of concentration, stress, and emotions, during business negotiations. The collected data is immediately encrypted and sent to the server.

[0238] Analysis and proposals for supporting business negotiations:

[0239] The server uses advanced algorithms based on biometric data to analyze emotional states. This identifies how interested the customer is in the deal, whether they are experiencing stress, and so on. Based on the analysis, it determines the optimal course of action and generates a proposal. This proposal includes an appropriate talk script, negotiation strategy, and closing timing.

[0240] Real-time notifications and feedback:

[0241] The proposal is sent from the server to the terminal and notified to the user in real time via audio through earphones. For example, the moment a customer shows interest in price negotiation, the optimal negotiation phrase is notified, allowing the user to change the topic at the appropriate time. After the negotiation is concluded, the server uses the aggregated data to analyze the factors contributing to the success of the negotiation and visualizes the results on a dashboard.

[0242] Machine learning for learning and improvement:

[0243] Sales negotiation data is continuously analyzed by machine learning algorithms. Based on this analysis, the server optimizes the support provided for subsequent sales negotiations, thereby improving the accuracy of proposals. This allows users to continuously improve themselves over time and enhance the quality of their sales activities.

[0244] Specific example:

[0245] For example, if a terminal acquires customer facial expression data during a business negotiation, and the server analyzes this data to determine that the customer is bored with the negotiation, it will notify the user in real time to introduce a new topic to change the flow of the negotiation. Also, if the customer shows interest in price, the server will provide voice guidance on the appropriate timing for closing the deal, helping to increase the closing rate. In this way, the present invention has a structure that optimizes sales activities based on data and supports the success of business negotiations.

[0246] The following describes the processing flow.

[0247] Step 1:

[0248] The device uses the user's brainwave sensor, microphone, and camera to acquire biometric data in real time during business negotiations. This includes measuring concentration levels using brainwaves, analyzing emotions from voice, and recognizing facial expressions using the camera.

[0249] Step 2:

[0250] The device encrypts the collected biometric data and transmits it to the server in real time. Buffering technology may be used during this process to ensure smooth data transmission.

[0251] Step 3:

[0252] The server analyzes the biometric data it receives. Here, a machine learning model is used to estimate the user's emotional state and score their level of interest, stress level, and attention focus.

[0253] Step 4:

[0254] The server generates suggestions for the user based on the results of sentiment analysis. This includes refining the talk script while considering the progress of the sales negotiation, and identifying the timing of negotiations and closing points.

[0255] Step 5:

[0256] The server sends the generated proposal to the terminal, which then provides real-time audio notifications to the user via earphones. The user is then provided with specific instructions to improve the flow of the sales negotiation.

[0257] Step 6:

[0258] Once a business negotiation is complete, the server analyzes the collected data to identify the factors that contributed to the negotiation's success and areas for improvement. These analysis results are then visualized on a dashboard.

[0259] Step 7:

[0260] The server uses machine learning based on past sales data to improve the accuracy of proposals for future sales meetings. This establishes a continuous improvement cycle.

[0261] Step 8:

[0262] Users develop self-improvement plans based on feedback from the dashboard and prepare for new business opportunities. This feedback helps improve users' sales skills.

[0263] (Example 1)

[0264] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0265] To ensure the success of business negotiations, it is crucial to accurately understand the customer's emotional state in real time and make actionable suggestions based on that understanding. However, traditional methods have made it difficult to grasp the situation in real time during negotiations and make appropriate suggestions, thus failing to maximize the outcome of negotiations.

[0266] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0267] In this invention, the server includes means for encrypting biometric information and transmitting it to a central device, means for analyzing the biometric information and evaluating the emotional state at the central device, and means for utilizing a generative model to propose the optimal action based on the evaluation results. This makes it possible to accurately grasp the emotional state of a customer in real time during a business negotiation and improve the outcome of the negotiation by proposing the optimal action.

[0268] "Biometric information" refers to data that indicates the physical state of an organism, and includes information such as brain waves, heart rate, facial expressions, and voice tone.

[0269] A "device" refers to an instrument or combination designed to perform a specific function, and in this context, it refers to a sensor device.

[0270] A "central system" refers to a core computer system used for data aggregation, processing, and analysis.

[0271] "Encryption" is a security technology that transforms information according to specific rules to prevent unauthorized access.

[0272] "Action suggestions" are guidelines and advice that instruct users on what they believe to be the most optimal actions based on the results of biometric data analysis.

[0273] A "generative model" is an artificial intelligence algorithm used to generate appropriate responses or outputs for a given input.

[0274] "Machine learning" is a data analysis technique aimed at enabling computers to automatically improve their performance through experience.

[0275] "Voice guidance" refers to audio messages used to convey information or instructions to users aurally.

[0276] This invention provides a system that supports business negotiations in real time through a terminal worn by the user. The terminal functions as a device that collects biometric information and provides feedback to the user as needed. Specifically, it is equipped with an electroencephalogram (EEG) sensor, a microphone, and a camera, which enable the acquisition of the user's brainwaves, voice, and facial expression data.

[0277] The device protects the collected biometric information using AES encryption technology and transmits it to the server via a secure HTTP protocol. After receiving this data, the server analyzes it using a generative AI model to evaluate the user's emotional state and level of interest. Based on the evaluation, prompts are used to generate optimal action suggestions for the progress of the negotiation. For example, a prompt such as, "Analyze the customer's level of interest in the price offered in real time and suggest the best way to proceed with the negotiation," might be used.

[0278] The proposed solutions are provided to the user in real time as voice guidance, enabling timely responses during sales negotiations. For example, if a customer shows interest in a particular product, the suggested closing phrase is notified to the user through their earphones. After the negotiation ends, the server analyzes the results, visualizes them, and provides feedback to the user. This allows the user to continuously make data-driven improvements in future negotiations.

[0279] The overall effect of this system is to improve user performance during sales negotiations and contribute to increasing the success rate of those negotiations.

[0280] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0281] Step 1:

[0282] The user wears the terminal and starts the negotiation. The terminal collects the user's biometric information in real time using an electroencephalogram sensor, a microphone, and a camera. This information includes electroencephalograms, voice tones, and facial expression data. The collected biometric information is used as input, and the data is temporarily stored in the terminal.

[0283] Step 2:

[0284] The terminal encrypts the collected biometric information using AES encryption technology. The encrypted data is used as output and is sent to the server using the HTTPS protocol. This process is important for ensuring the confidentiality and security of the data.

[0285] Step 3:

[0286] The server receives and decrypts the encrypted data sent from the terminal. Using the biometric information decrypted as input data, the server analyzes the emotional state and interest level by leveraging the generative AI model. In this analysis, it is checked whether the electroencephalogram pattern and voice tone meet specific criteria to evaluate the user's state.

[0287] Step 4:

[0288] Based on the analysis results, the server generates an optimal action proposal. Using the generative AI model, it generates prompt texts according to the progress of the negotiation. For example, by generating a prompt text such as "Please propose the most appropriate phrase when the customer shows strong interest in the price.", an appropriate action proposal is output.

[0289] Step 5:

[0290] The server sends the generated proposal to the terminal, and the terminal makes a voice notification to the user through the earphone. This notification is made in real time to support the user's decision-making during the negotiation. For example, when the customer's interest starts to fade, it proposes the timing to present a new topic.

[0291] Step 6:

[0292] After a sales negotiation concludes, the server aggregates and analyzes the data collected during the negotiation. Using the analysis results, it identifies success factors and areas for improvement, and visualizes them on a dashboard. This information helps users improve their sales activities.

[0293] Step 7:

[0294] The server applies machine learning algorithms using accumulated biometric data and analysis results. This learning process contributes to optimizing proposals for future business negotiations. A feedback loop is established to improve the accuracy of proposals based on the input data.

[0295] (Application Example 1)

[0296] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0297] Traditional security systems make it difficult for on-site security guards to instantly grasp the surrounding situation and potential threats, highlighting the need for improved security quality. Furthermore, if biometric information can be used to analyze the emotional state of security guards and encourage appropriate actions, more efficient security operations will be possible.

[0298] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0299] In this invention, the server includes means for acquiring the user's biometric information in real time, means for analyzing the user's emotional state based on the biometric information, and means for detecting the surrounding environment and potential threats. This enables real-time support for security guards and early detection of potential threats.

[0300] "User biometric information" refers to data that indicates the user's physical state and is acquired and analyzed in real time, including brain waves, heart rate, and skin potential.

[0301] The "means for analyzing emotional states" refers to the process and device for discriminating the user's emotions and psychological states based on the user's biological information.

[0302] The "means for proposing optimal actions" refers to the mechanism for showing the most appropriate action patterns and options to the user based on the analyzed emotional states and environmental information.

[0303] The "means for aggregating information and analyzing success factors" refers to the process of summarizing the collected data and extracting / analyzing the factors for leading to specific results.

[0304] The "means for visualizing analysis results and providing feedback" refers to the function and device for displaying the analyzed information in a user-friendly format and presenting advice and improvement plans based on it.

[0305] The "means for detecting the surrounding environment and potential threats" refers to the technology and device for collecting / analyzing information on the external environment and detecting potential threats at an early stage.

[0306] The present invention is a system that uses a smart device worn by a user to collect / analyze biological information in real time and propose optimal actions in combination with the surrounding environmental information. This system operates via a terminal such as smart glasses worn by a security guard and cooperates with a server through a network.

[0307] The server acquires biological data such as brain waves and heart rates transmitted from the smart glasses and analyzes the user's emotional state using advanced algorithms. The analysis result of the emotional state indicates the stress level and concentration that the user experiences, and based on this, appropriate actions are proposed in real time. Also, environmental information is collected through a camera and voice input and is also used for detecting potential threats. By grasping the changes in the surroundings, immediate alerts and attention calls to the security guard are made possible.

[0308] The analysis process includes sentiment analysis software and threat detection applications, which are executed using the server's processing power. Feedback to the user is provided in real time via audio devices based on the analysis results. For example, if a security guard determines that the surroundings are suspicious during a night patrol, a warning such as "Alert: Please check your surroundings" will be issued.

[0309] Next, the analyzed data is stored on a server and used to analyze the factors for success. Machine learning algorithms learn from this data and refine their suggestions for future security activities. In this way, the usefulness and accuracy of the system improve over time.

[0310] As a concrete example, we can support security guards' actions by using generative AI models to generate effective prompts using a large amount of data obtained from both the user and the environment. For example, consider the following prompt: "You are an AI assisting security guards. Based on your analysis of the surrounding environment, please suggest alerts in real time. Explain how to detect potential threats from EEG data and visual information of the environment."

[0311] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0312] Step 1:

[0313] The device uses sensors to acquire brainwaves, heart rate, and ambient audio and video data in real time. This biometric and environmental data is encrypted and transmitted to a server. The input for this step is the biometric and environmental data from the sensors, and the output is the encrypted data sent to the server.

[0314] Step 2:

[0315] The server analyzes the received data and applies an emotion analysis algorithm to identify the user's emotional state. The analysis process performs statistical calculations based on the data to determine stress levels, concentration levels, and other parameters. The input for this step is encrypted biometric data, and the output is an evaluation of the user's emotional state.

[0316] Step 3:

[0317] The server then combines the analysis results and environmental data to execute an algorithm for detecting potential threats. Image processing techniques are used to detect abnormal activity in the environment, and audio analysis is used to identify unusual sounds in the surroundings. The input for this step is the analysis results and environmental data, and the output is information about the detected potential threats.

[0318] Step 4:

[0319] The server generates optimal action suggestions for the user based on emotional state and threat information. This process uses a generative AI model to generate appropriate instructions and warnings as text, which are then prepared for voice notification. The input for this step is emotional state and threat information, and the output is text data as action suggestions.

[0320] Step 5:

[0321] The terminal notifies the user via voice of action suggestions received from the server. The user receives the instructions through earphones or a device, prompting immediate action. The input for this step is action suggestion data from the server, and the output is the voice notification provided to the user.

[0322] Step 6:

[0323] The user modifies or adjusts their actions based on the notifications they receive and performs their security duties. The input for this step is the voice notification, and the output is the user's actual actions.

[0324] Step 7:

[0325] The server aggregates all data and results and feeds them into a learning algorithm to improve the accuracy of future suggestions. The input for this step is the past data and records of user behavior, and the output is a learning model of optimized action suggestions.

[0326] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0327] This invention relates to a sales negotiation support system that combines an emotion engine that analyzes emotions based on the user's biometric data. This system analyzes the user's emotional state in real time and uses the analysis results to propose optimal actions during sales negotiations. After the negotiation is completed, the data is compiled, success factors are analyzed, and feedback can be provided. A specific embodiment of the system is shown below.

[0328] Data acquisition and analysis using an emotion engine:

[0329] First, the device acquires the user's biometric data—specifically, brainwaves, voice, and facial expression data—in real time. This data is a crucial source for accurately understanding the user's emotional state. The acquired biometric data is securely encrypted and transmitted to the server.

[0330] Analysis of emotional states using an emotion engine:

[0331] The server is equipped with an emotion engine that analyzes the transmitted biometric data. The emotion engine identifies the user's emotional state—for example, excitement, reassurance, stress, etc.—and evaluates these states in real time. The analysis results serve as indicators to optimize the user's behavior during business negotiations.

[0332] Business negotiation support and proposals:

[0333] Based on the analysis results provided by the emotion engine, the server generates optimal action suggestions during a business negotiation. These suggestions include timing for topic changes, closing the deal, and communication methods tailored to the customer's emotions. The optimized suggestions are sent to the device and delivered to the user as an audio notification through the earphones.

[0334] Post-sales analysis and feedback:

[0335] Once a business negotiation is complete, the server aggregates all the data and analyzes the factors that contributed to the negotiation's success and areas for improvement. This feedback is provided to the user via a dashboard, serving as valuable material for self-improvement.

[0336] Optimization using machine learning:

[0337] Sales negotiation data is fed into a machine learning model, improving the accuracy of proposals in subsequent negotiations. This process continuously enhances the user's sales skills.

[0338] Specific example:

[0339] For example, if the terminal collects the user's brainwave data and it indicates a high stress level, the emotion engine analyzes it, and the server suggests a topic change using a relaxing tone for the user. Also, if the customer's facial expression data indicates excitement, the server notifies the user of the appropriate time to close the deal, thereby supporting the effective progress of the business negotiation. In this way, the present invention provides an advanced support system for leading business negotiations to success.

[0340] The following describes the processing flow.

[0341] Step 1:

[0342] Since the device acquires the user's biometric data, it uses an electroencephalogram (EEG) sensor to collect brainwave data during business negotiations. In addition, it collects audio and facial expression data by recording voice with a microphone and capturing the user's facial expressions with a camera.

[0343] Step 2:

[0344] The device encrypts the biometric data it collects and sends it to the server in real time. The latest encryption technology is used to ensure data security.

[0345] Step 3:

[0346] The server utilizes an emotion engine to analyze the received biometric data. This identifies the user's emotional state, clarifying, for example, how excited, stressed, or anxious the user is during a business negotiation.

[0347] Step 4:

[0348] The server analyzes the results and suggests the optimal course of action during the sales negotiation. This suggestion includes automatically generated talk scripts and metrics for determining the flow of the negotiation. This information changes depending on the situation and is updated according to the progress of the negotiation.

[0349] Step 5:

[0350] The server sends the proposal to the terminal, and the terminal provides real-time audio notifications to the user via earphones. For example, if the customer's facial expression indicates excitement, the user is given instructions to move on to closing the deal.

[0351] Step 6:

[0352] After a business negotiation concludes, the server analyzes all collected data to identify success factors and areas for improvement. The analysis results are presented to the user as a dashboard, providing material for self-improvement.

[0353] Step 7:

[0354] The server uses sales negotiation data to perform machine learning and updates the model to optimize proposals for future sales negotiations. This improves the overall accuracy of the system and the quality of user support.

[0355] Step 8:

[0356] Based on feedback from the dashboard, users can refine their plans for future business meetings and prepare more effectively. This feedback contributes to continuous skill improvement, ultimately increasing the success rate of business meetings.

[0357] (Example 2)

[0358] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0359] In business negotiations, understanding a user's emotional state in real time and proposing the optimal course of action based on that understanding is a challenging task. Furthermore, there is a need for methods to analyze the success or failure of a negotiation after its conclusion and use that knowledge to improve future negotiations. Conventional methods have been unable to efficiently acquire and analyze biosignals in real time, provide voice notifications, and analyze emotional information; therefore, these challenges need to be addressed.

[0360] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0361] In this invention, the server includes means for acquiring the user's biometric signals in real time, means for analyzing the user's emotions based on the biometric signals, means for generating optimal actions during business negotiations based on the analysis results, means for aggregating information after the business negotiations are completed and analyzing the factors for success, and means for encrypting biometric signals to ensure data privacy. This enables real-time support and efficient feedback tailored to the user's emotional state during business negotiations.

[0362] "Biosignals" refer to data such as brain waves, voice, and facial expressions that can be obtained from the user's body.

[0363] "Emotional analysis" is the process of evaluating a user's emotional state based on biosignals and identifying mental states such as excitement, relief, and stress.

[0364] "Behavior generation" refers to the procedure for proposing the optimal response and actions during a business negotiation based on the analyzed emotional state.

[0365] "Information aggregation" is the process of compiling data recorded after a business negotiation and analyzing the factors contributing to its success and areas for improvement.

[0366] "Ensuring data privacy" refers to the technology of protecting the security and confidentiality of personal information by encrypting biometric signals.

[0367] This system is characterized by its ability to acquire and analyze biometric signals in real time to support users' business negotiations. Specifically, the terminal uses an electroencephalogram (EEG) sensor, built-in microphone, and camera to acquire the user's biometric signals (brainwaves, voice, and facial expression data) in real time. This data is encrypted using AES encryption technology and securely transmitted to the server.

[0368] The server is equipped with an emotion analysis engine built using a variety of machine learning algorithms. The server evaluates the biosignals received by this analysis engine to identify the user's emotional state. Based on this analysis, the server generates the optimal actions to take during a business negotiation. In this action generation process, a generative AI model is used, and by inputting prompt messages, the server outputs specific proposal content.

[0369] For example, the server generates optimal actions during a business negotiation by inputting a prompt message such as, "The user's brainwave data indicates high stress. Please generate suggestions to promote relaxation." The suggestions generated by the server are provided to the user as voice notifications through the terminal. The user can then proceed with the negotiation while listening to these suggestions through earphones.

[0370] After a sales negotiation concludes, the server aggregates and analyzes all data to identify success factors and areas for improvement. This information is then fed back to the user via a dashboard. Furthermore, the negotiation data is fed into a machine learning model, which is used to improve the accuracy of future proposals. This process enables users to continuously improve their sales skills.

[0371] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0372] Step 1:

[0373] The device acquires the user's biometric signals—brainwaves, voice, and facial expression data—in real time. Specifically, it acquires brainwave data using an EEG sensor, collects voice data through a built-in microphone, and acquires facial expression data using a camera. This input data serves as fundamental information for analyzing the user's emotional state.

[0374] Step 2:

[0375] The biometric signals acquired by the device are encrypted using AES encryption technology to ensure security. The encrypted data is then transmitted to the server via the internet. This process protects the privacy of the data.

[0376] Step 3:

[0377] The server decrypts the received encrypted data and inputs it into the emotion analysis engine. The analysis engine uses machine learning algorithms to identify the user's emotional state from biosignals. For example, it analyzes brainwave patterns to assess stress levels and analyzes voice tone to determine emotions. The output of the analysis is the user's current emotional state.

[0378] Step 4:

[0379] The server generates optimal actions during a business negotiation based on the analysis results. This process utilizes a generation AI model, taking prompt text as input to output specific proposals. These proposals can include elements such as timing for topic changes and adjusting the tone of speech. The generated proposals serve as a guide for users to effectively respond during business negotiations.

[0380] Step 5:

[0381] Action suggestions generated by the server are sent to the terminal and delivered as voice notifications to the user via earphones. Users receive these notifications in real time, enabling them to take the most appropriate action during the sales negotiation. This feedback is crucial support that directly impacts the success or failure of the negotiation.

[0382] Step 6:

[0383] Once a business negotiation concludes, the server aggregates all data and uses machine learning models to analyze success factors and areas for improvement. This clarifies the negotiation results and provides data to improve the accuracy of proposals in future negotiations. Users receive feedback through a dashboard, supporting continuous skill development.

[0384] (Application Example 2)

[0385] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0386] In current business negotiations and customer service situations, it is difficult to grasp customers' emotions and interests in real time and respond appropriately. This leads to a decline in the quality of communication with customers and a lack of improvement in customer satisfaction. Furthermore, there is a lack of means to use the results of negotiations and customer service to improve future interactions.

[0387] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0388] In this invention, the server includes means for acquiring the user's biosignals in real time, means for analyzing the user's emotional state based on those biosignals, means for proposing the optimal action during the conversation based on the analysis results, means for aggregating data after the conversation and analyzing the factors for success, and means for presenting optimal product information based on the customer's state during the conversation with the user. This enables accurate understanding of the customer's emotions during business negotiations and customer service, the proposal and execution of the optimal response, and post-conversation feedback for improvement in the future.

[0389] "Biosignals" refer to information obtained from the user's body, such as brainwaves, voice, and facial expressions, and are used to understand the user's emotional state in real time.

[0390] "Emotional state" refers to the user's psychological condition, such as excitement, reassurance, or stress, and is identified by analyzing biosignals.

[0391] "Optimal behavior during dialogue" refers to effective communication methods and guidelines for actions suggested to users during dialogues and business negotiations, based on the customer's emotional state.

[0392] "Success factors" are the elements that contributed to the outcome of business negotiations or discussions, analyzed and identified, and used for improvement in future events.

[0393] The "emotion engine" is a core technological tool for a system that analyzes biosignals to identify and evaluate a user's emotional state in real time.

[0394] "Product information" refers to the specific details, features, and benefits of a product or service that are presented to the user during a conversation.

[0395] "Feedback" refers to evaluations and suggestions for improvement based on analysis results provided to the user after a business negotiation or conversation has concluded, and is intended to contribute to improving the user's performance in future interactions.

[0396] To implement this invention, it is first necessary to build a system that acquires biosignals and analyzes the user's emotional state based on that information. Specifically, a wearable device such as smart glasses is used to acquire biosignals such as the user's brainwaves, voice, and facial expressions in real time. The acquired data is securely encrypted and transmitted to a server.

[0397] The server is equipped with an emotion engine that analyzes transmitted biosignals to identify the user's emotional state—for example, excitement, relief, or stress. Based on the analysis results, the server generates indicators to suggest optimal actions and product information during the interaction, and notifies the user of these indicators via smart glasses or earphones.

[0398] The software used to implement this system will be Python, combined with libraries such as OpenCV, TensorFlow, and PyTorch for emotion recognition. The hardware will utilize smart glasses such as Google Glass to efficiently collect and analyze a wide variety of biosignals.

[0399] For example, if a user detects an excited expression on a customer's face during a sales negotiation, the server analyzes the results and suggests a suitable time to close the deal, notifying the user. Conversely, if the customer shows signs of anxiety, the server suggests ways to explain the product's benefits in a reassuring tone.

[0400] By utilizing generative AI models, the accuracy of the proposed content gradually improves through machine learning.

[0401] Furthermore, the following example prompt messages can be used as a concrete example.

[0402] "Please tell me how to determine a customer's level of interest in a new product based on their facial expressions and voice data, and then notify staff in real time of the most appropriate product description."

[0403] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0404] Step 1:

[0405] The device acquires the user's biometric signals (brainwaves, voice, facial expressions) using sensors. This data is acquired in real time and encrypted for security. The input is biometric signal data, and the output is encrypted biometric signal data.

[0406] Step 2:

[0407] The terminal sends encrypted biometric data to the server. Here, data is transferred over a network, with encrypted biometric data as input and secure data transfer to the server as output.

[0408] Step 3:

[0409] The server decrypts the received biosignal data, and the emotion engine analyzes the emotional state. The input is the decrypted biosignal data, and the output is the analyzed emotional state (excitement, relief, stress). In this step, the data analysis algorithm is in operation.

[0410] Step 4:

[0411] The server uses the analysis results to generate optimal actions or product information for the interaction. This generation process utilizes emotional states and leverages a generative AI model. The input is the analyzed emotional state, and the output is action suggestions or product information.

[0412] Step 5:

[0413] The server sends suggested actions and product information to the terminal. The input is the suggested information on the server, and the output is the suggested information received by the terminal. This step involves processing related to data transmission and display.

[0414] Step 6:

[0415] The device notifies the user of the suggested information it has received. The user receives the information via smart glasses or earphones. The input is the suggested information received by the device, and the output is the information notification to the user.

[0416] Step 7:

[0417] After a business meeting or customer interaction concludes, the collected data is gathered again on the server, and the success factors are analyzed. In this step, the server processes the data using statistical methods, with the input being the business meeting data and the output being feedback information.

[0418] Step 8:

[0419] The server supplies feedback information to a machine learning model to improve the accuracy of future suggestions. The input consists of user sales data and feedback information, and the output is an optimized suggestion model. This process involves generative AI models and data calculations.

[0420] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0421] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0422] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0423] [Third Embodiment]

[0424] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0425] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0426] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0427] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0428] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0429] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0430] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0431] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0432] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0433] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0434] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0435] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0436] This invention is a system that optimizes user behavior during business negotiations, primarily characterized by real-time acquisition and analysis of biometric data, and action suggestions based on the results. This system operates via a terminal worn by the user and is centrally controlled by a server.

[0437] Data acquisition and sentiment analysis:

[0438] First, the device collects biometric data using sensors such as the user's brainwave sensor, microphone, and camera. This data is used to capture in real time the user's state of mind, such as their level of concentration, stress, and emotions, during business negotiations. The collected data is immediately encrypted and sent to the server.

[0439] Analysis and proposals for supporting business negotiations:

[0440] The server uses advanced algorithms based on biometric data to analyze emotional states. This identifies how interested the customer is in the deal, whether they are experiencing stress, and so on. Based on the analysis, it determines the optimal course of action and generates a proposal. This proposal includes an appropriate talk script, negotiation strategy, and closing timing.

[0441] Real-time notifications and feedback:

[0442] The proposal is sent from the server to the terminal and notified to the user in real time via audio through earphones. For example, the moment a customer shows interest in price negotiation, the optimal negotiation phrase is notified, allowing the user to change the topic at the appropriate time. After the negotiation is concluded, the server uses the aggregated data to analyze the factors contributing to the success of the negotiation and visualizes the results on a dashboard.

[0443] Machine learning for learning and improvement:

[0444] Sales negotiation data is continuously analyzed by machine learning algorithms. Based on this analysis, the server optimizes the support provided for subsequent sales negotiations, thereby improving the accuracy of proposals. This allows users to continuously improve themselves over time and enhance the quality of their sales activities.

[0445] Specific example:

[0446] For example, if a terminal acquires customer facial expression data during a business negotiation, and the server analyzes this data to determine that the customer is bored with the negotiation, it will notify the user in real time to introduce a new topic to change the flow of the negotiation. Also, if the customer shows interest in price, the server will provide voice guidance on the appropriate timing for closing the deal, helping to increase the closing rate. In this way, the present invention has a structure that optimizes sales activities based on data and supports the success of business negotiations.

[0447] The following describes the processing flow.

[0448] Step 1:

[0449] The device uses the user's brainwave sensor, microphone, and camera to acquire biometric data in real time during business negotiations. This includes measuring concentration levels using brainwaves, analyzing emotions from voice, and recognizing facial expressions using the camera.

[0450] Step 2:

[0451] The device encrypts the collected biometric data and transmits it to the server in real time. Buffering technology may be used during this process to ensure smooth data transmission.

[0452] Step 3:

[0453] The server analyzes the biometric data it receives. Here, a machine learning model is used to estimate the user's emotional state and score their level of interest, stress level, and attention focus.

[0454] Step 4:

[0455] The server generates suggestions for the user based on the results of sentiment analysis. This includes refining the talk script while considering the progress of the sales negotiation, and identifying the timing of negotiations and closing points.

[0456] Step 5:

[0457] The server sends the generated proposal to the terminal, which then provides real-time audio notifications to the user via earphones. The user is then provided with specific instructions to improve the flow of the sales negotiation.

[0458] Step 6:

[0459] Once a business negotiation is complete, the server analyzes the collected data to identify the factors that contributed to the negotiation's success and areas for improvement. These analysis results are then visualized on a dashboard.

[0460] Step 7:

[0461] The server uses machine learning based on past sales data to improve the accuracy of proposals for future sales meetings. This establishes a continuous improvement cycle.

[0462] Step 8:

[0463] Users develop self-improvement plans based on feedback from the dashboard and prepare for new business opportunities. This feedback helps improve users' sales skills.

[0464] (Example 1)

[0465] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0466] To ensure the success of business negotiations, it is crucial to accurately understand the customer's emotional state in real time and make actionable suggestions based on that understanding. However, traditional methods have made it difficult to grasp the situation in real time during negotiations and make appropriate suggestions, thus failing to maximize the outcome of negotiations.

[0467] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0468] In this invention, the server includes means for encrypting biometric information and transmitting it to a central device, means for analyzing the biometric information and evaluating the emotional state at the central device, and means for utilizing a generative model to propose the optimal action based on the evaluation results. This makes it possible to accurately grasp the emotional state of a customer in real time during a business negotiation and improve the outcome of the negotiation by proposing the optimal action.

[0469] "Biometric information" refers to data that indicates the physical state of an organism, and includes information such as brain waves, heart rate, facial expressions, and voice tone.

[0470] A "device" refers to an instrument or combination designed to perform a specific function, and in this context, it refers to a sensor device.

[0471] A "central system" refers to a core computer system used for data aggregation, processing, and analysis.

[0472] "Encryption" is a security technology that transforms information according to specific rules to prevent unauthorized access.

[0473] "Action suggestions" are guidelines and advice that instruct users on what they believe to be the most optimal actions based on the results of biometric data analysis.

[0474] A "generative model" is an artificial intelligence algorithm used to generate appropriate responses or outputs for a given input.

[0475] "Machine learning" is a data analysis technique aimed at enabling computers to automatically improve their performance through experience.

[0476] "Voice guidance" refers to audio messages used to convey information or instructions to users aurally.

[0477] This invention provides a system that supports business negotiations in real time through a terminal worn by the user. The terminal functions as a device that collects biometric information and provides feedback to the user as needed. Specifically, it is equipped with an electroencephalogram (EEG) sensor, a microphone, and a camera, which enable the acquisition of the user's brainwaves, voice, and facial expression data.

[0478] The device protects the collected biometric information using AES encryption technology and transmits it to the server via a secure HTTP protocol. After receiving this data, the server analyzes it using a generative AI model to evaluate the user's emotional state and level of interest. Based on the evaluation, prompts are used to generate optimal action suggestions for the progress of the negotiation. For example, a prompt such as, "Analyze the customer's level of interest in the price offered in real time and suggest the best way to proceed with the negotiation," might be used.

[0479] The proposed solutions are provided to the user in real time as voice guidance, enabling timely responses during sales negotiations. For example, if a customer shows interest in a particular product, the suggested closing phrase is notified to the user through their earphones. After the negotiation ends, the server analyzes the results, visualizes them, and provides feedback to the user. This allows the user to continuously make data-driven improvements in future negotiations.

[0480] The overall effect of this system is to improve user performance during sales negotiations and contribute to increasing the success rate of those negotiations.

[0481] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0482] Step 1:

[0483] The user puts on the device and begins the business negotiation. The device uses an electroencephalogram (EEG) sensor, microphone, and camera to collect the user's biometric information in real time. This information includes brainwaves, voice tone, and facial expression data. The collected biometric information is used as input, and the data is temporarily stored on the device.

[0484] Step 2:

[0485] The device encrypts the collected biometric information using AES encryption technology. The encrypted data is then sent to the server using the HTTPS protocol. This process is crucial to ensure the confidentiality and security of the data.

[0486] Step 3:

[0487] The server receives and decrypts encrypted data sent from the terminal. Using the decrypted biometric information as input data, it utilizes a generative AI model to analyze emotional states and interest levels. This analysis checks whether brainwave patterns and voice tone meet specific criteria to evaluate the user's state.

[0488] Step 4:

[0489] Based on the analysis results, the server generates optimal action suggestions. Using a generation AI model, it generates prompts tailored to the progress of the sales negotiation. For example, by generating a prompt such as, "Suggest the best phrase when the customer shows strong interest in price," appropriate action suggestions are output.

[0490] Step 5:

[0491] The server sends the generated suggestions to the terminal, which then provides an audio notification to the user via earphones. This notification is in real time and supports the user's decision-making during the business negotiation. For example, if the customer's interest begins to wane, the server suggests when to introduce a new topic.

[0492] Step 6:

[0493] After a sales negotiation concludes, the server aggregates and analyzes the data collected during the negotiation. Using the analysis results, it identifies success factors and areas for improvement, and visualizes them on a dashboard. This information helps users improve their sales activities.

[0494] Step 7:

[0495] The server applies machine learning algorithms using accumulated biometric data and analysis results. This learning process contributes to optimizing proposals for future business negotiations. A feedback loop is established to improve the accuracy of proposals based on the input data.

[0496] (Application Example 1)

[0497] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0498] Traditional security systems make it difficult for on-site security guards to instantly grasp the surrounding situation and potential threats, highlighting the need for improved security quality. Furthermore, if biometric information can be used to analyze the emotional state of security guards and encourage appropriate actions, more efficient security operations will be possible.

[0499] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0500] In this invention, the server includes means for acquiring the user's biometric information in real time, means for analyzing the user's emotional state based on the biometric information, and means for detecting the surrounding environment and potential threats. This enables real-time support for security guards and early detection of potential threats.

[0501] "User biometric information" refers to data that indicates the user's physical state and is acquired and analyzed in real time, including brain waves, heart rate, and skin potential.

[0502] "Means for analyzing emotional states" refers to processes and devices for determining a user's emotions and psychological state based on the user's biometric information.

[0503] "Means of suggesting optimal actions" refers to a system that, based on analyzed emotional state and environmental information, presents the user with the most appropriate behavioral patterns and choices.

[0504] "Methods for aggregating information and analyzing success factors" refers to the process of compiling collected data and extracting and analyzing factors that lead to specific results.

[0505] "Means for visualizing analysis results and providing feedback" refers to functions and devices that display analyzed information in a format that is easy for users to understand and that provide advice and suggestions for improvement based on that information.

[0506] "Means for detecting the surrounding environment and potential threats" refers to technologies and devices that collect and analyze information about the external environment to detect potential threats at an early stage.

[0507] This invention is a system that uses a smart device worn by the user to collect and analyze biometric information in real time, and proposes optimal actions in combination with information about the surrounding environment. This system operates via a terminal such as smart glasses worn by security guards and communicates with a server via a network.

[0508] The server acquires biometric data such as brainwaves and heart rate transmitted from the smart glasses and analyzes the user's emotional state using advanced algorithms. The analysis results indicate the user's stress level and concentration level, and based on this, it suggests appropriate actions in real time. It also collects environmental information through the camera and voice input, which is used to detect potential threats. By capturing changes in the surroundings, it enables immediate alerts and warnings to security personnel.

[0509] The analysis process includes sentiment analysis software and threat detection applications, which are executed using the server's processing power. Feedback to the user is provided in real time via audio devices based on the analysis results. For example, if a security guard determines that the surroundings are suspicious during a night patrol, a warning such as "Alert: Please check your surroundings" will be issued.

[0510] Next, the analyzed data is stored on a server and used to analyze the factors for success. Machine learning algorithms learn from this data and refine their suggestions for future security activities. In this way, the usefulness and accuracy of the system improve over time.

[0511] As a concrete example, we can support security guards' actions by using generative AI models to generate effective prompts using a large amount of data obtained from both the user and the environment. For example, consider the following prompt: "You are an AI assisting security guards. Based on your analysis of the surrounding environment, please suggest alerts in real time. Explain how to detect potential threats from EEG data and visual information of the environment."

[0512] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0513] Step 1:

[0514] The device uses sensors to acquire brainwaves, heart rate, and ambient audio and video data in real time. This biometric and environmental data is encrypted and transmitted to a server. The input for this step is the biometric and environmental data from the sensors, and the output is the encrypted data sent to the server.

[0515] Step 2:

[0516] The server analyzes the received data and applies an emotion analysis algorithm to identify the user's emotional state. The analysis process performs statistical calculations based on the data to determine stress levels, concentration levels, and other parameters. The input for this step is encrypted biometric data, and the output is an evaluation of the user's emotional state.

[0517] Step 3:

[0518] The server then combines the analysis results and environmental data to execute an algorithm for detecting potential threats. Image processing techniques are used to detect abnormal activity in the environment, and audio analysis is used to identify unusual sounds in the surroundings. The input for this step is the analysis results and environmental data, and the output is information about the detected potential threats.

[0519] Step 4:

[0520] The server generates optimal action suggestions for the user based on emotional state and threat information. This process uses a generative AI model to generate appropriate instructions and warnings as text, which are then prepared for voice notification. The input for this step is emotional state and threat information, and the output is text data as action suggestions.

[0521] Step 5:

[0522] The terminal notifies the user via voice of action suggestions received from the server. The user receives the instructions through earphones or a device, prompting immediate action. The input for this step is action suggestion data from the server, and the output is the voice notification provided to the user.

[0523] Step 6:

[0524] The user modifies or adjusts their actions based on the notifications they receive and performs their security duties. The input for this step is the voice notification, and the output is the user's actual actions.

[0525] Step 7:

[0526] The server aggregates all data and results and feeds them into a learning algorithm to improve the accuracy of future suggestions. The input for this step is the past data and records of user behavior, and the output is a learning model of optimized action suggestions.

[0527] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0528] This invention relates to a sales negotiation support system that combines an emotion engine that analyzes emotions based on the user's biometric data. This system analyzes the user's emotional state in real time and uses the analysis results to propose optimal actions during sales negotiations. After the negotiation is completed, the data is compiled, success factors are analyzed, and feedback can be provided. A specific embodiment of the system is shown below.

[0529] Data acquisition and analysis using an emotion engine:

[0530] First, the device acquires the user's biometric data—specifically, brainwaves, voice, and facial expression data—in real time. This data is a crucial source for accurately understanding the user's emotional state. The acquired biometric data is securely encrypted and transmitted to the server.

[0531] Analysis of emotional states using an emotion engine:

[0532] The server is equipped with an emotion engine that analyzes the transmitted biometric data. The emotion engine identifies the user's emotional state—for example, excitement, reassurance, stress, etc.—and evaluates these states in real time. The analysis results serve as indicators to optimize the user's behavior during business negotiations.

[0533] Business negotiation support and proposals:

[0534] Based on the analysis results provided by the emotion engine, the server generates optimal action suggestions during a business negotiation. These suggestions include timing for topic changes, closing the deal, and communication methods tailored to the customer's emotions. The optimized suggestions are sent to the device and delivered to the user as an audio notification through the earphones.

[0535] Post-sales analysis and feedback:

[0536] Once a business negotiation is complete, the server aggregates all the data and analyzes the factors that contributed to the negotiation's success and areas for improvement. This feedback is provided to the user via a dashboard, serving as valuable material for self-improvement.

[0537] Optimization using machine learning:

[0538] Sales negotiation data is fed into a machine learning model, improving the accuracy of proposals in subsequent negotiations. This process continuously enhances the user's sales skills.

[0539] Specific example:

[0540] For example, if the terminal collects the user's brainwave data and it indicates a high stress level, the emotion engine analyzes it, and the server suggests a topic change using a relaxing tone for the user. Also, if the customer's facial expression data indicates excitement, the server notifies the user of the appropriate time to close the deal, thereby supporting the effective progress of the business negotiation. In this way, the present invention provides an advanced support system for leading business negotiations to success.

[0541] The following describes the processing flow.

[0542] Step 1:

[0543] Since the device acquires the user's biometric data, it uses an electroencephalogram (EEG) sensor to collect brainwave data during business negotiations. In addition, it collects audio and facial expression data by recording voice with a microphone and capturing the user's facial expressions with a camera.

[0544] Step 2:

[0545] The device encrypts the biometric data it collects and sends it to the server in real time. The latest encryption technology is used to ensure data security.

[0546] Step 3:

[0547] The server utilizes an emotion engine to analyze the received biometric data. This identifies the user's emotional state, clarifying, for example, how excited, stressed, or anxious the user is during a business negotiation.

[0548] Step 4:

[0549] The server analyzes the results and suggests the optimal course of action during the sales negotiation. This suggestion includes automatically generated talk scripts and metrics for determining the flow of the negotiation. This information changes depending on the situation and is updated according to the progress of the negotiation.

[0550] Step 5:

[0551] The server sends the proposal to the terminal, and the terminal provides real-time audio notifications to the user via earphones. For example, if the customer's facial expression indicates excitement, the user is given instructions to move on to closing the deal.

[0552] Step 6:

[0553] After a business negotiation concludes, the server analyzes all collected data to identify success factors and areas for improvement. The analysis results are presented to the user as a dashboard, providing material for self-improvement.

[0554] Step 7:

[0555] The server uses sales negotiation data to perform machine learning and updates the model to optimize proposals for future sales negotiations. This improves the overall accuracy of the system and the quality of user support.

[0556] Step 8:

[0557] Based on feedback from the dashboard, users can refine their plans for future business meetings and prepare more effectively. This feedback contributes to continuous skill improvement, ultimately increasing the success rate of business meetings.

[0558] (Example 2)

[0559] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0560] In business negotiations, understanding a user's emotional state in real time and proposing the optimal course of action based on that understanding is a challenging task. Furthermore, there is a need for methods to analyze the success or failure of a negotiation after its conclusion and use that knowledge to improve future negotiations. Conventional methods have been unable to efficiently acquire and analyze biosignals in real time, provide voice notifications, and analyze emotional information; therefore, these challenges need to be addressed.

[0561] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0562] In this invention, the server includes means for acquiring the user's biometric signals in real time, means for analyzing the user's emotions based on the biometric signals, means for generating optimal actions during business negotiations based on the analysis results, means for aggregating information after the business negotiations are completed and analyzing the factors for success, and means for encrypting biometric signals to ensure data privacy. This enables real-time support and efficient feedback tailored to the user's emotional state during business negotiations.

[0563] "Biosignals" refer to data such as brain waves, voice, and facial expressions that can be obtained from the user's body.

[0564] "Emotional analysis" is the process of evaluating a user's emotional state based on biosignals and identifying mental states such as excitement, relief, and stress.

[0565] "Behavior generation" refers to the procedure for proposing the optimal response and actions during a business negotiation based on the analyzed emotional state.

[0566] "Information aggregation" is the process of compiling data recorded after a business negotiation and analyzing the factors contributing to its success and areas for improvement.

[0567] "Ensuring data privacy" refers to the technology of protecting the security and confidentiality of personal information by encrypting biometric signals.

[0568] This system is characterized by its ability to acquire and analyze biometric signals in real time to support users' business negotiations. Specifically, the terminal uses an electroencephalogram (EEG) sensor, built-in microphone, and camera to acquire the user's biometric signals (brainwaves, voice, and facial expression data) in real time. This data is encrypted using AES encryption technology and securely transmitted to the server.

[0569] The server is equipped with an emotion analysis engine built using a variety of machine learning algorithms. The server evaluates the biosignals received by this analysis engine to identify the user's emotional state. Based on this analysis, the server generates the optimal actions to take during a business negotiation. In this action generation process, a generative AI model is used, and by inputting prompt messages, the server outputs specific proposal content.

[0570] For example, the server generates optimal actions during a business negotiation by inputting a prompt message such as, "The user's brainwave data indicates high stress. Please generate suggestions to promote relaxation." The suggestions generated by the server are provided to the user as voice notifications through the terminal. The user can then proceed with the negotiation while listening to these suggestions through earphones.

[0571] After a sales negotiation concludes, the server aggregates and analyzes all data to identify success factors and areas for improvement. This information is then fed back to the user via a dashboard. Furthermore, the negotiation data is fed into a machine learning model, which is used to improve the accuracy of future proposals. This process enables users to continuously improve their sales skills.

[0572] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0573] Step 1:

[0574] The device acquires the user's biometric signals—brainwaves, voice, and facial expression data—in real time. Specifically, it acquires brainwave data using an EEG sensor, collects voice data through a built-in microphone, and acquires facial expression data using a camera. This input data serves as fundamental information for analyzing the user's emotional state.

[0575] Step 2:

[0576] The biometric signals acquired by the device are encrypted using AES encryption technology to ensure security. The encrypted data is then transmitted to the server via the internet. This process protects the privacy of the data.

[0577] Step 3:

[0578] The server decrypts the received encrypted data and inputs it into the emotion analysis engine. The analysis engine uses machine learning algorithms to identify the user's emotional state from biosignals. For example, it analyzes brainwave patterns to assess stress levels and analyzes voice tone to determine emotions. The output of the analysis is the user's current emotional state.

[0579] Step 4:

[0580] The server generates optimal actions during a business negotiation based on the analysis results. This process utilizes a generation AI model, taking prompt text as input to output specific proposals. These proposals can include elements such as timing for topic changes and adjusting the tone of speech. The generated proposals serve as a guide for users to effectively respond during business negotiations.

[0581] Step 5:

[0582] Action suggestions generated by the server are sent to the terminal and delivered as voice notifications to the user via earphones. Users receive these notifications in real time, enabling them to take the most appropriate action during the sales negotiation. This feedback is crucial support that directly impacts the success or failure of the negotiation.

[0583] Step 6:

[0584] Once a business negotiation concludes, the server aggregates all data and uses machine learning models to analyze success factors and areas for improvement. This clarifies the negotiation results and provides data to improve the accuracy of proposals in future negotiations. Users receive feedback through a dashboard, supporting continuous skill development.

[0585] (Application Example 2)

[0586] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0587] In current business negotiations and customer service situations, it is difficult to grasp customers' emotions and interests in real time and respond appropriately. This leads to a decline in the quality of communication with customers and a lack of improvement in customer satisfaction. Furthermore, there is a lack of means to use the results of negotiations and customer service to improve future interactions.

[0588] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0589] In this invention, the server includes means for acquiring the user's biosignals in real time, means for analyzing the user's emotional state based on those biosignals, means for proposing the optimal action during the conversation based on the analysis results, means for aggregating data after the conversation and analyzing the factors for success, and means for presenting optimal product information based on the customer's state during the conversation with the user. This enables accurate understanding of the customer's emotions during business negotiations and customer service, the proposal and execution of the optimal response, and post-conversation feedback for improvement in the future.

[0590] "Biosignals" refer to information obtained from the user's body, such as brainwaves, voice, and facial expressions, and are used to understand the user's emotional state in real time.

[0591] "Emotional state" refers to the user's psychological condition, such as excitement, reassurance, or stress, and is identified by analyzing biosignals.

[0592] "Optimal behavior during dialogue" refers to effective communication methods and guidelines for actions suggested to users during dialogues and business negotiations, based on the customer's emotional state.

[0593] "Success factors" are the elements that contributed to the outcome of business negotiations or discussions, analyzed and identified, and used for improvement in future events.

[0594] The "emotion engine" is a core technological tool for a system that analyzes biosignals to identify and evaluate a user's emotional state in real time.

[0595] "Product information" refers to the specific details, features, and benefits of a product or service that are presented to the user during a conversation.

[0596] "Feedback" refers to evaluations and suggestions for improvement based on analysis results provided to the user after a business negotiation or conversation has concluded, and is intended to contribute to improving the user's performance in future interactions.

[0597] To implement this invention, it is first necessary to build a system that acquires biosignals and analyzes the user's emotional state based on that information. Specifically, a wearable device such as smart glasses is used to acquire biosignals such as the user's brainwaves, voice, and facial expressions in real time. The acquired data is securely encrypted and transmitted to a server.

[0598] The server is equipped with an emotion engine that analyzes transmitted biosignals to identify the user's emotional state—for example, excitement, relief, or stress. Based on the analysis results, the server generates indicators to suggest optimal actions and product information during the interaction, and notifies the user of these indicators via smart glasses or earphones.

[0599] The software used to implement this system will be Python, combined with libraries such as OpenCV, TensorFlow, and PyTorch for emotion recognition. The hardware will utilize smart glasses such as Google Glass to efficiently collect and analyze a wide variety of biosignals.

[0600] For example, if a user detects an excited expression on a customer's face during a sales negotiation, the server analyzes the results and suggests a suitable time to close the deal, notifying the user. Conversely, if the customer shows signs of anxiety, the server suggests ways to explain the product's benefits in a reassuring tone.

[0601] By utilizing generative AI models, the accuracy of the proposed content gradually improves through machine learning.

[0602] Furthermore, the following example prompt messages can be used as a concrete example.

[0603] "Please tell me how to determine a customer's level of interest in a new product based on their facial expressions and voice data, and then notify staff in real time of the most appropriate product description."

[0604] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0605] Step 1:

[0606] The device acquires the user's biometric signals (brainwaves, voice, facial expressions) using sensors. This data is acquired in real time and encrypted for security. The input is biometric signal data, and the output is encrypted biometric signal data.

[0607] Step 2:

[0608] The terminal sends encrypted biometric data to the server. Here, data is transferred over a network, with encrypted biometric data as input and secure data transfer to the server as output.

[0609] Step 3:

[0610] The server decrypts the received biosignal data, and the emotion engine analyzes the emotional state. The input is the decrypted biosignal data, and the output is the analyzed emotional state (excitement, relief, stress). In this step, the data analysis algorithm is in operation.

[0611] Step 4:

[0612] The server uses the analysis results to generate optimal actions or product information for the interaction. This generation process utilizes emotional states and leverages a generative AI model. The input is the analyzed emotional state, and the output is action suggestions or product information.

[0613] Step 5:

[0614] The server sends suggested actions and product information to the terminal. The input is the suggested information on the server, and the output is the suggested information received by the terminal. This step involves processing related to data transmission and display.

[0615] Step 6:

[0616] The device notifies the user of the suggested information it has received. The user receives the information via smart glasses or earphones. The input is the suggested information received by the device, and the output is the information notification to the user.

[0617] Step 7:

[0618] After a business meeting or customer interaction concludes, the collected data is gathered again on the server, and the success factors are analyzed. In this step, the server processes the data using statistical methods, with the input being the business meeting data and the output being feedback information.

[0619] Step 8:

[0620] The server supplies feedback information to a machine learning model to improve the accuracy of future suggestions. The input consists of user sales data and feedback information, and the output is an optimized suggestion model. This process involves generative AI models and data calculations.

[0621] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0622] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0623] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0624] [Fourth Embodiment]

[0625] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0626] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0627] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0628] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0629] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0630] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0631] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0632] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0633] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0634] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0635] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0636] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0637] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0638] This invention is a system that optimizes user behavior during business negotiations, primarily characterized by real-time acquisition and analysis of biometric data, and action suggestions based on the results. This system operates via a terminal worn by the user and is centrally controlled by a server.

[0639] Data acquisition and sentiment analysis:

[0640] First, the device collects biometric data using sensors such as the user's brainwave sensor, microphone, and camera. This data is used to capture in real time the user's state of mind, such as their level of concentration, stress, and emotions, during business negotiations. The collected data is immediately encrypted and sent to the server.

[0641] Analysis and proposals for supporting business negotiations:

[0642] The server uses advanced algorithms based on biometric data to analyze emotional states. This identifies how interested the customer is in the deal, whether they are experiencing stress, and so on. Based on the analysis, it determines the optimal course of action and generates a proposal. This proposal includes an appropriate talk script, negotiation strategy, and closing timing.

[0643] Real-time notifications and feedback:

[0644] The proposal is sent from the server to the terminal and notified to the user in real time via audio through earphones. For example, the moment a customer shows interest in price negotiation, the optimal negotiation phrase is notified, allowing the user to change the topic at the appropriate time. After the negotiation is concluded, the server uses the aggregated data to analyze the factors contributing to the success of the negotiation and visualizes the results on a dashboard.

[0645] Machine learning for learning and improvement:

[0646] Sales negotiation data is continuously analyzed by machine learning algorithms. Based on this analysis, the server optimizes the support provided for subsequent sales negotiations, thereby improving the accuracy of proposals. This allows users to continuously improve themselves over time and enhance the quality of their sales activities.

[0647] Specific example:

[0648] For example, if a terminal acquires customer facial expression data during a business negotiation, and the server analyzes this data to determine that the customer is bored with the negotiation, it will notify the user in real time to introduce a new topic to change the flow of the negotiation. Also, if the customer shows interest in price, the server will provide voice guidance on the appropriate timing for closing the deal, helping to increase the closing rate. In this way, the present invention has a structure that optimizes sales activities based on data and supports the success of business negotiations.

[0649] The following describes the processing flow.

[0650] Step 1:

[0651] The device uses the user's brainwave sensor, microphone, and camera to acquire biometric data in real time during business negotiations. This includes measuring concentration levels using brainwaves, analyzing emotions from voice, and recognizing facial expressions using the camera.

[0652] Step 2:

[0653] The device encrypts the collected biometric data and transmits it to the server in real time. Buffering technology may be used during this process to ensure smooth data transmission.

[0654] Step 3:

[0655] The server analyzes the biometric data it receives. Here, a machine learning model is used to estimate the user's emotional state and score their level of interest, stress level, and attention focus.

[0656] Step 4:

[0657] The server generates suggestions for the user based on the results of sentiment analysis. This includes refining the talk script while considering the progress of the sales negotiation, and identifying the timing of negotiations and closing points.

[0658] Step 5:

[0659] The server sends the generated proposal to the terminal, which then provides real-time audio notifications to the user via earphones. The user is then provided with specific instructions to improve the flow of the sales negotiation.

[0660] Step 6:

[0661] Once a business negotiation is complete, the server analyzes the collected data to identify the factors that contributed to the negotiation's success and areas for improvement. These analysis results are then visualized on a dashboard.

[0662] Step 7:

[0663] The server uses machine learning based on past sales data to improve the accuracy of proposals for future sales meetings. This establishes a continuous improvement cycle.

[0664] Step 8:

[0665] Users develop self-improvement plans based on feedback from the dashboard and prepare for new business opportunities. This feedback helps improve users' sales skills.

[0666] (Example 1)

[0667] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0668] To ensure the success of business negotiations, it is crucial to accurately understand the customer's emotional state in real time and make actionable suggestions based on that understanding. However, traditional methods have made it difficult to grasp the situation in real time during negotiations and make appropriate suggestions, thus failing to maximize the outcome of negotiations.

[0669] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0670] In this invention, the server includes means for encrypting biometric information and transmitting it to a central device, means for analyzing the biometric information and evaluating the emotional state at the central device, and means for utilizing a generative model to propose the optimal action based on the evaluation results. This makes it possible to accurately grasp the emotional state of a customer in real time during a business negotiation and improve the outcome of the negotiation by proposing the optimal action.

[0671] "Biometric information" refers to data that indicates the physical state of an organism, and includes information such as brain waves, heart rate, facial expressions, and voice tone.

[0672] A "device" refers to an instrument or combination designed to perform a specific function, and in this context, it refers to a sensor device.

[0673] A "central system" refers to a core computer system used for data aggregation, processing, and analysis.

[0674] "Encryption" is a security technology that transforms information according to specific rules to prevent unauthorized access.

[0675] "Action suggestions" are guidelines and advice that instruct users on what they believe to be the most optimal actions based on the results of biometric data analysis.

[0676] A "generative model" is an artificial intelligence algorithm used to generate appropriate responses or outputs for a given input.

[0677] "Machine learning" is a data analysis technique aimed at enabling computers to automatically improve their performance through experience.

[0678] "Voice guidance" refers to audio messages used to convey information or instructions to users aurally.

[0679] This invention provides a system that supports business negotiations in real time through a terminal worn by the user. The terminal functions as a device that collects biometric information and provides feedback to the user as needed. Specifically, it is equipped with an electroencephalogram (EEG) sensor, a microphone, and a camera, which enable the acquisition of the user's brainwaves, voice, and facial expression data.

[0680] The device protects the collected biometric information using AES encryption technology and transmits it to the server via a secure HTTP protocol. After receiving this data, the server analyzes it using a generative AI model to evaluate the user's emotional state and level of interest. Based on the evaluation, prompts are used to generate optimal action suggestions for the progress of the negotiation. For example, a prompt such as, "Analyze the customer's level of interest in the price offered in real time and suggest the best way to proceed with the negotiation," might be used.

[0681] The proposed solutions are provided to the user in real time as voice guidance, enabling timely responses during sales negotiations. For example, if a customer shows interest in a particular product, the suggested closing phrase is notified to the user through their earphones. After the negotiation ends, the server analyzes the results, visualizes them, and provides feedback to the user. This allows the user to continuously make data-driven improvements in future negotiations.

[0682] The overall effect of this system is to improve user performance during sales negotiations and contribute to increasing the success rate of those negotiations.

[0683] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0684] Step 1:

[0685] The user puts on the device and begins the business negotiation. The device uses an electroencephalogram (EEG) sensor, microphone, and camera to collect the user's biometric information in real time. This information includes brainwaves, voice tone, and facial expression data. The collected biometric information is used as input, and the data is temporarily stored on the device.

[0686] Step 2:

[0687] The device encrypts the collected biometric information using AES encryption technology. The encrypted data is then sent to the server using the HTTPS protocol. This process is crucial to ensure the confidentiality and security of the data.

[0688] Step 3:

[0689] The server receives and decrypts encrypted data sent from the terminal. Using the decrypted biometric information as input data, it utilizes a generative AI model to analyze emotional states and interest levels. This analysis checks whether brainwave patterns and voice tone meet specific criteria to evaluate the user's state.

[0690] Step 4:

[0691] Based on the analysis results, the server generates optimal action suggestions. Using a generation AI model, it generates prompts tailored to the progress of the sales negotiation. For example, by generating a prompt such as, "Suggest the best phrase when the customer shows strong interest in price," appropriate action suggestions are output.

[0692] Step 5:

[0693] The server sends the generated suggestions to the terminal, which then provides an audio notification to the user via earphones. This notification is in real time and supports the user's decision-making during the business negotiation. For example, if the customer's interest begins to wane, the server suggests when to introduce a new topic.

[0694] Step 6:

[0695] After a sales negotiation concludes, the server aggregates and analyzes the data collected during the negotiation. Using the analysis results, it identifies success factors and areas for improvement, and visualizes them on a dashboard. This information helps users improve their sales activities.

[0696] Step 7:

[0697] The server applies machine learning algorithms using accumulated biometric data and analysis results. This learning process contributes to optimizing proposals for future business negotiations. A feedback loop is established to improve the accuracy of proposals based on the input data.

[0698] (Application Example 1)

[0699] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0700] Traditional security systems make it difficult for on-site security guards to instantly grasp the surrounding situation and potential threats, highlighting the need for improved security quality. Furthermore, if biometric information can be used to analyze the emotional state of security guards and encourage appropriate actions, more efficient security operations will be possible.

[0701] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0702] In this invention, the server includes means for acquiring the user's biometric information in real time, means for analyzing the user's emotional state based on the biometric information, and means for detecting the surrounding environment and potential threats. This enables real-time support for security guards and early detection of potential threats.

[0703] "User biometric information" refers to data that indicates the user's physical state and is acquired and analyzed in real time, including brain waves, heart rate, and skin potential.

[0704] "Means for analyzing emotional states" refers to processes and devices for determining a user's emotions and psychological state based on the user's biometric information.

[0705] "Means of suggesting optimal actions" refers to a system that, based on analyzed emotional state and environmental information, presents the user with the most appropriate behavioral patterns and choices.

[0706] "Methods for aggregating information and analyzing success factors" refers to the process of compiling collected data and extracting and analyzing factors that lead to specific results.

[0707] "Means for visualizing analysis results and providing feedback" refers to functions and devices that display analyzed information in a format that is easy for users to understand and that provide advice and suggestions for improvement based on that information.

[0708] "Means for detecting the surrounding environment and potential threats" refers to technologies and devices that collect and analyze information about the external environment to detect potential threats at an early stage.

[0709] This invention is a system that uses a smart device worn by the user to collect and analyze biometric information in real time, and proposes optimal actions in combination with information about the surrounding environment. This system operates via a terminal such as smart glasses worn by security guards and communicates with a server via a network.

[0710] The server acquires biometric data such as brainwaves and heart rate transmitted from the smart glasses and analyzes the user's emotional state using advanced algorithms. The analysis results indicate the user's stress level and concentration level, and based on this, it suggests appropriate actions in real time. It also collects environmental information through the camera and voice input, which is used to detect potential threats. By capturing changes in the surroundings, it enables immediate alerts and warnings to security personnel.

[0711] The analysis process includes sentiment analysis software and threat detection applications, which are executed using the server's processing power. Feedback to the user is provided in real time via audio devices based on the analysis results. For example, if a security guard determines that the surroundings are suspicious during a night patrol, a warning such as "Alert: Please check your surroundings" will be issued.

[0712] Next, the analyzed data is stored on a server and used to analyze the factors for success. Machine learning algorithms learn from this data and refine their suggestions for future security activities. In this way, the usefulness and accuracy of the system improve over time.

[0713] As a concrete example, we can support security guards' actions by using generative AI models to generate effective prompts using a large amount of data obtained from both the user and the environment. For example, consider the following prompt: "You are an AI assisting security guards. Based on your analysis of the surrounding environment, please suggest alerts in real time. Explain how to detect potential threats from EEG data and visual information of the environment."

[0714] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0715] Step 1:

[0716] The device uses sensors to acquire brainwaves, heart rate, and ambient audio and video data in real time. This biometric and environmental data is encrypted and transmitted to a server. The input for this step is the biometric and environmental data from the sensors, and the output is the encrypted data sent to the server.

[0717] Step 2:

[0718] The server analyzes the received data and applies an emotion analysis algorithm to identify the user's emotional state. The analysis process performs statistical calculations based on the data to determine stress levels, concentration levels, and other parameters. The input for this step is encrypted biometric data, and the output is an evaluation of the user's emotional state.

[0719] Step 3:

[0720] The server then combines the analysis results and environmental data to execute an algorithm for detecting potential threats. Image processing techniques are used to detect abnormal activity in the environment, and audio analysis is used to identify unusual sounds in the surroundings. The input for this step is the analysis results and environmental data, and the output is information about the detected potential threats.

[0721] Step 4:

[0722] The server generates optimal action suggestions for the user based on emotional state and threat information. This process uses a generative AI model to generate appropriate instructions and warnings as text, which are then prepared for voice notification. The input for this step is emotional state and threat information, and the output is text data as action suggestions.

[0723] Step 5:

[0724] The terminal notifies the user via voice of action suggestions received from the server. The user receives the instructions through earphones or a device, prompting immediate action. The input for this step is action suggestion data from the server, and the output is the voice notification provided to the user.

[0725] Step 6:

[0726] The user modifies or adjusts their actions based on the notifications they receive and performs their security duties. The input for this step is the voice notification, and the output is the user's actual actions.

[0727] Step 7:

[0728] The server aggregates all data and results and feeds them into a learning algorithm to improve the accuracy of future suggestions. The input for this step is the past data and records of user behavior, and the output is a learning model of optimized action suggestions.

[0729] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0730] This invention relates to a sales negotiation support system that combines an emotion engine that analyzes emotions based on the user's biometric data. This system analyzes the user's emotional state in real time and uses the analysis results to propose optimal actions during sales negotiations. After the negotiation is completed, the data is compiled, success factors are analyzed, and feedback can be provided. A specific embodiment of the system is shown below.

[0731] Data acquisition and analysis using an emotion engine:

[0732] First, the device acquires the user's biometric data—specifically, brainwaves, voice, and facial expression data—in real time. This data is a crucial source for accurately understanding the user's emotional state. The acquired biometric data is securely encrypted and transmitted to the server.

[0733] Analysis of emotional states using an emotion engine:

[0734] The server is equipped with an emotion engine that analyzes the transmitted biometric data. The emotion engine identifies the user's emotional state—for example, excitement, reassurance, stress, etc.—and evaluates these states in real time. The analysis results serve as indicators to optimize the user's behavior during business negotiations.

[0735] Business negotiation support and proposals:

[0736] Based on the analysis results provided by the emotion engine, the server generates optimal action suggestions during a business negotiation. These suggestions include timing for topic changes, closing the deal, and communication methods tailored to the customer's emotions. The optimized suggestions are sent to the device and delivered to the user as an audio notification through the earphones.

[0737] Post-sales analysis and feedback:

[0738] Once a business negotiation is complete, the server aggregates all the data and analyzes the factors that contributed to the negotiation's success and areas for improvement. This feedback is provided to the user via a dashboard, serving as valuable material for self-improvement.

[0739] Optimization using machine learning:

[0740] Sales negotiation data is fed into a machine learning model, improving the accuracy of proposals in subsequent negotiations. This process continuously enhances the user's sales skills.

[0741] Specific example:

[0742] For example, if the terminal collects the user's brainwave data and it indicates a high stress level, the emotion engine analyzes it, and the server suggests a topic change using a relaxing tone for the user. Also, if the customer's facial expression data indicates excitement, the server notifies the user of the appropriate time to close the deal, thereby supporting the effective progress of the business negotiation. In this way, the present invention provides an advanced support system for leading business negotiations to success.

[0743] The following describes the processing flow.

[0744] Step 1:

[0745] Since the device acquires the user's biometric data, it uses an electroencephalogram (EEG) sensor to collect brainwave data during business negotiations. In addition, it collects audio and facial expression data by recording voice with a microphone and capturing the user's facial expressions with a camera.

[0746] Step 2:

[0747] The device encrypts the biometric data it collects and sends it to the server in real time. The latest encryption technology is used to ensure data security.

[0748] Step 3:

[0749] The server utilizes an emotion engine to analyze the received biometric data. This identifies the user's emotional state, clarifying, for example, how excited, stressed, or anxious the user is during a business negotiation.

[0750] Step 4:

[0751] The server analyzes the results and suggests the optimal course of action during the sales negotiation. This suggestion includes automatically generated talk scripts and metrics for determining the flow of the negotiation. This information changes depending on the situation and is updated according to the progress of the negotiation.

[0752] Step 5:

[0753] The server sends the proposal to the terminal, and the terminal provides real-time audio notifications to the user via earphones. For example, if the customer's facial expression indicates excitement, the user is given instructions to move on to closing the deal.

[0754] Step 6:

[0755] After a business negotiation concludes, the server analyzes all collected data to identify success factors and areas for improvement. The analysis results are presented to the user as a dashboard, providing material for self-improvement.

[0756] Step 7:

[0757] The server uses sales negotiation data to perform machine learning and updates the model to optimize proposals for future sales negotiations. This improves the overall accuracy of the system and the quality of user support.

[0758] Step 8:

[0759] Based on feedback from the dashboard, users can refine their plans for future business meetings and prepare more effectively. This feedback contributes to continuous skill improvement, ultimately increasing the success rate of business meetings.

[0760] (Example 2)

[0761] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0762] In business negotiations, understanding a user's emotional state in real time and proposing the optimal course of action based on that understanding is a challenging task. Furthermore, there is a need for methods to analyze the success or failure of a negotiation after its conclusion and use that knowledge to improve future negotiations. Conventional methods have been unable to efficiently acquire and analyze biosignals in real time, provide voice notifications, and analyze emotional information; therefore, these challenges need to be addressed.

[0763] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0764] In this invention, the server includes means for acquiring the user's biometric signals in real time, means for analyzing the user's emotions based on the biometric signals, means for generating optimal actions during business negotiations based on the analysis results, means for aggregating information after the business negotiations are completed and analyzing the factors for success, and means for encrypting biometric signals to ensure data privacy. This enables real-time support and efficient feedback tailored to the user's emotional state during business negotiations.

[0765] "Biosignals" refer to data such as brain waves, voice, and facial expressions that can be obtained from the user's body.

[0766] "Emotional analysis" is the process of evaluating a user's emotional state based on biosignals and identifying mental states such as excitement, relief, and stress.

[0767] "Behavior generation" refers to the procedure for proposing the optimal response and actions during a business negotiation based on the analyzed emotional state.

[0768] "Information aggregation" is the process of compiling data recorded after a business negotiation and analyzing the factors contributing to its success and areas for improvement.

[0769] "Ensuring data privacy" refers to the technology of protecting the security and confidentiality of personal information by encrypting biometric signals.

[0770] This system is characterized by its ability to acquire and analyze biometric signals in real time to support users' business negotiations. Specifically, the terminal uses an electroencephalogram (EEG) sensor, built-in microphone, and camera to acquire the user's biometric signals (brainwaves, voice, and facial expression data) in real time. This data is encrypted using AES encryption technology and securely transmitted to the server.

[0771] The server is equipped with an emotion analysis engine built using a variety of machine learning algorithms. The server evaluates the biosignals received by this analysis engine to identify the user's emotional state. Based on this analysis, the server generates the optimal actions to take during a business negotiation. In this action generation process, a generative AI model is used, and by inputting prompt messages, the server outputs specific proposal content.

[0772] For example, the server generates optimal actions during a business negotiation by inputting a prompt message such as, "The user's brainwave data indicates high stress. Please generate suggestions to promote relaxation." The suggestions generated by the server are provided to the user as voice notifications through the terminal. The user can then proceed with the negotiation while listening to these suggestions through earphones.

[0773] After a sales negotiation concludes, the server aggregates and analyzes all data to identify success factors and areas for improvement. This information is then fed back to the user via a dashboard. Furthermore, the negotiation data is fed into a machine learning model, which is used to improve the accuracy of future proposals. This process enables users to continuously improve their sales skills.

[0774] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0775] Step 1:

[0776] The device acquires the user's biometric signals—brainwaves, voice, and facial expression data—in real time. Specifically, it acquires brainwave data using an EEG sensor, collects voice data through a built-in microphone, and acquires facial expression data using a camera. This input data serves as fundamental information for analyzing the user's emotional state.

[0777] Step 2:

[0778] The biometric signals acquired by the device are encrypted using AES encryption technology to ensure security. The encrypted data is then transmitted to the server via the internet. This process protects the privacy of the data.

[0779] Step 3:

[0780] The server decrypts the received encrypted data and inputs it into the emotion analysis engine. The analysis engine uses machine learning algorithms to identify the user's emotional state from biosignals. For example, it analyzes brainwave patterns to assess stress levels and analyzes voice tone to determine emotions. The output of the analysis is the user's current emotional state.

[0781] Step 4:

[0782] The server generates optimal actions during a business negotiation based on the analysis results. This process utilizes a generation AI model, taking prompt text as input to output specific proposals. These proposals can include elements such as timing for topic changes and adjusting the tone of speech. The generated proposals serve as a guide for users to effectively respond during business negotiations.

[0783] Step 5:

[0784] Action suggestions generated by the server are sent to the terminal and delivered as voice notifications to the user via earphones. Users receive these notifications in real time, enabling them to take the most appropriate action during the sales negotiation. This feedback is crucial support that directly impacts the success or failure of the negotiation.

[0785] Step 6:

[0786] Once a business negotiation concludes, the server aggregates all data and uses machine learning models to analyze success factors and areas for improvement. This clarifies the negotiation results and provides data to improve the accuracy of proposals in future negotiations. Users receive feedback through a dashboard, supporting continuous skill development.

[0787] (Application Example 2)

[0788] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0789] In current business negotiations and customer service situations, it is difficult to grasp customers' emotions and interests in real time and respond appropriately. This leads to a decline in the quality of communication with customers and a lack of improvement in customer satisfaction. Furthermore, there is a lack of means to use the results of negotiations and customer service to improve future interactions.

[0790] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0791] In this invention, the server includes means for acquiring the user's biosignals in real time, means for analyzing the user's emotional state based on those biosignals, means for proposing the optimal action during the conversation based on the analysis results, means for aggregating data after the conversation and analyzing the factors for success, and means for presenting optimal product information based on the customer's state during the conversation with the user. This enables accurate understanding of the customer's emotions during business negotiations and customer service, the proposal and execution of the optimal response, and post-conversation feedback for improvement in the future.

[0792] "Biosignals" refer to information obtained from the user's body, such as brainwaves, voice, and facial expressions, and are used to understand the user's emotional state in real time.

[0793] "Emotional state" refers to the user's psychological condition, such as excitement, reassurance, or stress, and is identified by analyzing biosignals.

[0794] "Optimal behavior during dialogue" refers to effective communication methods and guidelines for actions suggested to users during dialogues and business negotiations, based on the customer's emotional state.

[0795] "Success factors" are the elements that contributed to the outcome of business negotiations or discussions, analyzed and identified, and used for improvement in future events.

[0796] The "emotion engine" is a core technological tool for a system that analyzes biosignals to identify and evaluate a user's emotional state in real time.

[0797] "Product information" refers to the specific details, features, and benefits of a product or service that are presented to the user during a conversation.

[0798] "Feedback" refers to evaluations and suggestions for improvement based on analysis results provided to the user after a business negotiation or conversation has concluded, and is intended to contribute to improving the user's performance in future interactions.

[0799] To implement this invention, it is first necessary to build a system that acquires biosignals and analyzes the user's emotional state based on that information. Specifically, a wearable device such as smart glasses is used to acquire biosignals such as the user's brainwaves, voice, and facial expressions in real time. The acquired data is securely encrypted and transmitted to a server.

[0800] The server is equipped with an emotion engine that analyzes transmitted biosignals to identify the user's emotional state—for example, excitement, relief, or stress. Based on the analysis results, the server generates indicators to suggest optimal actions and product information during the interaction, and notifies the user of these indicators via smart glasses or earphones.

[0801] The software used to implement this system will be Python, combined with libraries such as OpenCV, TensorFlow, and PyTorch for emotion recognition. The hardware will utilize smart glasses such as Google Glass to efficiently collect and analyze a wide variety of biosignals.

[0802] For example, if a user detects an excited expression on a customer's face during a sales negotiation, the server analyzes the results and suggests a suitable time to close the deal, notifying the user. Conversely, if the customer shows signs of anxiety, the server suggests ways to explain the product's benefits in a reassuring tone.

[0803] By utilizing generative AI models, the accuracy of the proposed content gradually improves through machine learning.

[0804] Furthermore, the following example prompt messages can be used as a concrete example.

[0805] "Please tell me how to determine a customer's level of interest in a new product based on their facial expressions and voice data, and then notify staff in real time of the most appropriate product description."

[0806] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0807] Step 1:

[0808] The device acquires the user's biometric signals (brainwaves, voice, facial expressions) using sensors. This data is acquired in real time and encrypted for security. The input is biometric signal data, and the output is encrypted biometric signal data.

[0809] Step 2:

[0810] The terminal sends encrypted biometric data to the server. Here, data is transferred over a network, with encrypted biometric data as input and secure data transfer to the server as output.

[0811] Step 3:

[0812] The server decrypts the received biosignal data, and the emotion engine analyzes the emotional state. The input is the decrypted biosignal data, and the output is the analyzed emotional state (excitement, relief, stress). In this step, the data analysis algorithm is in operation.

[0813] Step 4:

[0814] The server uses the analysis results to generate optimal actions or product information for the interaction. This generation process utilizes emotional states and leverages a generative AI model. The input is the analyzed emotional state, and the output is action suggestions or product information.

[0815] Step 5:

[0816] The server sends suggested actions and product information to the terminal. The input is the suggested information on the server, and the output is the suggested information received by the terminal. This step involves processing related to data transmission and display.

[0817] Step 6:

[0818] The device notifies the user of the suggested information it has received. The user receives the information via smart glasses or earphones. The input is the suggested information received by the device, and the output is the information notification to the user.

[0819] Step 7:

[0820] After a business meeting or customer interaction concludes, the collected data is gathered again on the server, and the success factors are analyzed. In this step, the server processes the data using statistical methods, with the input being the business meeting data and the output being feedback information.

[0821] Step 8:

[0822] The server supplies feedback information to a machine learning model to improve the accuracy of future suggestions. The input consists of user sales data and feedback information, and the output is an optimized suggestion model. This process involves generative AI models and data calculations.

[0823] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0824] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0825] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0826] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0827] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0828] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0829] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0830] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0831] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0832] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0833] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0834] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0835] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0836] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0837] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0838] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0839] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0840] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0841] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0842] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0843] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0844] The following is further disclosed regarding the embodiments described above.

[0845] (Claim 1)

[0846] A means of acquiring user biometric data in real time,

[0847] A means for analyzing the user's emotional state based on the aforementioned biometric data,

[0848] A means of proposing the optimal course of action during business negotiations based on the aforementioned analysis results,

[0849] A method for aggregating data after a business negotiation and analyzing the factors for success,

[0850] A means for visualizing the aforementioned analysis results and providing feedback to the user,

[0851] A system that includes this.

[0852] (Claim 2)

[0853] The system according to claim 1, further comprising means for providing real-time voice notifications during business negotiations based on analysis results.

[0854] (Claim 3)

[0855] The system according to claim 1, further comprising means for performing machine learning using user negotiation data to optimize the content of future proposals.

[0856] "Example 1"

[0857] (Claim 1)

[0858] A device for measuring biological information,

[0859] Means for encrypting the aforementioned biometric information and transmitting it to a central device,

[0860] The aforementioned central device analyzes biological information and provides means for evaluating emotional states,

[0861] A means of utilizing a generative model to propose the optimal action based on the aforementioned evaluation results,

[0862] A means of analyzing data after the conclusion of commercial dialogues to identify the factors contributing to success,

[0863] A means for visualizing the aforementioned analysis results on a display device and providing the results to the user,

[0864] A system that includes this.

[0865] (Claim 2)

[0866] The system according to claim 1, which provides voice guidance during a commercial conversation based on the evaluation results.

[0867] (Claim 3)

[0868] The system according to claim 1, which performs machine learning based on the aforementioned biological information to improve subsequent suggestions.

[0869] "Application Example 1"

[0870] (Claim 1)

[0871] A means of acquiring a user's biometric information in real time,

[0872] A means for analyzing the user's emotional state based on the aforementioned biometric information,

[0873] A means for proposing the optimal action based on the aforementioned analysis results,

[0874] A method for compiling information after the work is completed and analyzing the factors for success,

[0875] A means for visualizing the aforementioned analysis results and providing feedback to the user,

[0876] Means for detecting the surrounding environment and potential threats,

[0877] A system that includes this.

[0878] (Claim 2)

[0879] The system according to claim 1, further comprising means for providing real-time voice notifications based on analysis results.

[0880] (Claim 3)

[0881] The system according to claim 1, further comprising means for performing machine learning using user business data to optimize the content of future suggestions.

[0882] "Example 2 of combining an emotion engine"

[0883] (Claim 1)

[0884] A means of acquiring the user's biometric signals in real time,

[0885] A means for analyzing the user's emotions based on the aforementioned biosignals,

[0886] A means for generating the optimal action to take during business negotiations based on the aforementioned analysis results,

[0887] A method for compiling information after a business negotiation and analyzing the factors for success,

[0888] A means of visualizing the aforementioned analysis results and providing information to the user,

[0889] Means of encrypting biometric signals to ensure data privacy,

[0890] A system that includes this.

[0891] (Claim 2)

[0892] The system according to claim 1, further comprising means for providing real-time voice notifications during business negotiations based on analysis results.

[0893] (Claim 3)

[0894] The system according to claim 1, further comprising a means for performing machine learning using user negotiation information to optimize future proposals.

[0895] "Application example 2 when combining with an emotional engine"

[0896] (Claim 1)

[0897] A means of acquiring the user's biometric signals in real time,

[0898] A means for analyzing the user's emotional state based on the aforementioned biosignals,

[0899] A means for proposing the optimal action during dialogue based on the aforementioned analysis results,

[0900] A method for aggregating data after the dialogue and analyzing the factors for success,

[0901] A means for visualizing the aforementioned analysis results and providing feedback to the user,

[0902] A means of presenting optimal product information based on the customer's status during interaction with the user,

[0903] A system that includes this.

[0904] (Claim 2)

[0905] The system according to claim 1, further comprising means for providing information in real time during dialogue based on the analysis results.

[0906] (Claim 3)

[0907] The system according to claim 1, further comprising means for performing machine learning using user dialogue data to optimize the content of suggestions for subsequent times. [Explanation of symbols]

[0908] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means of acquiring a user's biometric information in real time, A means for analyzing the user's emotional state based on the aforementioned biometric information, A means for proposing the optimal action based on the aforementioned analysis results, A method for compiling information after the work is completed and analyzing the factors for success, A means for visualizing the aforementioned analysis results and providing feedback to the user, Means for detecting the surrounding environment and potential threats, A system that includes this.

2. The system according to claim 1, further comprising means for providing real-time voice notifications based on analysis results.

3. The system according to claim 1, further comprising means for performing machine learning using user business data to optimize the content of future suggestions.

Images