System

Abstract

To provide a system capable of performing support in real time during business negotiation, and smoothly executing appropriate commodity proposal, presentation of hearing items, generation of additional confirmation items, and automatic creation and sharing of minutes.SOLUTION: The information processing apparatus includes a speech recognition unit that recognizes input speech in real time, a keyword extraction unit that extracts a keyword from recognized speech data, a search unit that searches for related product information on the basis of the extracted keyword, a display unit that displays the searched product information, an interview support unit that generates necessary interview items and provision conditions on the basis of the product information, an interview display unit that displays the generated interview items and provision conditions, an additional question generation unit that generates additional questions and confirmation items on the basis of interview contents, a minutes generation unit that automatically creates minutes on the basis of negotiation contents, a confirmation unit that displays the created minutes and performs confirmation and correction, and a sharing unit that shares the confirmed and corrected minutes with a customer.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In companies that handle many products, sales representatives with insufficient product expertise often find it difficult to immediately make appropriate proposals when a customer requests a new product during a sales negotiation. They also face the risk of leaking important information during the negotiation, and it is difficult to accurately organize the information to be shared with the customer or to create meeting minutes afterward. This can lead to missed sales opportunities and a loss of credibility. To solve these issues, a system is needed that can provide real-time support during sales negotiations, smoothly propose appropriate products, present questions for interviews, generate additional confirmation items, and automatically create and share meeting minutes. [Means for solving the problem]

[0005] The present invention solves the above-mentioned problems by providing a system including a voice recognition means for recognizing input voice in real time, a keyword extraction means for extracting keywords from the recognized voice data, a search means for searching for related product information based on the extracted keywords, a display means for displaying the searched product information, a hearing support means for generating necessary hearing items and provision conditions based on the product information, a hearing display means for displaying the generated hearing items and provision conditions, an additional question generation means for generating additional questions and confirmation items based on the hearing content during business negotiations, a minutes generation means for automatically creating minutes based on the content of the business negotiations, a confirmation means for displaying the minutes and confirming and correcting them, and a sharing means for sharing the confirmed and corrected minutes with the customer.

[0006] "Speech recognition means" refers to a device or software that recognizes speech during negotiations in real time and converts it into text data.

[0007] The "keyword extraction means" is a device or software that extracts important keywords from the text data converted by the voice recognition means.

[0008] The "search means" is a device or software that searches a database or the like for related product information based on the extracted keywords.

[0009] The "display means" refers to a device such as a monitor or screen, or software for displaying the searched product information to the salesperson.

[0010] The "hearing support means" is a device or software that generates the necessary hearing items and provision conditions based on the selected product information.

[0011] The "hearing display means" is a device or software that displays the generated hearing items and provision conditions to the sales representative.

[0012] The "additional question generating means" is a device or software that generates additional questions or confirmation items based on the content of hearings during business negotiations.

[0013] The "minutes generating means" is a device or software that automatically creates minutes based on the content of the business negotiations.

[0014] The "confirmation means" is a device or software that displays the generated minutes to the sales representative and prompts the sales representative to confirm and correct them.

[0015] "Sharing Instrument" means a device or software for sharing the confirmed and amended minutes with the client. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0017] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0020] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0021] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0022] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0024] [First embodiment]

[0025] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0026] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0029] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0032] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0036] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0037] The present invention relates to a business negotiation support system, and in particular to a system that effectively advances business negotiations by recognizing speech in real time during the negotiation, extracting important keywords, presenting related product information, and automatically creating records of the negotiations and minutes.

[0038] This system mainly includes the following elements: a speech recognition means, a keyword extraction means, a search means, a display means, a hearing support means, a hearing display means, a follow-up question generation means, a minutes generation means, a confirmation means, and a sharing means.

[0039] System Overview

[0040] Voice recognition means

[0041] The device uses voice recognition software to recognize speech during negotiations in real time and convert it into text data.

[0042] Keyword extraction method

[0043] The server extracts important keywords from the text data sent from the terminal, and the extracted keywords are used to identify product information related to the business negotiation.

[0044] Search methods

[0045] The server searches a database for relevant product information based on the extracted keywords, and provides candidate product information and related services as search results.

[0046] Display means

[0047] The terminal displays the product information and related services sent from the server on the salesperson's screen, allowing the salesperson to propose the most suitable product during the sales negotiation.

[0048] Hearing support measures

[0049] The server generates the necessary interview items and conditions for providing the product based on the selected product information, allowing the salesperson to ask the customer appropriate questions and collect the necessary information.

[0050] Hearing display means

[0051] The terminal displays the generated inquiry items and terms of service on the sales representative's screen, allowing the sales representative to smoothly proceed with the business negotiations.

[0052] Additional question generation means

[0053] The server generates follow-up questions and confirmations based on the conversations that take place during the sales meeting, allowing salespeople to gather the necessary details during the sales meeting.

[0054] Minutes generation method

[0055] The server automatically creates minutes based on the content of the business negotiations, and the minutes are provided to the sales representative at the end of the negotiations.

[0056] Verification method

[0057] The terminal displays the generated minutes to the sales representative, who then checks and corrects them.

[0058] means of sharing

[0059] The user (sales representative) can then share the confirmed and revised minutes with the customer, allowing them to clearly communicate the details of the business negotiations and the details of the next meeting.

[0060] Program description and examples

[0061] The program processing of this system will be explained below with specific examples.

[0062] Program processing

[0063] 1. The device uses voice recognition software to recognize speech during sales negotiations in real time. For example, if a salesperson says, "I'd like to know about cloud storage," the device converts this speech into text data that reads, "I'd like to know about cloud storage."

[0064] 2. The server analyzes the text data sent from the device and extracts the key keyword "cloud storage."

[0065] 3. The server searches the database for related products based on the extracted keywords. For example, for the keyword "cloud storage," it generates search results for "cloud storage service" and "data backup service."

[0066] 4. The terminal displays the search results sent from the server on the sales representative's screen, where the sales representative can select the products to suggest.

[0067] 5. The server generates the necessary inquiry items (e.g., storage capacity, access frequency, security requirements) and provision conditions based on the selected product information.

[0068] 6. The terminal displays the generated interview items and offer conditions on the sales representative's screen. The sales representative asks questions to the customer based on these items and collects the necessary information.

[0069] 7. The server generates additional questions and confirmations based on the interview content. For example, if the customer answers "I need 1TB of storage," the server generates an additional question such as "Please confirm the access frequency."

[0070] 8. The server records the details of the business negotiations and automatically creates minutes, which include proposed products, interview details, and additional confirmation items.

[0071] 9. The terminal provides the minutes to the sales representative, who then checks and corrects the contents.

[0072] 10. The user (sales representative) finally shares the confirmed and revised minutes with the customer, which clarifies the details of the negotiation and the next meeting.

[0073] This program streamlines information management during sales negotiations, allowing sales representatives to make prompt and appropriate proposals to customers. In addition, sales negotiation records and minutes are automatically generated, preventing information leaks and misunderstandings.

[0074] The processing flow will be explained below.

[0075] Step 1:

[0076] The device uses a microphone to capture speech during a business meeting and uses voice recognition software to convert the captured speech into text data in real time.

[0077] Step 2:

[0078] The terminal transmits the converted text data to the server via the network.

[0079] Step 3:

[0080] The server passes the received text data to a natural language processing engine and extracts important keywords. For example, it extracts "cloud storage" from the text "I want to know about cloud storage."

[0081] Step 4:

[0082] The server searches the database for relevant product information based on the extracted keywords, for example, to retrieve product information related to "cloud storage."

[0083] Step 5:

[0084] The server transmits the plurality of pieces of product information obtained as search results to the terminal.

[0085] Step 6:

[0086] The terminal displays the obtained product information and related services on the sales representative's screen, and the sales representative can select the products to suggest from the screen.

[0087] Step 7:

[0088] The server generates the necessary interview items and provision conditions based on the selected product information. For example, in the case of "cloud storage," it generates storage capacity, access frequency, security requirements, etc.

[0089] Step 8:

[0090] The terminal displays the generated interview items and terms of service on the screen of the sales representative, who then asks questions to the customer and collects the necessary information.

[0091] Step 9:

[0092] The terminal transmits the hearing response input by the sales representative or speech-recognized to the server.

[0093] Step 10:

[0094] The server analyzes the received answers and generates additional questions and confirmations. For example, in response to the answer "1 TB of storage space is required," the server adds a question such as "Please confirm the access frequency."

[0095] Step 11:

[0096] The terminal displays the generated additional questions and confirmation items on the screen of the sales representative, who then asks the customer further questions based on the displayed questions.

[0097] Step 12:

[0098] The server automatically creates minutes based on the details of the business negotiations, including proposed products, interview details, and additional confirmation items.

[0099] Step 13:

[0100] The terminal displays the generated minutes on the screen of the sales representative, allowing the sales representative to check and correct the contents.

[0101] Step 14:

[0102] The user (sales representative) shares the confirmed and corrected minutes with the customer, for example, by sending them by email to notify the customer of the contents of the minutes.

[0103] These steps enable real-time support for business negotiations, enabling efficient and accurate proposals and information sharing.

[0104] Example 1

[0105] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0106] In modern business negotiations, salespeople need to make quick and appropriate product proposals in real time and conduct interviews based on customer needs. However, it is difficult to instantly collect and record a large amount of information during a negotiation, and it also takes a lot of time to review the information later and create minutes. This places a heavy burden on salespeople, making it difficult to conduct negotiations efficiently.

[0107] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0108] In this invention, the server includes a speech recognition means for recognizing input speech in real time, a keyword extraction means for extracting keywords from speech data recognized by the speech recognition means, a search means for searching for related product information based on the keywords extracted by the keyword extraction means, a display means for displaying the product information searched by the search means, a hearing support means for generating necessary hearing items and provision conditions based on the product information, a hearing display means for displaying the generated hearing items and provision conditions, an additional question generation means for generating additional questions and confirmation items based on the hearing contents during the business negotiation, a minutes generation means for automatically creating minutes based on the contents of the business negotiation, and a display means for displaying the minutes created by the minutes generation means. The system includes a confirmation means for displaying minutes and confirming and correcting them, a sharing means for sharing the confirmed and corrected minutes with the customer, a database search means for identifying important keywords from the collected voice data and retrieving related product information from a database based on the keywords, a selection means for displaying the product information retrieved by the database search means on the sales representative's terminal and generating hearing items based on the selected product, a follow-up question generation means and a confirmation item generation means for generating follow-up questions based on the generated hearing items, a confirmation and correction means for automatically generating minutes at the end of the business negotiation and allowing the sales representative to confirm and correct the contents, and a sharing means for sharing the final confirmed and corrected minutes with the customer. This allows for efficient information gathering during the business negotiation, preventing oversight of results, and additional information confirmation, thereby reducing the burden on the sales representative and enabling the business negotiation to proceed more effectively.

[0109] The "voice recognition means" is a device or software that has the function of recognizing input voice in real time and converting voice data into text data.

[0110] The "keyword extraction means" is a device or software that has the function of extracting important keywords from the voice data recognized by the voice recognition means.

[0111] The "search means" is a device or software having a function of searching a database for related product information based on the keywords extracted by the keyword extraction means.

[0112] The "display means" is a device or software that has the function of displaying the product information acquired by the search means on the user's terminal.

[0113] The "hearing support means" is a device or software that has the function of generating necessary hearing items and provision conditions based on product information.

[0114] The "hearing display means" is a device or software that has the function of displaying the generated hearing items and provision conditions on the user's terminal.

[0115] The "additional question generating means" is a device or software that has the function of generating additional questions or confirmation items based on the content of hearings during business negotiations.

[0116] The "minutes generating means" is a device or software that has the function of automatically creating minutes based on the contents of the business negotiation.

[0117] The "checking means" is a device or software having a function of displaying the minutes created by the minutes creating means on the user's terminal and allowing the user to check and correct the contents.

[0118] "Sharing means" means a device or software that has the function of sharing the confirmed and corrected minutes with the client.

[0119] The "database search means" is a device or software that has the function of identifying important keywords from the collected voice data and retrieving related product information from a database based on those keywords.

[0120] The "selection means" is a device or software having the function of displaying the product information acquired by the database search means on the terminal and generating hearing items based on the selected product.

[0121] The "additional question generating means and confirmation item generating means" refers to a device or software having the function of generating additional questions and items to be confirmed based on the generated hearing items.

[0122] The "verification and correction means" is a device or software that has the function of allowing a user to verify the minutes that are automatically generated at the end of a business meeting and correct them as necessary.

[0123] "Sharing means" refers to a device or software that has the function of sharing the final confirmed and corrected minutes with the client.

[0124] The present invention relates to a sales negotiation support system, and in particular to a system that recognizes speech in real time during a sales negotiation, extracts important keywords, presents related product information, and automatically creates a record of the sales negotiation and minutes, thereby promoting the efficient progress of the sales negotiation and reducing the burden on sales representatives.

[0125] System configuration

[0126] The business negotiation support system of the present invention uses the following hardware and software.

[0127] 1. Devices: laptops, tablets, smartphones, etc.

[0128] 2. Speech recognition software: Google Cloud Speech-to-Text, IBM Watson Speech to Text, etc.

[0129] 3. Server: Cloud server (e.g., Amazon Web Services, Google Cloud Platform)

[0130] 4. Database: MySQL, PostgreSQL, etc.

[0131] Explanation of program processing

[0132] 1. The device uses voice recognition software to recognize speech during negotiations in real time and convert it into text data. For example, if a sales representative says, "I'd like to know about cloud storage," the speech is converted into text data that reads, "I'd like to know about cloud storage."

[0133] 2. The server receives the text data sent from the device and extracts important keywords from it. For example, the phrase "cloud storage" is extracted as a keyword. This is done using natural language processing technology (e.g., spaCy).

[0134] 3. The server searches the database for relevant product information based on the extracted keywords. For example, for the keyword "cloud storage," "cloud storage service" and "data backup service" are generated as search results.

[0135] 4. The terminal displays the search results for the product information and related services sent from the server on the sales representative's screen. The sales representative can check the product information displayed on the screen and select the products to suggest.

[0136] 5. The server generates the necessary interview items and terms of service based on the selected product information. For example, in the case of a cloud storage service, the server generates interview items such as storage capacity, access frequency, and security requirements.

[0137] 6. The terminal displays the generated interview items and terms of service on the sales representative's screen. The sales representative uses this information to ask questions of the customer and collect the necessary information.

[0138] 7. The server generates additional questions and confirmations based on the information gathered during the sales negotiation. For example, if the customer answers, "I need 1TB of storage space," the server generates an additional question, such as, "Please confirm the access frequency."

[0139] 8. The server automatically creates minutes based on the details of the business negotiations. The minutes include proposed products, interview details, and additional confirmation items.

[0140] 9. The terminal provides the minutes created by the server to the sales representative, who can then check and modify the contents.

[0141] 10. The user (sales representative) finally shares the confirmed and revised minutes with the customer, which clarifies the details of the negotiation and the next meeting.

[0142] Examples of concrete examples and prompts

[0143] Below are examples of specific actions and examples of input prompts to the generative AI model.

[0144] Specific examples of operation

[0145] If a salesperson says "I'd like to know about cloud storage" during a sales meeting, the voice recognition software converts this speech into text data, and the server extracts the keyword "cloud storage." The server then searches the database for relevant product information and displays the search results on the device. Based on this information, the salesperson can ask the customer appropriate questions, and the system automatically generates any necessary follow-up questions and meeting minutes.

[0146] Example of input prompt for generative AI model

[0147] Please explain how the sales support system works. Please provide a detailed explanation of the process from when a sales representative says, "I'd like to know about cloud storage," to when the meeting minutes are shared with the customer.

[0148] This sales negotiation support system streamlines information management during sales negotiations, allowing sales representatives to make prompt and appropriate proposals. In addition, sales negotiation records and minutes are automatically generated, preventing information leaks and misunderstandings.

[0149] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0150] Step 1:

[0151] The device uses voice recognition software to recognize speech during sales negotiations in real time and convert it into text data. Specifically, the device uses a microphone built into the laptop or tablet to record what the salesperson says and then sends the audio data to voice recognition software (e.g., Google Cloud Speech-to-Text). This software analyzes the audio data and outputs the corresponding text data (e.g., "I'd like to know about cloud storage").

[0152] Input: Voice data during business negotiations

[0153] Output: Text data (e.g., "I want to know about cloud storage")

[0154] Step 2:

[0155] The server receives the text data sent from the device and extracts important keywords from it. Specifically, it analyzes the text using natural language processing technology (e.g., spaCy) and identifies important keywords (e.g., "cloud storage").

[0156] Input: Text data (e.g., "I want to know about cloud storage")

[0157] Output: Keyword (e.g. "cloud storage")

[0158] Step 3:

[0159] The server searches for relevant product information from a database based on the extracted keywords. Specifically, it queries a database (e.g., MySQL) and retrieves product information (e.g., "cloud storage service" and "data backup service") that matches the keyword (e.g., "cloud storage").

[0160] Input: Keyword (e.g. "cloud storage")

[0161] Output: Product information (e.g. "Cloud storage service", "Data backup service")

[0162] Step 4:

[0163] The terminal displays the search results for the product information and related services sent from the server on the sales representative's screen. Specifically, the terminal uses a sales support application to display the product information on a user interface so that the sales representative can visually confirm it.

[0164] Input: Product information (e.g., "Cloud storage service," "Data backup service")

[0165] Output: Product information displayed in a user interface

[0166] Step 5:

[0167] The server generates the necessary interview items and provision conditions based on the selected product information. Specifically, it uses an algorithm to generate related interview items (e.g., "storage capacity," "access frequency," and "security requirements") based on the selected product (e.g., "cloud storage service").

[0168] Input: Selected product information (e.g., "Cloud storage service")

[0169] Output: Interview items and provision conditions (e.g., "storage capacity," "access frequency," "security requirements")

[0170] Step 6:

[0171] The terminal displays the generated hearing items and provision conditions on the sales representative's screen. Specifically, this information is updated on the user interface so that the sales representative can visually confirm it.

[0172] Input: Interview items and provision conditions (e.g., "storage capacity," "access frequency," "security requirements")

[0173] Output: Hearing items and provision conditions displayed on the user interface

[0174] Step 7:

[0175] The server generates additional questions and confirmation items based on the interview content during the business negotiation. Specifically, it analyzes the collected interview data (e.g., "1 TB of storage space is required") and generates corresponding additional questions (e.g., "Please confirm the access frequency").

[0176] Input: Interview details (e.g., "1TB of storage space required")

[0177] Output: Additional questions and confirmations (e.g., "Please confirm access frequency")

[0178] Step 8:

[0179] The server automatically creates minutes based on the content of the business negotiations. Specifically, it uses an algorithm that aggregates collected business negotiation data and generates minutes that include proposed products, interview details, and additional confirmation items.

[0180] Input: Negotiation data (e.g., proposed products, interview details, additional confirmation items)

[0181] Output: Auto-generated meeting minutes

[0182] Step 9:

[0183] The terminal provides the minutes created by the server to the sales representative, who can then check and modify the contents. Specifically, the minutes are displayed on a user interface, allowing the sales representative to make modifications.

[0184] Input: Auto-generated meeting minutes

[0185] Output: Meeting minutes reviewed and revised by sales representative

[0186] Step 10:

[0187] The user (sales representative) finally shares the minutes that have been confirmed and corrected with the customer. Specifically, the user exports the corrected minutes in PDF format or other format and sends them to the customer via email or a shared link.

[0188] Input: Confirmed and corrected minutes

[0189] Output: Meeting minutes shared with customer

[0190] This detailed process step set clarifies the overall flow of the system and the specific operations of each step, allowing sales representatives to efficiently collect, confirm, and share information during sales negotiations.

[0191] (Application example 1)

[0192] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0193] With conventional sales negotiation support systems, it took a lot of time and effort for sales representatives to quickly understand customer needs during negotiations and propose appropriate products. In addition, the process of recording the content of sales negotiations and creating minutes was often done manually, which increased the risk of information leaks and misunderstandings. Furthermore, it was not possible to present relevant information in real time during negotiations, which reduced the efficiency of sales negotiations.

[0194] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0195] In this invention, the server includes: a voice recognition means for recognizing voice in real time; a keyword extraction means for extracting keywords from voice data recognized by the voice recognition means; a search means for searching for related product information based on keywords extracted by the keyword extraction means; a display means for displaying the product information searched by the search means; a hearing support means for generating necessary hearing items and provision conditions based on the product information; a hearing display means for displaying the generated hearing items and provision conditions; an additional question generation means for generating additional questions and confirmation items based on the hearing content during the business negotiation; a minutes generation means for automatically creating minutes based on the business negotiation content; a confirmation means for displaying the minutes created by the minutes generation means and confirming and correcting them; a sharing means for sharing the confirmed and corrected minutes with the customer; a voice data conversion means for capturing voice during the business negotiation using a voice recognition device built into the smart device and converting it into text data in real time; and a visual display means for visually displaying the voice data during the business negotiation and related information using the smart device. This makes it possible to quickly grasp customer needs during sales negotiations and provide appropriate product information and related services in real time.Furthermore, by recording sales negotiations and automatically generating minutes, it is possible to prevent information leaks and misunderstandings, improving the efficiency and accuracy of sales negotiations.

[0196] The "voice recognition means" is a device that recognizes voices spoken during negotiations in real time and converts them into text data.

[0197] The "keyword extraction means" is a means for extracting important keywords from the text data recognized by the voice recognition means.

[0198] The "search means" is a device that searches a database for related product information based on the extracted keywords.

[0199] The "display means" is a device that displays the product information retrieved by the search means on the screen of the salesperson.

[0200] The "hearing support means" is a means for generating necessary hearing items and provision conditions based on product information.

[0201] The "hearing display means" is a device that displays the generated hearing items and provision conditions on the screen of the sales representative.

[0202] The "additional question generation means" is a means for generating additional questions or confirmation items based on the contents of the hearing during the business negotiation.

[0203] The "minutes generating means" is a device that automatically creates minutes based on the contents of the business negotiations.

[0204] The "checking means" is a device that displays the minutes created by the minutes creating means and allows confirmation and correction.

[0205] The "sharing means" is a device for sharing the confirmed and corrected minutes with the client.

[0206] The "voice data conversion means" is a means for capturing voices during business negotiations using a voice recognition device built into the smart device and converting them into text data in real time.

[0207] The "visual display means" is a device that visually displays voice data and related information during negotiations using a smart device.

[0208] The present invention relates to a business negotiation support system for a brick-and-mortar store, and in particular to a system that improves the efficiency and accuracy of business negotiations by using smart devices. Specific embodiments of the present invention are described below.

[0209] System configuration

[0210] The system includes a speech recognition means, a keyword extraction means, a search means, a display means, a hearing support means, a hearing display means, a follow-up question generation means, a minutes generation means, a confirmation means, a sharing means, a speech data conversion means, and a visual display means.

[0211] Voice recognition means

[0212] The smart device's built-in microphone captures the voice during the transaction and converts the voice into text data in real time using voice recognition software (e.g., Google Speech-to-Text API).

[0213] Keyword extraction method

[0214] The server extracts important keywords from the text data recognized by the speech recognition means, using natural language processing techniques.

[0215] Search methods

[0216] The server searches a database for relevant product information based on the extracted keywords, using a database management system such as MySQL.

[0217] Display means

[0218] The smart device (e.g., smart glasses) displays the product information sent from the server in the salesperson's field of vision, allowing the salesperson to recommend the most suitable product to the customer.

[0219] Hearing support measures

[0220] The server generates necessary interview items and provision conditions based on the selected product information. The generated interview items are used to collect necessary information from customers when proposing products.

[0221] Hearing display means

[0222] The smart device displays the generated inquiry items and offer conditions in the field of view of the salesperson, allowing the salesperson to efficiently ask questions to the customer.

[0223] Additional question generation means

[0224] The server generates follow-up questions and confirmations based on the conversations that take place during the sales meeting, a process that is dynamic based on the customer's responses.

[0225] Minutes generation method

[0226] The server automatically creates minutes based on the content of the business negotiations, including proposed products, interview details, and additional confirmation items.

[0227] Verification method

[0228] The smart device displays the generated minutes in the salesperson's field of view, allowing the salesperson to review and modify the contents.

[0229] means of sharing

[0230] The sales representative will then share the confirmed and revised minutes with the customer, which will help clarify the details of the negotiation and the next meeting.

[0231] Audio data conversion means

[0232] The smart device's built-in voice recognition device captures speech during sales negotiations and converts it into text data in real time. For example, if a salesperson says, "I want a new smartphone," the speech is converted into text data saying, "I want a new smartphone."

[0233] Visual display means

[0234] Smart devices are used to visually display voice data and related information during sales negotiations, allowing salespeople to respond quickly based on visual information.

[0235] Specific examples

[0236] For example, if a salesperson is in a sales meeting and thinks, "I want to recommend the latest smartphone to the customer," they can input the following prompt into the generative AI model:

[0237] User: “I want a new phone.”

[0238] System: "Find information about your new phone..."

[0239] System: "The latest smartphones: Cutting-edge smartphone A, feature-packed smartphone B, affordable smartphone C — choose the product you want."

[0240] Salesperson: (chooses from options displayed in his field of view through smart glasses)

[0241] System: "Would you like to see additional detailed information (specs, price) and information about the selected smartphone (budget, purpose of use, etc.)?"

[0242] Salesperson: "Yes"

[0243] System: "Displaying hearing items..."

[0244] This prompt sentence allows the salesperson to instantly obtain information to propose to the customer, and to effectively advance the sales negotiations.

[0245] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0246] Step 1:

[0247] The microphone in the smart device (terminal) captures the voice during the sales negotiation and sends the data to speech recognition software (e.g., Google Speech-to-Text API). The input is voice data, and the output is text data. Specifically, if a salesperson says, "I want a new smartphone," the voice data is converted into text data saying, "I want a new smartphone."

[0248] Step 2:

[0249] The server analyzes the text data received from the voice recognition software and extracts important keywords using natural language processing technology. The input is text data and the output is keywords. Specifically, the keyword "new smartphone" is extracted from the text data "I want a new smartphone."

[0250] Step 3:

[0251] The server searches a database (e.g., MySQL) for relevant product information based on the extracted keywords. The input is the keywords, and the output is the product information. Specifically, for the keyword "new smartphone," multiple smartphone models are generated as search results.

[0252] Step 4:

[0253] The server sends the search results to the smart device, and the search results are displayed in the salesperson's field of view by the display means of the smart device. The input is product information, and the output is display data. Specifically, a list of new smartphone models is displayed on the screen.

[0254] Step 5:

[0255] The salesperson (user) selects the product to be proposed from the options displayed in the field of view of the smart device. The selected product information is sent to the server. The input is the user's selection information, and the output is the selected product information.

[0256] Step 6:

[0257] The server generates the necessary questions and conditions for providing the product based on the selected product information. The input is the selected product information, and the output is the questions and conditions for providing the product. Specifically, questions about the selected smartphone model (e.g., desired storage capacity, color, etc.) are generated.

[0258] Step 7:

[0259] The smart device displays the generated hearing items and provision conditions in the field of view of the salesperson. The input is the hearing items and provision conditions, and the output is display data. Specifically, a list of questions for the hearing items is displayed on the screen.

[0260] Step 8:

[0261] The salesperson (user) asks questions to the customer based on the hearing items displayed in the field of view and collects the necessary information. The collected information is sent to the server. The input is the collected information, and the output is the hearing content data.

[0262] Step 9:

[0263] The server generates additional questions and confirmation items based on the interview content. The input is the interview content data, and the output is the additional questions and confirmation items. Specifically, if the answer is "1 TB of capacity is required," the server generates an additional question such as "Please confirm the access frequency."

[0264] Step 10:

[0265] The server records the details of the business negotiations and automatically creates minutes. The input is business negotiation data, and the output is minutes data. Specifically, the minutes include proposed products, interview details, additional confirmation items, etc.

[0266] Step 11:

[0267] The smart device displays the generated minutes in the field of view of the salesperson, who then checks and modifies the contents. The input is the minutes data, and the output is the checked and modified minutes data.

[0268] Step 12:

[0269] The sales representative (user) shares the confirmed and corrected minutes with the customer. The input is the confirmed and corrected minutes data, and the output is the minutes shared with the customer. Specifically, the final confirmed minutes are sent to the customer via email or other means.

[0270] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0271] The present invention relates to a business negotiation support system, and in particular to a system that recognizes voice in real time during business negotiations, extracts important keywords, presents related product information, automatically creates business negotiation records and minutes, and also recognizes the user's emotions and makes appropriate suggestions based on them.

[0272] This system mainly includes the following elements: a speech recognition means, a keyword extraction means, a search means, a display means, a hearing support means, a hearing display means, a follow-up question generation means, a minutes generation means, a confirmation means, a sharing means, and an emotion engine.

[0273] System Overview

[0274] Voice recognition means

[0275] The device uses a microphone to capture real-time audio during negotiations and converts it into text data using voice recognition software.

[0276] Keyword extraction method

[0277] The server extracts important keywords from the text data sent from the terminal, and the extracted keywords are used to identify product information related to the business negotiation.

[0278] Search methods

[0279] The server searches a database for relevant product information based on the extracted keywords, and provides candidate product information and related services as search results.

[0280] Display means

[0281] The terminal displays the product information and related services sent from the server on the salesperson's screen, allowing the salesperson to propose the most suitable product during the sales negotiation.

[0282] Hearing support measures

[0283] The server generates the necessary interview items and conditions for providing the product based on the selected product information, allowing the salesperson to ask the customer appropriate questions and collect the necessary information.

[0284] Hearing display means

[0285] The terminal displays the generated inquiry items and terms of service on the sales representative's screen, allowing the sales representative to smoothly proceed with the business negotiations.

[0286] Additional question generation means

[0287] The server generates follow-up questions and confirmations based on the conversations that take place during the sales meeting, allowing salespeople to gather the necessary details during the sales meeting.

[0288] Minutes generation method

[0289] The server automatically creates minutes based on the content of the business negotiations, and the minutes are provided to the sales representative at the end of the negotiations.

[0290] Verification method

[0291] The terminal displays the generated minutes to the sales representative, allowing the sales representative to check and correct the contents.

[0292] means of sharing

[0293] The user (sales representative) can then share the confirmed and revised minutes with the customer, allowing them to clearly communicate the details of the business negotiations and the details of the next meeting.

[0294] Emotion Engine

[0295] The server recognizes the user's emotions from data such as input voice and facial expressions. Based on the recognized emotions, the emotion engine adjusts the content of the product information it suggests, additional questions, and confirmation items.

[0296] Program description and examples

[0297] The program processing of this system will be explained below with specific examples.

[0298] Program processing

[0299] 1. The device uses voice recognition software to recognize speech during sales negotiations in real time. For example, if a salesperson says, "I'd like to know about cloud storage," the device converts this speech into text data that reads, "I'd like to know about cloud storage."

[0300] 2. The server analyzes the text data sent from the device and extracts the key keyword "cloud storage."

[0301] 3. The server searches the database for related products based on the extracted keywords. For example, for the keyword "cloud storage," it generates search results for "cloud storage service" and "data backup service."

[0302] 4. The server analyzes voice and facial expression data during the negotiation and uses an emotion engine to recognize the user's emotions. For example, it adjusts the content of the proposal depending on whether the customer is interested or skeptical.

[0303] 5. The server tailors the product information suggestions and follow-up questions based on the perceived sentiment, for example providing more detailed information if the customer expresses positive sentiment and a brief explanation if they are skeptical.

[0304] 6. The terminal displays the product information, related services, and emotion-based adjustment results sent from the server on the salesperson's screen. The salesperson can then select the products to recommend on the screen.

[0305] 7. The server generates the necessary inquiry items (for example, storage capacity, access frequency, security requirements) and supply conditions based on the selected product information.

[0306] 8. The terminal displays the generated interview items and offer conditions on the sales representative's screen. The sales representative then asks the customer questions based on this and collects the necessary information.

[0307] 9. The server generates additional questions and confirmations based on the interview content. For example, if the customer answers "I need 1TB of storage," the server generates an additional question such as "Please confirm the access frequency."

[0308] 10. The server records the details of the business negotiations and automatically creates minutes, which include proposed products, interview details, and additional confirmation items.

[0309] 11. The terminal provides the minutes to the sales representative, who then checks and corrects the contents.

[0310] 12. The user (sales representative) finally shares the confirmed and revised minutes with the customer, which clarifies the details of the negotiation and the next meeting.

[0311] This program streamlines information management during sales negotiations, allowing sales representatives to make quick and appropriate proposals to customers. It also recognizes users' emotions and makes adjustments based on them, leading to increased customer satisfaction.

[0312] The processing flow will be explained below.

[0313] Step 1:

[0314] The device uses a microphone to capture voices during sales negotiations in real time. It then uses voice recognition software to convert the captured voice into text data. For example, if a salesperson says, "I'd like to know about cloud storage," the device converts this voice into text data: "I'd like to know about cloud storage."

[0315] Step 2:

[0316] The terminal transmits the converted text data to the server via the network.

[0317] Step 3:

[0318] The server passes the received text data to a natural language processing engine and extracts important keywords. For example, it extracts "cloud storage" from the text "I want to know about cloud storage."

[0319] Step 4:

[0320] The server searches the database for relevant product information based on the extracted keywords, for example, to retrieve product information related to "cloud storage."

[0321] Step 5:

[0322] The terminal displays the search results (candidate product information and related services) sent from the server on the sales representative's screen, and the sales representative can select the products to suggest from the screen.

[0323] Step 6:

[0324] The server generates the necessary interview items and provision conditions based on the selected product information. For example, in the case of "cloud storage," it generates storage capacity, access frequency, security requirements, etc.

[0325] Step 7:

[0326] The terminal displays the generated interview items and terms of service on the screen of the sales representative, who then asks questions to the customer and collects the necessary information.

[0327] Step 8:

[0328] The terminal transmits the hearing response input by the sales representative or speech-recognized to the server.

[0329] Step 9:

[0330] The server analyzes the received answers and generates additional questions and confirmations. For example, in response to the answer "1 TB of storage space is required," the server adds a question such as "Please confirm the access frequency."

[0331] Step 10:

[0332] The terminal displays the generated additional questions and confirmation items on the screen of the sales representative, who then asks the customer further questions based on the displayed questions.

[0333] Step 11:

[0334] The server inputs voice and facial expression data from the business negotiation into an emotion engine to recognize the user's emotions.

[0335] Step 12:

[0336] The server then adjusts the product information and follow-up questions displayed based on the recognized emotion data, for example, providing detailed information if the customer expresses positive emotions, or providing a brief explanation if the customer is skeptical.

[0337] Step 13:

[0338] The server automatically creates minutes based on the details of the business negotiations, including proposed products, interview details, and additional confirmation items.

[0339] Step 14:

[0340] The terminal displays the generated minutes on the screen of the sales representative, allowing the sales representative to check and correct the contents.

[0341] Step 15:

[0342] The user (sales representative) shares the confirmed and corrected minutes with the customer, for example, by sending them by email to notify the customer of the contents of the minutes.

[0343] These steps enable the sales support system to manage information in real time, allowing sales representatives to make prompt and appropriate proposals to customers. Furthermore, by recognizing users' emotions and making adjustments based on them, it is possible to build trust with customers and increase the success rate of sales negotiations.

[0344] Example 2

[0345] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0346] While conventional sales negotiation support systems are effective in automatically recording sales negotiation content and proposing products, they lack the ability to recognize emotions in real time and adjust proposal content based on that, making it difficult to respond flexibly to customer emotions.In addition, there was a lack of means to improve the accuracy of sales negotiations while reducing the burden on sales representatives, such as generating follow-up questions needed during sales negotiations or automatically creating minutes.

[0347] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0348] In this invention, the server includes: a voice recognition means for recognizing input voice in real time; a keyword extraction means for extracting keywords from voice data recognized by the voice recognition means; a search means for searching for related product information based on keywords extracted by the keyword extraction means; a display means for displaying the product information searched by the search means; a hearing support means for generating necessary hearing items and provision conditions based on the product information; a hearing display means for displaying the generated hearing items and provision conditions; an additional question generation means for generating additional questions and confirmation items based on the hearing content during the business negotiation; a minutes generation means for automatically creating minutes based on the business negotiation content; a confirmation means for displaying the minutes created by the minutes generation means and confirming and correcting them; a sharing means for sharing the confirmed and corrected minutes with the customer; and an emotion recognition means for recognizing the user's emotions from the input voice and facial expression data and adjusting the provided product information and provision conditions based thereon. This makes it possible to efficiently perform real-time voice recognition and keyword extraction, related product searches, adjustment of proposal content based on emotion recognition, recording of business negotiations, and automatic generation of meeting minutes all within a single system.

[0349] The "voice recognition means" is a means for capturing input voice in real time and converting it into text data.

[0350] The "keyword extraction means" is a means for extracting important keywords from the voice data recognized by the voice recognition means.

[0351] The "search means" is a means for searching a database for related product information and service information based on the keywords extracted by the keyword extraction means.

[0352] The "display means" is a means for displaying product information and service information retrieved by the search means on the screen of the sales representative.

[0353] The "hearing support means" is a means for generating hearing items and provision conditions necessary for business negotiations based on the retrieved product information and service information.

[0354] The "hearing display means" is a means for displaying the generated hearing items and provision conditions on the screen of the sales representative.

[0355] The "additional question generation means" is a means for generating additional questions or confirmation items based on the contents of the interview obtained during the business negotiation.

[0356] The "minutes generating means" is a means for automatically creating minutes based on the contents of the business negotiation.

[0357] The "confirmation means" is a means for displaying the minutes created by the minutes creation means, and for the sales representative to confirm and correct the contents.

[0358] "Sharing means" means the means by which the confirmed and amended minutes are shared with the client.

[0359] The "emotion recognition means" is a means for recognizing the user's emotions from input voice and facial expression data, and adjusting the product information and terms of provision based on the emotions.

[0360] The present invention relates to a business negotiation support system that recognizes speech during business negotiations in real time, extracts important keywords, presents related product information, automatically records business negotiations and creates minutes, and also recognizes the user's emotions and makes appropriate suggestions based on those emotions. The specific configuration and operation of the system are described below.

[0361] Voice recognition means

[0362] The device uses a microphone to capture voices during sales negotiations in real time and converts them into text data using voice recognition software (e.g., Google's voice recognition API). For example, if a salesperson says, "I'd like to know about cloud storage," the device converts this voice into text data saying, "I'd like to know about cloud storage."

[0363] Keyword extraction method

[0364] The server receives the text data sent from the device and uses a natural language processing (NLP) tool (e.g., Python's NLTK library) to extract important keywords from this text data. For example, it extracts the keyword "cloud storage."

[0365] Search methods

[0366] The server searches a database (e.g., MySQL) for relevant product information based on the extracted keywords. For example, based on the keyword "cloud storage," it might generate search results such as "cloud storage service" and "data backup service."

[0367] emotion recognition means

[0368] The server analyzes voice and facial expression data and recognizes the user's emotions using an emotion engine (e.g., a deep learning model using Google's TensorFlow). For example, it adjusts its suggestions to provide detailed information if the customer expresses positive emotions, or a concise explanation if the customer expresses skepticism.

[0369] Adjusting the proposal

[0370] The server then tailors product suggestions and follow-up questions based on the perceived emotion, for example providing detailed information if the customer expresses positive emotion, or providing a clearer explanation if the customer is skeptical.

[0371] Displaying product information

[0372] The terminal displays product information, related services, and emotion-based adjustment results sent from the server on the salesperson's screen in real time, allowing the salesperson to use this information to recommend the most suitable product to the customer.

[0373] Hearing support

[0374] The server generates the necessary interview items (e.g., storage capacity, purpose of use) and terms of provision based on the selected product information.

[0375] Display of hearing items

[0376] The terminal displays the generated inquiry items and provision conditions on the sales representative's screen, allowing the sales representative to use this information to inquire about the necessary information from the customer.

[0377] Generate follow-up questions

[0378] The server generates additional questions and confirmations based on the information gathered during the sales negotiation. For example, if the customer answers, "I need 1 TB of storage space," the server generates additional questions such as, "Please confirm the access frequency."

[0379] Automatic generation of meeting minutes

[0380] The server records the details of the business negotiations and automatically creates minutes based on them, including proposed products, interview details, and additional confirmation items.

[0381] Checking the minutes

[0382] The terminal displays the generated minutes on the screen of the sales representative, allowing the sales representative to check and correct the contents.

[0383] Sharing meeting minutes

[0384] The user (sales representative) finally shares the confirmed and revised minutes with the customer. This can be done via an email system or a cloud storage service (e.g., Google Drive or Dropbox).

[0385] The program streamlines information management during sales negotiations, enabling salespeople to make quick and appropriate proposals to customers. It also recognizes users' emotions and adjusts accordingly, contributing to increased customer satisfaction.

[0386] Prompt Sentence Examples

[0387] Example prompts for inputting specific information into a generative AI model:

[0388] If a customer says "I want to know about cloud storage" during a sales meeting, explain how the system works. Include specific software and database names, and provide step-by-step details.

[0389] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0390] Step 1:

[0391] The device uses a microphone to capture voices during negotiations in real time and converts the input voice into text data using voice recognition software (e.g., Google's voice recognition API). The input is the voices being spoken during negotiations, and the output is the data converted from voice into text. Specifically, the device starts recording voices as soon as the negotiations begin, and sequentially sends the data to the voice recognition API to receive the text data.

[0392] Step 2:

[0393] The server receives the text data sent from the terminal and extracts important keywords using a natural language processing (NLP) tool (e.g., Python's NLTK library). The input is the text data, and the output is the extracted important keywords. The text data is subjected to morphological analysis, and keywords related to the pre-set business negotiations are extracted from it.

[0394] Step 3:

[0395] The server searches a database (e.g., MySQL) for relevant product information based on the extracted keywords. The input is the extracted keywords, and the output is the search result for related product information. A MySQL query is generated and executed to retrieve product information that matches the keywords.

[0396] Step 4:

[0397] The server analyzes voice data and facial expression data and recognizes the user's emotions using an emotion engine (e.g., a deep learning model using TensorFlow). The input is voice data and facial expression data, and the output is the recognized emotional information. Emotions are analyzed from the tone of voice and facial expressions, and the results are classified using an emotion model.

[0398] Step 5:

[0399] The server adjusts the product information suggestions and follow-up questions based on the recognized emotion. The input is emotion information and product information, and the output is the adjusted suggestions and question list. For example, if the emotion is positive, detailed information is provided, and if the emotion is skeptical, brief information is selected.

[0400] Step 6:

[0401] The terminal displays the product information and adjustment results sent from the server on the salesperson's screen. The input is the adjusted proposal content and question list, and the output is the information displayed on the salesperson's screen. The information is updated and displayed on the salesperson's display in real time.

[0402] Step 7:

[0403] The server generates the necessary interview items and provision conditions based on the selected product information. The input is the selected product information, and the output is the interview items and provision conditions. Based on the product information, the server selects appropriate interview items from a template and generates the provision conditions.

[0404] Step 8:

[0405] The terminal displays the generated inquiry items and provision conditions on the sales representative's screen. The input is the generated inquiry items and provision conditions, and the output is the information displayed on the sales representative's screen. This allows the sales representative to ask questions to the customer according to the inquiry items.

[0406] Step 9:

[0407] The server generates additional questions and confirmations based on the information gathered during the negotiation. The input is the information gathered during the negotiation, and the output is the additional questions and confirmations. For example, in response to the answer "1 TB of storage is required," the server generates an additional question such as "Please confirm the access frequency."

[0408] Step 10:

[0409] The server records the content of the business negotiations and automatically creates minutes based on that. The input is the text data of the business negotiations, and the output is the minutes. The minutes are created by dividing the business negotiation text into paragraphs and extracting important points and questions and answers.

[0410] Step 11:

[0411] The terminal displays the generated minutes on the sales representative's screen, allowing the sales representative to check and correct the contents. The input is the generated minutes, and the output is the checked and corrected minutes. The sales representative checks the displayed minutes and makes corrections as necessary.

[0412] Step 12:

[0413] The user (sales representative) finally shares the confirmed and revised minutes with the customer. The input is the revised minutes, and the output is the minutes shared with the customer. The sharing method is an email transmission system or cloud storage service.

[0414] This series of processes makes information management during negotiations more efficient, enabling sales representatives to make appropriate proposals to customers.

[0415] (Application example 2)

[0416] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0417] Conventional sales negotiation support systems can record the content of sales negotiations in real time and extract important keywords to present related information, but they have difficulty making accurate proposals based on customer sentiment. Furthermore, because they do not support automatic recording of sales negotiation content or automatic creation of meeting minutes, they have the problem of consuming a large amount of resources after the negotiation. This reduces the work efficiency of sales representatives and does not necessarily result in optimal sales negotiation outcomes.

[0418] The specification processing by the specification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: a speech recognition unit that recognizes input speech in real time; a keyword extraction unit that extracts keywords from speech data recognized by the speech recognition unit; a search unit that searches for related product information based on the keywords extracted by the keyword extraction unit; an emotion recognition unit that recognizes the customer's emotions and adjusts the proposal content based on the emotions; a display unit that displays the product information searched by the search unit; a hearing support unit that generates necessary hearing items and provision conditions based on the product information; a hearing display unit that displays the generated hearing items and provision conditions; a follow-up question generation unit that generates follow-up questions and confirmation items based on the hearing content during the business negotiation; a minutes generation unit that automatically creates minutes based on the business negotiation content; a confirmation unit that displays the minutes created by the minutes generation unit and confirms and modifies them; and a sharing unit that shares the confirmed and modified minutes with the customer. This enables efficient management of information during the business negotiation and optimal proposals based on the customer's emotions.

[0419] The "voice recognition means" is a means for recognizing input voice in real time and converting it into text data.

[0420] The "keyword extraction means" is a means for extracting important keywords from the voice data recognized by the voice recognition means.

[0421] The "search means" is a means for searching for related product information based on the keywords extracted by the keyword extraction means.

[0422] The "emotion recognition means" is a means for recognizing emotions from the customer's voice and facial expressions, and adjusting the content of the proposal based on those emotions.

[0423] The "display means" is a means for displaying product information and related services searched for by the search means, and is a means for visually presenting necessary information.

[0424] The "hearing support means" is a means for generating necessary hearing items and provision conditions based on product information.

[0425] The "hearing display means" is a means for displaying the generated hearing items and provision conditions.

[0426] The "additional question generation means" is a means for generating additional questions or confirmation items based on the contents of the hearing during the business negotiation.

[0427] The "minutes generation means" is a means for automatically creating minutes based on the contents of the business negotiations.

[0428] The "checking means" is a means for displaying the minutes created by the minutes creating means and for checking and correcting them.

[0429] "Sharing means" means the means by which the confirmed and amended minutes are shared with the client.

[0430] "Product information" is detailed information about products and services related to the content of the business negotiations.

[0431] "Emotion recognition" is a technology that analyzes and recognizes a customer's emotional state from their voice and facial expressions.

[0432] A "business negotiation" is a conversation or negotiation between a sales representative and a customer regarding a proposed product or service.

[0433] This invention realizes a shopping assistant system for brick-and-mortar stores, specifically, a system that uses smart glasses or an application installed on a smartphone to assist store clerks in conversations with customers. This system uses the following hardware and software:

[0434] Hardware and Software

[0435] 1. Hardware:

[0436] Smart glasses or smartphone: A device equipped with a microphone and camera.

[0437] 2. Software:

[0438] Speech recognition software (such as Google Cloud Speech-to-Text API)

[0439] Keyword extraction model (using TensorFlow, etc.)

[0440] Emotion recognition engine (e.g. NVIDIA Clara AI)

[0441] Database (e.g. Firebase Realtime Database)

[0442] System Procedures

[0443] Voice Capture and Recognition

[0444] The device uses the microphones in smart glasses or smartphones to capture customer interactions in real time, and the captured voice data is converted into text using the Google Cloud Speech-to-Text API.

[0445] Keyword extraction

[0446] The server extracts important keywords from the text data generated by the speech recognition means, using a keyword extraction model built with TensorFlow.

[0447] Searching and displaying product information

[0448] The server searches for relevant product information from a database such as Firebase based on the keywords extracted by the keyword extraction means. The searched product information is displayed on the terminal screen, allowing the store clerk to quickly provide appropriate product information.

[0449] Emotion recognition and suggestion adjustment

[0450] The server analyzes the customer's voice and facial expression data and uses NVIDIA Clara AI technology to recognize the customer's emotions. Based on the results of this emotion recognition, the server adjusts the product information suggestions and follow-up questions it asks. For example, if the customer expresses positive emotions, it provides more detailed product information, while if the customer expresses skepticism, it provides a concise, to-the-point explanation.

[0451] Hearing support and follow-up questions

[0452] The server generates the necessary inquiry items and provision conditions based on the selected product information. These inquiry items and provision conditions are displayed on the terminal screen, and the salesperson asks the customer appropriate questions based on them and collects the necessary information. Furthermore, the server generates additional questions and confirmation items based on the inquiry content during the sales negotiation.

[0453] Generate and review meeting minutes

[0454] The server automatically creates minutes based on the content of the business negotiation. The minutes are then displayed on the terminal screen, allowing the salesperson to check and modify the contents. After this, the minutes are shared with the customer.

[0455] Examples of concrete examples and prompts

[0456] Specific examples

[0457] For example, if a store clerk is talking to a customer and the customer says, "I want to see the most popular 4K TV," the system will recognize the speech and extract the keyword "4K TV." It will then search the database for related product information and display the search results on the store clerk's device. At the same time, it will recognize the customer's interest and suggest more detailed product descriptions. The store clerk can then make the best suggestions to the customer based on the displayed information and any follow-up questions.

[0458] Prompt Sentence Examples

[0459] You are a salesperson. If a customer says, "I want a new 4K TV," here's what you should do:

[0460] 1. Use speech recognition to extract the important keyword "4K TV."

[0461] 2. Search and suggest relevant product information from the database.

[0462] 3. Analyze customer sentiment to determine if they're interested or want to know more.

[0463] 4. View the suggestions and generate and ask follow-up questions as needed.

[0464] 5. Automatically record the contents of business negotiations and create minutes.

[0465] This system makes customer service in physical stores more efficient and effective, allowing store staff to make optimal suggestions based on customer needs. It also automatically records the details of sales negotiations, making post-negotiation management easier.

[0466] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0467] Step 1:

[0468] The terminal uses the microphone of smart glasses or smartphone to capture customer speech in real time, and the input data is the voice signal, which serves as the base data for subsequent processing. The output is the captured voice data.

[0469] Step 2:

[0470] The device converts the captured voice data into text data using speech recognition software (e.g., Google Cloud Speech-to-Text API). This process analyzes the voice signal and generates corresponding text. The input is the voice data acquired in step 1, and the output is text data.

[0471] Step 3:

[0472] The server extracts important keywords from the text data generated by the speech recognition means. This uses a keyword extraction model built with TensorFlow. The input is the text data generated in step 2, and the output is the extracted keywords.

[0473] Step 4:

[0474] The server searches for related product information from a database such as Firebase based on the extracted keywords. The system sends product information containing the keywords as a query to the database and retrieves the corresponding product information. The input is the keywords extracted in step 3, and the output is related product information.

[0475] Step 5:

[0476] The server uses NVIDIA Clara AI to recognize emotions using the customer's voice and facial expression data. Voice data and camera footage are used as inputs to analyze the customer's emotional state. The inputs are voice and video data, and the output is the emotion recognition results.

[0477] Step 6:

[0478] The terminal displays optimal suggestions to the customer based on the acquired product information and emotion recognition results. The system adjusts the suggestions based on the results of emotion analysis and displays them on the screen. The input is the output data from Steps 4 and 5, and the output is the adjusted suggestions.

[0479] Step 7:

[0480] The server generates the necessary interview items and provision conditions based on the selected product information. The interview support means automatically generates questions according to the customer's needs. The input is the output data of step 4, and the output is the interview items and provision conditions.

[0481] Step 8:

[0482] The terminal displays the generated interview items and conditions for provision, and the store clerk asks the customer appropriate questions based on this to collect information. The clerk checks the content displayed on the screen and collects the necessary information from the customer. The input is the output data of step 7, and the output is the collected customer information.

[0483] Step 9:

[0484] The server generates additional questions and confirmations based on the interview content. The system analyzes the answers from the customer and generates additional questions if more detailed information needs to be collected. The input is the customer information collected in step 8, and the output is additional questions and confirmations.

[0485] Step 10:

[0486] The server automatically creates minutes based on the content of the business negotiation. The system compiles all data acquired during the business negotiation and formats it as minutes. The input is all data from step 2 to step 9, and the output is the minutes of the business negotiation.

[0487] Step 11:

[0488] The terminal displays the generated minutes, which the clerk can review and modify. The system reflects the modifications and finalizes the minutes. The input is the minutes created in step 10, and the output is the reviewed and modified minutes.

[0489] Step 12:

[0490] The user (store clerk) shares the final confirmed and revised minutes with the customer. The minutes are provided to the customer and the customer confirms the details of the business negotiation. The input is the minutes confirmed and revised in step 11, and the output is the final minutes shared with the customer.

[0491] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0492] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0493] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0494] [Second embodiment]

[0495] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0496] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0497] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0498] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0499] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0500] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0501] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0502] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0503] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0504] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0505] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0506] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0507] The present invention relates to a business negotiation support system, and in particular to a system that effectively advances business negotiations by recognizing speech in real time during the negotiation, extracting important keywords, presenting related product information, and automatically creating records of the negotiations and minutes.

[0508] This system mainly includes the following elements: a speech recognition means, a keyword extraction means, a search means, a display means, a hearing support means, a hearing display means, a follow-up question generation means, a minutes generation means, a confirmation means, and a sharing means.

[0509] System Overview

[0510] Voice recognition means

[0511] The device uses voice recognition software to recognize speech during negotiations in real time and convert it into text data.

[0512] Keyword extraction method

[0513] The server extracts important keywords from the text data sent from the terminal, and the extracted keywords are used to identify product information related to the business negotiation.

[0514] Search methods

[0515] The server searches a database for relevant product information based on the extracted keywords, and provides candidate product information and related services as search results.

[0516] Display means

[0517] The terminal displays the product information and related services sent from the server on the salesperson's screen, allowing the salesperson to propose the most suitable product during the sales negotiation.

[0518] Hearing support measures

[0519] The server generates the necessary interview items and conditions for providing the product based on the selected product information, allowing the salesperson to ask the customer appropriate questions and collect the necessary information.

[0520] Hearing display means

[0521] The terminal displays the generated inquiry items and terms of service on the sales representative's screen, allowing the sales representative to smoothly proceed with the business negotiations.

[0522] Additional question generation means

[0523] The server generates follow-up questions and confirmations based on the conversations that take place during the sales meeting, allowing salespeople to gather the necessary details during the sales meeting.

[0524] Minutes generation method

[0525] The server automatically creates minutes based on the content of the business negotiations, and the minutes are provided to the sales representative at the end of the negotiations.

[0526] Verification method

[0527] The terminal displays the generated minutes to the sales representative, who then checks and corrects them.

[0528] means of sharing

[0529] The user (sales representative) can then share the confirmed and revised minutes with the customer, allowing them to clearly communicate the details of the business negotiations and the details of the next meeting.

[0530] Program description and examples

[0531] The program processing of this system will be explained below with specific examples.

[0532] Program processing

[0533] 1. The device uses voice recognition software to recognize speech during sales negotiations in real time. For example, if a salesperson says, "I'd like to know about cloud storage," the device converts this speech into text data that reads, "I'd like to know about cloud storage."

[0534] 2. The server analyzes the text data sent from the device and extracts the key keyword "cloud storage."

[0535] 3. The server searches the database for related products based on the extracted keywords. For example, for the keyword "cloud storage," it generates search results for "cloud storage service" and "data backup service."

[0536] 4. The terminal displays the search results sent from the server on the sales representative's screen, where the sales representative can select the products to suggest.

[0537] 5. The server generates the necessary inquiry items (e.g., storage capacity, access frequency, security requirements) and provision conditions based on the selected product information.

[0538] 6. The terminal displays the generated interview items and offer conditions on the sales representative's screen. The sales representative asks questions to the customer based on these items and collects the necessary information.

[0539] 7. The server generates additional questions and confirmations based on the interview content. For example, if the customer answers "I need 1TB of storage," the server generates an additional question such as "Please confirm the access frequency."

[0540] 8. The server records the details of the business negotiations and automatically creates minutes, which include proposed products, interview details, and additional confirmation items.

[0541] 9. The terminal provides the minutes to the sales representative, who then checks and corrects the contents.

[0542] 10. The user (sales representative) finally shares the confirmed and revised minutes with the customer, which clarifies the details of the negotiation and the next meeting.

[0543] This program streamlines information management during sales negotiations, allowing sales representatives to make prompt and appropriate proposals to customers. In addition, sales negotiation records and minutes are automatically generated, preventing information leaks and misunderstandings.

[0544] The processing flow will be explained below.

[0545] Step 1:

[0546] The device uses a microphone to capture speech during a business meeting and uses voice recognition software to convert the captured speech into text data in real time.

[0547] Step 2:

[0548] The terminal transmits the converted text data to the server via the network.

[0549] Step 3:

[0550] The server passes the received text data to a natural language processing engine and extracts important keywords. For example, it extracts "cloud storage" from the text "I want to know about cloud storage."

[0551] Step 4:

[0552] The server searches the database for relevant product information based on the extracted keywords, for example, to retrieve product information related to "cloud storage."

[0553] Step 5:

[0554] The server transmits the plurality of pieces of product information obtained as search results to the terminal.

[0555] Step 6:

[0556] The terminal displays the obtained product information and related services on the sales representative's screen, and the sales representative can select the products to suggest from the screen.

[0557] Step 7:

[0558] The server generates the necessary interview items and provision conditions based on the selected product information. For example, in the case of "cloud storage," it generates storage capacity, access frequency, security requirements, etc.

[0559] Step 8:

[0560] The terminal displays the generated interview items and terms of service on the screen of the sales representative, who then asks questions to the customer and collects the necessary information.

[0561] Step 9:

[0562] The terminal transmits the hearing response input by the sales representative or speech-recognized to the server.

[0563] Step 10:

[0564] The server analyzes the received answers and generates additional questions and confirmations. For example, in response to the answer "1 TB of storage space is required," the server adds a question such as "Please confirm the access frequency."

[0565] Step 11:

[0566] The terminal displays the generated additional questions and confirmation items on the screen of the sales representative, who then asks the customer further questions based on the displayed questions.

[0567] Step 12:

[0568] The server automatically creates minutes based on the details of the business negotiations, including proposed products, interview details, and additional confirmation items.

[0569] Step 13:

[0570] The terminal displays the generated minutes on the screen of the sales representative, allowing the sales representative to check and correct the contents.

[0571] Step 14:

[0572] The user (sales representative) shares the confirmed and corrected minutes with the customer, for example, by sending them by email to notify the customer of the contents of the minutes.

[0573] These steps enable real-time support for business negotiations, enabling efficient and accurate proposals and information sharing.

[0574] Example 1

[0575] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0576] In modern business negotiations, salespeople need to make quick and appropriate product proposals in real time and conduct interviews based on customer needs. However, it is difficult to instantly collect and record a large amount of information during a negotiation, and it also takes a lot of time to review the information later and create minutes. This places a heavy burden on salespeople, making it difficult to conduct negotiations efficiently.

[0577] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0578] In this invention, the server includes a speech recognition means for recognizing input speech in real time, a keyword extraction means for extracting keywords from speech data recognized by the speech recognition means, a search means for searching for related product information based on the keywords extracted by the keyword extraction means, a display means for displaying the product information searched by the search means, a hearing support means for generating necessary hearing items and provision conditions based on the product information, a hearing display means for displaying the generated hearing items and provision conditions, an additional question generation means for generating additional questions and confirmation items based on the hearing contents during the business negotiation, a minutes generation means for automatically creating minutes based on the contents of the business negotiation, and a display means for displaying the minutes created by the minutes generation means. The system includes a confirmation means for displaying minutes and confirming and correcting them, a sharing means for sharing the confirmed and corrected minutes with the customer, a database search means for identifying important keywords from the collected voice data and retrieving related product information from a database based on the keywords, a selection means for displaying the product information retrieved by the database search means on the sales representative's terminal and generating hearing items based on the selected product, a follow-up question generation means and a confirmation item generation means for generating follow-up questions based on the generated hearing items, a confirmation and correction means for automatically generating minutes at the end of the business negotiation and allowing the sales representative to confirm and correct the contents, and a sharing means for sharing the final confirmed and corrected minutes with the customer. This allows for efficient information gathering during the business negotiation, preventing oversight of results, and additional information confirmation, thereby reducing the burden on the sales representative and enabling the business negotiation to proceed more effectively.

[0579] The "voice recognition means" is a device or software that has the function of recognizing input voice in real time and converting voice data into text data.

[0580] The "keyword extraction means" is a device or software that has the function of extracting important keywords from the voice data recognized by the voice recognition means.

[0581] The "search means" is a device or software having a function of searching a database for related product information based on the keywords extracted by the keyword extraction means.

[0582] The "display means" is a device or software that has the function of displaying the product information acquired by the search means on the user's terminal.

[0583] The "hearing support means" is a device or software that has the function of generating necessary hearing items and provision conditions based on product information.

[0584] The "hearing display means" is a device or software that has the function of displaying the generated hearing items and provision conditions on the user's terminal.

[0585] The "additional question generating means" is a device or software that has the function of generating additional questions or confirmation items based on the content of hearings during business negotiations.

[0586] The "minutes generating means" is a device or software that has the function of automatically creating minutes based on the contents of the business negotiation.

[0587] The "checking means" is a device or software having a function of displaying the minutes created by the minutes creating means on the user's terminal and allowing the user to check and correct the contents.

[0588] "Sharing means" means a device or software that has the function of sharing the confirmed and corrected minutes with the client.

[0589] The "database search means" is a device or software that has the function of identifying important keywords from the collected voice data and retrieving related product information from a database based on those keywords.

[0590] The "selection means" is a device or software having the function of displaying the product information acquired by the database search means on the terminal and generating hearing items based on the selected product.

[0591] The "additional question generating means and confirmation item generating means" refers to a device or software having the function of generating additional questions and items to be confirmed based on the generated hearing items.

[0592] The "verification and correction means" is a device or software that has the function of allowing a user to verify the minutes that are automatically generated at the end of a business meeting and correct them as necessary.

[0593] "Sharing means" refers to a device or software that has the function of sharing the final confirmed and corrected minutes with the client.

[0594] The present invention relates to a sales negotiation support system, and in particular to a system that recognizes speech in real time during a sales negotiation, extracts important keywords, presents related product information, and automatically creates a record of the sales negotiation and minutes, thereby promoting the efficient progress of the sales negotiation and reducing the burden on sales representatives.

[0595] System configuration

[0596] The business negotiation support system of the present invention uses the following hardware and software.

[0597] 1. Devices: laptops, tablets, smartphones, etc.

[0598] 2. Speech recognition software: Google Cloud Speech-to-Text, IBM Watson Speech to Text, etc.

[0599] 3. Server: Cloud server (e.g., Amazon Web Services, Google Cloud Platform)

[0600] 4. Database: MySQL, PostgreSQL, etc.

[0601] Explanation of program processing

[0602] 1. The device uses voice recognition software to recognize speech during negotiations in real time and convert it into text data. For example, if a sales representative says, "I'd like to know about cloud storage," the speech is converted into text data that reads, "I'd like to know about cloud storage."

[0603] 2. The server receives the text data sent from the device and extracts important keywords from it. For example, the phrase "cloud storage" is extracted as a keyword. This is done using natural language processing technology (e.g., spaCy).

[0604] 3. The server searches the database for relevant product information based on the extracted keywords. For example, for the keyword "cloud storage," "cloud storage service" and "data backup service" are generated as search results.

[0605] 4. The terminal displays the search results for the product information and related services sent from the server on the sales representative's screen. The sales representative can check the product information displayed on the screen and select the products to suggest.

[0606] 5. The server generates the necessary interview items and terms of service based on the selected product information. For example, in the case of a cloud storage service, the server generates interview items such as storage capacity, access frequency, and security requirements.

[0607] 6. The terminal displays the generated interview items and terms of service on the sales representative's screen. The sales representative uses this information to ask questions of the customer and collect the necessary information.

[0608] 7. The server generates additional questions and confirmations based on the information gathered during the sales negotiation. For example, if the customer answers, "I need 1TB of storage space," the server generates an additional question, such as, "Please confirm the access frequency."

[0609] 8. The server automatically creates minutes based on the details of the business negotiations. The minutes include proposed products, interview details, and additional confirmation items.

[0610] 9. The terminal provides the minutes created by the server to the sales representative, who can then check and modify the contents.

[0611] 10. The user (sales representative) finally shares the confirmed and revised minutes with the customer, which clarifies the details of the negotiation and the next meeting.

[0612] Examples of concrete examples and prompts

[0613] Below are examples of specific actions and examples of input prompts to the generative AI model.

[0614] Specific examples of operation

[0615] If a salesperson says "I'd like to know about cloud storage" during a sales meeting, the voice recognition software converts this speech into text data, and the server extracts the keyword "cloud storage." The server then searches the database for relevant product information and displays the search results on the device. Based on this information, the salesperson can ask the customer appropriate questions, and the system automatically generates any necessary follow-up questions and meeting minutes.

[0616] Example of input prompt for generative AI model

[0617] Please explain how the sales support system works. Please provide a detailed explanation of the process from when a sales representative says, "I'd like to know about cloud storage," to when the meeting minutes are shared with the customer.

[0618] This sales negotiation support system streamlines information management during sales negotiations, allowing sales representatives to make prompt and appropriate proposals. In addition, sales negotiation records and minutes are automatically generated, preventing information leaks and misunderstandings.

[0619] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0620] Step 1:

[0621] The device uses voice recognition software to recognize speech during sales negotiations in real time and convert it into text data. Specifically, the device uses a microphone built into the laptop or tablet to record what the salesperson says and then sends the audio data to voice recognition software (e.g., Google Cloud Speech-to-Text). This software analyzes the audio data and outputs the corresponding text data (e.g., "I'd like to know about cloud storage").

[0622] Input: Voice data during business negotiations

[0623] Output: Text data (e.g., "I want to know about cloud storage")

[0624] Step 2:

[0625] The server receives the text data sent from the device and extracts important keywords from it. Specifically, it analyzes the text using natural language processing technology (e.g., spaCy) and identifies important keywords (e.g., "cloud storage").

[0626] Input: Text data (e.g., "I want to know about cloud storage")

[0627] Output: Keyword (e.g. "cloud storage")

[0628] Step 3:

[0629] The server searches for relevant product information from a database based on the extracted keywords. Specifically, it queries a database (e.g., MySQL) and retrieves product information (e.g., "cloud storage service" and "data backup service") that matches the keyword (e.g., "cloud storage").

[0630] Input: Keyword (e.g. "cloud storage")

[0631] Output: Product information (e.g. "Cloud storage service", "Data backup service")

[0632] Step 4:

[0633] The terminal displays the search results for the product information and related services sent from the server on the sales representative's screen. Specifically, the terminal uses a sales support application to display the product information on a user interface so that the sales representative can visually confirm it.

[0634] Input: Product information (e.g., "Cloud storage service," "Data backup service")

[0635] Output: Product information displayed in a user interface

[0636] Step 5:

[0637] The server generates the necessary interview items and provision conditions based on the selected product information. Specifically, it uses an algorithm to generate related interview items (e.g., "storage capacity," "access frequency," and "security requirements") based on the selected product (e.g., "cloud storage service").

[0638] Input: Selected product information (e.g., "Cloud storage service")

[0639] Output: Interview items and provision conditions (e.g., "storage capacity," "access frequency," "security requirements")

[0640] Step 6:

[0641] The terminal displays the generated hearing items and provision conditions on the sales representative's screen. Specifically, this information is updated on the user interface so that the sales representative can visually confirm it.

[0642] Input: Interview items and provision conditions (e.g., "storage capacity," "access frequency," "security requirements")

[0643] Output: Hearing items and provision conditions displayed on the user interface

[0644] Step 7:

[0645] The server generates additional questions and confirmation items based on the interview content during the business negotiation. Specifically, it analyzes the collected interview data (e.g., "1 TB of storage space is required") and generates corresponding additional questions (e.g., "Please confirm the access frequency").

[0646] Input: Interview details (e.g., "1TB of storage space required")

[0647] Output: Additional questions and confirmations (e.g., "Please confirm access frequency")

[0648] Step 8:

[0649] The server automatically creates minutes based on the content of the business negotiations. Specifically, it uses an algorithm that aggregates collected business negotiation data and generates minutes that include proposed products, interview details, and additional confirmation items.

[0650] Input: Negotiation data (e.g., proposed products, interview details, additional confirmation items)

[0651] Output: Auto-generated meeting minutes

[0652] Step 9:

[0653] The terminal provides the minutes created by the server to the sales representative, who can then check and modify the contents. Specifically, the minutes are displayed on a user interface, allowing the sales representative to make modifications.

[0654] Input: Auto-generated meeting minutes

[0655] Output: Meeting minutes reviewed and revised by sales representative

[0656] Step 10:

[0657] The user (sales representative) finally shares the minutes that have been confirmed and corrected with the customer. Specifically, the user exports the corrected minutes in PDF format or other format and sends them to the customer via email or a shared link.

[0658] Input: Confirmed and corrected minutes

[0659] Output: Meeting minutes shared with customer

[0660] This detailed process step set clarifies the overall flow of the system and the specific operations of each step, allowing sales representatives to efficiently collect, confirm, and share information during sales negotiations.

[0661] (Application example 1)

[0662] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0663] With conventional sales negotiation support systems, it took a lot of time and effort for sales representatives to quickly understand customer needs during negotiations and propose appropriate products. In addition, the process of recording the content of sales negotiations and creating minutes was often done manually, which increased the risk of information leaks and misunderstandings. Furthermore, it was not possible to present relevant information in real time during negotiations, which reduced the efficiency of sales negotiations.

[0664] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0665] In this invention, the server includes: a voice recognition means for recognizing voice in real time; a keyword extraction means for extracting keywords from voice data recognized by the voice recognition means; a search means for searching for related product information based on keywords extracted by the keyword extraction means; a display means for displaying the product information searched by the search means; a hearing support means for generating necessary hearing items and provision conditions based on the product information; a hearing display means for displaying the generated hearing items and provision conditions; an additional question generation means for generating additional questions and confirmation items based on the hearing content during the business negotiation; a minutes generation means for automatically creating minutes based on the business negotiation content; a confirmation means for displaying the minutes created by the minutes generation means and confirming and correcting them; a sharing means for sharing the confirmed and corrected minutes with the customer; a voice data conversion means for capturing voice during the business negotiation using a voice recognition device built into the smart device and converting it into text data in real time; and a visual display means for visually displaying the voice data during the business negotiation and related information using the smart device. This makes it possible to quickly grasp customer needs during sales negotiations and provide appropriate product information and related services in real time.Furthermore, by recording sales negotiations and automatically generating minutes, it is possible to prevent information leaks and misunderstandings, improving the efficiency and accuracy of sales negotiations.

[0666] The "voice recognition means" is a device that recognizes voices spoken during negotiations in real time and converts them into text data.

[0667] The "keyword extraction means" is a means for extracting important keywords from the text data recognized by the voice recognition means.

[0668] The "search means" is a device that searches a database for related product information based on the extracted keywords.

[0669] The "display means" is a device that displays the product information retrieved by the search means on the screen of the salesperson.

[0670] The "hearing support means" is a means for generating necessary hearing items and provision conditions based on product information.

[0671] The "hearing display means" is a device that displays the generated hearing items and provision conditions on the screen of the sales representative.

[0672] The "additional question generation means" is a means for generating additional questions or confirmation items based on the contents of the hearing during the business negotiation.

[0673] The "minutes generating means" is a device that automatically creates minutes based on the contents of the business negotiations.

[0674] The "checking means" is a device that displays the minutes created by the minutes creating means and allows confirmation and correction.

[0675] The "sharing means" is a device for sharing the confirmed and corrected minutes with the client.

[0676] The "voice data conversion means" is a means for capturing voices during business negotiations using a voice recognition device built into the smart device and converting them into text data in real time.

[0677] The "visual display means" is a device that visually displays voice data and related information during negotiations using a smart device.

[0678] The present invention relates to a business negotiation support system for a brick-and-mortar store, and in particular to a system that improves the efficiency and accuracy of business negotiations by using smart devices. Specific embodiments of the present invention are described below.

[0679] System configuration

[0680] The system includes a speech recognition means, a keyword extraction means, a search means, a display means, a hearing support means, a hearing display means, a follow-up question generation means, a minutes generation means, a confirmation means, a sharing means, a speech data conversion means, and a visual display means.

[0681] Voice recognition means

[0682] The smart device's built-in microphone captures the voice during the transaction and converts the voice into text data in real time using voice recognition software (e.g., Google Speech-to-Text API).

[0683] Keyword extraction method

[0684] The server extracts important keywords from the text data recognized by the speech recognition means, using natural language processing techniques.

[0685] Search methods

[0686] The server searches a database for relevant product information based on the extracted keywords, using a database management system such as MySQL.

[0687] Display means

[0688] The smart device (e.g., smart glasses) displays the product information sent from the server in the salesperson's field of vision, allowing the salesperson to recommend the most suitable product to the customer.

[0689] Hearing support measures

[0690] The server generates necessary interview items and provision conditions based on the selected product information. The generated interview items are used to collect necessary information from customers when proposing products.

[0691] Hearing display means

[0692] The smart device displays the generated inquiry items and offer conditions in the field of view of the salesperson, allowing the salesperson to efficiently ask questions to the customer.

[0693] Additional question generation means

[0694] The server generates follow-up questions and confirmations based on the conversations that take place during the sales meeting, a process that is dynamic based on the customer's responses.

[0695] Minutes generation method

[0696] The server automatically creates minutes based on the content of the business negotiations, including proposed products, interview details, and additional confirmation items.

[0697] Verification method

[0698] The smart device displays the generated minutes in the salesperson's field of view, allowing the salesperson to review and modify the contents.

[0699] means of sharing

[0700] The sales representative will then share the confirmed and revised minutes with the customer, which will help clarify the details of the negotiation and the next meeting.

[0701] Audio data conversion means

[0702] The smart device's built-in voice recognition device captures speech during sales negotiations and converts it into text data in real time. For example, if a salesperson says, "I want a new smartphone," the speech is converted into text data saying, "I want a new smartphone."

[0703] Visual display means

[0704] Smart devices are used to visually display voice data and related information during sales negotiations, allowing salespeople to respond quickly based on visual information.

[0705] Specific examples

[0706] For example, if a salesperson is in a sales meeting and thinks, "I want to recommend the latest smartphone to the customer," they can input the following prompt into the generative AI model:

[0707] User: "I want a new phone"

[0708] System: "Finding information about your new phone…"

[0709] System: "The latest smartphones: Cutting-edge smartphone A, feature-packed smartphone B, affordable smartphone C — choose the product you want."

[0710] Salesperson: (chooses from options displayed in his field of view through smart glasses)

[0711] System: "Would you like to see additional details about the selected smartphone (specs, price) and information about the items we asked you to consider (budget, purpose of use, etc.)?"

[0712] Salesperson: "Yes"

[0713] System: "Displaying hearing items…"

[0714] This prompt sentence allows the salesperson to instantly obtain information to propose to the customer, and to effectively advance the sales negotiations.

[0715] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0716] Step 1:

[0717] The microphone in the smart device (terminal) captures the voice during the sales negotiation and sends the data to speech recognition software (e.g., Google Speech-to-Text API). The input is voice data, and the output is text data. Specifically, if a salesperson says, "I want a new smartphone," the voice data is converted into text data saying, "I want a new smartphone."

[0718] Step 2:

[0719] The server analyzes the text data received from the voice recognition software and extracts important keywords using natural language processing technology. The input is text data and the output is keywords. Specifically, the keyword "new smartphone" is extracted from the text data "I want a new smartphone."

[0720] Step 3:

[0721] The server searches a database (e.g., MySQL) for relevant product information based on the extracted keywords. The input is the keywords, and the output is the product information. Specifically, for the keyword "new smartphone," multiple smartphone models are generated as search results.

[0722] Step 4:

[0723] The server sends the search results to the smart device, and the search results are displayed in the salesperson's field of view by the display means of the smart device. The input is product information, and the output is display data. Specifically, a list of new smartphone models is displayed on the screen.

[0724] Step 5:

[0725] The salesperson (user) selects the product to be proposed from the options displayed in the field of view of the smart device. The selected product information is sent to the server. The input is the user's selection information, and the output is the selected product information.

[0726] Step 6:

[0727] The server generates the necessary questions and conditions for providing the product based on the selected product information. The input is the selected product information, and the output is the questions and conditions for providing the product. Specifically, questions about the selected smartphone model (e.g., desired storage capacity, color, etc.) are generated.

[0728] Step 7:

[0729] The smart device displays the generated hearing items and provision conditions in the field of view of the salesperson. The input is the hearing items and provision conditions, and the output is display data. Specifically, a list of questions for the hearing items is displayed on the screen.

[0730] Step 8:

[0731] The salesperson (user) asks questions to the customer based on the hearing items displayed in the field of view and collects the necessary information. The collected information is sent to the server. The input is the collected information, and the output is the hearing content data.

[0732] Step 9:

[0733] The server generates additional questions and confirmation items based on the interview content. The input is the interview content data, and the output is the additional questions and confirmation items. Specifically, if the answer is "1 TB of capacity is required," the server generates an additional question such as "Please confirm the access frequency."

[0734] Step 10:

[0735] The server records the details of the business negotiations and automatically creates minutes. The input is business negotiation data, and the output is minutes data. Specifically, the minutes include proposed products, interview details, additional confirmation items, etc.

[0736] Step 11:

[0737] The smart device displays the generated minutes in the field of view of the salesperson, who then checks and modifies the contents. The input is the minutes data, and the output is the checked and modified minutes data.

[0738] Step 12:

[0739] The sales representative (user) shares the confirmed and corrected minutes with the customer. The input is the confirmed and corrected minutes data, and the output is the minutes shared with the customer. Specifically, the final confirmed minutes are sent to the customer via email or other means.

[0740] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0741] The present invention relates to a business negotiation support system, and in particular to a system that recognizes voice in real time during business negotiations, extracts important keywords, presents related product information, automatically creates business negotiation records and minutes, and also recognizes the user's emotions and makes appropriate suggestions based on them.

[0742] This system mainly includes the following elements: a speech recognition means, a keyword extraction means, a search means, a display means, a hearing support means, a hearing display means, a follow-up question generation means, a minutes generation means, a confirmation means, a sharing means, and an emotion engine.

[0743] System Overview

[0744] Voice recognition means

[0745] The device uses a microphone to capture real-time audio during negotiations and converts it into text data using voice recognition software.

[0746] Keyword extraction method

[0747] The server extracts important keywords from the text data sent from the terminal, and the extracted keywords are used to identify product information related to the business negotiation.

[0748] Search methods

[0749] The server searches a database for relevant product information based on the extracted keywords, and provides candidate product information and related services as search results.

[0750] Display means

[0751] The terminal displays the product information and related services sent from the server on the salesperson's screen, allowing the salesperson to propose the most suitable product during the sales negotiation.

[0752] Hearing support measures

[0753] The server generates the necessary interview items and conditions for providing the product based on the selected product information, allowing the salesperson to ask the customer appropriate questions and collect the necessary information.

[0754] Hearing display means

[0755] The terminal displays the generated inquiry items and terms of service on the sales representative's screen, allowing the sales representative to smoothly proceed with the business negotiations.

[0756] Additional question generation means

[0757] The server generates follow-up questions and confirmations based on the conversations that take place during the sales meeting, allowing salespeople to gather the necessary details during the sales meeting.

[0758] Minutes generation method

[0759] The server automatically creates minutes based on the content of the business negotiations, and the minutes are provided to the sales representative at the end of the negotiations.

[0760] Verification method

[0761] The terminal displays the generated minutes to the sales representative, allowing the sales representative to check and correct the contents.

[0762] means of sharing

[0763] The user (sales representative) can then share the confirmed and revised minutes with the customer, allowing them to clearly communicate the details of the business negotiations and the details of the next meeting.

[0764] Emotion Engine

[0765] The server recognizes the user's emotions from data such as input voice and facial expressions. Based on the recognized emotions, the emotion engine adjusts the content of the product information it suggests, additional questions, and confirmation items.

[0766] Program description and examples

[0767] The program processing of this system will be explained below with specific examples.

[0768] Program processing

[0769] 1. The device uses voice recognition software to recognize speech during sales negotiations in real time. For example, if a salesperson says, "I'd like to know about cloud storage," the device converts this speech into text data that reads, "I'd like to know about cloud storage."

[0770] 2. The server analyzes the text data sent from the device and extracts the key keyword "cloud storage."

[0771] 3. The server searches the database for related products based on the extracted keywords. For example, for the keyword "cloud storage," it generates search results for "cloud storage service" and "data backup service."

[0772] 4. The server analyzes voice and facial expression data during the negotiation and uses an emotion engine to recognize the user's emotions. For example, it adjusts the content of the proposal depending on whether the customer is interested or skeptical.

[0773] 5. The server tailors the product information suggestions and follow-up questions based on the perceived sentiment, for example providing more detailed information if the customer expresses positive sentiment and a brief explanation if they are skeptical.

[0774] 6. The terminal displays the product information, related services, and emotion-based adjustment results sent from the server on the salesperson's screen. The salesperson can then select the products to recommend on the screen.

[0775] 7. The server generates the necessary inquiry items (for example, storage capacity, access frequency, security requirements) and supply conditions based on the selected product information.

[0776] 8. The terminal displays the generated interview items and offer conditions on the sales representative's screen. The sales representative then asks the customer questions based on this and collects the necessary information.

[0777] 9. The server generates additional questions and confirmations based on the interview content. For example, if the customer answers "I need 1TB of storage," the server generates an additional question such as "Please confirm the access frequency."

[0778] 10. The server records the details of the business negotiations and automatically creates minutes, which include proposed products, interview details, and additional confirmation items.

[0779] 11. The terminal provides the minutes to the sales representative, who then checks and corrects the contents.

[0780] 12. The user (sales representative) finally shares the confirmed and revised minutes with the customer, which clarifies the details of the negotiation and the next meeting.

[0781] This program streamlines information management during sales negotiations, allowing sales representatives to make quick and appropriate proposals to customers. It also recognizes users' emotions and makes adjustments based on them, leading to increased customer satisfaction.

[0782] The processing flow will be explained below.

[0783] Step 1:

[0784] The device uses a microphone to capture voices during sales negotiations in real time. It then uses voice recognition software to convert the captured voice into text data. For example, if a salesperson says, "I'd like to know about cloud storage," the device converts this voice into text data: "I'd like to know about cloud storage."

[0785] Step 2:

[0786] The terminal transmits the converted text data to the server via the network.

[0787] Step 3:

[0788] The server passes the received text data to a natural language processing engine and extracts important keywords. For example, it extracts "cloud storage" from the text "I want to know about cloud storage."

[0789] Step 4:

[0790] The server searches the database for relevant product information based on the extracted keywords, for example, to retrieve product information related to "cloud storage."

[0791] Step 5:

[0792] The terminal displays the search results (candidate product information and related services) sent from the server on the sales representative's screen, and the sales representative can select the products to suggest from the screen.

[0793] Step 6:

[0794] The server generates the necessary interview items and provision conditions based on the selected product information. For example, in the case of "cloud storage," it generates storage capacity, access frequency, security requirements, etc.

[0795] Step 7:

[0796] The terminal displays the generated interview items and terms of service on the screen of the sales representative, who then asks questions to the customer and collects the necessary information.

[0797] Step 8:

[0798] The terminal transmits the hearing response input by the sales representative or speech-recognized to the server.

[0799] Step 9:

[0800] The server analyzes the received answers and generates additional questions and confirmations. For example, in response to the answer "1 TB of storage space is required," the server adds a question such as "Please confirm the access frequency."

[0801] Step 10:

[0802] The terminal displays the generated additional questions and confirmation items on the screen of the sales representative, who then asks the customer further questions based on the displayed questions.

[0803] Step 11:

[0804] The server inputs voice and facial expression data from the business negotiation into an emotion engine to recognize the user's emotions.

[0805] Step 12:

[0806] The server then adjusts the product information and follow-up questions displayed based on the recognized emotion data, for example, providing detailed information if the customer expresses positive emotions, or providing a brief explanation if the customer is skeptical.

[0807] Step 13:

[0808] The server automatically creates minutes based on the details of the business negotiations, including proposed products, interview details, and additional confirmation items.

[0809] Step 14:

[0810] The terminal displays the generated minutes on the screen of the sales representative, allowing the sales representative to check and correct the contents.

[0811] Step 15:

[0812] The user (sales representative) shares the confirmed and corrected minutes with the customer, for example, by sending them by email to notify the customer of the contents of the minutes.

[0813] These steps enable the sales support system to manage information in real time, allowing sales representatives to make prompt and appropriate proposals to customers. Furthermore, by recognizing users' emotions and making adjustments based on them, it is possible to build trust with customers and increase the success rate of sales negotiations.

[0814] Example 2

[0815] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0816] While conventional sales negotiation support systems are effective in automatically recording sales negotiation content and proposing products, they lack the ability to recognize emotions in real time and adjust proposal content based on that, making it difficult to respond flexibly to customer emotions.In addition, there was a lack of means to improve the accuracy of sales negotiations while reducing the burden on sales representatives, such as generating follow-up questions needed during sales negotiations or automatically creating minutes.

[0817] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0818] In this invention, the server includes: a voice recognition means for recognizing input voice in real time; a keyword extraction means for extracting keywords from voice data recognized by the voice recognition means; a search means for searching for related product information based on keywords extracted by the keyword extraction means; a display means for displaying the product information searched by the search means; a hearing support means for generating necessary hearing items and provision conditions based on the product information; a hearing display means for displaying the generated hearing items and provision conditions; an additional question generation means for generating additional questions and confirmation items based on the hearing content during the business negotiation; a minutes generation means for automatically creating minutes based on the business negotiation content; a confirmation means for displaying the minutes created by the minutes generation means and confirming and correcting them; a sharing means for sharing the confirmed and corrected minutes with the customer; and an emotion recognition means for recognizing the user's emotions from the input voice and facial expression data and adjusting the provided product information and provision conditions based thereon. This makes it possible to efficiently perform real-time voice recognition and keyword extraction, related product searches, adjustment of proposal content based on emotion recognition, recording of business negotiations, and automatic generation of meeting minutes all within a single system.

[0819] The "voice recognition means" is a means for capturing input voice in real time and converting it into text data.

[0820] The "keyword extraction means" is a means for extracting important keywords from the voice data recognized by the voice recognition means.

[0821] The "search means" is a means for searching a database for related product information and service information based on the keywords extracted by the keyword extraction means.

[0822] The "display means" is a means for displaying product information and service information retrieved by the search means on the screen of the sales representative.

[0823] The "hearing support means" is a means for generating hearing items and provision conditions necessary for business negotiations based on the retrieved product information and service information.

[0824] The "hearing display means" is a means for displaying the generated hearing items and provision conditions on the screen of the sales representative.

[0825] The "additional question generation means" is a means for generating additional questions or confirmation items based on the contents of the interview obtained during the business negotiation.

[0826] The "minutes generating means" is a means for automatically creating minutes based on the contents of the business negotiation.

[0827] The "confirmation means" is a means for displaying the minutes created by the minutes creation means, and for the sales representative to confirm and correct the contents.

[0828] "Sharing means" means the means by which the confirmed and amended minutes are shared with the client.

[0829] The "emotion recognition means" is a means for recognizing the user's emotions from input voice and facial expression data, and adjusting the product information and terms of provision based on the emotions.

[0830] The present invention relates to a business negotiation support system that recognizes speech during business negotiations in real time, extracts important keywords, presents related product information, automatically records business negotiations and creates minutes, and also recognizes the user's emotions and makes appropriate suggestions based on those emotions. The specific configuration and operation of the system are described below.

[0831] Voice recognition means

[0832] The device uses a microphone to capture voices during sales negotiations in real time and converts them into text data using voice recognition software (e.g., Google's voice recognition API). For example, if a salesperson says, "I'd like to know about cloud storage," the device converts this voice into text data saying, "I'd like to know about cloud storage."

[0833] Keyword extraction method

[0834] The server receives the text data sent from the device and uses a natural language processing (NLP) tool (e.g., Python's NLTK library) to extract important keywords from this text data. For example, it extracts the keyword "cloud storage."

[0835] Search methods

[0836] The server searches a database (e.g., MySQL) for relevant product information based on the extracted keywords. For example, based on the keyword "cloud storage," it might generate search results such as "cloud storage service" and "data backup service."

[0837] emotion recognition means

[0838] The server analyzes voice and facial expression data and recognizes the user's emotions using an emotion engine (e.g., a deep learning model using Google's TensorFlow). For example, it adjusts its suggestions to provide detailed information if the customer expresses positive emotions, or a concise explanation if the customer expresses skepticism.

[0839] Adjusting the proposal

[0840] The server then tailors product suggestions and follow-up questions based on the perceived emotion, for example providing detailed information if the customer expresses positive emotion, or providing a clearer explanation if the customer is skeptical.

[0841] Displaying product information

[0842] The terminal displays product information, related services, and emotion-based adjustment results sent from the server on the salesperson's screen in real time, allowing the salesperson to use this information to recommend the most suitable product to the customer.

[0843] Hearing support

[0844] The server generates the necessary interview items (e.g., storage capacity, purpose of use) and terms of provision based on the selected product information.

[0845] Display of hearing items

[0846] The terminal displays the generated inquiry items and provision conditions on the sales representative's screen, allowing the sales representative to use this information to inquire about the necessary information from the customer.

[0847] Generate follow-up questions

[0848] The server generates additional questions and confirmations based on the information gathered during the sales negotiation. For example, if the customer answers, "I need 1 TB of storage space," the server generates additional questions such as, "Please confirm the access frequency."

[0849] Automatic generation of meeting minutes

[0850] The server records the details of the business negotiations and automatically creates minutes based on them, including proposed products, interview details, and additional confirmation items.

[0851] Checking the minutes

[0852] The terminal displays the generated minutes on the screen of the sales representative, allowing the sales representative to check and correct the contents.

[0853] Sharing meeting minutes

[0854] The user (sales representative) finally shares the confirmed and revised minutes with the customer. This can be done via an email system or a cloud storage service (e.g., Google Drive or Dropbox).

[0855] The program streamlines information management during sales negotiations, enabling salespeople to make quick and appropriate proposals to customers. It also recognizes users' emotions and adjusts accordingly, contributing to increased customer satisfaction.

[0856] Prompt Sentence Examples

[0857] Example prompts for inputting specific information into a generative AI model:

[0858] If a customer says "I want to know about cloud storage" during a sales meeting, explain how the system works. Include specific software and database names, and provide step-by-step details.

[0859] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0860] Step 1:

[0861] The device uses a microphone to capture voices during negotiations in real time and converts the input voice into text data using voice recognition software (e.g., Google's voice recognition API). The input is the voices being spoken during negotiations, and the output is the data converted from voice into text. Specifically, the device starts recording voices as soon as the negotiations begin, and sequentially sends the data to the voice recognition API to receive the text data.

[0862] Step 2:

[0863] The server receives the text data sent from the terminal and extracts important keywords using a natural language processing (NLP) tool (e.g., Python's NLTK library). The input is the text data, and the output is the extracted important keywords. The text data is subjected to morphological analysis, and keywords related to the pre-set business negotiations are extracted from it.

[0864] Step 3:

[0865] The server searches a database (e.g., MySQL) for relevant product information based on the extracted keywords. The input is the extracted keywords, and the output is the search result for related product information. A MySQL query is generated and executed to retrieve product information that matches the keywords.

[0866] Step 4:

[0867] The server analyzes voice data and facial expression data and recognizes the user's emotions using an emotion engine (e.g., a deep learning model using TensorFlow). The input is voice data and facial expression data, and the output is the recognized emotional information. Emotions are analyzed from the tone of voice and facial expressions, and the results are classified using an emotion model.

[0868] Step 5:

[0869] The server adjusts the product information suggestions and follow-up questions based on the recognized emotion. The input is emotion information and product information, and the output is the adjusted suggestions and question list. For example, if the emotion is positive, detailed information is provided, and if the emotion is skeptical, brief information is selected.

[0870] Step 6:

[0871] The terminal displays the product information and adjustment results sent from the server on the salesperson's screen. The input is the adjusted proposal content and question list, and the output is the information displayed on the salesperson's screen. The information is updated and displayed on the salesperson's display in real time.

[0872] Step 7:

[0873] The server generates the necessary interview items and provision conditions based on the selected product information. The input is the selected product information, and the output is the interview items and provision conditions. Based on the product information, the server selects appropriate interview items from a template and generates the provision conditions.

[0874] Step 8:

[0875] The terminal displays the generated inquiry items and provision conditions on the sales representative's screen. The input is the generated inquiry items and provision conditions, and the output is the information displayed on the sales representative's screen. This allows the sales representative to ask questions to the customer according to the inquiry items.

[0876] Step 9:

[0877] The server generates additional questions and confirmations based on the information gathered during the negotiation. The input is the information gathered during the negotiation, and the output is the additional questions and confirmations. For example, in response to the answer "1 TB of storage is required," the server generates an additional question such as "Please confirm the access frequency."

[0878] Step 10:

[0879] The server records the content of the business negotiations and automatically creates minutes based on that. The input is the text data of the business negotiations, and the output is the minutes. The minutes are created by dividing the business negotiation text into paragraphs and extracting important points and questions and answers.

[0880] Step 11:

[0881] The terminal displays the generated minutes on the sales representative's screen, allowing the sales representative to check and correct the contents. The input is the generated minutes, and the output is the checked and corrected minutes. The sales representative checks the displayed minutes and makes corrections as necessary.

[0882] Step 12:

[0883] The user (sales representative) finally shares the confirmed and revised minutes with the customer. The input is the revised minutes, and the output is the minutes shared with the customer. The sharing method is an email transmission system or cloud storage service.

[0884] This series of processes makes information management during negotiations more efficient, enabling sales representatives to make appropriate proposals to customers.

[0885] (Application example 2)

[0886] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0887] Conventional sales negotiation support systems can record the content of sales negotiations in real time and extract important keywords to present related information, but they have difficulty making accurate proposals based on customer sentiment. Furthermore, because they do not support automatic recording of sales negotiation content or automatic creation of meeting minutes, they have the problem of consuming a large amount of resources after the negotiation. This reduces the work efficiency of sales representatives and does not necessarily result in optimal sales negotiation outcomes.

[0888] The specification processing by the specification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: a speech recognition unit that recognizes input speech in real time; a keyword extraction unit that extracts keywords from speech data recognized by the speech recognition unit; a search unit that searches for related product information based on the keywords extracted by the keyword extraction unit; an emotion recognition unit that recognizes the customer's emotions and adjusts the proposal content based on the emotions; a display unit that displays the product information searched by the search unit; a hearing support unit that generates necessary hearing items and provision conditions based on the product information; a hearing display unit that displays the generated hearing items and provision conditions; a follow-up question generation unit that generates follow-up questions and confirmation items based on the hearing content during the business negotiation; a minutes generation unit that automatically creates minutes based on the business negotiation content; a confirmation unit that displays the minutes created by the minutes generation unit and confirms and modifies them; and a sharing unit that shares the confirmed and modified minutes with the customer. This enables efficient management of information during the business negotiation and optimal proposals based on the customer's emotions.

[0889] The "voice recognition means" is a means for recognizing input voice in real time and converting it into text data.

[0890] The "keyword extraction means" is a means for extracting important keywords from the voice data recognized by the voice recognition means.

[0891] The "search means" is a means for searching for related product information based on the keywords extracted by the keyword extraction means.

[0892] The "emotion recognition means" is a means for recognizing emotions from the customer's voice and facial expressions, and adjusting the content of the proposal based on those emotions.

[0893] The "display means" is a means for displaying product information and related services searched for by the search means, and is a means for visually presenting necessary information.

[0894] The "hearing support means" is a means for generating necessary hearing items and provision conditions based on product information.

[0895] The "hearing display means" is a means for displaying the generated hearing items and provision conditions.

[0896] The "additional question generation means" is a means for generating additional questions or confirmation items based on the contents of the hearing during the business negotiation.

[0897] The "minutes generation means" is a means for automatically creating minutes based on the contents of the business negotiations.

[0898] The "checking means" is a means for displaying the minutes created by the minutes creating means and for checking and correcting them.

[0899] "Sharing means" means the means by which the confirmed and amended minutes are shared with the client.

[0900] "Product information" is detailed information about products and services related to the content of the business negotiations.

[0901] "Emotion recognition" is a technology that analyzes and recognizes a customer's emotional state from their voice and facial expressions.

[0902] A "business negotiation" is a conversation or negotiation between a sales representative and a customer regarding a proposed product or service.

[0903] This invention realizes a shopping assistant system for brick-and-mortar stores, specifically, a system that uses smart glasses or an application installed on a smartphone to assist store clerks in conversations with customers. This system uses the following hardware and software:

[0904] Hardware and Software

[0905] 1. Hardware:

[0906] Smart glasses or smartphone: A device equipped with a microphone and camera.

[0907] 2. Software:

[0908] Speech recognition software (such as Google Cloud Speech-to-Text API)

[0909] Keyword extraction model (using TensorFlow, etc.)

[0910] Emotion recognition engine (e.g. NVIDIA Clara AI)

[0911] Database (e.g. Firebase Realtime Database)

[0912] System Procedures

[0913] Voice Capture and Recognition

[0914] The device uses the microphones in smart glasses or smartphones to capture customer interactions in real time, and the captured voice data is converted into text using the Google Cloud Speech-to-Text API.

[0915] Keyword extraction

[0916] The server extracts important keywords from the text data generated by the speech recognition means, using a keyword extraction model built with TensorFlow.

[0917] Searching and displaying product information

[0918] The server searches for relevant product information from a database such as Firebase based on the keywords extracted by the keyword extraction means. The searched product information is displayed on the terminal screen, allowing the store clerk to quickly provide appropriate product information.

[0919] Emotion recognition and suggestion adjustment

[0920] The server analyzes the customer's voice and facial expression data and uses NVIDIA Clara AI technology to recognize the customer's emotions. Based on the results of this emotion recognition, the server adjusts the product information suggestions and follow-up questions it asks. For example, if the customer expresses positive emotions, it provides more detailed product information, while if the customer expresses skepticism, it provides a concise, to-the-point explanation.

[0921] Hearing support and follow-up questions

[0922] The server generates the necessary inquiry items and provision conditions based on the selected product information. These inquiry items and provision conditions are displayed on the terminal screen, and the salesperson asks the customer appropriate questions based on them and collects the necessary information. Furthermore, the server generates additional questions and confirmation items based on the inquiry content during the sales negotiation.

[0923] Generate and review meeting minutes

[0924] The server automatically creates minutes based on the content of the business negotiation. The minutes are then displayed on the terminal screen, allowing the salesperson to check and modify the contents. After this, the minutes are shared with the customer.

[0925] Examples of concrete examples and prompts

[0926] Specific examples

[0927] For example, if a store clerk is talking to a customer and the customer says, "I want to see the most popular 4K TV," the system will recognize the speech and extract the keyword "4K TV." It will then search the database for related product information and display the search results on the store clerk's device. At the same time, it will recognize the customer's interest and suggest more detailed product descriptions. The store clerk can then make the best suggestions to the customer based on the displayed information and any follow-up questions.

[0928] Prompt Sentence Examples

[0929] You are a salesperson. If a customer says, "I want a new 4K TV," here's what you should do:

[0930] 1. Use speech recognition to extract the important keyword "4K TV."

[0931] 2. Search and suggest relevant product information from the database.

[0932] 3. Analyze customer sentiment to determine if they're interested or want to know more.

[0933] 4. View the suggestions and generate and ask follow-up questions as needed.

[0934] 5. Automatically record the contents of business negotiations and create minutes.

[0935] This system makes customer service in physical stores more efficient and effective, allowing store staff to make optimal suggestions based on customer needs. It also automatically records the details of sales negotiations, making post-negotiation management easier.

[0936] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0937] Step 1:

[0938] The terminal uses the microphone of smart glasses or smartphone to capture customer speech in real time, and the input data is the voice signal, which serves as the base data for subsequent processing. The output is the captured voice data.

[0939] Step 2:

[0940] The device converts the captured voice data into text data using speech recognition software (e.g., Google Cloud Speech-to-Text API). This process analyzes the voice signal and generates corresponding text. The input is the voice data acquired in step 1, and the output is text data.

[0941] Step 3:

[0942] The server extracts important keywords from the text data generated by the speech recognition means. This uses a keyword extraction model built with TensorFlow. The input is the text data generated in step 2, and the output is the extracted keywords.

[0943] Step 4:

[0944] The server searches for related product information from a database such as Firebase based on the extracted keywords. The system sends product information containing the keywords as a query to the database and retrieves the corresponding product information. The input is the keywords extracted in step 3, and the output is related product information.

[0945] Step 5:

[0946] The server uses NVIDIA Clara AI to recognize emotions using the customer's voice and facial expression data. Voice data and camera footage are used as inputs to analyze the customer's emotional state. The inputs are voice and video data, and the output is the emotion recognition results.

[0947] Step 6:

[0948] The terminal displays optimal suggestions to the customer based on the acquired product information and emotion recognition results. The system adjusts the suggestions based on the results of emotion analysis and displays them on the screen. The input is the output data from Steps 4 and 5, and the output is the adjusted suggestions.

[0949] Step 7:

[0950] The server generates the necessary interview items and provision conditions based on the selected product information. The interview support means automatically generates questions according to the customer's needs. The input is the output data of step 4, and the output is the interview items and provision conditions.

[0951] Step 8:

[0952] The terminal displays the generated interview items and conditions for provision, and the store clerk asks the customer appropriate questions based on this to collect information. The clerk checks the content displayed on the screen and collects the necessary information from the customer. The input is the output data of step 7, and the output is the collected customer information.

[0953] Step 9:

[0954] The server generates additional questions and confirmations based on the interview content. The system analyzes the answers from the customer and generates additional questions if more detailed information needs to be collected. The input is the customer information collected in step 8, and the output is additional questions and confirmations.

[0955] Step 10:

[0956] The server automatically creates minutes based on the content of the business negotiation. The system compiles all data acquired during the business negotiation and formats it as minutes. The input is all data from step 2 to step 9, and the output is the minutes of the business negotiation.

[0957] Step 11:

[0958] The terminal displays the generated minutes, which the clerk can review and modify. The system reflects the modifications and finalizes the minutes. The input is the minutes created in step 10, and the output is the reviewed and modified minutes.

[0959] Step 12:

[0960] The user (store clerk) shares the final confirmed and revised minutes with the customer. The minutes are provided to the customer and the customer confirms the details of the business negotiation. The input is the minutes confirmed and revised in step 11, and the output is the final minutes shared with the customer.

[0961] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0962] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0963] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0964] [Third embodiment]

[0965] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0966] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0967] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0968] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0969] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0970] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0971] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0972] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0973] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0974] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0975] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0976] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0977] The present invention relates to a business negotiation support system, and in particular to a system that effectively advances business negotiations by recognizing speech in real time during the negotiation, extracting important keywords, presenting related product information, and automatically creating records of the negotiations and minutes.

[0978] This system mainly includes the following elements: a speech recognition means, a keyword extraction means, a search means, a display means, a hearing support means, a hearing display means, a follow-up question generation means, a minutes generation means, a confirmation means, and a sharing means.

[0979] System Overview

[0980] Voice recognition means

[0981] The device uses voice recognition software to recognize speech during negotiations in real time and convert it into text data.

[0982] Keyword extraction method

[0983] The server extracts important keywords from the text data sent from the terminal, and the extracted keywords are used to identify product information related to the business negotiation.

[0984] Search methods

[0985] The server searches a database for relevant product information based on the extracted keywords, and provides candidate product information and related services as search results.

[0986] Display means

[0987] The terminal displays the product information and related services sent from the server on the salesperson's screen, allowing the salesperson to propose the most suitable product during the sales negotiation.

[0988] Hearing support measures

[0989] The server generates the necessary interview items and conditions for providing the product based on the selected product information, allowing the salesperson to ask the customer appropriate questions and collect the necessary information.

[0990] Hearing display means

[0991] The terminal displays the generated inquiry items and terms of service on the sales representative's screen, allowing the sales representative to smoothly proceed with the business negotiations.

[0992] Additional question generation means

[0993] The server generates follow-up questions and confirmations based on the conversations that take place during the sales meeting, allowing salespeople to gather the necessary details during the sales meeting.

[0994] Minutes generation method

[0995] The server automatically creates minutes based on the content of the business negotiations, and the minutes are provided to the sales representative at the end of the negotiations.

[0996] Verification method

[0997] The terminal displays the generated minutes to the sales representative, who then checks and corrects them.

[0998] means of sharing

[0999] The user (sales representative) can then share the confirmed and revised minutes with the customer, allowing them to clearly communicate the details of the business negotiations and the details of the next meeting.

[1000] Program description and examples

[1001] The program processing of this system will be explained below with specific examples.

[1002] Program processing

[1003] 1. The device uses voice recognition software to recognize speech during sales negotiations in real time. For example, if a salesperson says, "I'd like to know about cloud storage," the device converts this speech into text data that reads, "I'd like to know about cloud storage."

[1004] 2. The server analyzes the text data sent from the device and extracts the key keyword "cloud storage."

[1005] 3. The server searches the database for related products based on the extracted keywords. For example, for the keyword "cloud storage," it generates search results for "cloud storage service" and "data backup service."

[1006] 4. The terminal displays the search results sent from the server on the sales representative's screen, where the sales representative can select the products to suggest.

[1007] 5. The server generates the necessary inquiry items (e.g., storage capacity, access frequency, security requirements) and provision conditions based on the selected product information.

[1008] 6. The terminal displays the generated interview items and offer conditions on the sales representative's screen. The sales representative asks questions to the customer based on these items and collects the necessary information.

[1009] 7. The server generates additional questions and confirmations based on the interview content. For example, if the customer answers "I need 1TB of storage," the server generates an additional question such as "Please confirm the access frequency."

[1010] 8. The server records the details of the business negotiations and automatically creates minutes, which include proposed products, interview details, and additional confirmation items.

[1011] 9. The terminal provides the minutes to the sales representative, who then checks and corrects the contents.

[1012] 10. The user (sales representative) finally shares the confirmed and revised minutes with the customer, which clarifies the details of the negotiation and the next meeting.

[1013] This program streamlines information management during sales negotiations, allowing sales representatives to make prompt and appropriate proposals to customers. In addition, sales negotiation records and minutes are automatically generated, preventing information leaks and misunderstandings.

[1014] The processing flow will be explained below.

[1015] Step 1:

[1016] The device uses a microphone to capture speech during a business meeting and uses voice recognition software to convert the captured speech into text data in real time.

[1017] Step 2:

[1018] The terminal transmits the converted text data to the server via the network.

[1019] Step 3:

[1020] The server passes the received text data to a natural language processing engine and extracts important keywords. For example, it extracts "cloud storage" from the text "I want to know about cloud storage."

[1021] Step 4:

[1022] The server searches the database for relevant product information based on the extracted keywords, for example, to retrieve product information related to "cloud storage."

[1023] Step 5:

[1024] The server transmits the plurality of pieces of product information obtained as search results to the terminal.

[1025] Step 6:

[1026] The terminal displays the obtained product information and related services on the sales representative's screen, and the sales representative can select the products to suggest from the screen.

[1027] Step 7:

[1028] The server generates the necessary interview items and provision conditions based on the selected product information. For example, in the case of "cloud storage," it generates storage capacity, access frequency, security requirements, etc.

[1029] Step 8:

[1030] The terminal displays the generated interview items and terms of service on the screen of the sales representative, who then asks questions to the customer and collects the necessary information.

[1031] Step 9:

[1032] The terminal transmits the hearing response input by the sales representative or speech-recognized to the server.

[1033] Step 10:

[1034] The server analyzes the received answers and generates additional questions and confirmations. For example, in response to the answer "1 TB of storage space is required," the server adds a question such as "Please confirm the access frequency."

[1035] Step 11:

[1036] The terminal displays the generated additional questions and confirmation items on the screen of the sales representative, who then asks the customer further questions based on the displayed questions.

[1037] Step 12:

[1038] The server automatically creates minutes based on the details of the business negotiations, including proposed products, interview details, and additional confirmation items.

[1039] Step 13:

[1040] The terminal displays the generated minutes on the screen of the sales representative, allowing the sales representative to check and correct the contents.

[1041] Step 14:

[1042] The user (sales representative) shares the confirmed and corrected minutes with the customer, for example, by sending them by email to notify the customer of the contents of the minutes.

[1043] These steps enable real-time support for business negotiations, enabling efficient and accurate proposals and information sharing.

[1044] Example 1

[1045] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1046] In modern business negotiations, salespeople need to make quick and appropriate product proposals in real time and conduct interviews based on customer needs. However, it is difficult to instantly collect and record a large amount of information during a negotiation, and it also takes a lot of time to review the information later and create minutes. This places a heavy burden on salespeople, making it difficult to conduct negotiations efficiently.

[1047] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1048] In this invention, the server includes a speech recognition means for recognizing input speech in real time, a keyword extraction means for extracting keywords from speech data recognized by the speech recognition means, a search means for searching for related product information based on the keywords extracted by the keyword extraction means, a display means for displaying the product information searched by the search means, a hearing support means for generating necessary hearing items and provision conditions based on the product information, a hearing display means for displaying the generated hearing items and provision conditions, an additional question generation means for generating additional questions and confirmation items based on the hearing contents during the business negotiation, a minutes generation means for automatically creating minutes based on the contents of the business negotiation, and a display means for displaying the minutes created by the minutes generation means. The system includes a confirmation means for displaying minutes and confirming and correcting them, a sharing means for sharing the confirmed and corrected minutes with the customer, a database search means for identifying important keywords from the collected voice data and retrieving related product information from a database based on the keywords, a selection means for displaying the product information retrieved by the database search means on the sales representative's terminal and generating hearing items based on the selected product, a follow-up question generation means and a confirmation item generation means for generating follow-up questions based on the generated hearing items, a confirmation and correction means for automatically generating minutes at the end of the business negotiation and allowing the sales representative to confirm and correct the contents, and a sharing means for sharing the final confirmed and corrected minutes with the customer. This allows for efficient information gathering during the business negotiation, preventing oversight of results, and additional information confirmation, thereby reducing the burden on the sales representative and enabling the business negotiation to proceed more effectively.

[1049] The "voice recognition means" is a device or software that has the function of recognizing input voice in real time and converting voice data into text data.

[1050] The "keyword extraction means" is a device or software that has the function of extracting important keywords from the voice data recognized by the voice recognition means.

[1051] The "search means" is a device or software having a function of searching a database for related product information based on the keywords extracted by the keyword extraction means.

[1052] The "display means" is a device or software that has the function of displaying the product information acquired by the search means on the user's terminal.

[1053] The "hearing support means" is a device or software that has the function of generating necessary hearing items and provision conditions based on product information.

[1054] The "hearing display means" is a device or software that has the function of displaying the generated hearing items and provision conditions on the user's terminal.

[1055] The "additional question generating means" is a device or software that has the function of generating additional questions or confirmation items based on the content of hearings during business negotiations.

[1056] The "minutes generating means" is a device or software that has the function of automatically creating minutes based on the contents of the business negotiation.

[1057] The "checking means" is a device or software having a function of displaying the minutes created by the minutes creating means on the user's terminal and allowing the user to check and correct the contents.

[1058] "Sharing means" means a device or software that has the function of sharing the confirmed and corrected minutes with the client.

[1059] The "database search means" is a device or software that has the function of identifying important keywords from the collected voice data and retrieving related product information from a database based on those keywords.

[1060] The "selection means" is a device or software having the function of displaying the product information acquired by the database search means on the terminal and generating hearing items based on the selected product.

[1061] The "additional question generating means and confirmation item generating means" refers to a device or software having the function of generating additional questions and items to be confirmed based on the generated hearing items.

[1062] The "verification and correction means" is a device or software that has the function of allowing a user to verify the minutes that are automatically generated at the end of a business meeting and correct them as necessary.

[1063] "Sharing means" refers to a device or software that has the function of sharing the final confirmed and corrected minutes with the client.

[1064] The present invention relates to a sales negotiation support system, and in particular to a system that recognizes speech in real time during a sales negotiation, extracts important keywords, presents related product information, and automatically creates a record of the sales negotiation and minutes, thereby promoting the efficient progress of the sales negotiation and reducing the burden on sales representatives.

[1065] System configuration

[1066] The business negotiation support system of the present invention uses the following hardware and software.

[1067] 1. Devices: laptops, tablets, smartphones, etc.

[1068] 2. Speech recognition software: Google Cloud Speech-to-Text, IBM Watson Speech to Text, etc.

[1069] 3. Server: Cloud server (e.g., Amazon Web Services, Google Cloud Platform)

[1070] 4. Database: MySQL, PostgreSQL, etc.

[1071] Explanation of program processing

[1072] 1. The device uses voice recognition software to recognize speech during negotiations in real time and convert it into text data. For example, if a sales representative says, "I'd like to know about cloud storage," the speech is converted into text data that reads, "I'd like to know about cloud storage."

[1073] 2. The server receives the text data sent from the device and extracts important keywords from it. For example, the phrase "cloud storage" is extracted as a keyword. This is done using natural language processing technology (e.g., spaCy).

[1074] 3. The server searches the database for relevant product information based on the extracted keywords. For example, for the keyword "cloud storage," "cloud storage service" and "data backup service" are generated as search results.

[1075] 4. The terminal displays the search results for the product information and related services sent from the server on the sales representative's screen. The sales representative can check the product information displayed on the screen and select the products to suggest.

[1076] 5. The server generates the necessary interview items and terms of service based on the selected product information. For example, in the case of a cloud storage service, the server generates interview items such as storage capacity, access frequency, and security requirements.

[1077] 6. The terminal displays the generated interview items and terms of service on the sales representative's screen. The sales representative uses this information to ask questions of the customer and collect the necessary information.

[1078] 7. The server generates additional questions and confirmations based on the information gathered during the sales negotiation. For example, if the customer answers, "I need 1TB of storage space," the server generates an additional question, such as, "Please confirm the access frequency."

[1079] 8. The server automatically creates minutes based on the details of the business negotiations. The minutes include proposed products, interview details, and additional confirmation items.

[1080] 9. The terminal provides the minutes created by the server to the sales representative, who can then check and modify the contents.

[1081] 10. The user (sales representative) finally shares the confirmed and revised minutes with the customer, which clarifies the details of the negotiation and the next meeting.

[1082] Examples of concrete examples and prompts

[1083] Below are examples of specific actions and examples of input prompts to the generative AI model.

[1084] Specific examples of operation

[1085] If a salesperson says "I'd like to know about cloud storage" during a sales meeting, the voice recognition software converts this speech into text data, and the server extracts the keyword "cloud storage." The server then searches the database for relevant product information and displays the search results on the device. Based on this information, the salesperson can ask the customer appropriate questions, and the system automatically generates any necessary follow-up questions and meeting minutes.

[1086] Example of input prompt for generative AI model

[1087] Please explain how the sales support system works. Please provide a detailed explanation of the process from when a sales representative says, "I'd like to know about cloud storage," to when the meeting minutes are shared with the customer.

[1088] This sales negotiation support system streamlines information management during sales negotiations, allowing sales representatives to make prompt and appropriate proposals. In addition, sales negotiation records and minutes are automatically generated, preventing information leaks and misunderstandings.

[1089] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1090] Step 1:

[1091] The device uses voice recognition software to recognize speech during sales negotiations in real time and convert it into text data. Specifically, the device uses a microphone built into the laptop or tablet to record what the salesperson says and then sends the audio data to voice recognition software (e.g., Google Cloud Speech-to-Text). This software analyzes the audio data and outputs the corresponding text data (e.g., "I'd like to know about cloud storage").

[1092] Input: Voice data during business negotiations

[1093] Output: Text data (e.g., "I want to know about cloud storage")

[1094] Step 2:

[1095] The server receives the text data sent from the device and extracts important keywords from it. Specifically, it analyzes the text using natural language processing technology (e.g., spaCy) and identifies important keywords (e.g., "cloud storage").

[1096] Input: Text data (e.g., "I want to know about cloud storage")

[1097] Output: Keyword (e.g. "cloud storage")

[1098] Step 3:

[1099] The server searches for relevant product information from a database based on the extracted keywords. Specifically, it queries a database (e.g., MySQL) and retrieves product information (e.g., "cloud storage service" and "data backup service") that matches the keyword (e.g., "cloud storage").

[1100] Input: Keyword (e.g. "cloud storage")

[1101] Output: Product information (e.g. "Cloud storage service", "Data backup service")

[1102] Step 4:

[1103] The terminal displays the search results for the product information and related services sent from the server on the sales representative's screen. Specifically, the terminal uses a sales support application to display the product information on a user interface so that the sales representative can visually confirm it.

[1104] Input: Product information (e.g., "Cloud storage service," "Data backup service")

[1105] Output: Product information displayed in a user interface

[1106] Step 5:

[1107] The server generates the necessary interview items and provision conditions based on the selected product information. Specifically, it uses an algorithm to generate related interview items (e.g., "storage capacity," "access frequency," and "security requirements") based on the selected product (e.g., "cloud storage service").

[1108] Input: Selected product information (e.g., "Cloud storage service")

[1109] Output: Interview items and provision conditions (e.g., "storage capacity," "access frequency," "security requirements")

[1110] Step 6:

[1111] The terminal displays the generated hearing items and provision conditions on the sales representative's screen. Specifically, this information is updated on the user interface so that the sales representative can visually confirm it.

[1112] Input: Interview items and provision conditions (e.g., "storage capacity," "access frequency," "security requirements")

[1113] Output: Hearing items and provision conditions displayed on the user interface

[1114] Step 7:

[1115] The server generates additional questions and confirmation items based on the interview content during the business negotiation. Specifically, it analyzes the collected interview data (e.g., "1 TB of storage space is required") and generates corresponding additional questions (e.g., "Please confirm the access frequency").

[1116] Input: Interview details (e.g., "1TB of storage space required")

[1117] Output: Additional questions and confirmations (e.g., "Please confirm access frequency")

[1118] Step 8:

[1119] The server automatically creates minutes based on the content of the business negotiations. Specifically, it uses an algorithm that aggregates collected business negotiation data and generates minutes that include proposed products, interview details, and additional confirmation items.

[1120] Input: Negotiation data (e.g., proposed products, interview details, additional confirmation items)

[1121] Output: Auto-generated meeting minutes

[1122] Step 9:

[1123] The terminal provides the minutes created by the server to the sales representative, who can then check and modify the contents. Specifically, the minutes are displayed on a user interface, allowing the sales representative to make modifications.

[1124] Input: Auto-generated meeting minutes

[1125] Output: Meeting minutes reviewed and revised by sales representative

[1126] Step 10:

[1127] The user (sales representative) finally shares the minutes that have been confirmed and corrected with the customer. Specifically, the user exports the corrected minutes in PDF format or other format and sends them to the customer via email or a shared link.

[1128] Input: Confirmed and corrected minutes

[1129] Output: Meeting minutes shared with customer

[1130] This detailed process step set clarifies the overall flow of the system and the specific operations of each step, allowing sales representatives to efficiently collect, confirm, and share information during sales negotiations.

[1131] (Application example 1)

[1132] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1133] With conventional sales negotiation support systems, it took a lot of time and effort for sales representatives to quickly understand customer needs during negotiations and propose appropriate products. In addition, the process of recording the content of sales negotiations and creating minutes was often done manually, which increased the risk of information leaks and misunderstandings. Furthermore, it was not possible to present relevant information in real time during negotiations, which reduced the efficiency of sales negotiations.

[1134] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1135] In this invention, the server includes: a voice recognition means for recognizing voice in real time; a keyword extraction means for extracting keywords from voice data recognized by the voice recognition means; a search means for searching for related product information based on keywords extracted by the keyword extraction means; a display means for displaying the product information searched by the search means; a hearing support means for generating necessary hearing items and provision conditions based on the product information; a hearing display means for displaying the generated hearing items and provision conditions; an additional question generation means for generating additional questions and confirmation items based on the hearing content during the business negotiation; a minutes generation means for automatically creating minutes based on the business negotiation content; a confirmation means for displaying the minutes created by the minutes generation means and confirming and correcting them; a sharing means for sharing the confirmed and corrected minutes with the customer; a voice data conversion means for capturing voice during the business negotiation using a voice recognition device built into the smart device and converting it into text data in real time; and a visual display means for visually displaying the voice data during the business negotiation and related information using the smart device. This makes it possible to quickly grasp customer needs during sales negotiations and provide appropriate product information and related services in real time.Furthermore, by recording sales negotiations and automatically generating minutes, it is possible to prevent information leaks and misunderstandings, improving the efficiency and accuracy of sales negotiations.

[1136] The "voice recognition means" is a device that recognizes voices spoken during negotiations in real time and converts them into text data.

[1137] The "keyword extraction means" is a means for extracting important keywords from the text data recognized by the voice recognition means.

[1138] The "search means" is a device that searches a database for related product information based on the extracted keywords.

[1139] The "display means" is a device that displays the product information retrieved by the search means on the screen of the salesperson.

[1140] The "hearing support means" is a means for generating necessary hearing items and provision conditions based on product information.

[1141] The "hearing display means" is a device that displays the generated hearing items and provision conditions on the screen of the sales representative.

[1142] The "additional question generation means" is a means for generating additional questions or confirmation items based on the contents of the hearing during the business negotiation.

[1143] The "minutes generating means" is a device that automatically creates minutes based on the contents of the business negotiations.

[1144] The "checking means" is a device that displays the minutes created by the minutes creating means and allows confirmation and correction.

[1145] The "sharing means" is a device for sharing the confirmed and corrected minutes with the client.

[1146] The "voice data conversion means" is a means for capturing voices during business negotiations using a voice recognition device built into the smart device and converting them into text data in real time.

[1147] The "visual display means" is a device that visually displays voice data and related information during negotiations using a smart device.

[1148] The present invention relates to a business negotiation support system for a brick-and-mortar store, and in particular to a system that improves the efficiency and accuracy of business negotiations by using smart devices. Specific embodiments of the present invention are described below.

[1149] System configuration

[1150] The system includes a speech recognition means, a keyword extraction means, a search means, a display means, a hearing support means, a hearing display means, a follow-up question generation means, a minutes generation means, a confirmation means, a sharing means, a speech data conversion means, and a visual display means.

[1151] Voice recognition means

[1152] The smart device's built-in microphone captures the voice during the transaction and converts the voice into text data in real time using voice recognition software (e.g., Google Speech-to-Text API).

[1153] Keyword extraction method

[1154] The server extracts important keywords from the text data recognized by the speech recognition means, using natural language processing techniques.

[1155] Search methods

[1156] The server searches a database for relevant product information based on the extracted keywords, using a database management system such as MySQL.

[1157] Display means

[1158] The smart device (e.g., smart glasses) displays the product information sent from the server in the salesperson's field of vision, allowing the salesperson to recommend the most suitable product to the customer.

[1159] Hearing support measures

[1160] The server generates necessary interview items and provision conditions based on the selected product information. The generated interview items are used to collect necessary information from customers when proposing products.

[1161] Hearing display means

[1162] The smart device displays the generated inquiry items and offer conditions in the field of view of the salesperson, allowing the salesperson to efficiently ask questions to the customer.

[1163] Additional question generation means

[1164] The server generates follow-up questions and confirmations based on the conversations that take place during the sales meeting, a process that is dynamic based on the customer's responses.

[1165] Minutes generation method

[1166] The server automatically creates minutes based on the content of the business negotiations, including proposed products, interview details, and additional confirmation items.

[1167] Verification method

[1168] The smart device displays the generated minutes in the salesperson's field of view, allowing the salesperson to review and modify the contents.

[1169] means of sharing

[1170] The sales representative will then share the confirmed and revised minutes with the customer, which will help clarify the details of the negotiation and the next meeting.

[1171] Audio data conversion means

[1172] The smart device's built-in voice recognition device captures speech during sales negotiations and converts it into text data in real time. For example, if a salesperson says, "I want a new smartphone," the speech is converted into text data saying, "I want a new smartphone."

[1173] Visual display means

[1174] Smart devices are used to visually display voice data and related information during sales negotiations, allowing salespeople to respond quickly based on visual information.

[1175] Specific examples

[1176] For example, if a salesperson is in a sales meeting and thinks, "I want to recommend the latest smartphone to the customer," they can input the following prompt into the generative AI model:

[1177] User: "I want a new phone"

[1178] System: "Finding information about your new phone…"

[1179] System: "The latest smartphones: Cutting-edge smartphone A, feature-packed smartphone B, affordable smartphone C — choose the product you want."

[1180] Salesperson: (chooses from options displayed in his field of view through smart glasses)

[1181] System: "Would you like to see additional details about the selected smartphone (specs, price) and information about the items we asked you to consider (budget, purpose of use, etc.)?"

[1182] Salesperson: "Yes"

[1183] System: "Displaying hearing items…"

[1184] This prompt sentence allows the salesperson to instantly obtain information to propose to the customer, and to effectively advance the sales negotiations.

[1185] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1186] Step 1:

[1187] The microphone in the smart device (terminal) captures the voice during the sales negotiation and sends the data to speech recognition software (e.g., Google Speech-to-Text API). The input is voice data, and the output is text data. Specifically, if a salesperson says, "I want a new smartphone," the voice data is converted into text data saying, "I want a new smartphone."

[1188] Step 2:

[1189] The server analyzes the text data received from the voice recognition software and extracts important keywords using natural language processing technology. The input is text data and the output is keywords. Specifically, the keyword "new smartphone" is extracted from the text data "I want a new smartphone."

[1190] Step 3:

[1191] The server searches a database (e.g., MySQL) for relevant product information based on the extracted keywords. The input is the keywords, and the output is the product information. Specifically, for the keyword "new smartphone," multiple smartphone models are generated as search results.

[1192] Step 4:

[1193] The server sends the search results to the smart device, and the search results are displayed in the salesperson's field of view by the display means of the smart device. The input is product information, and the output is display data. Specifically, a list of new smartphone models is displayed on the screen.

[1194] Step 5:

[1195] The salesperson (user) selects the product to be proposed from the options displayed in the field of view of the smart device. The selected product information is sent to the server. The input is the user's selection information, and the output is the selected product information.

[1196] Step 6:

[1197] The server generates the necessary questions and conditions for providing the product based on the selected product information. The input is the selected product information, and the output is the questions and conditions for providing the product. Specifically, questions about the selected smartphone model (e.g., desired storage capacity, color, etc.) are generated.

[1198] Step 7:

[1199] The smart device displays the generated hearing items and provision conditions in the field of view of the salesperson. The input is the hearing items and provision conditions, and the output is display data. Specifically, a list of questions for the hearing items is displayed on the screen.

[1200] Step 8:

[1201] The salesperson (user) asks questions to the customer based on the hearing items displayed in the field of view and collects the necessary information. The collected information is sent to the server. The input is the collected information, and the output is the hearing content data.

[1202] Step 9:

[1203] The server generates additional questions and confirmation items based on the interview content. The input is the interview content data, and the output is the additional questions and confirmation items. Specifically, if the answer is "1 TB of capacity is required," the server generates an additional question such as "Please confirm the access frequency."

[1204] Step 10:

[1205] The server records the details of the business negotiations and automatically creates minutes. The input is business negotiation data, and the output is minutes data. Specifically, the minutes include proposed products, interview details, additional confirmation items, etc.

[1206] Step 11:

[1207] The smart device displays the generated minutes in the field of view of the salesperson, who then checks and modifies the contents. The input is the minutes data, and the output is the checked and modified minutes data.

[1208] Step 12:

[1209] The sales representative (user) shares the confirmed and corrected minutes with the customer. The input is the confirmed and corrected minutes data, and the output is the minutes shared with the customer. Specifically, the final confirmed minutes are sent to the customer via email or other means.

[1210] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1211] The present invention relates to a business negotiation support system, and in particular to a system that recognizes voice in real time during business negotiations, extracts important keywords, presents related product information, automatically creates business negotiation records and minutes, and also recognizes the user's emotions and makes appropriate suggestions based on them.

[1212] This system mainly includes the following elements: a speech recognition means, a keyword extraction means, a search means, a display means, a hearing support means, a hearing display means, a follow-up question generation means, a minutes generation means, a confirmation means, a sharing means, and an emotion engine.

[1213] System Overview

[1214] Voice recognition means

[1215] The device uses a microphone to capture real-time audio during negotiations and converts it into text data using voice recognition software.

[1216] Keyword extraction method

[1217] The server extracts important keywords from the text data sent from the terminal, and the extracted keywords are used to identify product information related to the business negotiation.

[1218] Search methods

[1219] The server searches a database for relevant product information based on the extracted keywords, and provides candidate product information and related services as search results.

[1220] Display means

[1221] The terminal displays the product information and related services sent from the server on the salesperson's screen, allowing the salesperson to propose the most suitable product during the sales negotiation.

[1222] Hearing support measures

[1223] The server generates the necessary interview items and conditions for providing the product based on the selected product information, allowing the salesperson to ask the customer appropriate questions and collect the necessary information.

[1224] Hearing display means

[1225] The terminal displays the generated inquiry items and terms of service on the sales representative's screen, allowing the sales representative to smoothly proceed with the business negotiations.

[1226] Additional question generation means

[1227] The server generates follow-up questions and confirmations based on the conversations that take place during the sales meeting, allowing salespeople to gather the necessary details during the sales meeting.

[1228] Minutes generation method

[1229] The server automatically creates minutes based on the content of the business negotiations, and the minutes are provided to the sales representative at the end of the negotiations.

[1230] Verification method

[1231] The terminal displays the generated minutes to the sales representative, allowing the sales representative to check and correct the contents.

[1232] means of sharing

[1233] The user (sales representative) can then share the confirmed and revised minutes with the customer, allowing them to clearly communicate the details of the business negotiations and the details of the next meeting.

[1234] Emotion Engine

[1235] The server recognizes the user's emotions from data such as input voice and facial expressions. Based on the recognized emotions, the emotion engine adjusts the content of the product information it suggests, additional questions, and confirmation items.

[1236] Program description and examples

[1237] The program processing of this system will be explained below with specific examples.

[1238] Program processing

[1239] 1. The device uses voice recognition software to recognize speech during sales negotiations in real time. For example, if a salesperson says, "I'd like to know about cloud storage," the device converts this speech into text data that reads, "I'd like to know about cloud storage."

[1240] 2. The server analyzes the text data sent from the device and extracts the key keyword "cloud storage."

[1241] 3. The server searches the database for related products based on the extracted keywords. For example, for the keyword "cloud storage," it generates search results for "cloud storage service" and "data backup service."

[1242] 4. The server analyzes voice and facial expression data during the negotiation and uses an emotion engine to recognize the user's emotions. For example, it adjusts the content of the proposal depending on whether the customer is interested or skeptical.

[1243] 5. The server tailors the product information suggestions and follow-up questions based on the perceived sentiment, for example providing more detailed information if the customer expresses positive sentiment and a brief explanation if they are skeptical.

[1244] 6. The terminal displays the product information, related services, and emotion-based adjustment results sent from the server on the salesperson's screen. The salesperson can then select the products to recommend on the screen.

[1245] 7. The server generates the necessary inquiry items (for example, storage capacity, access frequency, security requirements) and supply conditions based on the selected product information.

[1246] 8. The terminal displays the generated interview items and offer conditions on the sales representative's screen. The sales representative then asks the customer questions based on this and collects the necessary information.

[1247] 9. The server generates additional questions and confirmations based on the interview content. For example, if the customer answers "I need 1TB of storage," the server generates an additional question such as "Please confirm the access frequency."

[1248] 10. The server records the details of the business negotiations and automatically creates minutes, which include proposed products, interview details, and additional confirmation items.

[1249] 11. The terminal provides the minutes to the sales representative, who then checks and corrects the contents.

[1250] 12. The user (sales representative) finally shares the confirmed and revised minutes with the customer, which clarifies the details of the negotiation and the next meeting.

[1251] This program streamlines information management during sales negotiations, allowing sales representatives to make quick and appropriate proposals to customers. It also recognizes users' emotions and makes adjustments based on them, leading to increased customer satisfaction.

[1252] The processing flow will be explained below.

[1253] Step 1:

[1254] The device uses a microphone to capture voices during sales negotiations in real time. It then uses voice recognition software to convert the captured voice into text data. For example, if a salesperson says, "I'd like to know about cloud storage," the device converts this voice into text data: "I'd like to know about cloud storage."

[1255] Step 2:

[1256] The terminal transmits the converted text data to the server via the network.

[1257] Step 3:

[1258] The server passes the received text data to a natural language processing engine and extracts important keywords. For example, it extracts "cloud storage" from the text "I want to know about cloud storage."

[1259] Step 4:

[1260] The server searches the database for relevant product information based on the extracted keywords, for example, to retrieve product information related to "cloud storage."

[1261] Step 5:

[1262] The terminal displays the search results (candidate product information and related services) sent from the server on the sales representative's screen, and the sales representative can select the products to suggest from the screen.

[1263] Step 6:

[1264] The server generates the necessary interview items and provision conditions based on the selected product information. For example, in the case of "cloud storage," it generates storage capacity, access frequency, security requirements, etc.

[1265] Step 7:

[1266] The terminal displays the generated interview items and terms of service on the screen of the sales representative, who then asks questions to the customer and collects the necessary information.

[1267] Step 8:

[1268] The terminal transmits the hearing response input by the sales representative or speech-recognized to the server.

[1269] Step 9:

[1270] The server analyzes the received answers and generates additional questions and confirmations. For example, in response to the answer "1 TB of storage space is required," the server adds a question such as "Please confirm the access frequency."

[1271] Step 10:

[1272] The terminal displays the generated additional questions and confirmation items on the screen of the sales representative, who then asks the customer further questions based on the displayed questions.

[1273] Step 11:

[1274] The server inputs voice and facial expression data from the business negotiation into an emotion engine to recognize the user's emotions.

[1275] Step 12:

[1276] The server then adjusts the product information and follow-up questions displayed based on the recognized emotion data, for example, providing detailed information if the customer expresses positive emotions, or providing a brief explanation if the customer is skeptical.

[1277] Step 13:

[1278] The server automatically creates minutes based on the details of the business negotiations, including proposed products, interview details, and additional confirmation items.

[1279] Step 14:

[1280] The terminal displays the generated minutes on the screen of the sales representative, allowing the sales representative to check and correct the contents.

[1281] Step 15:

[1282] The user (sales representative) shares the confirmed and corrected minutes with the customer, for example, by sending them by email to notify the customer of the contents of the minutes.

[1283] These steps enable the sales support system to manage information in real time, allowing sales representatives to make prompt and appropriate proposals to customers. Furthermore, by recognizing users' emotions and making adjustments based on them, it is possible to build trust with customers and increase the success rate of sales negotiations.

[1284] Example 2

[1285] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1286] While conventional sales negotiation support systems are effective in automatically recording sales negotiation content and proposing products, they lack the ability to recognize emotions in real time and adjust proposal content based on that, making it difficult to respond flexibly to customer emotions.In addition, there was a lack of means to improve the accuracy of sales negotiations while reducing the burden on sales representatives, such as generating follow-up questions needed during sales negotiations or automatically creating minutes.

[1287] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1288] In this invention, the server includes: a voice recognition means for recognizing input voice in real time; a keyword extraction means for extracting keywords from voice data recognized by the voice recognition means; a search means for searching for related product information based on keywords extracted by the keyword extraction means; a display means for displaying the product information searched by the search means; a hearing support means for generating necessary hearing items and provision conditions based on the product information; a hearing display means for displaying the generated hearing items and provision conditions; an additional question generation means for generating additional questions and confirmation items based on the hearing content during the business negotiation; a minutes generation means for automatically creating minutes based on the business negotiation content; a confirmation means for displaying the minutes created by the minutes generation means and confirming and correcting them; a sharing means for sharing the confirmed and corrected minutes with the customer; and an emotion recognition means for recognizing the user's emotions from the input voice and facial expression data and adjusting the provided product information and provision conditions based thereon. This makes it possible to efficiently perform real-time voice recognition and keyword extraction, related product searches, adjustment of proposal content based on emotion recognition, recording of business negotiations, and automatic generation of meeting minutes all within a single system.

[1289] The "voice recognition means" is a means for capturing input voice in real time and converting it into text data.

[1290] The "keyword extraction means" is a means for extracting important keywords from the voice data recognized by the voice recognition means.

[1291] The "search means" is a means for searching a database for related product information and service information based on the keywords extracted by the keyword extraction means.

[1292] The "display means" is a means for displaying product information and service information retrieved by the search means on the screen of the sales representative.

[1293] The "hearing support means" is a means for generating hearing items and provision conditions necessary for business negotiations based on the retrieved product information and service information.

[1294] The "hearing display means" is a means for displaying the generated hearing items and provision conditions on the screen of the sales representative.

[1295] The "additional question generation means" is a means for generating additional questions or confirmation items based on the contents of the interview obtained during the business negotiation.

[1296] The "minutes generating means" is a means for automatically creating minutes based on the contents of the business negotiation.

[1297] The "confirmation means" is a means for displaying the minutes created by the minutes creation means, and for the sales representative to confirm and correct the contents.

[1298] "Sharing means" means the means by which the confirmed and amended minutes are shared with the client.

[1299] The "emotion recognition means" is a means for recognizing the user's emotions from input voice and facial expression data, and adjusting the product information and terms of provision based on the emotions.

[1300] The present invention relates to a business negotiation support system that recognizes speech during business negotiations in real time, extracts important keywords, presents related product information, automatically records business negotiations and creates minutes, and also recognizes the user's emotions and makes appropriate suggestions based on those emotions. The specific configuration and operation of the system are described below.

[1301] Voice recognition means

[1302] The device uses a microphone to capture voices during sales negotiations in real time and converts them into text data using voice recognition software (e.g., Google's voice recognition API). For example, if a salesperson says, "I'd like to know about cloud storage," the device converts this voice into text data saying, "I'd like to know about cloud storage."

[1303] Keyword extraction method

[1304] The server receives the text data sent from the device and uses a natural language processing (NLP) tool (e.g., Python's NLTK library) to extract important keywords from this text data. For example, it extracts the keyword "cloud storage."

[1305] Search methods

[1306] The server searches a database (e.g., MySQL) for relevant product information based on the extracted keywords. For example, based on the keyword "cloud storage," it might generate search results such as "cloud storage service" and "data backup service."

[1307] emotion recognition means

[1308] The server analyzes voice and facial expression data and recognizes the user's emotions using an emotion engine (e.g., a deep learning model using Google's TensorFlow). For example, it adjusts its suggestions to provide detailed information if the customer expresses positive emotions, or a concise explanation if the customer expresses skepticism.

[1309] Adjusting the proposal

[1310] The server then tailors product suggestions and follow-up questions based on the perceived emotion, for example providing detailed information if the customer expresses positive emotion, or providing a clearer explanation if the customer is skeptical.

[1311] Displaying product information

[1312] The terminal displays product information, related services, and emotion-based adjustment results sent from the server on the salesperson's screen in real time, allowing the salesperson to use this information to recommend the most suitable product to the customer.

[1313] Hearing support

[1314] The server generates the necessary interview items (e.g., storage capacity, purpose of use) and terms of provision based on the selected product information.

[1315] Display of hearing items

[1316] The terminal displays the generated inquiry items and provision conditions on the sales representative's screen, allowing the sales representative to use this information to inquire about the necessary information from the customer.

[1317] Generate follow-up questions

[1318] The server generates additional questions and confirmations based on the information gathered during the sales negotiation. For example, if the customer answers, "I need 1 TB of storage space," the server generates additional questions such as, "Please confirm the access frequency."

[1319] Automatic generation of meeting minutes

[1320] The server records the details of the business negotiations and automatically creates minutes based on them, including proposed products, interview details, and additional confirmation items.

[1321] Checking the minutes

[1322] The terminal displays the generated minutes on the screen of the sales representative, allowing the sales representative to check and correct the contents.

[1323] Sharing meeting minutes

[1324] The user (sales representative) finally shares the confirmed and revised minutes with the customer. This can be done via an email system or a cloud storage service (e.g., Google Drive or Dropbox).

[1325] The program streamlines information management during sales negotiations, enabling salespeople to make quick and appropriate proposals to customers. It also recognizes users' emotions and adjusts accordingly, contributing to increased customer satisfaction.

[1326] Prompt Sentence Examples

[1327] Example prompts for inputting specific information into a generative AI model:

[1328] If a customer says "I want to know about cloud storage" during a sales meeting, explain how the system works. Include specific software and database names, and provide step-by-step details.

[1329] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1330] Step 1:

[1331] The device uses a microphone to capture voices during negotiations in real time and converts the input voice into text data using voice recognition software (e.g., Google's voice recognition API). The input is the voices being spoken during negotiations, and the output is the data converted from voice into text. Specifically, the device starts recording voices as soon as the negotiations begin, and sequentially sends the data to the voice recognition API to receive the text data.

[1332] Step 2:

[1333] The server receives the text data sent from the terminal and extracts important keywords using a natural language processing (NLP) tool (e.g., Python's NLTK library). The input is the text data, and the output is the extracted important keywords. The text data is subjected to morphological analysis, and keywords related to the pre-set business negotiations are extracted from it.

[1334] Step 3:

[1335] The server searches a database (e.g., MySQL) for relevant product information based on the extracted keywords. The input is the extracted keywords, and the output is the search result for related product information. A MySQL query is generated and executed to retrieve product information that matches the keywords.

[1336] Step 4:

[1337] The server analyzes voice data and facial expression data and recognizes the user's emotions using an emotion engine (e.g., a deep learning model using TensorFlow). The input is voice data and facial expression data, and the output is the recognized emotional information. Emotions are analyzed from the tone of voice and facial expressions, and the results are classified using an emotion model.

[1338] Step 5:

[1339] The server adjusts the product information suggestions and follow-up questions based on the recognized emotion. The input is emotion information and product information, and the output is the adjusted suggestions and question list. For example, if the emotion is positive, detailed information is provided, and if the emotion is skeptical, brief information is selected.

[1340] Step 6:

[1341] The terminal displays the product information and adjustment results sent from the server on the salesperson's screen. The input is the adjusted proposal content and question list, and the output is the information displayed on the salesperson's screen. The information is updated and displayed on the salesperson's display in real time.

[1342] Step 7:

[1343] The server generates the necessary interview items and provision conditions based on the selected product information. The input is the selected product information, and the output is the interview items and provision conditions. Based on the product information, the server selects appropriate interview items from a template and generates the provision conditions.

[1344] Step 8:

[1345] The terminal displays the generated inquiry items and provision conditions on the sales representative's screen. The input is the generated inquiry items and provision conditions, and the output is the information displayed on the sales representative's screen. This allows the sales representative to ask questions to the customer according to the inquiry items.

[1346] Step 9:

[1347] The server generates additional questions and confirmations based on the information gathered during the negotiation. The input is the information gathered during the negotiation, and the output is the additional questions and confirmations. For example, in response to the answer "1 TB of storage is required," the server generates an additional question such as "Please confirm the access frequency."

[1348] Step 10:

[1349] The server records the content of the business negotiations and automatically creates minutes based on that. The input is the text data of the business negotiations, and the output is the minutes. The minutes are created by dividing the business negotiation text into paragraphs and extracting important points and questions and answers.

[1350] Step 11:

[1351] The terminal displays the generated minutes on the sales representative's screen, allowing the sales representative to check and correct the contents. The input is the generated minutes, and the output is the checked and corrected minutes. The sales representative checks the displayed minutes and makes corrections as necessary.

[1352] Step 12:

[1353] The user (sales representative) finally shares the confirmed and revised minutes with the customer. The input is the revised minutes, and the output is the minutes shared with the customer. The sharing method is an email transmission system or cloud storage service.

[1354] This series of processes makes information management during negotiations more efficient, enabling sales representatives to make appropriate proposals to customers.

[1355] (Application example 2)

[1356] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1357] Conventional sales negotiation support systems can record the content of sales negotiations in real time and extract important keywords to present related information, but they have difficulty making accurate proposals based on customer sentiment. Furthermore, because they do not support automatic recording of sales negotiation content or automatic creation of meeting minutes, they have the problem of consuming a large amount of resources after the negotiation. This reduces the work efficiency of sales representatives and does not necessarily result in optimal sales negotiation outcomes.

[1358] The specification processing by the specification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: a speech recognition unit that recognizes input speech in real time; a keyword extraction unit that extracts keywords from speech data recognized by the speech recognition unit; a search unit that searches for related product information based on the keywords extracted by the keyword extraction unit; an emotion recognition unit that recognizes the customer's emotions and adjusts the proposal content based on the emotions; a display unit that displays the product information searched by the search unit; a hearing support unit that generates necessary hearing items and provision conditions based on the product information; a hearing display unit that displays the generated hearing items and provision conditions; a follow-up question generation unit that generates follow-up questions and confirmation items based on the hearing content during the business negotiation; a minutes generation unit that automatically creates minutes based on the business negotiation content; a confirmation unit that displays the minutes created by the minutes generation unit and confirms and modifies them; and a sharing unit that shares the confirmed and modified minutes with the customer. This enables efficient management of information during the business negotiation and optimal proposals based on the customer's emotions.

[1359] The "voice recognition means" is a means for recognizing input voice in real time and converting it into text data.

[1360] The "keyword extraction means" is a means for extracting important keywords from the voice data recognized by the voice recognition means.

[1361] The "search means" is a means for searching for related product information based on the keywords extracted by the keyword extraction means.

[1362] The "emotion recognition means" is a means for recognizing emotions from the customer's voice and facial expressions, and adjusting the content of the proposal based on those emotions.

[1363] The "display means" is a means for displaying product information and related services searched for by the search means, and is a means for visually presenting necessary information.

[1364] The "hearing support means" is a means for generating necessary hearing items and provision conditions based on product information.

[1365] The "hearing display means" is a means for displaying the generated hearing items and provision conditions.

[1366] The "additional question generation means" is a means for generating additional questions or confirmation items based on the contents of the hearing during the business negotiation.

[1367] The "minutes generation means" is a means for automatically creating minutes based on the contents of the business negotiations.

[1368] The "checking means" is a means for displaying the minutes created by the minutes creating means and for checking and correcting them.

[1369] "Sharing means" means the means by which the confirmed and amended minutes are shared with the client.

[1370] "Product information" is detailed information about products and services related to the content of the business negotiations.

[1371] "Emotion recognition" is a technology that analyzes and recognizes a customer's emotional state from their voice and facial expressions.

[1372] A "business negotiation" is a conversation or negotiation between a sales representative and a customer regarding a proposed product or service.

[1373] This invention realizes a shopping assistant system for brick-and-mortar stores, specifically, a system that uses smart glasses or an application installed on a smartphone to assist store clerks in conversations with customers. This system uses the following hardware and software:

[1374] Hardware and Software

[1375] 1. Hardware:

[1376] Smart glasses or smartphone: A device equipped with a microphone and camera.

[1377] 2. Software:

[1378] Speech recognition software (such as Google Cloud Speech-to-Text API)

[1379] Keyword extraction model (using TensorFlow, etc.)

[1380] Emotion recognition engine (e.g. NVIDIA Clara AI)

[1381] Database (e.g. Firebase Realtime Database)

[1382] System Procedures

[1383] Voice Capture and Recognition

[1384] The device uses the microphones in smart glasses or smartphones to capture customer interactions in real time, and the captured voice data is converted into text using the Google Cloud Speech-to-Text API.

[1385] Keyword extraction

[1386] The server extracts important keywords from the text data generated by the speech recognition means, using a keyword extraction model built with TensorFlow.

[1387] Searching and displaying product information

[1388] The server searches for relevant product information from a database such as Firebase based on the keywords extracted by the keyword extraction means. The searched product information is displayed on the terminal screen, allowing the store clerk to quickly provide appropriate product information.

[1389] Emotion recognition and suggestion adjustment

[1390] The server analyzes the customer's voice and facial expression data and uses NVIDIA Clara AI technology to recognize the customer's emotions. Based on the results of this emotion recognition, the server adjusts the product information suggestions and follow-up questions it asks. For example, if the customer expresses positive emotions, it provides more detailed product information, while if the customer expresses skepticism, it provides a concise, to-the-point explanation.

[1391] Hearing support and follow-up questions

[1392] The server generates the necessary inquiry items and provision conditions based on the selected product information. These inquiry items and provision conditions are displayed on the terminal screen, and the salesperson asks the customer appropriate questions based on them and collects the necessary information. Furthermore, the server generates additional questions and confirmation items based on the inquiry content during the sales negotiation.

[1393] Generate and review meeting minutes

[1394] The server automatically creates minutes based on the content of the business negotiation. The minutes are then displayed on the terminal screen, allowing the salesperson to check and modify the contents. After this, the minutes are shared with the customer.

[1395] Examples of concrete examples and prompts

[1396] Specific examples

[1397] For example, if a store clerk is talking to a customer and the customer says, "I want to see the most popular 4K TV," the system will recognize the speech and extract the keyword "4K TV." It will then search the database for related product information and display the search results on the store clerk's device. At the same time, it will recognize the customer's interest and suggest more detailed product descriptions. The store clerk can then make the best suggestions to the customer based on the displayed information and any follow-up questions.

[1398] Prompt Sentence Examples

[1399] You are a salesperson. If a customer says, "I want a new 4K TV," here's what you should do:

[1400] 1. Use speech recognition to extract the important keyword "4K TV."

[1401] 2. Search and suggest relevant product information from the database.

[1402] 3. Analyze customer sentiment to determine if they're interested or want to know more.

[1403] 4. View the suggestions and generate and ask follow-up questions as needed.

[1404] 5. Automatically record the contents of business negotiations and create minutes.

[1405] This system makes customer service in physical stores more efficient and effective, allowing store staff to make optimal suggestions based on customer needs. It also automatically records the details of sales negotiations, making post-negotiation management easier.

[1406] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1407] Step 1:

[1408] The terminal uses the microphone of smart glasses or smartphone to capture customer speech in real time, and the input data is the voice signal, which serves as the base data for subsequent processing. The output is the captured voice data.

[1409] Step 2:

[1410] The device converts the captured voice data into text data using speech recognition software (e.g., Google Cloud Speech-to-Text API). This process analyzes the voice signal and generates corresponding text. The input is the voice data acquired in step 1, and the output is text data.

[1411] Step 3:

[1412] The server extracts important keywords from the text data generated by the speech recognition means. This uses a keyword extraction model built with TensorFlow. The input is the text data generated in step 2, and the output is the extracted keywords.

[1413] Step 4:

[1414] The server searches for related product information from a database such as Firebase based on the extracted keywords. The system sends product information containing the keywords as a query to the database and retrieves the corresponding product information. The input is the keywords extracted in step 3, and the output is related product information.

[1415] Step 5:

[1416] The server uses NVIDIA Clara AI to recognize emotions using the customer's voice and facial expression data. Voice data and camera footage are used as inputs to analyze the customer's emotional state. The inputs are voice and video data, and the output is the emotion recognition results.

[1417] Step 6:

[1418] The terminal displays optimal suggestions to the customer based on the acquired product information and emotion recognition results. The system adjusts the suggestions based on the results of emotion analysis and displays them on the screen. The input is the output data from Steps 4 and 5, and the output is the adjusted suggestions.

[1419] Step 7:

[1420] The server generates the necessary interview items and provision conditions based on the selected product information. The interview support means automatically generates questions according to the customer's needs. The input is the output data of step 4, and the output is the interview items and provision conditions.

[1421] Step 8:

[1422] The terminal displays the generated interview items and conditions for provision, and the store clerk asks the customer appropriate questions based on this to collect information. The clerk checks the content displayed on the screen and collects the necessary information from the customer. The input is the output data of step 7, and the output is the collected customer information.

[1423] Step 9:

[1424] The server generates additional questions and confirmations based on the interview content. The system analyzes the answers from the customer and generates additional questions if more detailed information needs to be collected. The input is the customer information collected in step 8, and the output is additional questions and confirmations.

[1425] Step 10:

[1426] The server automatically creates minutes based on the content of the business negotiation. The system compiles all data acquired during the business negotiation and formats it as minutes. The input is all data from step 2 to step 9, and the output is the minutes of the business negotiation.

[1427] Step 11:

[1428] The terminal displays the generated minutes, which the clerk can review and modify. The system reflects the modifications and finalizes the minutes. The input is the minutes created in step 10, and the output is the reviewed and modified minutes.

[1429] Step 12:

[1430] The user (store clerk) shares the final confirmed and revised minutes with the customer. The minutes are provided to the customer and the customer confirms the details of the business negotiation. The input is the minutes confirmed and revised in step 11, and the output is the final minutes shared with the customer.

[1431] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1432] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1433] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1434] [Fourth embodiment]

[1435] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1436] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1437] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1438] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1439] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1440] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1441] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1442] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1443] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1444] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1445] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1446] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1447] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1448] The present invention relates to a business negotiation support system, and in particular to a system that effectively advances business negotiations by recognizing speech in real time during the negotiation, extracting important keywords, presenting related product information, and automatically creating records of the negotiations and minutes.

[1449] This system mainly includes the following elements: a speech recognition means, a keyword extraction means, a search means, a display means, a hearing support means, a hearing display means, a follow-up question generation means, a minutes generation means, a confirmation means, and a sharing means.

[1450] System Overview

[1451] Voice recognition means

[1452] The device uses voice recognition software to recognize speech during negotiations in real time and convert it into text data.

[1453] Keyword extraction method

[1454] The server extracts important keywords from the text data sent from the terminal, and the extracted keywords are used to identify product information related to the business negotiation.

[1455] Search methods

[1456] The server searches a database for relevant product information based on the extracted keywords, and provides candidate product information and related services as search results.

[1457] Display means

[1458] The terminal displays the product information and related services sent from the server on the salesperson's screen, allowing the salesperson to propose the most suitable product during the sales negotiation.

[1459] Hearing support measures

[1460] The server generates the necessary interview items and conditions for providing the product based on the selected product information, allowing the salesperson to ask the customer appropriate questions and collect the necessary information.

[1461] Hearing display means

[1462] The terminal displays the generated inquiry items and terms of service on the sales representative's screen, allowing the sales representative to smoothly proceed with the business negotiations.

[1463] Additional question generation means

[1464] The server generates follow-up questions and confirmations based on the conversations that take place during the sales meeting, allowing salespeople to gather the necessary details during the sales meeting.

[1465] Minutes generation method

[1466] The server automatically creates minutes based on the content of the business negotiations, and the minutes are provided to the sales representative at the end of the negotiations.

[1467] Verification method

[1468] The terminal displays the generated minutes to the sales representative, who then checks and corrects them.

[1469] means of sharing

[1470] The user (sales representative) can then share the confirmed and revised minutes with the customer, allowing them to clearly communicate the details of the business negotiations and the details of the next meeting.

[1471] Program description and examples

[1472] The program processing of this system will be explained below with specific examples.

[1473] Program processing

[1474] 1. The device uses voice recognition software to recognize speech during sales negotiations in real time. For example, if a salesperson says, "I'd like to know about cloud storage," the device converts this speech into text data that reads, "I'd like to know about cloud storage."

[1475] 2. The server analyzes the text data sent from the device and extracts the key keyword "cloud storage."

[1476] 3. The server searches the database for related products based on the extracted keywords. For example, for the keyword "cloud storage," it generates search results for "cloud storage service" and "data backup service."

[1477] 4. The terminal displays the search results sent from the server on the sales representative's screen, where the sales representative can select the products to suggest.

[1478] 5. The server generates the necessary inquiry items (e.g., storage capacity, access frequency, security requirements) and provision conditions based on the selected product information.

[1479] 6. The terminal displays the generated interview items and offer conditions on the sales representative's screen. The sales representative asks questions to the customer based on these items and collects the necessary information.

[1480] 7. The server generates additional questions and confirmations based on the interview content. For example, if the customer answers "I need 1TB of storage," the server generates an additional question such as "Please confirm the access frequency."

[1481] 8. The server records the details of the business negotiations and automatically creates minutes, which include proposed products, interview details, and additional confirmation items.

[1482] 9. The terminal provides the minutes to the sales representative, who then checks and corrects the contents.

[1483] 10. The user (sales representative) finally shares the confirmed and revised minutes with the customer, which clarifies the details of the negotiation and the next meeting.

[1484] This program streamlines information management during sales negotiations, allowing sales representatives to make prompt and appropriate proposals to customers. In addition, sales negotiation records and minutes are automatically generated, preventing information leaks and misunderstandings.

[1485] The processing flow will be explained below.

[1486] Step 1:

[1487] The device uses a microphone to capture speech during a business meeting and uses voice recognition software to convert the captured speech into text data in real time.

[1488] Step 2:

[1489] The terminal transmits the converted text data to the server via the network.

[1490] Step 3:

[1491] The server passes the received text data to a natural language processing engine and extracts important keywords. For example, it extracts "cloud storage" from the text "I want to know about cloud storage."

[1492] Step 4:

[1493] The server searches the database for relevant product information based on the extracted keywords, for example, to retrieve product information related to "cloud storage."

[1494] Step 5:

[1495] The server transmits the plurality of pieces of product information obtained as search results to the terminal.

[1496] Step 6:

[1497] The terminal displays the obtained product information and related services on the sales representative's screen, and the sales representative can select the products to suggest from the screen.

[1498] Step 7:

[1499] The server generates the necessary interview items and provision conditions based on the selected product information. For example, in the case of "cloud storage," it generates storage capacity, access frequency, security requirements, etc.

[1500] Step 8:

[1501] The terminal displays the generated interview items and terms of service on the screen of the sales representative, who then asks questions to the customer and collects the necessary information.

[1502] Step 9:

[1503] The terminal transmits the hearing response input by the sales representative or speech-recognized to the server.

[1504] Step 10:

[1505] The server analyzes the received answers and generates additional questions and confirmations. For example, in response to the answer "1 TB of storage space is required," the server adds a question such as "Please confirm the access frequency."

[1506] Step 11:

[1507] The terminal displays the generated additional questions and confirmation items on the screen of the sales representative, who then asks the customer further questions based on the displayed questions.

[1508] Step 12:

[1509] The server automatically creates minutes based on the details of the business negotiations, including proposed products, interview details, and additional confirmation items.

[1510] Step 13:

[1511] The terminal displays the generated minutes on the screen of the sales representative, allowing the sales representative to check and correct the contents.

[1512] Step 14:

[1513] The user (sales representative) shares the confirmed and corrected minutes with the customer, for example, by sending them by email to notify the customer of the contents of the minutes.

[1514] These steps enable real-time support for business negotiations, enabling efficient and accurate proposals and information sharing.

[1515] Example 1

[1516] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1517] In modern business negotiations, salespeople need to make quick and appropriate product proposals in real time and conduct interviews based on customer needs. However, it is difficult to instantly collect and record a large amount of information during a negotiation, and it also takes a lot of time to review the information later and create minutes. This places a heavy burden on salespeople, making it difficult to conduct negotiations efficiently.

[1518] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1519] In this invention, the server includes a speech recognition means for recognizing input speech in real time, a keyword extraction means for extracting keywords from speech data recognized by the speech recognition means, a search means for searching for related product information based on the keywords extracted by the keyword extraction means, a display means for displaying the product information searched by the search means, a hearing support means for generating necessary hearing items and provision conditions based on the product information, a hearing display means for displaying the generated hearing items and provision conditions, an additional question generation means for generating additional questions and confirmation items based on the hearing contents during the business negotiation, a minutes generation means for automatically creating minutes based on the contents of the business negotiation, and a display means for displaying the minutes created by the minutes generation means. The system includes a confirmation means for displaying minutes and confirming and correcting them, a sharing means for sharing the confirmed and corrected minutes with the customer, a database search means for identifying important keywords from the collected voice data and retrieving related product information from a database based on the keywords, a selection means for displaying the product information retrieved by the database search means on the sales representative's terminal and generating hearing items based on the selected product, a follow-up question generation means and a confirmation item generation means for generating follow-up questions based on the generated hearing items, a confirmation and correction means for automatically generating minutes at the end of the business negotiation and allowing the sales representative to confirm and correct the contents, and a sharing means for sharing the final confirmed and corrected minutes with the customer. This allows for efficient information gathering during the business negotiation, preventing oversight of results, and additional information confirmation, thereby reducing the burden on the sales representative and enabling the business negotiation to proceed more effectively.

[1520] The "voice recognition means" is a device or software that has the function of recognizing input voice in real time and converting voice data into text data.

[1521] The "keyword extraction means" is a device or software that has the function of extracting important keywords from the voice data recognized by the voice recognition means.

[1522] The "search means" is a device or software having a function of searching a database for related product information based on the keywords extracted by the keyword extraction means.

[1523] The "display means" is a device or software that has the function of displaying the product information acquired by the search means on the user's terminal.

[1524] The "hearing support means" is a device or software that has the function of generating necessary hearing items and provision conditions based on product information.

[1525] The "hearing display means" is a device or software that has the function of displaying the generated hearing items and provision conditions on the user's terminal.

[1526] The "additional question generating means" is a device or software that has the function of generating additional questions or confirmation items based on the content of hearings during business negotiations.

[1527] The "minutes generating means" is a device or software that has the function of automatically creating minutes based on the contents of the business negotiation.

[1528] The "checking means" is a device or software having a function of displaying the minutes created by the minutes creating means on the user's terminal and allowing the user to check and correct the contents.

[1529] "Sharing means" means a device or software that has the function of sharing the confirmed and corrected minutes with the client.

[1530] The "database search means" is a device or software that has the function of identifying important keywords from the collected voice data and retrieving related product information from a database based on those keywords.

[1531] The "selection means" is a device or software having the function of displaying the product information acquired by the database search means on the terminal and generating hearing items based on the selected product.

[1532] The "additional question generating means and confirmation item generating means" refers to a device or software having the function of generating additional questions and items to be confirmed based on the generated hearing items.

[1533] The "verification and correction means" is a device or software that has the function of allowing a user to verify the minutes that are automatically generated at the end of a business meeting and correct them as necessary.

[1534] "Sharing means" refers to a device or software that has the function of sharing the final confirmed and corrected minutes with the client.

[1535] The present invention relates to a sales negotiation support system, and in particular to a system that recognizes speech in real time during a sales negotiation, extracts important keywords, presents related product information, and automatically creates a record of the sales negotiation and minutes, thereby promoting the efficient progress of the sales negotiation and reducing the burden on sales representatives.

[1536] System configuration

[1537] The business negotiation support system of the present invention uses the following hardware and software.

[1538] 1. Devices: laptops, tablets, smartphones, etc.

[1539] 2. Speech recognition software: Google Cloud Speech-to-Text, IBM Watson Speech to Text, etc.

[1540] 3. Server: Cloud server (e.g., Amazon Web Services, Google Cloud Platform)

[1541] 4. Database: MySQL, PostgreSQL, etc.

[1542] Explanation of program processing

[1543] 1. The device uses voice recognition software to recognize speech during negotiations in real time and convert it into text data. For example, if a sales representative says, "I'd like to know about cloud storage," the speech is converted into text data that reads, "I'd like to know about cloud storage."

[1544] 2. The server receives the text data sent from the device and extracts important keywords from it. For example, the phrase "cloud storage" is extracted as a keyword. This is done using natural language processing technology (e.g., spaCy).

[1545] 3. The server searches the database for relevant product information based on the extracted keywords. For example, for the keyword "cloud storage," "cloud storage service" and "data backup service" are generated as search results.

[1546] 4. The terminal displays the search results for the product information and related services sent from the server on the sales representative's screen. The sales representative can check the product information displayed on the screen and select the products to suggest.

[1547] 5. The server generates the necessary interview items and terms of service based on the selected product information. For example, in the case of a cloud storage service, the server generates interview items such as storage capacity, access frequency, and security requirements.

[1548] 6. The terminal displays the generated interview items and terms of service on the sales representative's screen. The sales representative uses this information to ask questions of the customer and collect the necessary information.

[1549] 7. The server generates additional questions and confirmations based on the information gathered during the sales negotiation. For example, if the customer answers, "I need 1TB of storage space," the server generates an additional question, such as, "Please confirm the access frequency."

[1550] 8. The server automatically creates minutes based on the details of the business negotiations. The minutes include proposed products, interview details, and additional confirmation items.

[1551] 9. The terminal provides the minutes created by the server to the sales representative, who can then check and modify the contents.

[1552] 10. The user (sales representative) finally shares the confirmed and revised minutes with the customer, which clarifies the details of the negotiation and the next meeting.

[1553] Examples of concrete examples and prompts

[1554] Below are examples of specific actions and examples of input prompts to the generative AI model.

[1555] Specific examples of operation

[1556] If a salesperson says "I'd like to know about cloud storage" during a sales meeting, the voice recognition software converts this speech into text data, and the server extracts the keyword "cloud storage." The server then searches the database for relevant product information and displays the search results on the device. Based on this information, the salesperson can ask the customer appropriate questions, and the system automatically generates any necessary follow-up questions and meeting minutes.

[1557] Example of input prompt for generative AI model

[1558] Please explain how the sales support system works. Please provide a detailed explanation of the process from when a sales representative says, "I'd like to know about cloud storage," to when the meeting minutes are shared with the customer.

[1559] This sales negotiation support system streamlines information management during sales negotiations, allowing sales representatives to make prompt and appropriate proposals. In addition, sales negotiation records and minutes are automatically generated, preventing information leaks and misunderstandings.

[1560] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1561] Step 1:

[1562] The device uses voice recognition software to recognize speech during sales negotiations in real time and convert it into text data. Specifically, the device uses a microphone built into the laptop or tablet to record what the salesperson says and then sends the audio data to voice recognition software (e.g., Google Cloud Speech-to-Text). This software analyzes the audio data and outputs the corresponding text data (e.g., "I'd like to know about cloud storage").

[1563] Input: Voice data during business negotiations

[1564] Output: Text data (e.g., "I want to know about cloud storage")

[1565] Step 2:

[1566] The server receives the text data sent from the device and extracts important keywords from it. Specifically, it analyzes the text using natural language processing technology (e.g., spaCy) and identifies important keywords (e.g., "cloud storage").

[1567] Input: Text data (e.g., "I want to know about cloud storage")

[1568] Output: Keyword (e.g. "cloud storage")

[1569] Step 3:

[1570] The server searches for relevant product information from a database based on the extracted keywords. Specifically, it queries a database (e.g., MySQL) and retrieves product information (e.g., "cloud storage service" and "data backup service") that matches the keyword (e.g., "cloud storage").

[1571] Input: Keyword (e.g. "cloud storage")

[1572] Output: Product information (e.g. "Cloud storage service", "Data backup service")

[1573] Step 4:

[1574] The terminal displays the search results for the product information and related services sent from the server on the sales representative's screen. Specifically, the terminal uses a sales support application to display the product information on a user interface so that the sales representative can visually confirm it.

[1575] Input: Product information (e.g., "Cloud storage service," "Data backup service")

[1576] Output: Product information displayed in a user interface

[1577] Step 5:

[1578] The server generates the necessary interview items and provision conditions based on the selected product information. Specifically, it uses an algorithm to generate related interview items (e.g., "storage capacity," "access frequency," and "security requirements") based on the selected product (e.g., "cloud storage service").

[1579] Input: Selected product information (e.g., "Cloud storage service")

[1580] Output: Interview items and provision conditions (e.g., "storage capacity," "access frequency," "security requirements")

[1581] Step 6:

[1582] The terminal displays the generated hearing items and provision conditions on the sales representative's screen. Specifically, this information is updated on the user interface so that the sales representative can visually confirm it.

[1583] Input: Interview items and provision conditions (e.g., "storage capacity," "access frequency," "security requirements")

[1584] Output: Hearing items and provision conditions displayed on the user interface

[1585] Step 7:

[1586] The server generates additional questions and confirmation items based on the interview content during the business negotiation. Specifically, it analyzes the collected interview data (e.g., "1 TB of storage space is required") and generates corresponding additional questions (e.g., "Please confirm the access frequency").

[1587] Input: Interview details (e.g., "1TB of storage space required")

[1588] Output: Additional questions and confirmations (e.g., "Please confirm access frequency")

[1589] Step 8:

[1590] The server automatically creates minutes based on the content of the business negotiations. Specifically, it uses an algorithm that aggregates collected business negotiation data and generates minutes that include proposed products, interview details, and additional confirmation items.

[1591] Input: Negotiation data (e.g., proposed products, interview details, additional confirmation items)

[1592] Output: Auto-generated meeting minutes

[1593] Step 9:

[1594] The terminal provides the minutes created by the server to the sales representative, who can then check and modify the contents. Specifically, the minutes are displayed on a user interface, allowing the sales representative to make modifications.

[1595] Input: Auto-generated meeting minutes

[1596] Output: Meeting minutes reviewed and revised by sales representative

[1597] Step 10:

[1598] The user (sales representative) finally shares the minutes that have been confirmed and corrected with the customer. Specifically, the user exports the corrected minutes in PDF format or other format and sends them to the customer via email or a shared link.

[1599] Input: Confirmed and corrected minutes

[1600] Output: Meeting minutes shared with customer

[1601] This detailed process step set clarifies the overall flow of the system and the specific operations of each step, allowing sales representatives to efficiently collect, confirm, and share information during sales negotiations.

[1602] (Application example 1)

[1603] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1604] With conventional sales negotiation support systems, it took a lot of time and effort for sales representatives to quickly understand customer needs during negotiations and propose appropriate products. In addition, the process of recording the content of sales negotiations and creating minutes was often done manually, which increased the risk of information leaks and misunderstandings. Furthermore, it was not possible to present relevant information in real time during negotiations, which reduced the efficiency of sales negotiations.

[1605] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1606] In this invention, the server includes: a voice recognition means for recognizing voice in real time; a keyword extraction means for extracting keywords from voice data recognized by the voice recognition means; a search means for searching for related product information based on keywords extracted by the keyword extraction means; a display means for displaying the product information searched by the search means; a hearing support means for generating necessary hearing items and provision conditions based on the product information; a hearing display means for displaying the generated hearing items and provision conditions; an additional question generation means for generating additional questions and confirmation items based on the hearing content during the business negotiation; a minutes generation means for automatically creating minutes based on the business negotiation content; a confirmation means for displaying the minutes created by the minutes generation means and confirming and correcting them; a sharing means for sharing the confirmed and corrected minutes with the customer; a voice data conversion means for capturing voice during the business negotiation using a voice recognition device built into the smart device and converting it into text data in real time; and a visual display means for visually displaying the voice data during the business negotiation and related information using the smart device. This makes it possible to quickly grasp customer needs during sales negotiations and provide appropriate product information and related services in real time.Furthermore, by recording sales negotiations and automatically generating minutes, it is possible to prevent information leaks and misunderstandings, improving the efficiency and accuracy of sales negotiations.

[1607] The "voice recognition means" is a device that recognizes voices spoken during negotiations in real time and converts them into text data.

[1608] The "keyword extraction means" is a means for extracting important keywords from the text data recognized by the voice recognition means.

[1609] The "search means" is a device that searches a database for related product information based on the extracted keywords.

[1610] The "display means" is a device that displays the product information retrieved by the search means on the screen of the salesperson.

[1611] The "hearing support means" is a means for generating necessary hearing items and provision conditions based on product information.

[1612] The "hearing display means" is a device that displays the generated hearing items and provision conditions on the screen of the sales representative.

[1613] The "additional question generation means" is a means for generating additional questions or confirmation items based on the contents of the hearing during the business negotiation.

[1614] The "minutes generating means" is a device that automatically creates minutes based on the contents of the business negotiations.

[1615] The "checking means" is a device that displays the minutes created by the minutes creating means and allows confirmation and correction.

[1616] The "sharing means" is a device for sharing the confirmed and corrected minutes with the client.

[1617] The "voice data conversion means" is a means for capturing voices during business negotiations using a voice recognition device built into the smart device and converting them into text data in real time.

[1618] The "visual display means" is a device that visually displays voice data and related information during negotiations using a smart device.

[1619] The present invention relates to a business negotiation support system for a brick-and-mortar store, and in particular to a system that improves the efficiency and accuracy of business negotiations by using smart devices. Specific embodiments of the present invention are described below.

[1620] System configuration

[1621] The system includes a speech recognition means, a keyword extraction means, a search means, a display means, a hearing support means, a hearing display means, a follow-up question generation means, a minutes generation means, a confirmation means, a sharing means, a speech data conversion means, and a visual display means.

[1622] Voice recognition means

[1623] The smart device's built-in microphone captures the voice during the transaction and converts the voice into text data in real time using voice recognition software (e.g., Google Speech-to-Text API).

[1624] Keyword extraction method

[1625] The server extracts important keywords from the text data recognized by the speech recognition means, using natural language processing techniques.

[1626] Search methods

[1627] The server searches a database for relevant product information based on the extracted keywords, using a database management system such as MySQL.

[1628] Display means

[1629] The smart device (e.g., smart glasses) displays the product information sent from the server in the salesperson's field of vision, allowing the salesperson to recommend the most suitable product to the customer.

[1630] Hearing support measures

[1631] The server generates necessary interview items and provision conditions based on the selected product information. The generated interview items are used to collect necessary information from customers when proposing products.

[1632] Hearing display means

[1633] The smart device displays the generated inquiry items and offer conditions in the field of view of the salesperson, allowing the salesperson to efficiently ask questions to the customer.

[1634] Additional question generation means

[1635] The server generates follow-up questions and confirmations based on the conversations that take place during the sales meeting, a process that is dynamic based on the customer's responses.

[1636] Minutes generation method

[1637] The server automatically creates minutes based on the content of the business negotiations, including proposed products, interview details, and additional confirmation items.

[1638] Verification method

[1639] The smart device displays the generated minutes in the salesperson's field of view, allowing the salesperson to review and modify the contents.

[1640] means of sharing

[1641] The sales representative will then share the confirmed and revised minutes with the customer, which will help clarify the details of the negotiation and the next meeting.

[1642] Audio data conversion means

[1643] The smart device's built-in voice recognition device captures speech during sales negotiations and converts it into text data in real time. For example, if a salesperson says, "I want a new smartphone," the speech is converted into text data saying, "I want a new smartphone."

[1644] Visual display means

[1645] Smart devices are used to visually display voice data and related information during sales negotiations, allowing salespeople to respond quickly based on visual information.

[1646] Specific examples

[1647] For example, if a salesperson is in a sales meeting and thinks, "I want to recommend the latest smartphone to the customer," they can input the following prompt into the generative AI model:

[1648] User: "I want a new phone"

[1649] System: "Finding information about your new phone…"

[1650] System: "The latest smartphones: Cutting-edge smartphone A, feature-packed smartphone B, affordable smartphone C — choose the product you want."

[1651] Salesperson: (chooses from options displayed in his field of view through smart glasses)

[1652] System: "Would you like to see additional details about the selected smartphone (specs, price) and information about the items we asked you to consider (budget, purpose of use, etc.)?"

[1653] Salesperson: "Yes"

[1654] System: "Displaying hearing items…"

[1655] This prompt sentence allows the salesperson to instantly obtain information to propose to the customer, and to effectively advance the sales negotiations.

[1656] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1657] Step 1:

[1658] The microphone in the smart device (terminal) captures the voice during the sales negotiation and sends the data to speech recognition software (e.g., Google Speech-to-Text API). The input is voice data, and the output is text data. Specifically, if a salesperson says, "I want a new smartphone," the voice data is converted into text data saying, "I want a new smartphone."

[1659] Step 2:

[1660] The server analyzes the text data received from the voice recognition software and extracts important keywords using natural language processing technology. The input is text data and the output is keywords. Specifically, the keyword "new smartphone" is extracted from the text data "I want a new smartphone."

[1661] Step 3:

[1662] The server searches a database (e.g., MySQL) for relevant product information based on the extracted keywords. The input is the keywords, and the output is the product information. Specifically, for the keyword "new smartphone," multiple smartphone models are generated as search results.

[1663] Step 4:

[1664] The server sends the search results to the smart device, and the search results are displayed in the salesperson's field of view by the display means of the smart device. The input is product information, and the output is display data. Specifically, a list of new smartphone models is displayed on the screen.

[1665] Step 5:

[1666] The salesperson (user) selects the product to be proposed from the options displayed in the field of view of the smart device. The selected product information is sent to the server. The input is the user's selection information, and the output is the selected product information.

[1667] Step 6:

[1668] The server generates the necessary questions and conditions for providing the product based on the selected product information. The input is the selected product information, and the output is the questions and conditions for providing the product. Specifically, questions about the selected smartphone model (e.g., desired storage capacity, color, etc.) are generated.

[1669] Step 7:

[1670] The smart device displays the generated hearing items and provision conditions in the field of view of the salesperson. The input is the hearing items and provision conditions, and the output is display data. Specifically, a list of questions for the hearing items is displayed on the screen.

[1671] Step 8:

[1672] The salesperson (user) asks questions to the customer based on the hearing items displayed in the field of view and collects the necessary information. The collected information is sent to the server. The input is the collected information, and the output is the hearing content data.

[1673] Step 9:

[1674] The server generates additional questions and confirmation items based on the interview content. The input is the interview content data, and the output is the additional questions and confirmation items. Specifically, if the answer is "1 TB of capacity is required," the server generates an additional ...

Claims

1. a speech recognition means for recognizing input speech in real time; a keyword extraction means for extracting keywords from the voice data recognized by the voice recognition means; a search means for searching for related product information based on the keywords extracted by the keyword extraction means; a display means for displaying the product information searched by the search means; a hearing support means for generating necessary hearing items and provision conditions based on the product information; hearing display means for displaying the generated hearing items and provision conditions; additional question generation means for generating additional questions and confirmation items based on the contents of the hearing during the business negotiation; a minutes generating means for automatically generating minutes based on the business negotiation content; a confirmation means for displaying the minutes created by the minutes creation means and for confirming and correcting the minutes; A means of sharing the confirmed and amended minutes with the client; A system including:

2. 2. The system according to claim 1, wherein the display means is for presenting candidate product information and related services.

3. 2. The system according to claim 1, wherein said hearing support means generates necessary hearing items and provision conditions based on the selected product.

Images