System

Abstract

A system is provided.SOLUTION: The system includes a means for recording a conversation with a customer, a means for converting recorded voice data into character data by voice recognition, a natural language processing means for analyzing the converted character data and extracting an important point, and a means for generating the extracted point as a summary and distributing it to a user.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] There is a demand for support to improve the efficiency of sales activities, such as recording and analyzing conversations with customers, optimizing sales pitches, advising on next actions, and recommending appropriate products. However, performing these tasks manually takes time and effort and can lead to errors. Furthermore, improving sales pitches and selecting effective next actions requires specialized knowledge, which is dependent on the skills of each individual salesperson. Therefore, there is a demand for a system that provides comprehensive support for all sales activities and improves efficiency. [Means for solving the problem]

[0005] In order to solve this problem, the present invention provides the following means.

[0006] 1. This system includes a means for recording conversations with customers and converting the audio data into text data using speech recognition, and a natural language processing means for analyzing the text data, extracting important points, and summarizing them. This allows for quick and accurate recording of conversation content and providing a summary to the user.

[0007] 2. The system includes a database of past successful sales talks, a means to record new sales talks, convert them into text data, and compare them with past successful examples to generate advice on areas for improvement, thereby enabling the effective improvement of sales talks.

[0008] 3. A system that includes a means for collecting and analyzing customer profile and sales status data, a means for evaluating potential next sales actions, selecting the most appropriate action, and delivering it to the user as advice, thereby enabling the next steps in sales activities to be determined effectively.

[0009] 4. The system includes a means for collecting customer profiles and sales histories, analyzing the purchasing patterns of similar customers, and extracting related products from the database, generating recommendation information, and distributing it to users. This allows the system to propose the most suitable products to customers.

[0010] By combining these means, we provide a system that can comprehensively support each process of sales activities and make them more efficient.

[0011] A "customer profile" is data that includes detailed information about a customer, such as the customer's name, company name, contact information, past transaction history, interests, and purchasing habits.

[0012] "Speech recognition" is a technology that analyzes recorded voice data and converts the spoken content into text data.

[0013] "Natural language processing" is a technology that analyzes text data to understand meaning, summarize, and extract keywords.

[0014] "Sales talk" refers to the content of explanations, proposals, and conversations given to customers during sales activities.

[0015] A "database" is a system that stores related data in an organized manner and allows for quick access and retrieval.

[0016] "Related products" refers to products and services that should be suggested to customers based on their customer profile, sales history, purchasing patterns of similar customers, etc.

[0017] "Recorded data" refers to data that records audio information such as conversations in digital format.

[0018] "Important points" refer to matters, key phrases, keywords, etc. that should be particularly noted in the content of the conversation.

[0019] "Sales status data" refers to data including the progress of the current sales process, the stage of the negotiation, the contact history with the customer, the next schedule, and so on.

[0020] "Recommendations" are information that recommends specific products or actions based on analysis results.

[0021] "Extracting the main points" means extracting and summarizing the most important parts from a large amount of information. [Brief explanation of the drawings]

[0022] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4]FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0023] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0024] First, the terms used in the following description will be explained.

[0025] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0026] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0027] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0028] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0029] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0030] [First embodiment]

[0031] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0032] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0033] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0034] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0035] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0036] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0037] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0038] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0039] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0040] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0041] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0042] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0043] An embodiment of the present invention is an integrated sales support system that automatically summarizes conversations with customers, provides advice on improving sales talks, advises on effective sales actions for the next time, and recommends appropriate proposed products. The program and processing of this system are described below with specific examples.

[0044] 1. Automatically summarize customer conversations

[0045] When a user starts a conversation with a customer, the device records the conversation. When the conversation ends, the device sends the recorded data to the server. The server uses a speech recognition engine to convert the voice data into text data. The server then analyzes the text data using a natural language processing algorithm, extracts important points, and generates a summary. The server then sends the generated summary to the device and notifies the user. As a concrete example, a salesperson discusses a new product with a customer, and a summary is generated based on the recording.

[0046] 2. Sales pitch training

[0047] The server maintains a database of past successful sales pitches. When a user makes a new sales pitch, the device records the content and sends it to the server. The server converts the received data into text data using voice recognition, compares it with past successful examples, and analyzes areas for improvement. Based on the analysis results, the server generates advice and sends it to the user's device. As a concrete example, the server analyzes a presentation a salesperson gave to a customer and provides advice such as, "It would be more effective if you gave more specific examples."

[0048] 3. Advice on next steps

[0049] The user inputs a customer profile and current sales status into the terminal. The terminal sends this data to the server, which analyzes the customer profile and sales status data and evaluates potential next sales actions. The server selects the optimal next action, generates advice, and sends it to the terminal. The user can effectively decide on the next step based on this advice. As a specific example, the advice provided to the user is, "It would be effective to demonstrate a new product at the next interview."

[0050] 4. Recommendation of proposed products

[0051] The user enters their customer profile and sales history into the terminal, which then sends the data to the server. Based on this data, the server analyzes the purchasing patterns of similar customers and extracts related products from a database. The server generates recommended products and sends the recommendation information to the terminal. Based on this information, the user can propose appropriate products to the customer. As a specific example, the server recommends related products such as "new printers" and "high-performance computers."

[0052] This system will link the above functions to comprehensively support sales activities and make them more efficient.It is also anticipated that these functions will be provided in a SaaS (Software as a Service) format, and that they will be available to a wide range of salespeople via the Internet.

[0053] The processing flow will be explained below.

[0054] Automatically summarize customer conversations

[0055] Step 1:

[0056] When the user starts a conversation with the customer, the terminal activates the recording function and starts recording the audio of the conversation.

[0057] Step 2:

[0058] After the conversation ends, the device sends the recorded data to the server.

[0059] Step 3:

[0060] The server sends the received voice data to a voice recognition engine, which converts the voice data into text data.

[0061] Step 4:

[0062] The server runs the converted text data through a natural language processing algorithm to analyze the content of the conversation.

[0063] Step 5:

[0064] The server extracts important keywords and phrases and generates a summary.

[0065] Step 6:

[0066] The server sends the generated summary to the user's terminal.

[0067] Step 7:

[0068] The terminal displays the summary on the screen for the user to confirm.

[0069] Sales talk skills training

[0070] Step 1:

[0071] The server stores a database of past successful sales pitches.

[0072] Step 2:

[0073] The user starts a new sales conversation, and the device records the conversation.

[0074] Step 3:

[0075] After the sales talk ends, the device sends the recorded data to the server.

[0076] Step 4:

[0077] The server converts the received voice data into text data using a voice recognition engine.

[0078] Step 5:

[0079] The server runs the converted text data through an analysis algorithm that compares it with past successes.

[0080] Step 6:

[0081] The server generates advice on how to improve the sales pitch based on the analysis results.

[0082] Step 7:

[0083] The server transmits the generated advice to the user's terminal.

[0084] Step 8:

[0085] The terminal notifies the user of the advice, which the user confirms.

[0086] Advice for next action

[0087] Step 1:

[0088] The user enters a customer profile and current business status into the terminal.

[0089] Step 2:

[0090] The terminal transmits the entered customer profile and business status data to the server.

[0091] Step 3:

[0092] The server runs the received data through an analytical algorithm to analyze the customer's situation.

[0093] Step 4:

[0094] The server generates multiple next action candidates based on the analysis results.

[0095] Step 5:

[0096] The server ranks the next action candidates based on their deviation scores and success rates.

[0097] Step 6:

[0098] The server selects the best next action and generates advice.

[0099] Step 7:

[0100] The server sends the generated advice to the user's terminal.

[0101] Step 8:

[0102] The terminal notifies the user of the advice, which the user confirms.

[0103] Recommendation of proposed products

[0104] Step 1:

[0105] The user enters their customer profile and sales history into the terminal.

[0106] Step 2:

[0107] The terminal transmits the input data to the server.

[0108] Step 3:

[0109] The server runs customer profiles and sales history data through analytical algorithms.

[0110] Step 4:

[0111] The server analyzes the purchasing patterns of similar customers and extracts related products from the database.

[0112] Step 5:

[0113] The server generates recommendation information based on the analysis results.

[0114] Step 6:

[0115] The server transmits the generated recommendation information to the user's terminal.

[0116] Step 7:

[0117] The device notifies the user of the recommended information, which the user can then confirm.

[0118] With the above processing flow, the system of the present invention can comprehensively support sales activities and improve efficiency.

[0119] Example 1

[0120] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0121] Conventional sales support systems lacked the means to efficiently summarize conversations with customers and provide specific guidance to improve the quality of sales talks. Furthermore, they lacked a system that could provide advice on the next sales action or recommend appropriate products, which meant that sales activities were not sufficiently streamlined. This created challenges that made it difficult for salespeople to conduct effective sales activities.

[0122] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0123] In this invention, the server includes a means for recording conversations with customers, a means for converting the recorded voice data into character data using a recognition means, a processing means for analyzing the converted character data and extracting important points, and a means for generating a summary of the extracted points and distributing it to the user. This makes it possible to efficiently summarize the content of conversations with customers, allowing the user to quickly grasp the main points.

[0124] The server also includes a means for maintaining a database of past successful sales talks, a means for collecting new sales talks from users and converting the recorded data into text data, a means for comparing the converted text data with past successful examples and generating advice on areas for improvement, and a means for delivering the generated advice to users, thereby enabling specific guidance for improving the quality of sales talks.

[0125] Furthermore, the server includes a means for collecting and analyzing customer profiles and situation data, a means for evaluating next action candidates and selecting the most appropriate action, and a means for delivering the selected action to the user as advice, thereby enabling appropriate advice on the next sales action to be provided, thereby achieving efficiency in sales activities.

[0126] "Customer" means any person or entity to whom products or services are provided.

[0127] "Conversation" refers to the process of information exchange between a customer and a salesperson, which is usually conducted verbally.

[0128] "Recording" refers to the method of recording audio data in digital or analog form.

[0129] "Audio data" refers to data that records human voice as a digital signal.

[0130] "Recognition means" refers to the function of a device or software for converting voice data into text data.

[0131] "Character data" refers to data expressed in text format.

[0132] "Analysis" refers to the process of examining the contents of data in detail to extract meaning and characteristics.

[0133] "Key points" refer to particularly important information or key phrases in the conversation.

[0134] "Processing means" refers to the functionality of a device or software to perform specific data operations or algorithms.

[0135] A "summary" is a short summary that extracts important elements from a long piece of text or a large amount of data.

[0136] "Distribution" refers to the act of delivering information or data to a specific recipient.

[0137] "Talk" refers to the sales pitch or presentation that a salesperson gives to a customer.

[0138] "Database" refers to a system or software for efficiently storing, managing, and retrieving data.

[0139] A "new sales pitch" refers to a new type of sales presentation that has never been done before.

[0140] "Success stories" refer to specific sales pitches or approaches that have produced results in the past.

[0141] "Areas for improvement" refers to areas where changes are needed to make the current way of doing things more effective.

[0142] "Advice" refers to advice or instructions for achieving a specific purpose.

[0143] "Profile" refers to information that compiles a customer's attributes, past transaction history, and other characteristics.

[0144] "Contextual data" refers to data related to current sales activities and customer status.

[0145] "Potential actions" refer to the next possible sales activity options that may be taken.

[0146] "Optimal action" refers to the sales activity that is considered to be the most effective among multiple options.

[0147] "Advice" refers to information or instructions that help you decide what to do next.

[0148] This invention is an integrated sales support system that automatically summarizes conversations with customers, provides advice on improving sales talks, advises on effective sales actions for the next time, and recommends appropriate products to propose. This system is provided in the form of SaaS (Software as a Service) and is expected to be used by a wide range of salespeople via the Internet.

[0149] The specific configuration and operation of the system will be described below.

[0150] Automatically summarize customer conversations

[0151] When a user starts a conversation with a customer, the device (such as a smartphone or PC) records the conversation. When the conversation ends, the device sends the recorded data to a server. The server uses a speech recognition engine such as Google® Cloud Speech-to-Text to convert the voice data into text data. The server then analyzes the text data using a natural language processing algorithm such as NLTK (Natural Language Toolkit), extracts key points, and generates a summary. Finally, the generated summary is sent to the device and notified to the user.

[0152] As a concrete example, a salesperson discusses a new product with a customer and records the conversation. After the conversation ends, the system generates a summary such as "I'm interested in the new product's features" and "I'd like to know more about the price." Example prompts:

[0153] Generate a summary of a conversation with a customer. Here is the transcript of the conversation:

[0154] Sales talk skills training

[0155] The server maintains a database of past successful sales pitches, and when a user makes a new sales pitch, the device records the content and sends it to the server. The server uses Google Cloud Speech-to-Text to convert the audio data into text data. The server then compares it with past successes and analyzes areas for improvement. Advice is generated based on the analysis results and sent to the user's device.

[0156] For example, the system analyzes a salesperson's presentation and provides advice such as, "It would be more effective if you provided more specific examples." Example prompt:

[0157] Please offer some advice to improve the following sales pitch:...

[0158] Advice for next action

[0159] When a user enters their customer profile and current sales status into the device, the device sends this to the server. The server uses a database such as MongoDB to analyze the customer profile and sales status data, evaluate potential next sales actions, select the most appropriate next action, generate advice, and send it to the device.

[0160] As a concrete example, the generated advice is "It would be effective to demonstrate the new product at the next interview." Example prompt sentence:

[0161] Suggest the best next action based on the following customer profile and sales situation:...

[0162] Recommendation of proposed products

[0163] The user enters their customer profile and sales history into the terminal, which then sends it to the server. The server uses a database such as Salesforce to analyze the purchasing patterns of similar customers and extract related products. Recommended products are generated and the recommendation information is sent to the terminal.

[0164] For example, a "new printer" or a "high-performance computer" may be recommended. Example prompts:

[0165] Please recommend the appropriate products based on the following customer profile and sales history:

[0166] By linking the above functions, sales activities can be comprehensively supported and made more efficient.

[0167] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0168] Automatically summarize customer conversations

[0169] Step 1:

[0170] When the user starts a conversation with the customer, the terminal records the conversation.

[0171] Input: Voice of conversation with customer

[0172] Output: Recorded audio data

[0173] Specific operation: A salesperson uses a smartphone to record a discussion with a customer.

[0174] Step 2:

[0175] When the conversation is finished, the user presses the recording end button and the terminal sends the recorded data to the server.

[0176] Input: Recorded audio data

[0177] Output: Audio data sent to the server

[0178] Specific operation: The salesperson presses the end recording button, and the device uploads the audio data to the server.

[0179] Step 3:

[0180] The server uses a speech recognition engine such as Google Cloud Speech-to-Text to convert the voice data into text data.

[0181] Input: Audio data

[0182] Output: Character data

[0183] Specific operation: The server starts a speech recognition engine and converts the voice data into text.

[0184] Step 4:

[0185] The server uses natural language processing algorithms such as NLTK to analyze the text data, extract key points, and generate a summary.

[0186] Input: Character data

[0187] Output: Summary data

[0188] What it does: The server runs a natural language processing algorithm to extract customer points of interest and generate a summary.

[0189] Step 5:

[0190] The generated summary is sent from the server to the terminal, which notifies the user of it.

[0191] Input: Summary data

[0192] Output: Summary data reported to the user

[0193] Specific operation: The terminal uses the notification function to display the summary content to the user.

[0194] Sales talk skills training

[0195] Step 1:

[0196] When a user gives a sales pitch, the device records the content.

[0197] Input: Sales talk audio

[0198] Output: Recorded audio data

[0199] Specific operation: Start recording when the salesperson starts talking.

[0200] Step 2:

[0201] After the recording is completed, the device sends the recorded data to the server.

[0202] Input: Recorded audio data

[0203] Output: Audio data sent to the server

[0204] Specific operation: When the talk ends, the data is automatically sent to the server.

[0205] Step 3:

[0206] The server uses a voice recognition engine to convert the recorded data into text data.

[0207] Input: Audio data

[0208] Output: Character data

[0209] Specific operation: The server converts the voice data into text using Google Cloud Speech-to-Text.

[0210] Step 4:

[0211] The server compares the newly received text data with past successes and analyzes areas for improvement.

[0212] Input: Text data, past success data

[0213] Output: Improvements

[0214] How it works: The server compares new data with past databases and uses algorithms to identify areas for improvement.

[0215] Step 5:

[0216] The server transmits the generated advice to the terminal, which notifies the user of it.

[0217] Input: Advice data

[0218] Output: Advice given to the user

[0219] Specific action: The device displays a notification to the user and suggests specific improvements.

[0220] Advice for next action

[0221] Step 1:

[0222] The user enters a customer profile and current business status into the terminal.

[0223] Input: Customer profile, sales status

[0224] Output: Data entered into the terminal

[0225] Specific operation: A salesperson enters customer information and sales progress through the application.

[0226] Step 2:

[0227] The terminal transmits customer profile and sales status data to the server.

[0228] Input: Customer profile, sales status

[0229] Output: Data sent to the server

[0230] Specific operation: The input data is automatically sent to the server.

[0231] Step 3:

[0232] The server uses a database such as MongoDB to analyze the data and evaluate potential next sales actions.

[0233] Input: Customer profile, sales status data

[0234] Output: Evaluated action candidates

[0235] What happens: The server takes in the data and uses an algorithm to determine the next step.

[0236] Step 4:

[0237] The server selects the optimal next action, generates advice, and sends it to the terminal.

[0238] Input: Evaluated action candidates

[0239] Output: Generated advisory data

[0240] Specific operation: The server uses an algorithm to determine the most effective action for the customer and generates it as advice.

[0241] Step 5:

[0242] The terminal notifies the user of the advice content.

[0243] Input: Advisory data

[0244] Output: Advice given to the user

[0245] Specific operation: The terminal notifies the user of advice on the next action.

[0246] Recommendation of proposed products

[0247] Step 1:

[0248] The user enters their customer profile and sales history into the terminal.

[0249] Input: Customer profile, sales history

[0250] Output: Data entered into the terminal

[0251] What happens: A salesperson uses the application to enter customer and past purchase history.

[0252] Step 2:

[0253] The terminal transmits this data to the server.

[0254] Input: Customer profile, sales history

[0255] Output: Data sent to the server

[0256] Specific operation: The entered data is transferred from the terminal to the server.

[0257] Step 3:

[0258] The server uses databases such as Salesforce to analyze the purchasing patterns of similar customers and extract related products.

[0259] Input: Customer profile, sales history

[0260] Output: List of related products

[0261] What happens: The server queries the database and extracts the most relevant products.

[0262] Step 4:

[0263] The server generates recommended products and sends the recommendation information to the terminal.

[0264] Input: List of related products

[0265] Output: Recommended product data

[0266] Specific operation: The server uses a recommendation algorithm to select the most suitable product for the customer.

[0267] Step 5:

[0268] The terminal notifies the user of the recommended information.

[0269] Input: Recommended product data

[0270] Output: Recommendation information notified to the user

[0271] Specific operation: The terminal displays a list of suggested products to the user.

[0272] This detailed processing flow allows the functions of the present invention to work together, comprehensively supporting sales activities and making them more efficient.

[0273] (Application example 1)

[0274] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0275] In modern sales and marketing activities, it is important to effectively manage customer interactions, efficiently summarize their content, and improve sales pitches by referencing past success stories. However, performing these processes manually takes time and effort, placing a heavy burden on sales representatives. Furthermore, advice on next sales actions and proposals for appropriate products must also be provided quickly and accurately, but current systems are unable to meet these needs.

[0276] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0277] In this invention, the server includes a means for recording conversations with customers, a means for converting the recorded voice data into text data using speech recognition, and a natural language processing means for analyzing the converted text data and extracting key points. This enables efficient summaries of conversations with customers and delivering them to users. The server also includes a means for maintaining a database of past successful sales pitches, a means for collecting new sales pitches from users and converting the recorded data into text data, a means for comparing the converted text data with past successes and generating advice on areas for improvement, a means for providing advice in real time, and a means for utilizing a generative AI model. This allows users to receive effective improvement advice in real time based on comparisons with past successes. The server also includes a means for collecting and analyzing customer profiles and sales status data, a means for evaluating potential next sales actions and selecting the optimal action, a means for analyzing user input through prompts, and a means for recommending related products based on the customer profile. This enables quick and accurate recommendations for next sales actions and proposed products.

[0278] "Customer profile" refers to information about a customer, such as their attributes, purchasing history, and interests.

[0279] "Sales talk" refers to the way salespeople talk and the explanations they give to customers to promote sales.

[0280] "Speech recognition" refers to the technology of analyzing recorded voice data and converting it into text data.

[0281] "Natural language processing" refers to the technology of analyzing text data and understanding its meaning and structure.

[0282] A "summary" refers to information that has been extracted from analyzed text data and concisely summarized.

[0283] "Database" refers to a system for efficiently storing information and making it searchable and accessible.

[0284] "Real-time" refers to processing or response occurring immediately, without delay.

[0285] A "generative AI model" refers to an artificial intelligence model that uses machine learning to learn patterns from large amounts of data, and then generates and analyzes new data.

[0286] A "prompt sentence" refers to an instruction sentence or question sentence that a user inputs to a system.

[0287] "Recommendation" refers to the act of suggesting optimal products or services based on a user's preferences and behavioral history.

[0288] The present invention relates to a sales support system, and in particular to a system that summarizes conversations with customers, provides advice on improving sales talks, gives advice on the next sales action, and recommends appropriate products through an application that runs on a device such as a smartphone.

[0289] The details of the system embodiment will be described below.

[0290] 1. Automatic summarization of customer conversations

[0291] When a user starts a conversation with a customer, the device (e.g., a smartphone) records the conversation. When the conversation ends, the device sends the recorded data to a server. The server converts the voice data into text data using a speech recognition engine (e.g., Google Speech Recognition). The server then analyzes the converted text data using a natural language processing algorithm (e.g., the "T5" model from the Transformers library), extracts important points, and generates a summary. The generated summary is distributed to the device via an intranet or the Internet, and is notified to the user. As a concrete example, a sales representative may discuss the features of a new product with a customer, and a summary may be generated based on the recording.

[0292] 2. Sales pitch training

[0293] The server maintains a database of past successful sales pitches. When a user gives a new sales pitch, the device records the content and sends it to the server. The server converts the received data into text data using voice recognition, compares it with past successful examples, and analyzes areas for improvement. Based on the analysis results, the server generates advice and delivers it to the user's device. As a concrete example, the server analyzes a presentation a sales representative gave to a customer and provides advice such as "it would be good to include more specific examples."

[0294] 3. Advice on next steps

[0295] The user inputs the customer profile and current sales situation into the device. The device sends this data to the server, which uses a generative AI model to analyze the customer profile and sales situation data and evaluate potential next sales actions. The server selects the optimal next action, generates advice, and delivers it to the device. The user can use this advice to effectively decide on the next step. As a specific example, the advice provided to the user is, "It would be effective to demonstrate a new product at the next interview."

[0296] 4. Recommendation of proposed products

[0297] The user enters their customer profile and sales history into the device, and the device sends the data to the server. The server uses a generative AI model to analyze the purchasing patterns of similar customers based on the customer profile and extract related products from a database. The server analyzes the information through prompt text, generates recommended products, and delivers the recommended information to the device. Based on this information, the user proposes appropriate products to the customer. As a specific example, the server recommends related products such as "new printers" and "high-performance computers."

[0298] Prompt Sentence Examples

[0299] "After a five-minute conversation with a customer about new product features, you have the following transcript. Please summarize it:

[0300] "Hello, customer. Today I'd like to introduce you to our new X900 series printer. This printer is extremely high-performance... (omitted)"

[0301] This system enables users to efficiently manage conversations with customers and conduct effective sales activities.

[0302] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0303] Step 1:

[0304] When a user starts a conversation with a customer, the user uses a terminal (smartphone) to record the conversation.

[0305] Input: Voice of conversation with customer

[0306] Output: Recorded audio file

[0307] What it does: Uses the device's recording feature to save a conversation with a customer as an audio file.

[0308] Step 2:

[0309] When the conversation ends, the terminal transmits the recorded voice data to the server.

[0310] Input: Recorded audio file

[0311] Output: Audio data uploaded to the server

[0312] How it works: Uploads recorded data from the device to a cloud server or database.

[0313] Step 3:

[0314] The server uses a speech recognition engine to convert the voice data into text data.

[0315] Input: Audio file

[0316] Output: Converted text data

[0317] What it does: Calls a speech recognition engine (e.g., Google Speech Recognition API) to convert an audio file into text.

[0318] Step 4:

[0319] The server analyzes the text data using natural language processing algorithms, extracts key points, and generates a summary.

[0320] Input: Text data

[0321] Output: Summary text

[0322] How it works: It uses natural language processing algorithms (e.g., Hugging Face Transformers' "T5" model) to extract key points and generate a summary.

[0323] Step 5:

[0324] The server delivers the generated summary to the terminal.

[0325] Input: Summary text

[0326] Output: Summary text delivered to the user's device

[0327] Action: Sends summary text to the terminal and notifies the user.

[0328] Step 6:

[0329] When a user makes a new sales pitch, the device records the content and sends it to the server.

[0330] Input: Sales pitch audio file

[0331] Output: Audio data uploaded to the server

[0332] Operation: Uploads the recorded audio data of the sales talk to the server.

[0333] Step 7:

[0334] The server converts the received voice data into text data using voice recognition.

[0335] Input: Audio file

[0336] Output: Converted text data

[0337] How it works: Uses a speech recognition engine to convert voice data into text data.

[0338] Step 8:

[0339] The server compares the text data with past success stories, analyzes areas for improvement, and generates advice.

[0340] Input: Text data, past success data

[0341] Output: Advice text

[0342] How it works: Text is vectorized using TfidfVectorizer, compared to past successes using cosine similarity, and a generative AI model is used to identify areas for improvement and generate advice.

[0343] Step 9:

[0344] The server distributes the generated advice to the user's terminal.

[0345] Input: Advice text

[0346] Output: Advice text delivered to the user's device

[0347] Action: Sends advice text to the user's device and notifies them.

[0348] Step 10:

[0349] The user enters a customer profile and current business status into the terminal.

[0350] Input: Customer profile, sales status data

[0351] Output: Profile and status data sent from the device to the server

[0352] How it works: A user enters their profile and status into a terminal and sends it to a database.

[0353] Step 11:

[0354] The server uses the generative AI model to analyze customer profiles and sales status data and select potential next sales actions.

[0355] Input: Customer profile, sales status data

[0356] Output: Text of advice on optimal sales actions

[0357] How it works: Analyzes data using a generative AI model to evaluate and select the next sales action.

[0358] Step 12:

[0359] The server delivers the selected action to the user as a recommendation.

[0360] Input: Sales action advice text

[0361] Output: Advice text delivered to the user's device

[0362] Operation: The advice is sent to the terminal and notified.

[0363] Step 13:

[0364] The user enters a customer profile and sales history into the terminal.

[0365] Input: Customer profile, sales history data

[0366] Output: Profile and sales history data sent from the device to the server

[0367] How it works: A user enters their profile and history into a terminal and sends it to a database.

[0368] Step 14:

[0369] The server uses a generative AI model to analyze the purchasing patterns of similar customers based on the customer profile and recommend related products.

[0370] Input: Customer profile, sales history data

[0371] Output: Recommended product list

[0372] How it works: Analyzes data using a generative AI model to recommend relevant products.

[0373] Step 15:

[0374] The server delivers a list of recommended products to the user's terminal.

[0375] Input: Recommended product list

[0376] Output: Product list delivered to the user's device

[0377] Operation: The recommended product list is sent to the device and notified.

[0378] For example, if you set the prompt text as follows:

[0379] "After a five-minute conversation with a customer about new product features, you have the following transcript. Please summarize it:

[0380] "Hello, customer. Today I'd like to introduce you to our new X900 series printer. This printer is extremely high-performance... (omitted)"

[0381] By using prompts in this way, users can efficiently issue instructions to the system and receive optimal summaries, advice, and recommendations.

[0382] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0383] This embodiment of the present invention combines an emotion engine that recognizes the user's emotions with an integrated sales support system that automatically summarizes conversations with customers, provides advice on improving sales talks, advises on effective next sales actions, and recommends appropriate products to propose. The program and processing of this system are described below with specific examples.

[0384] 1. Automatically summarize customer conversations

[0385] When a user starts a conversation with a customer, the device activates the recording function and records the audio of the conversation. When the conversation ends, the device sends the recorded data to the server. The server converts the received audio data into text data using a speech recognition engine. The server then runs the text data through a natural language processing algorithm to extract key points and generate a summary. Furthermore, an emotion engine analyzes the user's emotions from the converted text and audio data and adjusts the summary content accordingly. Finally, the server sends the generated summary to the user's device, where the user can review it. As a concrete example, a salesperson discusses a new product with a customer, and a summary is generated based on the recording, reflecting the content in line with the emotions.

[0386] 2. Sales pitch training

[0387] The server maintains a database of past successful sales pitches. When a user gives a new sales pitch, the device records the content and sends it to the server. The server converts the received data into text data using a voice recognition engine and runs an analysis algorithm that compares the converted text data with past successful examples. The server generates advice on how to improve the sales pitch based on the user's emotional data. The advice generated by the server is sent to the user's device, where the user can confirm it. It is also possible to provide flexible advice based on the user's emotions. As a specific example, a salesperson's presentation to a customer is analyzed and emotionally sensitive advice such as "it would be more effective if you gave more specific examples" is provided.

[0388] 3. Advice on next steps

[0389] The user enters a customer profile and current sales status into the terminal. The terminal sends this data to the server. The server analyzes the customer profile and sales status data and runs it through an algorithm that evaluates potential next sales actions. The server selects the optimal next action based on the evaluation results and uses an emotion engine to generate advice that takes the user's emotions into consideration. The server sends the generated advice to the user's terminal, where the user can confirm it. As a specific example, advice such as "It would be effective to demonstrate a new product at the next interview" is provided in a form that is appropriate to the user's emotions.

[0390] 4. Recommendation of proposed products

[0391] The user enters their customer profile and sales history into the terminal, which then sends the data to the server. The server uses this data to analyze the purchasing patterns of similar customers and extracts related products from a database. The server uses an emotion engine to generate recommendation information that takes the emotional data into account. The server then sends the recommendation information to the user's terminal, where the user confirms it. As a specific example, the server provides related products such as "new printers" and "high-performance computers" as recommendation information tailored to the user's emotions.

[0392] By linking the above functions, this system comprehensively supports sales activities, improving efficiency and effectiveness. In addition, the combination of an emotion engine enables flexible responses that take user emotions into consideration, allowing for more personalized support.

[0393] The processing flow will be explained below.

[0394] Automatically summarize customer conversations

[0395] Step 1:

[0396] The user starts a conversation with the customer. The device activates the recording function and starts recording the audio of the conversation.

[0397] Step 2:

[0398] After the conversation ends, the device sends the recorded data to the server.

[0399] Step 3:

[0400] The server sends the received voice data to a voice recognition engine, which converts the voice data into text data.

[0401] Step 4:

[0402] The server sends the converted text data to a natural language processing algorithm, which analyzes the text data.

[0403] Step 5:

[0404] The server extracts important keywords and phrases and generates a summary.

[0405] Step 6:

[0406] The server sends the voice and text data to the emotion engine, which analyzes the user's emotions.

[0407] Step 7:

[0408] The server adjusts the summary content based on the analysis results of the emotion engine.

[0409] Step 8:

[0410] The server sends the generated summary to the user's terminal.

[0411] Step 9:

[0412] The terminal displays the summary and the user confirms it.

[0413] Sales talk skills training

[0414] Step 1:

[0415] A server maintains a database of past successful sales pitches.

[0416] Step 2:

[0417] The user makes a new sales pitch, and the device records the content.

[0418] Step 3:

[0419] After the sales talk ends, the device sends the recorded data to the server.

[0420] Step 4:

[0421] The server sends the received voice data to a voice recognition engine, which converts it into text data.

[0422] Step 5:

[0423] The server sends the converted text data to an analysis algorithm that compares it with past successes.

[0424] Step 6:

[0425] The server generates advice on how to improve the sales pitch based on the analysis results.

[0426] Step 7:

[0427] The server sends text data to the emotion engine, which analyzes the user's emotions.

[0428] Step 8:

[0429] The server adjusts the advice content based on the analysis results of the emotion engine.

[0430] Step 9:

[0431] The server transmits the generated advice to the user's terminal.

[0432] Step 10:

[0433] The terminal notifies the user of the advice, which is then confirmed by the user.

[0434] Advice for next action

[0435] Step 1:

[0436] The user enters a customer profile and current business status into the terminal.

[0437] Step 2:

[0438] The terminal transmits the input data to the server.

[0439] Step 3:

[0440] The server sends customer profile and sales data to an analytics algorithm.

[0441] Step 4:

[0442] The server generates the next sales action candidates based on the analysis results.

[0443] Step 5:

[0444] The server sends the user's emotional data to the emotion engine, which analyzes the emotions.

[0445] Step 6:

[0446] Based on the analysis results of the emotion engine, the server ranks and adjusts potential actions.

[0447] Step 7:

[0448] The server selects the best next action and generates advice.

[0449] Step 8:

[0450] The server sends the generated advice to the user's terminal.

[0451] Step 9:

[0452] The terminal notifies the user of the advice and the user confirms it.

[0453] Recommendation of proposed products

[0454] Step 1:

[0455] The user enters their customer profile and sales history into the terminal.

[0456] Step 2:

[0457] The terminal transmits the input data to the server.

[0458] Step 3:

[0459] The server sends customer profiles and sales histories to an analytics algorithm.

[0460] Step 4:

[0461] The server analyzes the purchasing patterns of similar customers and extracts related products from the database.

[0462] Step 5:

[0463] The server sends the user's emotional data to the emotion engine, which analyzes the emotions.

[0464] Step 6:

[0465] The server adjusts the recommendations based on the analysis results of the emotion engine.

[0466] Step 7:

[0467] The server transmits the generated recommendation information to the user's terminal.

[0468] Step 8:

[0469] The device notifies the user of the recommended information, which is then confirmed by the user.

[0470] As a result, the system of the present invention, which incorporates an emotion engine, can comprehensively support sales activities and respond flexibly while taking into account the user's emotions. As a specific example, the emotion engine can analyze the emotional state of a salesperson, and if the salesperson is feeling stressed, it can provide personalized support, such as advice on how to speak calmly.

[0471] Example 2

[0472] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0473] Conventional sales support systems have the problem of being unable to automatically and efficiently summarize conversations with customers, give advice on improving sales talks, or advise on next actions.In addition, these functions are not adjusted according to the user's emotions, and only standardized suggestions and advice are provided, which has led to issues that do not provide sufficient satisfaction to users.

[0474] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for recording a conversation with a customer, a means for converting the recorded voice data into text data using a voice recognition engine, a means for analyzing the converted text data using a natural language processing algorithm and extracting important points, a means for generating a summary from the extracted points and adjusting the summary content according to the user's emotions using an emotion engine, and a means for delivering the adjusted summary to the user. This makes it possible to provide summaries and advice that take the user's emotions into consideration, which is expected to improve the efficiency and effectiveness of sales activities.

[0475] The "means for recording conversations with customers" refers to a device or software for recording conversations as audio data when a user starts a conversation with a customer.

[0476] A "speech recognition engine" is software or algorithms that convert recorded voice data into text data.

[0477] A "natural language processing algorithm" is an algorithm for analyzing text data and extracting important information and patterns.

[0478] An "emotion engine" is software or an algorithm that analyzes emotions from text and voice data and makes adjustments based on those emotions.

[0479] A "summary generation means" is a device or software that extracts important points and creates a short summary of the text.

[0480] The "distribution means" refers to a device or software for transmitting the generated information or advice to the user's terminal and displaying it.

[0481] A "database" is an information accumulation system that stores data such as past successful sales pitches, customer profiles, and sales history, and can be searched and used as needed.

[0482] An "analysis algorithm" is an algorithm that analyzes collected data and finds patterns and trends.

[0483] "Potential sales actions" are suggestions for specific actions that can be taken in the next sales activity.

[0484] The "means for delivering advice" is a device or software that notifies the user of the optimal action to take in the next sales activity and provides it as a guideline for action.

[0485] The present invention is an integrated sales support system that automatically summarizes conversations with customers, provides advice on improving sales talks, advises on effective next sales actions, and recommends appropriate products. By combining this system with an emotion engine that recognizes the user's emotions, the present invention can provide more personalized support. The specific configuration and operation of this system are described below.

[0486] 1. Automatically summarize customer conversations

[0487] When a user starts a conversation with a customer, the device activates the recording function and records the audio of the conversation. A recording application (e.g., Recorder App) is installed on the device. When the conversation ends, the device sends the recorded data to the server. The recorded data is sent to the server using an HTTP request (e.g., in MP3 format). The server converts the received audio data into text data using a speech recognition engine (e.g., Google Cloud Speech-to-Text). The server then uses a natural language processing algorithm (e.g., BERT or GPT-3®) to analyze the text data, extract key points, and generate a summary. After that, an emotion engine (e.g., Microsoft® Azure® Emotion API) analyzes the user's emotions from the converted text and audio data and adjusts the summary content. Finally, the server sends the generated summary to the user's device, where the user can review it.

[0488] For example, if a salesperson is discussing a new product with a customer, a summary is generated based on the recording, and the content reflects the customer's emotions.

[0489] Example prompt sentence:

[0490] "Summarize key points from recordings of customer conversations, analyze sentiment, and adjust accordingly."

[0491] 2. Sales pitch training

[0492] The server maintains a database of past successful sales pitches. When a user gives a new sales pitch, the device records the content and sends it to the server. The server converts the received data into text data using a voice recognition engine and runs an analysis algorithm (e.g., a similarity calculation algorithm) that compares the converted text data with past successful examples. The server generates advice on how to improve the sales pitch based on the user's emotional data. The generated advice is adjusted according to the user's emotions using an emotion engine. The server sends the adjusted advice to the user's device, where the user confirms it.

[0493] As a concrete example, the system analyzes a presentation a salesperson gives to a customer and provides emotionally sensitive advice such as "It would be more effective if you provided more specific examples."

[0494] Example prompt sentence:

[0495] "Please compare my current sales pitch with past successful ones and give me advice on how to improve it."

[0496] 3. Advice on next steps

[0497] The user inputs their customer profile and current sales situation into their device. The device then sends this data to the server. The server then runs this data through an analytical algorithm (e.g., a machine learning model) to evaluate potential next sales actions. The server then selects the optimal next action based on the evaluation results and uses an emotion engine to generate advice that takes the user's emotions into account. The server then sends the generated advice to the user's device, where the user confirms it.

[0498] As a specific example, advice such as "It would be effective to demonstrate a new product at the next interview" is provided in accordance with the user's emotions.

[0499] Example prompt sentence:

[0500] "Please advise me on my next action based on my customer profile and sales situation."

[0501] 4. Recommendation of proposed products

[0502] The user enters their customer profile and sales history into the terminal, which then sends the data to the server. The server uses this data to analyze the purchasing patterns of similar customers and extracts related products from the database. The server then uses an emotion engine to generate recommendation information that takes into account the emotion data. The generated recommendation information is then sent by the server to the user's terminal, where the user can confirm it.

[0503] As a specific example, the server provides related products such as "new printers" and "high-performance computers" as recommended information tailored to the user's emotions.

[0504] Example prompt sentence:

[0505] "Recommend relevant products based on customer profile and sales history."

[0506] By linking the above functions, this system comprehensively supports sales activities, improving efficiency and effectiveness. In addition, the combination of an emotion engine enables flexible responses that take into account the user's emotions, allowing for more personalized support.

[0507] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0508] Step 1:

[0509] When a user starts a conversation with a customer, the device automatically activates the recording function and collects the audio data of the conversation. Specifically, the recording application installed on the device starts running in the background. The recording application receives audio data as input and outputs a recorded audio file (e.g., MP3 format).

[0510] Step 2:

[0511] Once the conversation is finished, the device sends the recording to the server using an HTTP POST request to the server's API endpoint. The device receives the recorded audio file as input and outputs an HTTP request.

[0512] Step 3:

[0513] The server converts the received voice data into text data using a voice recognition engine. Specifically, the server calls the API of the voice recognition engine (for example, Google Cloud Speech-to-Text), provides the voice data as input, and receives the converted text data as output.

[0514] Step 4:

[0515] The server applies a natural language processing algorithm to the acquired text data to extract key points. Specifically, the server inputs the text data into a natural language processing algorithm (e.g., BERT or GPT-3) and outputs key points as the analysis results.

[0516] Step 5:

[0517] The server uses an emotion engine to analyze emotions from text data and voice data. Specifically, the server calls the API of the emotion engine (for example, Microsoft Azure Emotion API), provides text data and voice data as input, and receives the emotion analysis results as output.

[0518] Step 6:

[0519] The server adjusts the summary content based on the sentiment analysis results and generates the final summary. Specifically, the server receives the sentiment analysis results and key points as input and outputs the adjusted summary.

[0520] Step 7:

[0521] The server sends the generated summary to the user's device. Specifically, the server uses an HTTP response to send the summary data to the user's device. The user's device receives the summary data as input and outputs a custom UI to display to the user.

[0522] This allows the content of the conversation with the customer to be summarized and presented in a form that matches the user's emotions.

[0523] (Application example 2)

[0524] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0525] Conventional sales support systems are limited in their functionality, such as efficiently summarizing conversations with customers, providing advice on improving sales talks, advising on next actions, and recommending proposed products. Furthermore, few systems can take user emotions into account when it comes to these functions, and more personalized support is desired. Therefore, there is a need for a comprehensive sales support system that can summarize conversations with customers, provide guidance on sales talks, advise on next actions, recommend proposed products, and also consider user emotions.

[0526] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0527] In this invention, the server includes means for recording conversations with customers, means for converting the recorded voice data into text data using voice recognition, natural language processing means for analyzing the converted text data and extracting important points, means for generating a summary of the extracted points and distributing it to the user, and means for recognizing the user's emotions and reflecting them in the summary content, thereby enabling efficient sales support that takes the user's emotions into consideration.

[0528] The server also includes means for maintaining a database of past successful sales talks, means for collecting new sales talks from users and converting the recorded data into text data, means for comparing the converted text data with past successful examples and generating advice on areas for improvement, means for delivering the generated advice to users, and means for generating and delivering flexible advice taking into account the user's emotions, thereby enabling the quality of sales talks to be improved while taking into account emotions.

[0529] Furthermore, the server includes means for collecting and analyzing customer profiles and sales status data, means for evaluating next sales action candidates and selecting the optimal action, means for delivering the selected action to the user as advice, and means for generating and delivering advice content that takes the user's emotions into consideration, thereby making the next sales activity more effective and personalized.

[0530] "Customer" refers to the person to whom goods or services are provided in a business transaction or transaction.

[0531] "Conversation recording means" refers to a device or function for recording a conversation with a customer as audio data.

[0532] "Speech recognition means" refers to technology or software that analyzes recorded voice data and converts it into text data.

[0533] "Character data" refers to text information converted by a speech recognition means.

[0534] "Natural language processing means" refers to algorithms and technologies for analyzing text data and extracting important points.

[0535] "Summary generation means" refers to a function that concisely summarizes a text based on extracted important points.

[0536] "Emotion recognition means" refers to the technology and algorithms that determine and analyze a user's emotions from text data and voice data.

[0537] A "sales pitch database" refers to a system or device that stores records and data of past successful sales pitches.

[0538] "Means for generating improvement advice" refers to a function that compares new sales pitches with existing successful examples and suggests more effective methods.

[0539] "Customer profile" refers to detailed data such as basic customer information and past transaction history.

[0540] "Sales status data" refers to information regarding the progress and results of current sales activities.

[0541] "Action evaluation means" refers to a function for evaluating potential next sales actions and selecting the most appropriate course of action.

[0542] The "advice delivery means" refers to a function for transmitting the generated advice or advice to the user.

[0543] This invention combines an emotion engine that recognizes user emotions with an integrated sales support system that automatically summarizes conversations with customers, provides advice on improving sales talks, advises on effective next sales actions, and recommends appropriate products. This system is designed to support customer service in brick-and-mortar stores in particular.

[0544] The entire system functions by linking the server and the terminal. Specifically, it has the following configuration:

[0545] 1. Automatically summarize customer conversations

[0546] The user wears smart glasses as a device and records conversations with customers. The recorded audio data is sent to a server in real time. The server converts the audio data into text data using Google Cloud Speech-to-Text. The converted text data is analyzed using natural language processing with the BERT model to extract key points. A summary is then generated, and Affectiva's emotion recognition engine analyzes the user's emotions and reflects them in the summary. The generated summary is then sent back to the user's device, where the user can review it.

[0547] 2. Sales pitch training

[0548] The server maintains a database of past successful sales pitches. When a user makes a new sales pitch, the smart glasses record it and send the data to the server. The server converts the recorded data into text data using Google Cloud Speech-to-Text, analyzes the converted data using the BERT model, and compares it with past successful examples. Based on the analysis results, advice on areas for improvement is generated, and an emotion engine provides advice based on the user's emotions. As a result, flexible advice is delivered to the user's device.

[0549] 3. Advice on next steps

[0550] When a user enters their customer profile and current sales status into their device, the data is sent to the server. The server analyzes the profile and sales status and evaluates the next sales action candidate using the BERT model and machine learning algorithm (K-means clustering). The emotion engine generates advice on the optimal next action that reflects the user's emotions and delivers it to the user's device.

[0551] 4. Recommendation of proposed products

[0552] Users enter their customer profile and sales history, and the data is sent to the server. The server analyzes the purchasing patterns of similar customers and extracts related products. The emotion engine then generates recommendation information that takes emotional data into account, and delivers it to the user's device. This allows users to recommend suggested products to customers at the appropriate time.

[0553] Example prompt sentence:

[0554] "Please convert the following Japanese audio data into text and summarize it."

[0555] "Perform a sentiment analysis of this text and generate sentiment-based advice."

[0556] "Recommend the next most effective sales action based on your customer profile."

[0557] This system will improve the efficiency of sales activities in brick-and-mortar stores and enable effective responses to customers. By instantly analyzing information obtained during conversations with customers and providing advice based on their emotions, the quality of sales activities can be improved.

[0558] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0559] Step 1:

[0560] A user uses smart glasses to record a conversation with a customer. The recorded voice data is sent to the server in real time. The input is the voice data, and the output is the raw voice data sent to the server.

[0561] Step 2:

[0562] The server receives the voice data and converts it into text data using Google Cloud Speech-to-Text. The input is voice data and the output is text data. Voice analysis is performed as part of the data processing.

[0563] Step 3:

[0564] The server analyzes the converted text data using the BERT model and extracts key points. The input is text data, and the output is a list of key points. Natural language processing is performed on the data.

[0565] Step 4:

[0566] The server generates a summary based on the extracted key points. The input is a list of key points, and the output is a summary. The data processing performed here is the generation of a summary.

[0567] Step 5:

[0568] The server uses Affectiva's emotion recognition engine to analyze the user's emotions from text and voice data. The input is text and voice data, and the output is emotion data. The process performed is emotion analysis.

[0569] Step 6:

[0570] The generated summary is adjusted by reflecting emotional data. The input is the summary and emotional data, and the output is the summary that reflects the emotion. The emotional data is reflected as a data calculation.

[0571] Step 7:

[0572] The final summary is delivered to the user's device. The input is the summary that reflects the emotion, and the output is the summary delivered to the user. The specific operation is that the device receives the summary and displays it to the user.

[0573] Step 8:

[0574] A database of past successful sales pitches is maintained by the server. The input is the database, and the output is the data of successful cases. The operation is to execute a query on the database.

[0575] Step 9:

[0576] The server converts newly recorded sales talks by the user into text data and compares them with past successes. The input is the newly recorded data, and the output is the comparison results. Data processing involves voice analysis and comparison calculations.

[0577] Step 10:

[0578] The server generates improvement advice and adjusts it based on the emotion data. The input is the comparison result and emotion data, and the output is advice based on the emotion. The processing involves generating advice and reflecting the emotion data.

[0579] Step 11:

[0580] The generated advice is delivered to the user's device. The input is the advice according to the emotion, and the output is the advice delivered to the user. The specific operation is that the device receives the advice and displays it to the user.

[0581] Step 12:

[0582] Customer profile and sales status data are sent from the terminal to the server. The input is the profile data and sales status data, and the output is the data sent to the server. The action is sending the data.

[0583] Step 13:

[0584] The server analyzes the profile and sales status data and evaluates the next optimal sales action. The input is the customer profile and sales status data, and the output is the next action candidate. The data processing is evaluated using machine learning.

[0585] Step 14:

[0586] Based on the evaluation results, the system generates advice on the optimal next action using emotional data. The input is the next action candidate and emotional data, and the output is advice based on the emotion. The processing involves generating advice and reflecting the emotional data.

[0587] Step 15:

[0588] The generated advice is delivered to the user's device. The input is advice corresponding to the emotion, and the output is the advice delivered to the user. The specific operation is for the device to receive the advice and display it to the user.

[0589] Step 16:

[0590] The user enters a customer profile and sales history, which is sent to the server. The input is the profile data and sales history, and the output is the data sent to the server. The action is sending the data.

[0591] Step 17:

[0592] The server analyzes the purchasing patterns of similar customers and extracts related products. The input is profile data and sales history, and the output is a list of related products. The data processing is done using machine learning analysis.

[0593] Step 18:

[0594] Recommendation information is generated using an emotion engine and delivered to users. The input is a list of related products and emotion data, and the output is recommendation information based on the emotion. The processing involves generating recommendation information and reflecting the emotion data.

[0595] Step 19:

[0596] The final recommendation information is delivered to the user's device. The input is recommendation information based on emotions, and the output is the recommendation information delivered to the user. The operation is for the device to receive the recommendation information and display it to the user.

[0597] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0598] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0599] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0600] [Second embodiment]

[0601] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0602] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0603] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0604] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0605] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0606] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0607] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0608] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0609] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0610] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0611] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0612] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0613] An embodiment of the present invention is an integrated sales support system that automatically summarizes conversations with customers, provides advice on improving sales talks, advises on effective sales actions for the next time, and recommends appropriate proposed products. The program and processing of this system are described below with specific examples.

[0614] 1. Automatically summarize customer conversations

[0615] When a user starts a conversation with a customer, the device records the conversation. When the conversation ends, the device sends the recorded data to the server. The server uses a speech recognition engine to convert the voice data into text data. The server then analyzes the text data using a natural language processing algorithm, extracts important points, and generates a summary. The server then sends the generated summary to the device and notifies the user. As a concrete example, a salesperson discusses a new product with a customer, and a summary is generated based on the recording.

[0616] 2. Sales pitch training

[0617] The server maintains a database of past successful sales pitches. When a user makes a new sales pitch, the device records the content and sends it to the server. The server converts the received data into text data using voice recognition, compares it with past successful examples, and analyzes areas for improvement. Based on the analysis results, the server generates advice and sends it to the user's device. As a concrete example, the server analyzes a presentation a salesperson gave to a customer and provides advice such as, "It would be more effective if you gave more specific examples."

[0618] 3. Advice on next steps

[0619] The user inputs a customer profile and current sales status into the terminal. The terminal sends this data to the server, which analyzes the customer profile and sales status data and evaluates potential next sales actions. The server selects the optimal next action, generates advice, and sends it to the terminal. The user can effectively decide on the next step based on this advice. As a specific example, the advice provided to the user is, "It would be effective to demonstrate a new product at the next interview."

[0620] 4. Recommendation of proposed products

[0621] The user enters their customer profile and sales history into the terminal, which then sends the data to the server. Based on this data, the server analyzes the purchasing patterns of similar customers and extracts related products from a database. The server generates recommended products and sends the recommendation information to the terminal. Based on this information, the user can propose appropriate products to the customer. As a specific example, the server recommends related products such as "new printers" and "high-performance computers."

[0622] This system will link the above functions to comprehensively support sales activities and make them more efficient.It is also anticipated that these functions will be provided in a SaaS (Software as a Service) format, and that they will be available to a wide range of salespeople via the Internet.

[0623] The processing flow will be explained below.

[0624] Automatically summarize customer conversations

[0625] Step 1:

[0626] When the user starts a conversation with the customer, the terminal activates the recording function and starts recording the audio of the conversation.

[0627] Step 2:

[0628] After the conversation ends, the device sends the recorded data to the server.

[0629] Step 3:

[0630] The server sends the received voice data to a voice recognition engine, which converts the voice data into text data.

[0631] Step 4:

[0632] The server runs the converted text data through a natural language processing algorithm to analyze the content of the conversation.

[0633] Step 5:

[0634] The server extracts important keywords and phrases and generates a summary.

[0635] Step 6:

[0636] The server sends the generated summary to the user's terminal.

[0637] Step 7:

[0638] The terminal displays the summary on the screen for the user to confirm.

[0639] Sales talk skills training

[0640] Step 1:

[0641] The server stores a database of past successful sales pitches.

[0642] Step 2:

[0643] The user starts a new sales conversation, and the device records the conversation.

[0644] Step 3:

[0645] After the sales talk ends, the device sends the recorded data to the server.

[0646] Step 4:

[0647] The server converts the received voice data into text data using a voice recognition engine.

[0648] Step 5:

[0649] The server runs the converted text data through an analysis algorithm that compares it with past successes.

[0650] Step 6:

[0651] The server generates advice on how to improve the sales pitch based on the analysis results.

[0652] Step 7:

[0653] The server transmits the generated advice to the user's terminal.

[0654] Step 8:

[0655] The terminal notifies the user of the advice, which the user confirms.

[0656] Advice for next action

[0657] Step 1:

[0658] The user enters a customer profile and current business status into the terminal.

[0659] Step 2:

[0660] The terminal transmits the entered customer profile and business status data to the server.

[0661] Step 3:

[0662] The server runs the received data through an analytical algorithm to analyze the customer's situation.

[0663] Step 4:

[0664] The server generates multiple next action candidates based on the analysis results.

[0665] Step 5:

[0666] The server ranks the next action candidates based on their deviation scores and success rates.

[0667] Step 6:

[0668] The server selects the best next action and generates advice.

[0669] Step 7:

[0670] The server sends the generated advice to the user's terminal.

[0671] Step 8:

[0672] The terminal notifies the user of the advice, which the user confirms.

[0673] Recommendation of proposed products

[0674] Step 1:

[0675] The user enters their customer profile and sales history into the terminal.

[0676] Step 2:

[0677] The terminal transmits the input data to the server.

[0678] Step 3:

[0679] The server runs customer profiles and sales history data through analytical algorithms.

[0680] Step 4:

[0681] The server analyzes the purchasing patterns of similar customers and extracts related products from the database.

[0682] Step 5:

[0683] The server generates recommendation information based on the analysis results.

[0684] Step 6:

[0685] The server transmits the generated recommendation information to the user's terminal.

[0686] Step 7:

[0687] The device notifies the user of the recommended information, which the user can then confirm.

[0688] With the above processing flow, the system of the present invention can comprehensively support sales activities and improve efficiency.

[0689] Example 1

[0690] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0691] Conventional sales support systems lacked the means to efficiently summarize conversations with customers and provide specific guidance to improve the quality of sales talks. Furthermore, they lacked a system that could provide advice on the next sales action or recommend appropriate products, which meant that sales activities were not sufficiently streamlined. This created challenges that made it difficult for salespeople to conduct effective sales activities.

[0692] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0693] In this invention, the server includes a means for recording conversations with customers, a means for converting the recorded voice data into character data using a recognition means, a processing means for analyzing the converted character data and extracting important points, and a means for generating a summary of the extracted points and distributing it to the user. This makes it possible to efficiently summarize the content of conversations with customers, allowing the user to quickly grasp the main points.

[0694] The server also includes a means for maintaining a database of past successful sales talks, a means for collecting new sales talks from users and converting the recorded data into text data, a means for comparing the converted text data with past successful examples and generating advice on areas for improvement, and a means for delivering the generated advice to users, thereby enabling specific guidance for improving the quality of sales talks.

[0695] Furthermore, the server includes a means for collecting and analyzing customer profiles and situation data, a means for evaluating next action candidates and selecting the most appropriate action, and a means for delivering the selected action to the user as advice, thereby enabling appropriate advice on the next sales action to be provided, thereby achieving efficiency in sales activities.

[0696] "Customer" means any person or entity to whom products or services are provided.

[0697] "Conversation" refers to the process of information exchange between a customer and a salesperson, which is usually conducted verbally.

[0698] "Recording" refers to the method of recording audio data in digital or analog form.

[0699] "Audio data" refers to data that records human voice as a digital signal.

[0700] "Recognition means" refers to the function of a device or software for converting voice data into text data.

[0701] "Character data" refers to data expressed in text format.

[0702] "Analysis" refers to the process of examining the contents of data in detail to extract meaning and characteristics.

[0703] "Key points" refer to particularly important information or key phrases in the conversation.

[0704] "Processing means" refers to the functionality of a device or software to perform specific data operations or algorithms.

[0705] A "summary" is a short summary that extracts important elements from a long piece of text or a large amount of data.

[0706] "Distribution" refers to the act of delivering information or data to a specific recipient.

[0707] "Talk" refers to the sales pitch or presentation that a salesperson gives to a customer.

[0708] "Database" refers to a system or software for efficiently storing, managing, and retrieving data.

[0709] A "new sales pitch" refers to a new type of sales presentation that has never been done before.

[0710] "Success stories" refer to specific sales pitches or approaches that have produced results in the past.

[0711] "Areas for improvement" refers to areas where changes are needed to make the current way of doing things more effective.

[0712] "Advice" refers to advice or instructions for achieving a specific purpose.

[0713] "Profile" refers to information that compiles a customer's attributes, past transaction history, and other characteristics.

[0714] "Contextual data" refers to data related to current sales activities and customer status.

[0715] "Potential actions" refer to the next possible sales activity options that may be taken.

[0716] "Optimal action" refers to the sales activity that is considered to be the most effective among multiple options.

[0717] "Advice" refers to information or instructions that help you decide what to do next.

[0718] This invention is an integrated sales support system that automatically summarizes conversations with customers, provides advice on improving sales talks, advises on effective sales actions for the next time, and recommends appropriate products to propose. This system is provided in the form of SaaS (Software as a Service) and is expected to be used by a wide range of salespeople via the Internet.

[0719] The specific configuration and operation of the system will be described below.

[0720] Automatically summarize customer conversations

[0721] When a user starts a conversation with a customer, the device (such as a smartphone or PC) records the conversation. When the conversation ends, the device sends the recorded data to a server. The server uses a speech recognition engine such as Google Cloud Speech-to-Text to convert the voice data into text data. The server then analyzes the text data using a natural language processing algorithm such as NLTK (Natural Language Toolkit), extracts key points, and generates a summary. Finally, the generated summary is sent to the device and notified to the user.

[0722] As a concrete example, a salesperson discusses a new product with a customer and records the conversation. After the conversation ends, the system generates a summary such as "I'm interested in the new product's features" and "I'd like to know more about the price." Example prompts:

[0723] Generate a summary of a conversation with a customer. Here is the transcript of the conversation:

[0724] Sales talk skills training

[0725] The server maintains a database of past successful sales pitches, and when a user makes a new sales pitch, the device records the content and sends it to the server. The server uses Google Cloud Speech-to-Text to convert the audio data into text data. The server then compares it with past successes and analyzes areas for improvement. Advice is generated based on the analysis results and sent to the user's device.

[0726] For example, the system analyzes a salesperson's presentation and provides advice such as, "It would be more effective if you provided more specific examples." Example prompt:

[0727] Please offer some advice to improve the following sales pitch:...

[0728] Advice for next action

[0729] When a user enters their customer profile and current sales status into the device, the device sends this to the server. The server uses a database such as MongoDB to analyze the customer profile and sales status data, evaluate potential next sales actions, select the most appropriate next action, generate advice, and send it to the device.

[0730] As a concrete example, the generated advice is "It would be effective to demonstrate the new product at the next interview." Example prompt sentence:

[0731] Suggest the best next action based on the following customer profile and sales situation:...

[0732] Recommendation of proposed products

[0733] The user enters their customer profile and sales history into the terminal, which then sends it to the server. The server uses a database such as Salesforce to analyze the purchasing patterns of similar customers and extract related products. Recommended products are generated and the recommendation information is sent to the terminal.

[0734] For example, a "new printer" or a "high-performance computer" may be recommended. Example prompts:

[0735] Please recommend the appropriate products based on the following customer profile and sales history:

[0736] By linking the above functions, sales activities can be comprehensively supported and made more efficient.

[0737] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0738] Automatically summarize customer conversations

[0739] Step 1:

[0740] When the user starts a conversation with the customer, the terminal records the conversation.

[0741] Input: Voice of conversation with customer

[0742] Output: Recorded audio data

[0743] Specific operation: A salesperson uses a smartphone to record a discussion with a customer.

[0744] Step 2:

[0745] When the conversation is finished, the user presses the recording end button and the terminal sends the recorded data to the server.

[0746] Input: Recorded audio data

[0747] Output: Audio data sent to the server

[0748] Specific operation: The salesperson presses the end recording button, and the device uploads the audio data to the server.

[0749] Step 3:

[0750] The server uses a speech recognition engine such as Google Cloud Speech-to-Text to convert the voice data into text data.

[0751] Input: Audio data

[0752] Output: Character data

[0753] Specific operation: The server starts a speech recognition engine and converts the voice data into text.

[0754] Step 4:

[0755] The server uses natural language processing algorithms such as NLTK to analyze the text data, extract key points, and generate a summary.

[0756] Input: Character data

[0757] Output: Summary data

[0758] What it does: The server runs a natural language processing algorithm to extract customer points of interest and generate a summary.

[0759] Step 5:

[0760] The generated summary is sent from the server to the terminal, which notifies the user of it.

[0761] Input: Summary data

[0762] Output: Summary data reported to the user

[0763] Specific operation: The terminal uses the notification function to display the summary content to the user.

[0764] Sales talk skills training

[0765] Step 1:

[0766] When a user gives a sales pitch, the device records the content.

[0767] Input: Sales talk audio

[0768] Output: Recorded audio data

[0769] Specific operation: Start recording when the salesperson starts talking.

[0770] Step 2:

[0771] After the recording is completed, the device sends the recorded data to the server.

[0772] Input: Recorded audio data

[0773] Output: Audio data sent to the server

[0774] Specific operation: When the talk ends, the data is automatically sent to the server.

[0775] Step 3:

[0776] The server uses a voice recognition engine to convert the recorded data into text data.

[0777] Input: Audio data

[0778] Output: Character data

[0779] Specific operation: The server converts the voice data into text using Google Cloud Speech-to-Text.

[0780] Step 4:

[0781] The server compares the newly received text data with past successes and analyzes areas for improvement.

[0782] Input: Text data, past success data

[0783] Output: Improvements

[0784] How it works: The server compares new data with past databases and uses algorithms to identify areas for improvement.

[0785] Step 5:

[0786] The server transmits the generated advice to the terminal, which notifies the user of it.

[0787] Input: Advice data

[0788] Output: Advice given to the user

[0789] Specific action: The device displays a notification to the user and suggests specific improvements.

[0790] Advice for next action

[0791] Step 1:

[0792] The user enters a customer profile and current business status into the terminal.

[0793] Input: Customer profile, sales status

[0794] Output: Data entered into the terminal

[0795] Specific operation: A salesperson enters customer information and sales progress through the application.

[0796] Step 2:

[0797] The terminal transmits customer profile and sales status data to the server.

[0798] Input: Customer profile, sales status

[0799] Output: Data sent to the server

[0800] Specific operation: The input data is automatically sent to the server.

[0801] Step 3:

[0802] The server uses a database such as MongoDB to analyze the data and evaluate potential next sales actions.

[0803] Input: Customer profile, sales status data

[0804] Output: Evaluated action candidates

[0805] What happens: The server takes in the data and uses an algorithm to determine the next step.

[0806] Step 4:

[0807] The server selects the optimal next action, generates advice, and sends it to the terminal.

[0808] Input: Evaluated action candidates

[0809] Output: Generated advisory data

[0810] Specific operation: The server uses an algorithm to determine the most effective action for the customer and generates it as advice.

[0811] Step 5:

[0812] The terminal notifies the user of the advice content.

[0813] Input: Advisory data

[0814] Output: Advice given to the user

[0815] Specific operation: The terminal notifies the user of advice on the next action.

[0816] Recommendation of proposed products

[0817] Step 1:

[0818] The user enters their customer profile and sales history into the terminal.

[0819] Input: Customer profile, sales history

[0820] Output: Data entered into the terminal

[0821] What happens: A salesperson uses the application to enter customer and past purchase history.

[0822] Step 2:

[0823] The terminal transmits this data to the server.

[0824] Input: Customer profile, sales history

[0825] Output: Data sent to the server

[0826] Specific operation: The entered data is transferred from the terminal to the server.

[0827] Step 3:

[0828] The server uses databases such as Salesforce to analyze the purchasing patterns of similar customers and extract related products.

[0829] Input: Customer profile, sales history

[0830] Output: List of related products

[0831] What happens: The server queries the database and extracts the most relevant products.

[0832] Step 4:

[0833] The server generates recommended products and sends the recommendation information to the terminal.

[0834] Input: List of related products

[0835] Output: Recommended product data

[0836] Specific operation: The server uses a recommendation algorithm to select the most suitable product for the customer.

[0837] Step 5:

[0838] The terminal notifies the user of the recommended information.

[0839] Input: Recommended product data

[0840] Output: Recommendation information notified to the user

[0841] Specific operation: The terminal displays a list of suggested products to the user.

[0842] This detailed processing flow allows the functions of the present invention to work together, comprehensively supporting sales activities and making them more efficient.

[0843] (Application example 1)

[0844] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0845] In modern sales and marketing activities, it is important to effectively manage customer interactions, efficiently summarize their content, and improve sales pitches by referencing past success stories. However, performing these processes manually takes time and effort, placing a heavy burden on sales representatives. Furthermore, advice on next sales actions and proposals for appropriate products must also be provided quickly and accurately, but current systems are unable to meet these needs.

[0846] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0847] In this invention, the server includes a means for recording conversations with customers, a means for converting the recorded voice data into text data using speech recognition, and a natural language processing means for analyzing the converted text data and extracting key points. This enables efficient summaries of conversations with customers and delivering them to users. The server also includes a means for maintaining a database of past successful sales pitches, a means for collecting new sales pitches from users and converting the recorded data into text data, a means for comparing the converted text data with past successes and generating advice on areas for improvement, a means for providing advice in real time, and a means for utilizing a generative AI model. This allows users to receive effective improvement advice in real time based on comparisons with past successes. The server also includes a means for collecting and analyzing customer profiles and sales status data, a means for evaluating potential next sales actions and selecting the optimal action, a means for analyzing user input through prompts, and a means for recommending related products based on the customer profile. This enables quick and accurate recommendations for next sales actions and proposed products.

[0848] "Customer profile" refers to information about a customer, such as their attributes, purchasing history, and interests.

[0849] "Sales talk" refers to the way salespeople talk and the explanations they give to customers to promote sales.

[0850] "Speech recognition" refers to the technology of analyzing recorded voice data and converting it into text data.

[0851] "Natural language processing" refers to the technology of analyzing text data and understanding its meaning and structure.

[0852] A "summary" refers to information that has been extracted from analyzed text data and concisely summarized.

[0853] "Database" refers to a system for efficiently storing information and making it searchable and accessible.

[0854] "Real-time" refers to processing or response occurring immediately, without delay.

[0855] A "generative AI model" refers to an artificial intelligence model that uses machine learning to learn patterns from large amounts of data, and then generates and analyzes new data.

[0856] A "prompt sentence" refers to an instruction sentence or question sentence that a user inputs to a system.

[0857] "Recommendation" refers to the act of suggesting optimal products or services based on a user's preferences and behavioral history.

[0858] The present invention relates to a sales support system, and in particular to a system that summarizes conversations with customers, provides advice on improving sales talks, gives advice on the next sales action, and recommends appropriate products through an application that runs on a device such as a smartphone.

[0859] The details of the system embodiment will be described below.

[0860] 1. Automatic summarization of customer conversations

[0861] When a user starts a conversation with a customer, the device (e.g., a smartphone) records the conversation. When the conversation ends, the device sends the recorded data to a server. The server converts the voice data into text data using a speech recognition engine (e.g., Google Speech Recognition). The server then analyzes the converted text data using a natural language processing algorithm (e.g., the "T5" model from the Transformers library), extracts important points, and generates a summary. The generated summary is distributed to the device via an intranet or the Internet, and is notified to the user. As a concrete example, a sales representative may discuss the features of a new product with a customer, and a summary may be generated based on the recording.

[0862] 2. Sales pitch training

[0863] The server maintains a database of past successful sales pitches. When a user gives a new sales pitch, the device records the content and sends it to the server. The server converts the received data into text data using voice recognition, compares it with past successful examples, and analyzes areas for improvement. Based on the analysis results, the server generates advice and delivers it to the user's device. As a concrete example, the server analyzes a presentation a sales representative gave to a customer and provides advice such as "it would be good to include more specific examples."

[0864] 3. Advice on next steps

[0865] The user inputs the customer profile and current sales situation into the device. The device sends this data to the server, which uses a generative AI model to analyze the customer profile and sales situation data and evaluate potential next sales actions. The server selects the optimal next action, generates advice, and delivers it to the device. The user can use this advice to effectively decide on the next step. As a specific example, the advice provided to the user is, "It would be effective to demonstrate a new product at the next interview."

[0866] 4. Recommendation of proposed products

[0867] The user enters their customer profile and sales history into the device, and the device sends the data to the server. The server uses a generative AI model to analyze the purchasing patterns of similar customers based on the customer profile and extract related products from a database. The server analyzes the information through prompt text, generates recommended products, and delivers the recommended information to the device. Based on this information, the user proposes appropriate products to the customer. As a specific example, the server recommends related products such as "new printers" and "high-performance computers."

[0868] Prompt Sentence Examples

[0869] "After a five-minute conversation with a customer about new product features, you have the following transcript. Please summarize it:

[0870] "Hello, customer. Today I'd like to introduce you to our new X900 series printer. This printer is extremely high-performance... (omitted)"

[0871] This system enables users to efficiently manage conversations with customers and conduct effective sales activities.

[0872] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0873] Step 1:

[0874] When a user starts a conversation with a customer, the user uses a terminal (smartphone) to record the conversation.

[0875] Input: Voice of conversation with customer

[0876] Output: Recorded audio file

[0877] What it does: Uses the device's recording feature to save a conversation with a customer as an audio file.

[0878] Step 2:

[0879] When the conversation ends, the terminal transmits the recorded voice data to the server.

[0880] Input: Recorded audio file

[0881] Output: Audio data uploaded to the server

[0882] How it works: Uploads recorded data from the device to a cloud server or database.

[0883] Step 3:

[0884] The server uses a speech recognition engine to convert the voice data into text data.

[0885] Input: Audio file

[0886] Output: Converted text data

[0887] What it does: Calls a speech recognition engine (e.g., Google Speech Recognition API) to convert an audio file into text.

[0888] Step 4:

[0889] The server analyzes the text data using natural language processing algorithms, extracts key points, and generates a summary.

[0890] Input: Text data

[0891] Output: Summary text

[0892] How it works: It uses natural language processing algorithms (e.g., Hugging Face Transformers' "T5" model) to extract key points and generate a summary.

[0893] Step 5:

[0894] The server delivers the generated summary to the terminal.

[0895] Input: Summary text

[0896] Output: Summary text delivered to the user's device

[0897] Action: Sends summary text to the terminal and notifies the user.

[0898] Step 6:

[0899] When a user makes a new sales pitch, the device records the content and sends it to the server.

[0900] Input: Sales pitch audio file

[0901] Output: Audio data uploaded to the server

[0902] Operation: Uploads the recorded audio data of the sales talk to the server.

[0903] Step 7:

[0904] The server converts the received voice data into text data using voice recognition.

[0905] Input: Audio file

[0906] Output: Converted text data

[0907] How it works: Uses a speech recognition engine to convert voice data into text data.

[0908] Step 8:

[0909] The server compares the text data with past success stories, analyzes areas for improvement, and generates advice.

[0910] Input: Text data, past success data

[0911] Output: Advice text

[0912] How it works: Text is vectorized using TfidfVectorizer, compared to past successes using cosine similarity, and a generative AI model is used to identify areas for improvement and generate advice.

[0913] Step 9:

[0914] The server distributes the generated advice to the user's terminal.

[0915] Input: Advice text

[0916] Output: Advice text delivered to the user's device

[0917] Action: Sends advice text to the user's device and notifies them.

[0918] Step 10:

[0919] The user enters a customer profile and current business status into the terminal.

[0920] Input: Customer profile, sales status data

[0921] Output: Profile and status data sent from the device to the server

[0922] How it works: A user enters their profile and status into a terminal and sends it to a database.

[0923] Step 11:

[0924] The server uses the generative AI model to analyze customer profiles and sales status data and select potential next sales actions.

[0925] Input: Customer profile, sales status data

[0926] Output: Text of advice on optimal sales actions

[0927] How it works: Analyzes data using a generative AI model to evaluate and select the next sales action.

[0928] Step 12:

[0929] The server delivers the selected action to the user as a recommendation.

[0930] Input: Sales action advice text

[0931] Output: Advice text delivered to the user's device

[0932] Operation: The advice is sent to the terminal and notified.

[0933] Step 13:

[0934] The user enters a customer profile and sales history into the terminal.

[0935] Input: Customer profile, sales history data

[0936] Output: Profile and sales history data sent from the device to the server

[0937] How it works: A user enters their profile and history into a terminal and sends it to a database.

[0938] Step 14:

[0939] The server uses a generative AI model to analyze the purchasing patterns of similar customers based on the customer profile and recommend related products.

[0940] Input: Customer profile, sales history data

[0941] Output: Recommended product list

[0942] How it works: Analyzes data using a generative AI model to recommend relevant products.

[0943] Step 15:

[0944] The server delivers a list of recommended products to the user's terminal.

[0945] Input: Recommended product list

[0946] Output: Product list delivered to the user's device

[0947] Operation: The recommended product list is sent to the device and notified.

[0948] For example, if you set the prompt text as follows:

[0949] "After a five-minute conversation with a customer about new product features, you have the following transcript. Please summarize it:

[0950] "Hello, customer. Today I'd like to introduce you to our new X900 series printer. This printer is extremely high-performance... (omitted)"

[0951] By using prompts in this way, users can efficiently issue instructions to the system and receive optimal summaries, advice, and recommendations.

[0952] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0953] This embodiment of the present invention combines an emotion engine that recognizes the user's emotions with an integrated sales support system that automatically summarizes conversations with customers, provides advice on improving sales talks, advises on effective next sales actions, and recommends appropriate products to propose. The program and processing of this system are described below with specific examples.

[0954] 1. Automatically summarize customer conversations

[0955] When a user starts a conversation with a customer, the device activates the recording function and records the audio of the conversation. When the conversation ends, the device sends the recorded data to the server. The server converts the received audio data into text data using a speech recognition engine. The server then runs the text data through a natural language processing algorithm to extract key points and generate a summary. Furthermore, an emotion engine analyzes the user's emotions from the converted text and audio data and adjusts the summary content accordingly. Finally, the server sends the generated summary to the user's device, where the user can review it. As a concrete example, a salesperson discusses a new product with a customer, and a summary is generated based on the recording, reflecting the content in line with the emotions.

[0956] 2. Sales pitch training

[0957] The server maintains a database of past successful sales pitches. When a user gives a new sales pitch, the device records the content and sends it to the server. The server converts the received data into text data using a voice recognition engine and runs an analysis algorithm that compares the converted text data with past successful examples. The server generates advice on how to improve the sales pitch based on the user's emotional data. The advice generated by the server is sent to the user's device, where the user can confirm it. It is also possible to provide flexible advice based on the user's emotions. As a specific example, a salesperson's presentation to a customer is analyzed and emotionally sensitive advice such as "it would be more effective if you gave more specific examples" is provided.

[0958] 3. Advice on next steps

[0959] The user enters a customer profile and current sales status into the terminal. The terminal sends this data to the server. The server analyzes the customer profile and sales status data and runs it through an algorithm that evaluates potential next sales actions. The server selects the optimal next action based on the evaluation results and uses an emotion engine to generate advice that takes the user's emotions into consideration. The server sends the generated advice to the user's terminal, where the user can confirm it. As a specific example, advice such as "It would be effective to demonstrate a new product at the next interview" is provided in a form that is appropriate to the user's emotions.

[0960] 4. Recommendation of proposed products

[0961] The user enters their customer profile and sales history into the terminal, which then sends the data to the server. The server uses this data to analyze the purchasing patterns of similar customers and extracts related products from a database. The server uses an emotion engine to generate recommendation information that takes the emotional data into account. The server then sends the recommendation information to the user's terminal, where the user confirms it. As a specific example, the server provides related products such as "new printers" and "high-performance computers" as recommendation information tailored to the user's emotions.

[0962] By linking the above functions, this system comprehensively supports sales activities, improving efficiency and effectiveness. In addition, the combination of an emotion engine enables flexible responses that take user emotions into consideration, allowing for more personalized support.

[0963] The processing flow will be explained below.

[0964] Automatically summarize customer conversations

[0965] Step 1:

[0966] The user starts a conversation with the customer. The device activates the recording function and starts recording the audio of the conversation.

[0967] Step 2:

[0968] After the conversation ends, the device sends the recorded data to the server.

[0969] Step 3:

[0970] The server sends the received voice data to a voice recognition engine, which converts the voice data into text data.

[0971] Step 4:

[0972] The server sends the converted text data to a natural language processing algorithm, which analyzes the text data.

[0973] Step 5:

[0974] The server extracts important keywords and phrases and generates a summary.

[0975] Step 6:

[0976] The server sends the voice and text data to the emotion engine, which analyzes the user's emotions.

[0977] Step 7:

[0978] The server adjusts the summary content based on the analysis results of the emotion engine.

[0979] Step 8:

[0980] The server sends the generated summary to the user's terminal.

[0981] Step 9:

[0982] The terminal displays the summary and the user confirms it.

[0983] Sales talk skills training

[0984] Step 1:

[0985] A server maintains a database of past successful sales pitches.

[0986] Step 2:

[0987] The user makes a new sales pitch, and the device records the content.

[0988] Step 3:

[0989] After the sales talk ends, the device sends the recorded data to the server.

[0990] Step 4:

[0991] The server sends the received voice data to a voice recognition engine, which converts it into text data.

[0992] Step 5:

[0993] The server sends the converted text data to an analysis algorithm that compares it with past successes.

[0994] Step 6:

[0995] The server generates advice on how to improve the sales pitch based on the analysis results.

[0996] Step 7:

[0997] The server sends text data to the emotion engine, which analyzes the user's emotions.

[0998] Step 8:

[0999] The server adjusts the advice content based on the analysis results of the emotion engine.

[1000] Step 9:

[1001] The server transmits the generated advice to the user's terminal.

[1002] Step 10:

[1003] The terminal notifies the user of the advice, which is then confirmed by the user.

[1004] Advice for next action

[1005] Step 1:

[1006] The user enters a customer profile and current business status into the terminal.

[1007] Step 2:

[1008] The terminal transmits the input data to the server.

[1009] Step 3:

[1010] The server sends customer profile and sales data to an analytics algorithm.

[1011] Step 4:

[1012] The server generates the next sales action candidates based on the analysis results.

[1013] Step 5:

[1014] The server sends the user's emotional data to the emotion engine, which analyzes the emotions.

[1015] Step 6:

[1016] Based on the analysis results of the emotion engine, the server ranks and adjusts potential actions.

[1017] Step 7:

[1018] The server selects the best next action and generates advice.

[1019] Step 8:

[1020] The server sends the generated advice to the user's terminal.

[1021] Step 9:

[1022] The terminal notifies the user of the advice and the user confirms it.

[1023] Recommendation of proposed products

[1024] Step 1:

[1025] The user enters their customer profile and sales history into the terminal.

[1026] Step 2:

[1027] The terminal transmits the input data to the server.

[1028] Step 3:

[1029] The server sends customer profiles and sales histories to an analytics algorithm.

[1030] Step 4:

[1031] The server analyzes the purchasing patterns of similar customers and extracts related products from the database.

[1032] Step 5:

[1033] The server sends the user's emotional data to the emotion engine, which analyzes the emotions.

[1034] Step 6:

[1035] The server adjusts the recommendations based on the analysis results of the emotion engine.

[1036] Step 7:

[1037] The server transmits the generated recommendation information to the user's terminal.

[1038] Step 8:

[1039] The device notifies the user of the recommended information, which is then confirmed by the user.

[1040] As a result, the system of the present invention, which incorporates an emotion engine, can comprehensively support sales activities and respond flexibly while taking into account the user's emotions. As a specific example, the emotion engine can analyze the emotional state of a salesperson, and if the salesperson is feeling stressed, it can provide personalized support, such as advice on how to speak calmly.

[1041] Example 2

[1042] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1043] Conventional sales support systems have the problem of being unable to automatically and efficiently summarize conversations with customers, give advice on improving sales talks, or advise on next actions.In addition, these functions are not adjusted according to the user's emotions, and only standardized suggestions and advice are provided, which has led to issues that do not provide sufficient satisfaction to users.

[1044] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for recording a conversation with a customer, a means for converting the recorded voice data into text data using a voice recognition engine, a means for analyzing the converted text data using a natural language processing algorithm and extracting important points, a means for generating a summary from the extracted points and adjusting the summary content according to the user's emotions using an emotion engine, and a means for delivering the adjusted summary to the user. This makes it possible to provide summaries and advice that take the user's emotions into consideration, which is expected to improve the efficiency and effectiveness of sales activities.

[1045] The "means for recording conversations with customers" refers to a device or software for recording conversations as audio data when a user starts a conversation with a customer.

[1046] A "speech recognition engine" is software or algorithms that convert recorded voice data into text data.

[1047] A "natural language processing algorithm" is an algorithm for analyzing text data and extracting important information and patterns.

[1048] An "emotion engine" is software or an algorithm that analyzes emotions from text and voice data and makes adjustments based on those emotions.

[1049] A "summary generation means" is a device or software that extracts important points and creates a short summary of the text.

[1050] The "distribution means" refers to a device or software for transmitting the generated information or advice to the user's terminal and displaying it.

[1051] A "database" is an information accumulation system that stores data such as past successful sales pitches, customer profiles, and sales history, and can be searched and used as needed.

[1052] An "analysis algorithm" is an algorithm that analyzes collected data and finds patterns and trends.

[1053] "Potential sales actions" are suggestions for specific actions that can be taken in the next sales activity.

[1054] The "means for delivering advice" is a device or software that notifies the user of the optimal action to take in the next sales activity and provides it as a guideline for action.

[1055] The present invention is an integrated sales support system that automatically summarizes conversations with customers, provides advice on improving sales talks, advises on effective next sales actions, and recommends appropriate products. By combining this system with an emotion engine that recognizes the user's emotions, the present invention can provide more personalized support. The specific configuration and operation of this system are described below.

[1056] 1. Automatically summarize customer conversations

[1057] When a user starts a conversation with a customer, the device activates the recording function and records the audio of the conversation. A recording application (e.g., Recorder App) is installed on the device. When the conversation ends, the device sends the recorded data to the server. The recorded data is sent to the server using an HTTP request (e.g., in MP3 format). The server converts the received audio data into text data using a speech recognition engine (e.g., Google Cloud Speech-to-Text). The server then uses a natural language processing algorithm (e.g., BERT or GPT-3) to analyze the text data, extract key points, and generate a summary. After that, an emotion engine (e.g., Microsoft Azure Emotion API) analyzes the user's emotions from the converted text and audio data and adjusts the summary content. Finally, the server sends the generated summary to the user's device, where the user can review it.

[1058] For example, if a salesperson is discussing a new product with a customer, a summary is generated based on the recording, and the content reflects the customer's emotions.

[1059] Example prompt sentence:

[1060] "Summarize key points from recordings of customer conversations, analyze sentiment, and adjust accordingly."

[1061] 2. Sales pitch training

[1062] The server maintains a database of past successful sales pitches. When a user gives a new sales pitch, the device records the content and sends it to the server. The server converts the received data into text data using a voice recognition engine and runs an analysis algorithm (e.g., a similarity calculation algorithm) that compares the converted text data with past successful examples. The server generates advice on how to improve the sales pitch based on the user's emotional data. The generated advice is adjusted according to the user's emotions using an emotion engine. The server sends the adjusted advice to the user's device, where the user confirms it.

[1063] As a concrete example, the system analyzes a presentation a salesperson gives to a customer and provides emotionally sensitive advice such as "It would be more effective if you provided more specific examples."

[1064] Example prompt sentence:

[1065] "Please compare my current sales pitch with past successful ones and give me advice on how to improve it."

[1066] 3. Advice on next steps

[1067] The user inputs their customer profile and current sales situation into their device. The device then sends this data to the server. The server then runs this data through an analytical algorithm (e.g., a machine learning model) to evaluate potential next sales actions. The server then selects the optimal next action based on the evaluation results and uses an emotion engine to generate advice that takes the user's emotions into account. The server then sends the generated advice to the user's device, where the user confirms it.

[1068] As a specific example, advice such as "It would be effective to demonstrate a new product at the next interview" is provided in accordance with the user's emotions.

[1069] Example prompt sentence:

[1070] "Please advise me on my next action based on my customer profile and sales situation."

[1071] 4. Recommendation of proposed products

[1072] The user enters their customer profile and sales history into the terminal, which then sends the data to the server. The server uses this data to analyze the purchasing patterns of similar customers and extracts related products from the database. The server then uses an emotion engine to generate recommendation information that takes into account the emotion data. The generated recommendation information is then sent by the server to the user's terminal, where the user can confirm it.

[1073] As a specific example, the server provides related products such as "new printers" and "high-performance computers" as recommended information tailored to the user's emotions.

[1074] Example prompt sentence:

[1075] "Recommend relevant products based on customer profile and sales history."

[1076] By linking the above functions, this system comprehensively supports sales activities, improving efficiency and effectiveness. In addition, the combination of an emotion engine enables flexible responses that take into account the user's emotions, allowing for more personalized support.

[1077] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1078] Step 1:

[1079] When a user starts a conversation with a customer, the device automatically activates the recording function and collects the audio data of the conversation. Specifically, the recording application installed on the device starts running in the background. The recording application receives audio data as input and outputs a recorded audio file (e.g., MP3 format).

[1080] Step 2:

[1081] Once the conversation is finished, the device sends the recording to the server using an HTTP POST request to the server's API endpoint. The device receives the recorded audio file as input and outputs an HTTP request.

[1082] Step 3:

[1083] The server converts the received voice data into text data using a voice recognition engine. Specifically, the server calls the API of the voice recognition engine (for example, Google Cloud Speech-to-Text), provides the voice data as input, and receives the converted text data as output.

[1084] Step 4:

[1085] The server applies a natural language processing algorithm to the acquired text data to extract key points. Specifically, the server inputs the text data into a natural language processing algorithm (e.g., BERT or GPT-3) and outputs key points as the analysis results.

[1086] Step 5:

[1087] The server uses an emotion engine to analyze emotions from text data and voice data. Specifically, the server calls the API of the emotion engine (for example, Microsoft Azure Emotion API), provides text data and voice data as input, and receives the emotion analysis results as output.

[1088] Step 6:

[1089] The server adjusts the summary content based on the sentiment analysis results and generates the final summary. Specifically, the server receives the sentiment analysis results and key points as input and outputs the adjusted summary.

[1090] Step 7:

[1091] The server sends the generated summary to the user's device. Specifically, the server uses an HTTP response to send the summary data to the user's device. The user's device receives the summary data as input and outputs a custom UI to display to the user.

[1092] This allows the content of the conversation with the customer to be summarized and presented in a form that matches the user's emotions.

[1093] (Application example 2)

[1094] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1095] Conventional sales support systems are limited in their functionality, such as efficiently summarizing conversations with customers, providing advice on improving sales talks, advising on next actions, and recommending proposed products. Furthermore, few systems can take user emotions into account when it comes to these functions, and more personalized support is desired. Therefore, there is a need for a comprehensive sales support system that can summarize conversations with customers, provide guidance on sales talks, advise on next actions, recommend proposed products, and also consider user emotions.

[1096] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1097] In this invention, the server includes means for recording conversations with customers, means for converting the recorded voice data into text data using voice recognition, natural language processing means for analyzing the converted text data and extracting important points, means for generating a summary of the extracted points and distributing it to the user, and means for recognizing the user's emotions and reflecting them in the summary content, thereby enabling efficient sales support that takes the user's emotions into consideration.

[1098] The server also includes means for maintaining a database of past successful sales talks, means for collecting new sales talks from users and converting the recorded data into text data, means for comparing the converted text data with past successful examples and generating advice on areas for improvement, means for delivering the generated advice to users, and means for generating and delivering flexible advice taking into account the user's emotions, thereby enabling the quality of sales talks to be improved while taking into account emotions.

[1099] Furthermore, the server includes means for collecting and analyzing customer profiles and sales status data, means for evaluating next sales action candidates and selecting the optimal action, means for delivering the selected action to the user as advice, and means for generating and delivering advice content that takes the user's emotions into consideration, thereby making the next sales activity more effective and personalized.

[1100] "Customer" refers to the person to whom goods or services are provided in a business transaction or transaction.

[1101] "Conversation recording means" refers to a device or function for recording a conversation with a customer as audio data.

[1102] "Speech recognition means" refers to technology or software that analyzes recorded voice data and converts it into text data.

[1103] "Character data" refers to text information converted by a speech recognition means.

[1104] "Natural language processing means" refers to algorithms and technologies for analyzing text data and extracting important points.

[1105] "Summary generation means" refers to a function that concisely summarizes a text based on extracted important points.

[1106] "Emotion recognition means" refers to the technology and algorithms that determine and analyze a user's emotions from text data and voice data.

[1107] A "sales pitch database" refers to a system or device that stores records and data of past successful sales pitches.

[1108] "Means for generating improvement advice" refers to a function that compares new sales pitches with existing successful examples and suggests more effective methods.

[1109] "Customer profile" refers to detailed data such as basic customer information and past transaction history.

[1110] "Sales status data" refers to information regarding the progress and results of current sales activities.

[1111] "Action evaluation means" refers to a function for evaluating potential next sales actions and selecting the most appropriate course of action.

[1112] The "advice delivery means" refers to a function for transmitting the generated advice or advice to the user.

[1113] This invention combines an emotion engine that recognizes user emotions with an integrated sales support system that automatically summarizes conversations with customers, provides advice on improving sales talks, advises on effective next sales actions, and recommends appropriate products. This system is designed to support customer service in brick-and-mortar stores in particular.

[1114] The entire system functions by linking the server and the terminal. Specifically, it has the following configuration:

[1115] 1. Automatically summarize customer conversations

[1116] The user wears smart glasses as a device and records conversations with customers. The recorded audio data is sent to a server in real time. The server converts the audio data into text data using Google Cloud Speech-to-Text. The converted text data is analyzed using natural language processing with the BERT model to extract key points. A summary is then generated, and Affectiva's emotion recognition engine analyzes the user's emotions and reflects them in the summary. The generated summary is then sent back to the user's device, where the user can review it.

[1117] 2. Sales pitch training

[1118] The server maintains a database of past successful sales pitches. When a user makes a new sales pitch, the smart glasses record it and send the data to the server. The server converts the recorded data into text data using Google Cloud Speech-to-Text, analyzes the converted data using the BERT model, and compares it with past successful examples. Based on the analysis results, advice on areas for improvement is generated, and an emotion engine provides advice based on the user's emotions. As a result, flexible advice is delivered to the user's device.

[1119] 3. Advice on next steps

[1120] When a user enters their customer profile and current sales status into their device, the data is sent to the server. The server analyzes the profile and sales status and evaluates the next sales action candidate using the BERT model and machine learning algorithm (K-means clustering). The emotion engine generates advice on the optimal next action that reflects the user's emotions and delivers it to the user's device.

[1121] 4. Recommendation of proposed products

[1122] Users enter their customer profile and sales history, and the data is sent to the server. The server analyzes the purchasing patterns of similar customers and extracts related products. The emotion engine then generates recommendation information that takes emotional data into account, and delivers it to the user's device. This allows users to recommend suggested products to customers at the appropriate time.

[1123] Example prompt sentence:

[1124] "Please convert the following Japanese audio data into text and summarize it."

[1125] "Perform a sentiment analysis of this text and generate sentiment-based advice."

[1126] "Recommend the next most effective sales action based on your customer profile."

[1127] This system will improve the efficiency of sales activities in brick-and-mortar stores and enable effective responses to customers. By instantly analyzing information obtained during conversations with customers and providing advice based on their emotions, the quality of sales activities can be improved.

[1128] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1129] Step 1:

[1130] A user uses smart glasses to record a conversation with a customer. The recorded voice data is sent to the server in real time. The input is the voice data, and the output is the raw voice data sent to the server.

[1131] Step 2:

[1132] The server receives the voice data and converts it into text data using Google Cloud Speech-to-Text. The input is voice data and the output is text data. Voice analysis is performed as part of the data processing.

[1133] Step 3:

[1134] The server analyzes the converted text data using the BERT model and extracts key points. The input is text data, and the output is a list of key points. Natural language processing is performed on the data.

[1135] Step 4:

[1136] The server generates a summary based on the extracted key points. The input is a list of key points, and the output is a summary. The data processing performed here is the generation of a summary.

[1137] Step 5:

[1138] The server uses Affectiva's emotion recognition engine to analyze the user's emotions from text and voice data. The input is text and voice data, and the output is emotion data. The process performed is emotion analysis.

[1139] Step 6:

[1140] The generated summary is adjusted by reflecting emotional data. The input is the summary and emotional data, and the output is the summary that reflects the emotion. The emotional data is reflected as a data calculation.

[1141] Step 7:

[1142] The final summary is delivered to the user's device. The input is the summary that reflects the emotion, and the output is the summary delivered to the user. The specific operation is that the device receives the summary and displays it to the user.

[1143] Step 8:

[1144] A database of past successful sales pitches is maintained by the server. The input is the database, and the output is the data of successful cases. The operation is to execute a query on the database.

[1145] Step 9:

[1146] The server converts newly recorded sales talks by the user into text data and compares them with past successes. The input is the newly recorded data, and the output is the comparison results. Data processing involves voice analysis and comparison calculations.

[1147] Step 10:

[1148] The server generates improvement advice and adjusts it based on the emotion data. The input is the comparison result and emotion data, and the output is advice based on the emotion. The processing involves generating advice and reflecting the emotion data.

[1149] Step 11:

[1150] The generated advice is delivered to the user's device. The input is the advice according to the emotion, and the output is the advice delivered to the user. The specific operation is that the device receives the advice and displays it to the user.

[1151] Step 12:

[1152] Customer profile and sales status data are sent from the terminal to the server. The input is the profile data and sales status data, and the output is the data sent to the server. The action is sending the data.

[1153] Step 13:

[1154] The server analyzes the profile and sales status data and evaluates the next optimal sales action. The input is the customer profile and sales status data, and the output is the next action candidate. The data processing is evaluated using machine learning.

[1155] Step 14:

[1156] Based on the evaluation results, the system generates advice on the optimal next action using emotional data. The input is the next action candidate and emotional data, and the output is advice based on the emotion. The processing involves generating advice and reflecting the emotional data.

[1157] Step 15:

[1158] The generated advice is delivered to the user's device. The input is advice corresponding to the emotion, and the output is the advice delivered to the user. The specific operation is for the device to receive the advice and display it to the user.

[1159] Step 16:

[1160] The user enters a customer profile and sales history, which is sent to the server. The input is the profile data and sales history, and the output is the data sent to the server. The action is sending the data.

[1161] Step 17:

[1162] The server analyzes the purchasing patterns of similar customers and extracts related products. The input is profile data and sales history, and the output is a list of related products. The data processing is done using machine learning analysis.

[1163] Step 18:

[1164] Recommendation information is generated using an emotion engine and delivered to users. The input is a list of related products and emotion data, and the output is recommendation information based on the emotion. The processing involves generating recommendation information and reflecting the emotion data.

[1165] Step 19:

[1166] The final recommendation information is delivered to the user's device. The input is recommendation information based on emotions, and the output is the recommendation information delivered to the user. The operation is for the device to receive the recommendation information and display it to the user.

[1167] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1168] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1169] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1170] [Third embodiment]

[1171] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1172] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1173] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1174] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1175] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1176] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1177] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1178] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1179] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1180] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1181] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1182] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1183] An embodiment of the present invention is an integrated sales support system that automatically summarizes conversations with customers, provides advice on improving sales talks, advises on effective sales actions for the next time, and recommends appropriate proposed products. The program and processing of this system are described below with specific examples.

[1184] 1. Automatically summarize customer conversations

[1185] When a user starts a conversation with a customer, the device records the conversation. When the conversation ends, the device sends the recorded data to the server. The server uses a speech recognition engine to convert the voice data into text data. The server then analyzes the text data using a natural language processing algorithm, extracts important points, and generates a summary. The server then sends the generated summary to the device and notifies the user. As a concrete example, a salesperson discusses a new product with a customer, and a summary is generated based on the recording.

[1186] 2. Sales pitch training

[1187] The server maintains a database of past successful sales pitches. When a user makes a new sales pitch, the device records the content and sends it to the server. The server converts the received data into text data using voice recognition, compares it with past successful examples, and analyzes areas for improvement. Based on the analysis results, the server generates advice and sends it to the user's device. As a concrete example, the server analyzes a presentation a salesperson gave to a customer and provides advice such as, "It would be more effective if you gave more specific examples."

[1188] 3. Advice on next steps

[1189] The user inputs a customer profile and current sales status into the terminal. The terminal sends this data to the server, which analyzes the customer profile and sales status data and evaluates potential next sales actions. The server selects the optimal next action, generates advice, and sends it to the terminal. The user can effectively decide on the next step based on this advice. As a specific example, the advice provided to the user is, "It would be effective to demonstrate a new product at the next interview."

[1190] 4. Recommendation of proposed products

[1191] The user enters their customer profile and sales history into the terminal, which then sends the data to the server. Based on this data, the server analyzes the purchasing patterns of similar customers and extracts related products from a database. The server generates recommended products and sends the recommendation information to the terminal. Based on this information, the user can propose appropriate products to the customer. As a specific example, the server recommends related products such as "new printers" and "high-performance computers."

[1192] This system will link the above functions to comprehensively support sales activities and make them more efficient.It is also anticipated that these functions will be provided in a SaaS (Software as a Service) format, and that they will be available to a wide range of salespeople via the Internet.

[1193] The processing flow will be explained below.

[1194] Automatically summarize customer conversations

[1195] Step 1:

[1196] When the user starts a conversation with the customer, the terminal activates the recording function and starts recording the audio of the conversation.

[1197] Step 2:

[1198] After the conversation ends, the device sends the recorded data to the server.

[1199] Step 3:

[1200] The server sends the received voice data to a voice recognition engine, which converts the voice data into text data.

[1201] Step 4:

[1202] The server runs the converted text data through a natural language processing algorithm to analyze the content of the conversation.

[1203] Step 5:

[1204] The server extracts important keywords and phrases and generates a summary.

[1205] Step 6:

[1206] The server sends the generated summary to the user's terminal.

[1207] Step 7:

[1208] The terminal displays the summary on the screen for the user to confirm.

[1209] Sales talk skills training

[1210] Step 1:

[1211] The server stores a database of past successful sales pitches.

[1212] Step 2:

[1213] The user starts a new sales conversation, and the device records the conversation.

[1214] Step 3:

[1215] After the sales talk ends, the device sends the recorded data to the server.

[1216] Step 4:

[1217] The server converts the received voice data into text data using a voice recognition engine.

[1218] Step 5:

[1219] The server runs the converted text data through an analysis algorithm that compares it with past successes.

[1220] Step 6:

[1221] The server generates advice on how to improve the sales pitch based on the analysis results.

[1222] Step 7:

[1223] The server transmits the generated advice to the user's terminal.

[1224] Step 8:

[1225] The terminal notifies the user of the advice, which the user confirms.

[1226] Advice for next action

[1227] Step 1:

[1228] The user enters a customer profile and current business status into the terminal.

[1229] Step 2:

[1230] The terminal transmits the entered customer profile and business status data to the server.

[1231] Step 3:

[1232] The server runs the received data through an analytical algorithm to analyze the customer's situation.

[1233] Step 4:

[1234] The server generates multiple next action candidates based on the analysis results.

[1235] Step 5:

[1236] The server ranks the next action candidates based on their deviation scores and success rates.

[1237] Step 6:

[1238] The server selects the best next action and generates advice.

[1239] Step 7:

[1240] The server sends the generated advice to the user's terminal.

[1241] Step 8:

[1242] The terminal notifies the user of the advice, which the user confirms.

[1243] Recommendation of proposed products

[1244] Step 1:

[1245] The user enters their customer profile and sales history into the terminal.

[1246] Step 2:

[1247] The terminal transmits the input data to the server.

[1248] Step 3:

[1249] The server runs customer profiles and sales history data through analytical algorithms.

[1250] Step 4:

[1251] The server analyzes the purchasing patterns of similar customers and extracts related products from the database.

[1252] Step 5:

[1253] The server generates recommendation information based on the analysis results.

[1254] Step 6:

[1255] The server transmits the generated recommendation information to the user's terminal.

[1256] Step 7:

[1257] The device notifies the user of the recommended information, which the user can then confirm.

[1258] With the above processing flow, the system of the present invention can comprehensively support sales activities and improve efficiency.

[1259] Example 1

[1260] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1261] Conventional sales support systems lacked the means to efficiently summarize conversations with customers and provide specific guidance to improve the quality of sales talks. Furthermore, they lacked a system that could provide advice on the next sales action or recommend appropriate products, which meant that sales activities were not sufficiently streamlined. This created challenges that made it difficult for salespeople to conduct effective sales activities.

[1262] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1263] In this invention, the server includes a means for recording conversations with customers, a means for converting the recorded voice data into character data using a recognition means, a processing means for analyzing the converted character data and extracting important points, and a means for generating a summary of the extracted points and distributing it to the user. This makes it possible to efficiently summarize the content of conversations with customers, allowing the user to quickly grasp the main points.

[1264] The server also includes a means for maintaining a database of past successful sales talks, a means for collecting new sales talks from users and converting the recorded data into text data, a means for comparing the converted text data with past successful examples and generating advice on areas for improvement, and a means for delivering the generated advice to users, thereby enabling specific guidance for improving the quality of sales talks.

[1265] Furthermore, the server includes a means for collecting and analyzing customer profiles and situation data, a means for evaluating next action candidates and selecting the most appropriate action, and a means for delivering the selected action to the user as advice, thereby enabling appropriate advice on the next sales action to be provided, thereby achieving efficiency in sales activities.

[1266] "Customer" means any person or entity to whom products or services are provided.

[1267] "Conversation" refers to the process of information exchange between a customer and a salesperson, which is usually conducted verbally.

[1268] "Recording" refers to the method of recording audio data in digital or analog form.

[1269] "Audio data" refers to data that records human voice as a digital signal.

[1270] "Recognition means" refers to the function of a device or software for converting voice data into text data.

[1271] "Character data" refers to data expressed in text format.

[1272] "Analysis" refers to the process of examining the contents of data in detail to extract meaning and characteristics.

[1273] "Key points" refer to particularly important information or key phrases in the conversation.

[1274] "Processing means" refers to the functionality of a device or software to perform specific data operations or algorithms.

[1275] A "summary" is a short summary that extracts important elements from a long piece of text or a large amount of data.

[1276] "Distribution" refers to the act of delivering information or data to a specific recipient.

[1277] "Talk" refers to the sales pitch or presentation that a salesperson gives to a customer.

[1278] "Database" refers to a system or software for efficiently storing, managing, and retrieving data.

[1279] A "new sales pitch" refers to a new type of sales presentation that has never been done before.

[1280] "Success stories" refer to specific sales pitches or approaches that have produced results in the past.

[1281] "Areas for improvement" refers to areas where changes are needed to make the current way of doing things more effective.

[1282] "Advice" refers to advice or instructions for achieving a specific purpose.

[1283] "Profile" refers to information that compiles a customer's attributes, past transaction history, and other characteristics.

[1284] "Contextual data" refers to data related to current sales activities and customer status.

[1285] "Potential actions" refer to the next possible sales activity options that may be taken.

[1286] "Optimal action" refers to the sales activity that is considered to be the most effective among multiple options.

[1287] "Advice" refers to information or instructions that help you decide what to do next.

[1288] This invention is an integrated sales support system that automatically summarizes conversations with customers, provides advice on improving sales talks, advises on effective sales actions for the next time, and recommends appropriate products to propose. This system is provided in the form of SaaS (Software as a Service) and is expected to be used by a wide range of salespeople via the Internet.

[1289] The specific configuration and operation of the system will be described below.

[1290] Automatically summarize customer conversations

[1291] When a user starts a conversation with a customer, the device (such as a smartphone or PC) records the conversation. When the conversation ends, the device sends the recorded data to a server. The server uses a speech recognition engine such as Google Cloud Speech-to-Text to convert the voice data into text data. The server then analyzes the text data using a natural language processing algorithm such as NLTK (Natural Language Toolkit), extracts key points, and generates a summary. Finally, the generated summary is sent to the device and notified to the user.

[1292] As a concrete example, a salesperson discusses a new product with a customer and records the conversation. After the conversation ends, the system generates a summary such as "I'm interested in the new product's features" and "I'd like to know more about the price." Example prompts:

[1293] Generate a summary of a conversation with a customer. Here is the transcript of the conversation:

[1294] Sales talk skills training

[1295] The server maintains a database of past successful sales pitches, and when a user makes a new sales pitch, the device records the content and sends it to the server. The server uses Google Cloud Speech-to-Text to convert the audio data into text data. The server then compares it with past successes and analyzes areas for improvement. Advice is generated based on the analysis results and sent to the user's device.

[1296] For example, the system analyzes a salesperson's presentation and provides advice such as, "It would be more effective if you provided more specific examples." Example prompt:

[1297] Please offer some advice to improve the following sales pitch:...

[1298] Advice for next action

[1299] When a user enters their customer profile and current sales status into the device, the device sends this to the server. The server uses a database such as MongoDB to analyze the customer profile and sales status data, evaluate potential next sales actions, select the most appropriate next action, generate advice, and send it to the device.

[1300] As a concrete example, the generated advice is "It would be effective to demonstrate the new product at the next interview." Example prompt sentence:

[1301] Suggest the best next action based on the following customer profile and sales situation:...

[1302] Recommendation of proposed products

[1303] The user enters their customer profile and sales history into the terminal, which then sends it to the server. The server uses a database such as Salesforce to analyze the purchasing patterns of similar customers and extract related products. Recommended products are generated and the recommendation information is sent to the terminal.

[1304] For example, a "new printer" or a "high-performance computer" may be recommended. Example prompts:

[1305] Please recommend the appropriate products based on the following customer profile and sales history:

[1306] By linking the above functions, sales activities can be comprehensively supported and made more efficient.

[1307] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1308] Automatically summarize customer conversations

[1309] Step 1:

[1310] When the user starts a conversation with the customer, the terminal records the conversation.

[1311] Input: Voice of conversation with customer

[1312] Output: Recorded audio data

[1313] Specific operation: A salesperson uses a smartphone to record a discussion with a customer.

[1314] Step 2:

[1315] When the conversation is finished, the user presses the recording end button and the terminal sends the recorded data to the server.

[1316] Input: Recorded audio data

[1317] Output: Audio data sent to the server

[1318] Specific operation: The salesperson presses the end recording button, and the device uploads the audio data to the server.

[1319] Step 3:

[1320] The server uses a speech recognition engine such as Google Cloud Speech-to-Text to convert the voice data into text data.

[1321] Input: Audio data

[1322] Output: Character data

[1323] Specific operation: The server starts a speech recognition engine and converts the voice data into text.

[1324] Step 4:

[1325] The server uses natural language processing algorithms such as NLTK to analyze the text data, extract key points, and generate a summary.

[1326] Input: Character data

[1327] Output: Summary data

[1328] What it does: The server runs a natural language processing algorithm to extract customer points of interest and generate a summary.

[1329] Step 5:

[1330] The generated summary is sent from the server to the terminal, which notifies the user of it.

[1331] Input: Summary data

[1332] Output: Summary data reported to the user

[1333] Specific operation: The terminal uses the notification function to display the summary content to the user.

[1334] Sales talk skills training

[1335] Step 1:

[1336] When a user gives a sales pitch, the device records the content.

[1337] Input: Sales talk audio

[1338] Output: Recorded audio data

[1339] Specific operation: Start recording when the salesperson starts talking.

[1340] Step 2:

[1341] After the recording is completed, the device sends the recorded data to the server.

[1342] Input: Recorded audio data

[1343] Output: Audio data sent to the server

[1344] Specific operation: When the talk ends, the data is automatically sent to the server.

[1345] Step 3:

[1346] The server uses a voice recognition engine to convert the recorded data into text data.

[1347] Input: Audio data

[1348] Output: Character data

[1349] Specific operation: The server converts the voice data into text using Google Cloud Speech-to-Text.

[1350] Step 4:

[1351] The server compares the newly received text data with past successes and analyzes areas for improvement.

[1352] Input: Text data, past success data

[1353] Output: Improvements

[1354] How it works: The server compares new data with past databases and uses algorithms to identify areas for improvement.

[1355] Step 5:

[1356] The server transmits the generated advice to the terminal, which notifies the user of it.

[1357] Input: Advice data

[1358] Output: Advice given to the user

[1359] Specific action: The device displays a notification to the user and suggests specific improvements.

[1360] Advice for next action

[1361] Step 1:

[1362] The user enters a customer profile and current business status into the terminal.

[1363] Input: Customer profile, sales status

[1364] Output: Data entered into the terminal

[1365] Specific operation: A salesperson enters customer information and sales progress through the application.

[1366] Step 2:

[1367] The terminal transmits customer profile and sales status data to the server.

[1368] Input: Customer profile, sales status

[1369] Output: Data sent to the server

[1370] Specific operation: The input data is automatically sent to the server.

[1371] Step 3:

[1372] The server uses a database such as MongoDB to analyze the data and evaluate potential next sales actions.

[1373] Input: Customer profile, sales status data

[1374] Output: Evaluated action candidates

[1375] What happens: The server takes in the data and uses an algorithm to determine the next step.

[1376] Step 4:

[1377] The server selects the optimal next action, generates advice, and sends it to the terminal.

[1378] Input: Evaluated action candidates

[1379] Output: Generated advisory data

[1380] Specific operation: The server uses an algorithm to determine the most effective action for the customer and generates it as advice.

[1381] Step 5:

[1382] The terminal notifies the user of the advice content.

[1383] Input: Advisory data

[1384] Output: Advice given to the user

[1385] Specific operation: The terminal notifies the user of advice on the next action.

[1386] Recommendation of proposed products

[1387] Step 1:

[1388] The user enters their customer profile and sales history into the terminal.

[1389] Input: Customer profile, sales history

[1390] Output: Data entered into the terminal

[1391] What happens: A salesperson uses the application to enter customer and past purchase history.

[1392] Step 2:

[1393] The terminal transmits this data to the server.

[1394] Input: Customer profile, sales history

[1395] Output: Data sent to the server

[1396] Specific operation: The entered data is transferred from the terminal to the server.

[1397] Step 3:

[1398] The server uses databases such as Salesforce to analyze the purchasing patterns of similar customers and extract related products.

[1399] Input: Customer profile, sales history

[1400] Output: List of related products

[1401] What happens: The server queries the database and extracts the most relevant products.

[1402] Step 4:

[1403] The server generates recommended products and sends the recommendation information to the terminal.

[1404] Input: List of related products

[1405] Output: Recommended product data

[1406] Specific operation: The server uses a recommendation algorithm to select the most suitable product for the customer.

[1407] Step 5:

[1408] The terminal notifies the user of the recommended information.

[1409] Input: Recommended product data

[1410] Output: Recommendation information notified to the user

[1411] Specific operation: The terminal displays a list of suggested products to the user.

[1412] This detailed processing flow allows the functions of the present invention to work together, comprehensively supporting sales activities and making them more efficient.

[1413] (Application example 1)

[1414] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1415] In modern sales and marketing activities, it is important to effectively manage customer interactions, efficiently summarize their content, and improve sales pitches by referencing past success stories. However, performing these processes manually takes time and effort, placing a heavy burden on sales representatives. Furthermore, advice on next sales actions and proposals for appropriate products must also be provided quickly and accurately, but current systems are unable to meet these needs.

[1416] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1417] In this invention, the server includes a means for recording conversations with customers, a means for converting the recorded voice data into text data using speech recognition, and a natural language processing means for analyzing the converted text data and extracting key points. This enables efficient summaries of conversations with customers and delivering them to users. The server also includes a means for maintaining a database of past successful sales pitches, a means for collecting new sales pitches from users and converting the recorded data into text data, a means for comparing the converted text data with past successes and generating advice on areas for improvement, a means for providing advice in real time, and a means for utilizing a generative AI model. This allows users to receive effective improvement advice in real time based on comparisons with past successes. The server also includes a means for collecting and analyzing customer profiles and sales status data, a means for evaluating potential next sales actions and selecting the optimal action, a means for analyzing user input through prompts, and a means for recommending related products based on the customer profile. This enables quick and accurate recommendations for next sales actions and proposed products.

[1418] "Customer profile" refers to information about a customer, such as their attributes, purchasing history, and interests.

[1419] "Sales talk" refers to the way salespeople talk and the explanations they give to customers to promote sales.

[1420] "Speech recognition" refers to the technology of analyzing recorded voice data and converting it into text data.

[1421] "Natural language processing" refers to the technology of analyzing text data and understanding its meaning and structure.

[1422] A "summary" refers to information that has been extracted from analyzed text data and concisely summarized.

[1423] "Database" refers to a system for efficiently storing information and making it searchable and accessible.

[1424] "Real-time" refers to processing or response occurring immediately, without delay.

[1425] A "generative AI model" refers to an artificial intelligence model that uses machine learning to learn patterns from large amounts of data, and then generates and analyzes new data.

[1426] A "prompt sentence" refers to an instruction sentence or question sentence that a user inputs to a system.

[1427] "Recommendation" refers to the act of suggesting optimal products or services based on a user's preferences and behavioral history.

[1428] The present invention relates to a sales support system, and in particular to a system that summarizes conversations with customers, provides advice on improving sales talks, gives advice on the next sales action, and recommends appropriate products through an application that runs on a device such as a smartphone.

[1429] The details of the system embodiment will be described below.

[1430] 1. Automatic summarization of customer conversations

[1431] When a user starts a conversation with a customer, the device (e.g., a smartphone) records the conversation. When the conversation ends, the device sends the recorded data to a server. The server converts the voice data into text data using a speech recognition engine (e.g., Google Speech Recognition). The server then analyzes the converted text data using a natural language processing algorithm (e.g., the "T5" model from the Transformers library), extracts important points, and generates a summary. The generated summary is distributed to the device via an intranet or the Internet, and is notified to the user. As a concrete example, a sales representative may discuss the features of a new product with a customer, and a summary may be generated based on the recording.

[1432] 2. Sales pitch training

[1433] The server maintains a database of past successful sales pitches. When a user gives a new sales pitch, the device records the content and sends it to the server. The server converts the received data into text data using voice recognition, compares it with past successful examples, and analyzes areas for improvement. Based on the analysis results, the server generates advice and delivers it to the user's device. As a concrete example, the server analyzes a presentation a sales representative gave to a customer and provides advice such as "it would be good to include more specific examples."

[1434] 3. Advice on next steps

[1435] The user inputs the customer profile and current sales situation into the device. The device sends this data to the server, which uses a generative AI model to analyze the customer profile and sales situation data and evaluate potential next sales actions. The server selects the optimal next action, generates advice, and delivers it to the device. The user can use this advice to effectively decide on the next step. As a specific example, the advice provided to the user is, "It would be effective to demonstrate a new product at the next interview."

[1436] 4. Recommendation of proposed products

[1437] The user enters their customer profile and sales history into the device, and the device sends the data to the server. The server uses a generative AI model to analyze the purchasing patterns of similar customers based on the customer profile and extract related products from a database. The server analyzes the information through prompt text, generates recommended products, and delivers the recommended information to the device. Based on this information, the user proposes appropriate products to the customer. As a specific example, the server recommends related products such as "new printers" and "high-performance computers."

[1438] Prompt Sentence Examples

[1439] "After a five-minute conversation with a customer about new product features, you have the following transcript. Please summarize it:

[1440] "Hello, customer. Today I'd like to introduce you to our new X900 series printer. This printer is extremely high-performance... (omitted)"

[1441] This system enables users to efficiently manage conversations with customers and conduct effective sales activities.

[1442] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1443] Step 1:

[1444] When a user starts a conversation with a customer, the user uses a terminal (smartphone) to record the conversation.

[1445] Input: Voice of conversation with customer

[1446] Output: Recorded audio file

[1447] What it does: Uses the device's recording feature to save a conversation with a customer as an audio file.

[1448] Step 2:

[1449] When the conversation ends, the terminal transmits the recorded voice data to the server.

[1450] Input: Recorded audio file

[1451] Output: Audio data uploaded to the server

[1452] How it works: Uploads recorded data from the device to a cloud server or database.

[1453] Step 3:

[1454] The server uses a speech recognition engine to convert the voice data into text data.

[1455] Input: Audio file

[1456] Output: Converted text data

[1457] What it does: Calls a speech recognition engine (e.g., Google Speech Recognition API) to convert an audio file into text.

[1458] Step 4:

[1459] The server analyzes the text data using natural language processing algorithms, extracts key points, and generates a summary.

[1460] Input: Text data

[1461] Output: Summary text

[1462] How it works: It uses natural language processing algorithms (e.g., Hugging Face Transformers' "T5" model) to extract key points and generate a summary.

[1463] Step 5:

[1464] The server delivers the generated summary to the terminal.

[1465] Input: Summary text

[1466] Output: Summary text delivered to the user's device

[1467] Action: Sends summary text to the terminal and notifies the user.

[1468] Step 6:

[1469] When a user makes a new sales pitch, the device records the content and sends it to the server.

[1470] Input: Sales pitch audio file

[1471] Output: Audio data uploaded to the server

[1472] Operation: Uploads the recorded audio data of the sales talk to the server.

[1473] Step 7:

[1474] The server converts the received voice data into text data using voice recognition.

[1475] Input: Audio file

[1476] Output: Converted text data

[1477] How it works: Uses a speech recognition engine to convert voice data into text data.

[1478] Step 8:

[1479] The server compares the text data with past success stories, analyzes areas for improvement, and generates advice.

[1480] Input: Text data, past success data

[1481] Output: Advice text

[1482] How it works: Text is vectorized using TfidfVectorizer, compared to past successes using cosine similarity, and a generative AI model is used to identify areas for improvement and generate advice.

[1483] Step 9:

[1484] The server distributes the generated advice to the user's terminal.

[1485] Input: Advice text

[1486] Output: Advice text delivered to the user's device

[1487] Action: Sends advice text to the user's device and notifies them.

[1488] Step 10:

[1489] The user enters a customer profile and current business status into the terminal.

[1490] Input: Customer profile, sales status data

[1491] Output: Profile and status data sent from the device to the server

[1492] How it works: A user enters their profile and status into a terminal and sends it to a database.

[1493] Step 11:

[1494] The server uses the generative AI model to analyze customer profiles and sales status data and select potential next sales actions.

[1495] Input: Customer profile, sales status data

[1496] Output: Text of advice on optimal sales actions

[1497] How it works: Analyzes data using a generative AI model to evaluate and select the next sales action.

[1498] Step 12:

[1499] The server delivers the selected action to the user as a recommendation.

[1500] Input: Sales action advice text

[1501] Output: Advice text delivered to the user's device

[1502] Operation: The advice is sent to the terminal and notified.

[1503] Step 13:

[1504] The user enters a customer profile and sales history into the terminal.

[1505] Input: Customer profile, sales history data

[1506] Output: Profile and sales history data sent from the device to the server

[1507] How it works: A user enters their profile and history into a terminal and sends it to a database.

[1508] Step 14:

[1509] The server uses a generative AI model to analyze the purchasing patterns of similar customers based on the customer profile and recommend related products.

[1510] Input: Customer profile, sales history data

[1511] Output: Recommended product list

[1512] How it works: Analyzes data using a generative AI model to recommend relevant products.

[1513] Step 15:

[1514] The server delivers a list of recommended products to the user's terminal.

[1515] Input: Recommended product list

[1516] Output: Product list delivered to the user's device

[1517] Operation: The recommended product list is sent to the device and notified.

[1518] For example, if you set the prompt text as follows:

[1519] "After a five-minute conversation with a customer about new product features, you have the following transcript. Please summarize it:

[1520] "Hello, customer. Today I'd like to introduce you to our new X900 series printer. This printer is extremely high-performance... (omitted)"

[1521] By using prompts in this way, users can efficiently issue instructions to the system and receive optimal summaries, advice, and recommendations.

[1522] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1523] This embodiment of the present invention combines an emotion engine that recognizes the user's emotions with an integrated sales support system that automatically summarizes conversations with customers, provides advice on improving sales talks, advises on effective next sales actions, and recommends appropriate products to propose. The program and processing of this system are described below with specific examples.

[1524] 1. Automatically summarize customer conversations

[1525] When a user starts a conversation with a customer, the device activates the recording function and records the audio of the conversation. When the conversation ends, the device sends the recorded data to the server. The server converts the received audio data into text data using a speech recognition engine. The server then runs the text data through a natural language processing algorithm to extract key points and generate a summary. Furthermore, an emotion engine analyzes the user's emotions from the converted text and audio data and adjusts the summary content accordingly. Finally, the server sends the generated summary to the user's device, where the user can review it. As a concrete example, a salesperson discusses a new product with a customer, and a summary is generated based on the recording, reflecting the content in line with the emotions.

[1526] 2. Sales pitch training

[1527] The server maintains a database of past successful sales pitches. When a user gives a new sales pitch, the device records the content and sends it to the server. The server converts the received data into text data using a voice recognition engine and runs an analysis algorithm that compares the converted text data with past successful examples. The server generates advice on how to improve the sales pitch based on the user's emotional data. The advice generated by the server is sent to the user's device, where the user can confirm it. It is also possible to provide flexible advice based on the user's emotions. As a specific example, a salesperson's presentation to a customer is analyzed and emotionally sensitive advice such as "it would be more effective if you gave more specific examples" is provided.

[1528] 3. Advice on next steps

[1529] The user enters a customer profile and current sales status into the terminal. The terminal sends this data to the server. The server analyzes the customer profile and sales status data and runs it through an algorithm that evaluates potential next sales actions. The server selects the optimal next action based on the evaluation results and uses an emotion engine to generate advice that takes the user's emotions into consideration. The server sends the generated advice to the user's terminal, where the user can confirm it. As a specific example, advice such as "It would be effective to demonstrate a new product at the next interview" is provided in a form that is appropriate to the user's emotions.

[1530] 4. Recommendation of proposed products

[1531] The user enters their customer profile and sales history into the terminal, which then sends the data to the server. The server uses this data to analyze the purchasing patterns of similar customers and extracts related products from a database. The server uses an emotion engine to generate recommendation information that takes the emotional data into account. The server then sends the recommendation information to the user's terminal, where the user confirms it. As a specific example, the server provides related products such as "new printers" and "high-performance computers" as recommendation information tailored to the user's emotions.

[1532] By linking the above functions, this system comprehensively supports sales activities, improving efficiency and effectiveness. In addition, the combination of an emotion engine enables flexible responses that take user emotions into consideration, allowing for more personalized support.

[1533] The processing flow will be explained below.

[1534] Automatically summarize customer conversations

[1535] Step 1:

[1536] The user starts a conversation with the customer. The device activates the recording function and starts recording the audio of the conversation.

[1537] Step 2:

[1538] After the conversation ends, the device sends the recorded data to the server.

[1539] Step 3:

[1540] The server sends the received voice data to a voice recognition engine, which converts the voice data into text data.

[1541] Step 4:

[1542] The server sends the converted text data to a natural language processing algorithm, which analyzes the text data.

[1543] Step 5:

[1544] The server extracts important keywords and phrases and generates a summary.

[1545] Step 6:

[1546] The server sends the voice and text data to the emotion engine, which analyzes the user's emotions.

[1547] Step 7:

[1548] The server adjusts the summary content based on the analysis results of the emotion engine.

[1549] Step 8:

[1550] The server sends the generated summary to the user's terminal.

[1551] Step 9:

[1552] The terminal displays the summary and the user confirms it.

[1553] Sales talk skills training

[1554] Step 1:

[1555] A server maintains a database of past successful sales pitches.

[1556] Step 2:

[1557] The user makes a new sales pitch, and the device records the content.

[1558] Step 3:

[1559] After the sales talk ends, the device sends the recorded data to the server.

[1560] Step 4:

[1561] The server sends the received voice data to a voice recognition engine, which converts it into text data.

[1562] Step 5:

[1563] The server sends the converted text data to an analysis algorithm that compares it with past successes.

[1564] Step 6:

[1565] The server generates advice on how to improve the sales pitch based on the analysis results.

[1566] Step 7:

[1567] The server sends text data to the emotion engine, which analyzes the user's emotions.

[1568] Step 8:

[1569] The server adjusts the advice content based on the analysis results of the emotion engine.

[1570] Step 9:

[1571] The server transmits the generated advice to the user's terminal.

[1572] Step 10:

[1573] The terminal notifies the user of the advice, which is then confirmed by the user.

[1574] Advice for next action

[1575] Step 1:

[1576] The user enters a customer profile and current business status into the terminal.

[1577] Step 2:

[1578] The terminal transmits the input data to the server.

[1579] Step 3:

[1580] The server sends customer profile and sales data to an analytics algorithm.

[1581] Step 4:

[1582] The server generates the next sales action candidates based on the analysis results.

[1583] Step 5:

[1584] The server sends the user's emotional data to the emotion engine, which analyzes the emotions.

[1585] Step 6:

[1586] Based on the analysis results of the emotion engine, the server ranks and adjusts potential actions.

[1587] Step 7:

[1588] The server selects the best next action and generates advice.

[1589] Step 8:

[1590] The server sends the generated advice to the user's terminal.

[1591] Step 9:

[1592] The terminal notifies the user of the advice and the user confirms it.

[1593] Recommendation of proposed products

[1594] Step 1:

[1595] The user enters their customer profile and sales history into the terminal.

[1596] Step 2:

[1597] The terminal transmits the input data to the server.

[1598] Step 3:

[1599] The server sends customer profiles and sales histories to an analytics algorithm.

[1600] Step 4:

[1601] The server analyzes the purchasing patterns of similar customers and extracts related products from the database.

[1602] Step 5:

[1603] The server sends the user's emotional data to the emotion engine, which analyzes the emotions.

[1604] Step 6:

[1605] The server adjusts the recommendations based on the analysis results of the emotion engine.

[1606] Step 7:

[1607] The server transmits the generated recommendation information to the user's terminal.

[1608] Step 8:

[1609] The device notifies the user of the recommended information, which is then confirmed by the user.

[1610] As a result, the system of the present invention, which incorporates an emotion engine, can comprehensively support sales activities and respond flexibly while taking into account the user's emotions. As a specific example, the emotion engine can analyze the emotional state of a salesperson, and if the salesperson is feeling stressed, it can provide personalized support, such as advice on how to speak calmly.

[1611] Example 2

[1612] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1613] Conventional sales support systems have the problem of being unable to automatically and efficiently summarize conversations with customers, give advice on improving sales talks, or advise on next actions.In addition, these functions are not adjusted according to the user's emotions, and only standardized suggestions and advice are provided, which has led to issues that do not provide sufficient satisfaction to users.

[1614] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for recording a conversation with a customer, a means for converting the recorded voice data into text data using a voice recognition engine, a means for analyzing the converted text data using a natural language processing algorithm and extracting important points, a means for generating a summary from the extracted points and adjusting the summary content according to the user's emotions using an emotion engine, and a means for delivering the adjusted summary to the user. This makes it possible to provide summaries and advice that take the user's emotions into consideration, which is expected to improve the efficiency and effectiveness of sales activities.

[1615] The "means for recording conversations with customers" refers to a device or software for recording conversations as audio data when a user starts a conversation with a customer.

[1616] A "speech recognition engine" is software or algorithms that convert recorded voice data into text data.

[1617] A "natural language processing algorithm" is an algorithm for analyzing text data and extracting important information and patterns.

[1618] An "emotion engine" is software or an algorithm that analyzes emotions from text and voice data and makes adjustments based on those emotions.

[1619] A "summary generation means" is a device or software that extracts important points and creates a short summary of the text.

[1620] The "distribution means" refers to a device or software for transmitting the generated information or advice to the user's terminal and displaying it.

[1621] A "database" is an information accumulation system that stores data such as past successful sales pitches, customer profiles, and sales history, and can be searched and used as needed.

[1622] An "analysis algorithm" is an algorithm that analyzes collected data and finds patterns and trends.

[1623] "Potential sales actions" are suggestions for specific actions that can be taken in the next sales activity.

[1624] The "means for delivering advice" is a device or software that notifies the user of the optimal action to take in the next sales activity and provides it as a guideline for action.

[1625] The present invention is an integrated sales support system that automatically summarizes conversations with customers, provides advice on improving sales talks, advises on effective next sales actions, and recommends appropriate products. By combining this system with an emotion engine that recognizes the user's emotions, the present invention can provide more personalized support. The specific configuration and operation of this system are described below.

[1626] 1. Automatically summarize customer conversations

[1627] When a user starts a conversation with a customer, the device activates the recording function and records the audio of the conversation. A recording application (e.g., Recorder App) is installed on the device. When the conversation ends, the device sends the recorded data to the server. The recorded data is sent to the server using an HTTP request (e.g., in MP3 format). The server converts the received audio data into text data using a speech recognition engine (e.g., Google Cloud Speech-to-Text). The server then uses a natural language processing algorithm (e.g., BERT or GPT-3) to analyze the text data, extract key points, and generate a summary. After that, an emotion engine (e.g., Microsoft Azure Emotion API) analyzes the user's emotions from the converted text and audio data and adjusts the summary content. Finally, the server sends the generated summary to the user's device, where the user can review it.

[1628] For example, if a salesperson is discussing a new product with a customer, a summary is generated based on the recording, and the content reflects the customer's emotions.

[1629] Example prompt sentence:

[1630] "Summarize key points from recordings of customer conversations, analyze sentiment, and adjust accordingly."

[1631] 2. Sales pitch training

[1632] The server maintains a database of past successful sales pitches. When a user gives a new sales pitch, the device records the content and sends it to the server. The server converts the received data into text data using a voice recognition engine and runs an analysis algorithm (e.g., a similarity calculation algorithm) that compares the converted text data with past successful examples. The server generates advice on how to improve the sales pitch based on the user's emotional data. The generated advice is adjusted according to the user's emotions using an emotion engine. The server sends the adjusted advice to the user's device, where the user confirms it.

[1633] As a concrete example, the system analyzes a presentation a salesperson gives to a customer and provides emotionally sensitive advice such as "It would be more effective if you provided more specific examples."

[1634] Example prompt sentence:

[1635] "Please compare my current sales pitch with past successful ones and give me advice on how to improve it."

[1636] 3. Advice on next steps

[1637] The user inputs their customer profile and current sales situation into their device. The device then sends this data to the server. The server then runs this data through an analytical algorithm (e.g., a machine learning model) to evaluate potential next sales actions. The server then selects the optimal next action based on the evaluation results and uses an emotion engine to generate advice that takes the user's emotions into account. The server then sends the generated advice to the user's device, where the user confirms it.

[1638] As a specific example, advice such as "It would be effective to demonstrate a new product at the next interview" is provided in accordance with the user's emotions.

[1639] Example prompt sentence:

[1640] "Please advise me on my next action based on my customer profile and sales situation."

[1641] 4. Recommendation of proposed products

[1642] The user enters their customer profile and sales history into the terminal, which then sends the data to the server. The server uses this data to analyze the purchasing patterns of similar customers and extracts related products from the database. The server then uses an emotion engine to generate recommendation information that takes into account the emotion data. The generated recommendation information is then sent by the server to the user's terminal, where the user can confirm it.

[1643] As a specific example, the server provides related products such as "new printers" and "high-performance computers" as recommended information tailored to the user's emotions.

[1644] Example prompt sentence:

[1645] "Recommend relevant products based on customer profile and sales history."

[1646] By linking the above functions, this system comprehensively supports sales activities, improving efficiency and effectiveness. In addition, the combination of an emotion engine enables flexible responses that take into account the user's emotions, allowing for more personalized support.

[1647] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1648] Step 1:

[1649] When a user starts a conversation with a customer, the device automatically activates the recording function and collects the audio data of the conversation. Specifically, the recording application installed on the device starts running in the background. The recording application receives audio data as input and outputs a recorded audio file (e.g., MP3 format).

[1650] Step 2:

[1651] Once the conversation is finished, the device sends the recording to the server using an HTTP POST request to the server's API endpoint. The device receives the recorded audio file as input and outputs an HTTP request.

[1652] Step 3:

[1653] The server converts the received voice data into text data using a voice recognition engine. Specifically, the server calls the API of the voice recognition engine (for example, Google Cloud Speech-to-Text), provides the voice data as input, and receives the converted text data as output.

[1654] Step 4:

[1655] The server applies a natural language processing algorithm to the acquired text data to extract key points. Specifically, the server inputs the text data into a natural language processing algorithm (e.g., BERT or GPT-3) and outputs key points as the analysis results.

[1656] Step 5:

[1657] The server uses an emotion engine to analyze emotions from text data and voice data. Specifically, the server calls the API of the emotion engine (for example, Microsoft Azure Emotion API), provides text data and voice data as input, and receives the emotion analysis results as output.

[1658] Step 6:

[1659] The server adjusts the summary content based on the sentiment analysis results and generates the final summary. Specifically, the server receives the sentiment analysis results and key points as input and outputs the adjusted summary.

[1660] Step 7:

[1661] The server sends the generated summary to the user's device. Specifically, the server uses an HTTP response to send the summary data to the user's device. The user's device receives the summary data as input and outputs a custom UI to display to the user.

[1662] This allows the content of the conversation with the customer to be summarized and presented in a form that matches the user's emotions.

[1663] (Application example 2)

[1664] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1665] Conventional sales support systems are limited in their functionality, such as efficiently summarizing conversations with customers, providing advice on improving sales talks, advising on next actions, and recommending proposed products. Furthermore, few systems can take user emotions into account when it comes to these functions, and more personalized support is desired. Therefore, there is a need for a comprehensive sales support system that can summarize conversations with customers, provide guidance on sales talks, advise on next actions, recommend proposed products, and also consider user emotions.

[1666] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1667] In this invention, the server includes means for recording conversations with customers, means for converting the recorded voice data into text data using voice recognition, natural language processing means for analyzing the converted text data and extracting important points, means for generating a summary of the extracted points and distributing it to the user, and means for recognizing the user's emotions and reflecting them in the summary content, thereby enabling efficient sales support that takes the user's emotions into consideration.

[1668] The server also includes means for maintaining a database of past successful sales talks, means for collecting new sales talks from users and converting the recorded data into text data, means for comparing the converted text data with past successful examples and generating advice on areas for improvement, means for delivering the generated advice to users, and means for generating and delivering flexible advice taking into account the user's emotions, thereby enabling the quality of sales talks to be improved while taking into account emotions.

[1669] Furthermore, the server includes means for collecting and analyzing customer profiles and sales status data, means for evaluating next sales action candidates and selecting the optimal action, means for delivering the selected action to the user as advice, and means for generating and delivering advice content that takes the user's emotions into consideration, thereby making the next sales activity more effective and personalized.

[1670] "Customer" refers to the person to whom goods or services are provided in a business transaction or transaction.

[1671] "Conversation recording means" refers to a device or function for recording a conversation with a customer as audio data.

[1672] "Speech recognition means" refers to technology or software that analyzes recorded voice data and converts it into text data.

[1673] "Character data" refers to text information converted by a speech recognition means.

[1674] "Natural language processing means" refers to algorithms and technologies for analyzing text data and extracting important points.

[1675] "Summary generation means" refers to a function that concisely summarizes a text based on extracted important points.

[1676] "Emotion recognition means" refers to the technology and algorithms that determine and analyze a user's emotions from text data and voice data.

[1677] A "sales pitch database" refers to a system or device that stores records and data of past successful sales pitches.

[1678] "Means for generating improvement advice" refers to a function that compares new sales pitches with existing successful examples and suggests more effective methods.

[1679] "Customer profile" refers to detailed data such as basic customer information and past transaction history.

[1680] "Sales status data" refers to information regarding the progress and results of current sales activities.

[1681] "Action evaluation means" refers to a function for evaluating potential next sales actions and selecting the most appropriate course of action.

[1682] The "advice delivery means" refers to a function for transmitting the generated advice or advice to the user.

[1683] This invention combines an emotion engine that recognizes user emotions with an integrated sales support system that automatically summarizes conversations with customers, provides advice on improving sales talks, advises on effective next sales actions, and recommends appropriate products. This system is designed to support customer service in brick-and-mortar stores in particular.

[1684] The entire system functions by linking the server and the terminal. Specifically, it has the following configuration:

[1685] 1. Automatically summarize customer conversations

[1686] The user wears smart glasses as a device and records conversations with customers. The recorded audio data is sent to a server in real time. The server converts the audio data into text data using Google Cloud Speech-to-Text. The converted text data is analyzed using natural language processing with the BERT model to extract key points. A summary is then generated, and Affectiva's emotion recognition engine analyzes the user's emotions and reflects them in the summary. The generated summary is then sent back to the user's device, where the user can review it.

[1687] 2. Sales pitch training

[1688] The server maintains a database of past successful sales pitches. When a user makes a new sales pitch, the smart glasses record it and send the data to the server. The server converts the recorded data into text data using Google Cloud Speech-to-Text, analyzes the converted data using the BERT model, and compares it with past successful examples. Based on the analysis results, advice on areas for improvement is generated, and an emotion engine provides advice based on the user's emotions. As a result, flexible advice is delivered to the user's device.

[1689] 3. Advice on next steps

[1690] When a user enters their customer profile and current sales status into their device, the data is sent to the server. The server analyzes the profile and sales status and evaluates the next sales action candidate using the BERT model and machine learning algorithm (K-means clustering). The emotion engine generates advice on the optimal next action that reflects the user's emotions and delivers it to the user's device.

[1691] 4. Recommendation of proposed products

[1692] Users enter their customer profile and sales history, and the data is sent to the server. The server analyzes the purchasing patterns of similar customers and extracts related products. The emotion engine then generates recommendation information that takes emotional data into account, and delivers it to the user's device. This allows users to recommend suggested products to customers at the appropriate time.

[1693] Example prompt sentence:

[1694] "Please convert the following Japanese audio data into text and summarize it."

[1695] "Perform a sentiment analysis of this text and generate sentiment-based advice."

[1696] "Recommend the next most effective sales action based on your customer profile."

[1697] This system will improve the efficiency of sales activities in brick-and-mortar stores and enable effective responses to customers. By instantly analyzing information obtained during conversations with customers and providing advice based on their emotions, the quality of sales activities can be improved.

[1698] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1699] Step 1:

[1700] A user uses smart glasses to record a conversation with a customer. The recorded voice data is sent to the server in real time. The input is the voice data, and the output is the raw voice data sent to the server.

[1701] Step 2:

[1702] The server receives the voice data and converts it into text data using Google Cloud Speech-to-Text. The input is voice data and the output is text data. Voice analysis is performed as part of the data processing.

[1703] Step 3:

[1704] The server analyzes the converted text data using the BERT model and extracts key points. The input is text data, and the output is a list of key points. Natural language processing is performed on the data.

[1705] Step 4:

[1706] The server generates a summary based on the extracted key points. The input is a list of key points, and the output is a summary. The data processing performed here is the generation of a summary.

[1707] Step 5:

[1708] The server uses Affectiva's emotion recognition engine to analyze the user's emotions from text and voice data. The input is text and voice data, and the output is emotion data. The process performed is emotion analysis.

[1709] Step 6:

[1710] The generated summary is adjusted by reflecting emotional data. The input is the summary and emotional data, and the output is the summary that reflects the emotion. The emotional data is reflected as a data calculation.

[1711] Step 7:

[1712] The final summary is delivered to the user's device. The input is the summary that reflects the emotion, and the output is the summary delivered to the user. The specific operation is that the device receives the summary and displays it to the user.

[1713] Step 8:

[1714] A database of past successful sales pitches is maintained by the server. The input is the database, and the output is the data of successful cases. The operation is to execute a query on the database.

[1715] Step 9:

[1716] The server converts newly recorded sales talks by the user into text data and compares them with past successes. The input is the newly recorded data, and the output is the comparison results. Data processing involves voice analysis and comparison calculations.

[1717] Step 10:

[1718] The server generates improvement advice and adjusts it based on the emotion data. The input is the comparison result and emotion data, and the output is advice based on the emotion. The processing involves generating advice and reflecting the emotion data.

[1719] Step 11:

[1720] The generated advice is delivered to the user's device. The input is the advice according to the emotion, and the output is the advice delivered to the user. The specific operation is that the device receives the advice and displays it to the user.

[1721] Step 12:

[1722] Customer profile and sales status data are sent from the terminal to the server. The input is the profile data and sales status data, and the output is the data sent to the server. The action is sending the data.

[1723] Step 13:

[1724] The server analyzes the profile and sales status data and evaluates the next optimal sales action. The input is the customer profile and sales status data, and the output is the next action candidate. The data processing is evaluated using machine learning.

[1725] Step 14:

[1726] Based on the evaluation results, the system generates advice on the optimal next action using emotional data. The input is the next action candidate and emotional data, and the output is advice based on the emotion. The processing involves generating advice and reflecting the emotional data.

[1727] Step 15:

[1728] The generated advice is delivered to the user's device. The input is advice corresponding to the emotion, and the output is the advice delivered to the user. The specific operation is for the device to receive the advice and display it to the user.

[1729] Step 16:

[1730] The user enters a customer profile and sales history, which is sent to the server. The input is the profile data and sales history, and the output is the data sent to the server. The action is sending the data.

[1731] Step 17:

[1732] The server analyzes the purchasing patterns of similar customers and extracts related products. The input is profile data and sales history, and the output is a list of related products. The data processing is done using machine learning analysis.

[1733] Step 18:

[1734] Recommendation information is generated using an emotion engine and delivered to users. The input is a list of related products and emotion data, and the output is recommendation information based on the emotion. The processing involves generating recommendation information and reflecting the emotion data.

[1735] Step 19:

[1736] The final recommendation information is delivered to the user's device. The input is recommendation information based on emotions, and the output is the recommendation information delivered to the user. The operation is for the device to receive the recommendation information and display it to the user.

[1737] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1738] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1739] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1740] [Fourth embodiment]

[1741] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1742] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1743] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1744] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1745] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1746] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1747] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1748] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1749] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1750] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1751] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1752] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1753] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1754] An embodiment of the present invention is an integrated sales support system that automatically summarizes conversations with customers, provides advice on improving sales talks, advises on effective sales actions for the next time, and recommends appropriate proposed products. The program and processing of this system are described below with specific examples.

[1755] 1. Automatically summarize customer conversations

[1756] When a user starts a conversation with a customer, the device records the conversation. When the conversation ends, the device sends the recorded data to the server. The server uses a speech recognition engine to convert the voice data into text data. The server then analyzes the text data using a natural language processing algorithm, extracts important points, and generates a summary. The server then sends the generated summary to the device and notifies the user. As a concrete example, a salesperson discusses a new product with a customer, and a summary is generated based on the recording.

[1757] 2. Sales pitch training

[1758] The server maintains a database of past successful sales pitches. When a user makes a new sales pitch, the device records the content and sends it to the server. The server converts the received data into text data using voice recognition, compares it with past successful examples, and analyzes areas for improvement. Based on the analysis results, the server generates advice and sends it to the user's device. As a concrete example, the server analyzes a presentation a salesperson gave to a customer and provides advice such as, "It would be more effective if you gave more specific examples."

[1759] 3. Advice on next steps

[1760] The user inputs a customer profile and current sales status into the terminal. The terminal sends this data to the server, which analyzes the customer profile and sales status data and evaluates potential next sales actions. The server selects the optimal next action, generates advice, and sends it to the terminal. The user can effectively decide on the next step based on this advice. As a specific example, the advice provided to the user is, "It would be effective to demonstrate a new product at the next interview."

[1761] 4. Recommendation of proposed products

[1762] The user enters their customer profile and sales history into the terminal, which then sends the data to the server. Based on this data, the server analyzes the purchasing patterns of similar customers and extracts related products from a database. The server generates recommended products and sends the recommendation information to the terminal. Based on this information, the user can propose appropriate products to the customer. As a specific example, the server recommends related products such as "new printers" and "high-performance computers."

[1763] This system will link the above functions to comprehensively support sales activities and make them more efficient.It is also anticipated that these functions will be provided in a SaaS (Software as a Service) format, and that they will be available to a wide range of salespeople via the Internet.

[1764] The processing flow will be explained below.

[1765] Automatically summarize customer conversations

[1766] Step 1:

[1767] When the user starts a conversation with the customer, the terminal activates the recording function and starts recording the audio of the conversation.

[1768] Step 2:

[1769] After the conversation ends, the device sends the recorded data to the server.

[1770] Step 3:

[1771] The server sends the received voice data to a voice recognition engine, which converts the voice data into text data.

[1772] Step 4:

[1773] The server runs the converted text data through a natural language processing algorithm to analyze the content of the conversation.

[1774] Step 5:

[1775] The server extracts important keywords and phrases and generates a summary.

[1776] Step 6:

[1777] The server sends the generated summary to the user's terminal.

[1778] Step 7:

[1779] The terminal displays the summary on the screen for the user to confirm.

[1780] Sales talk skills training

[1781] Step 1:

[1782] The server stores a database of past successful sales pitches.

[1783] Step 2:

[1784] The user starts a new sales conversation, and the device records the conversation.

[1785] Step 3:

[1786] After the sales talk ends, the device sends the recorded data to the server.

[1787] Step 4:

[1788] The server converts the received voice data into text data using a voice recognition engine.

[1789] Step 5:

[1790] The server runs the converted text data through an analysis algorithm that compares it with past successes.

[1791] Step 6:

[1792] The server generates advice on how to improve the sales pitch based on the analysis results.

[1793] Step 7:

[1794] The server transmits the generated advice to the user's terminal.

[1795] Step 8:

[1796] The terminal notifies the user of the advice, which the user confirms.

[1797] Advice for next action

[1798] Step 1:

[1799] The user enters a customer profile and current business status into the terminal.

[1800] Step 2:

[1801] The terminal transmits the entered customer profile and business status data to the server.

[1802] Step 3:

[1803] The server runs the received data through an analytical algorithm to analyze the customer's situation.

[1804] Step 4:

[1805] The server generates multiple next action candidates based on the analysis results.

[1806] Step 5:

[1807] The server ranks the next action candidates based on their deviation scores and success rates.

[1808] Step 6:

[1809] The server selects the best next action and generates advice.

[1810] Step 7:

[1811] The server sends the generated advice to the user's terminal.

[1812] Step 8:

[1813] The terminal notifies the user of the advice, which the user confirms.

[1814] Recommendation of proposed products

[1815] Step 1:

[1816] The user enters their customer profile and sales history into the terminal.

[1817] Step 2:

[1818] The terminal transmits the input data to the server.

[1819] Step 3:

[1820] The server runs customer profiles and sales history data through analytical algorithms.

[1821] Step 4:

[1822] The server analyzes the purchasing patterns of similar customers and extracts related products from the database.

[1823] Step 5:

[1824] The server generates recommendation information based on the analysis results.

[1825] Step 6:

[1826] The server transmits the generated recommendation information to the user's terminal.

[1827] Step 7:

[1828] The device notifies the user of the recommended information, which the user can then confirm.

[1829] With the above processing flow, the system of the present invention can comprehensively support sales activities and improve efficiency.

[1830] Example 1

[1831] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1832] Conventional sales support systems lacked the means to efficiently summarize conversations with customers and provide specific guidance to improve the quality of sales talks. Furthermore, they lacked a system that could provide advice on the next sales action or recommend appropriate products, which meant that sales activities were not sufficiently streamlined. This created challenges that made it difficult for salespeople to conduct effective sales activities.

[1833] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1834] In this invention, the server includes a means for recording conversations with customers, a means for converting the recorded voice data into character data using a recognition means, a processing means for analyzing the converted character data and extracting important points, and a means for generating a summary of the extracted points and distributing it to the user. This makes it possible to efficiently summarize the content of conversations with customers, allowing the user to quickly grasp the main points.

[1835] The server also includes a means for maintaining a database of past successful sales talks, a means for collecting new sales talks from users and converting the recorded data into text data, a means for comparing the converted text data with past successful examples and generating advice on areas for improvement, and a means for delivering the generated advice to users, thereby enabling specific guidance for improving the quality of sales talks.

[1836] Furthermore, the server includes a means for collecting and analyzing customer profiles and situation data, a means for evaluating next action candidates and selecting the most appropriate action, and a means for delivering the selected action to the user as advice, thereby enabling appropriate advice on the next sales action to be provided, thereby achieving efficiency in sales activities.

[1837] "Customer" means any person or entity to whom products or services are provided.

[1838] "Conversation" refers to the process of information exchange between a customer and a salesperson, which is usually conducted verbally.

[1839] "Recording" refers to the method of recording audio data in digital or analog form.

[1840] "Audio data" refers to data that records human voice as a digital signal.

[1841] "Recognition means" refers to the function of a device or software for converting voice data into text data.

[1842] "Character data" refers to data expressed in text format.

[1843] "Analysis" refers to the process of examining the contents of data in detail to extract meaning and characteristics.

[1844] "Key points" refer to particularly important information or key phrases in the conversation.

[1845] "Processing means" refers to the functionality of a device or software to perform specific data operations or algorithms.

[1846] A "summary" is a short summary that extracts important elements from a long piece of text or a large amount of data.

[1847] "Distribution" refers to the act of delivering information or data to a specific recipient.

[1848] "Talk" refers to the sales pitch or presentation that a salesperson gives to a customer.

[1849] "Database" refers to a system or software for efficiently storing, managing, and retrieving data.

[1850] A "new sales pitch" refers to a new type of sales presentation that has never been done before.

[1851] "Success stories" refer to specific sales pitches or approaches that have produced results in the past.

[1852] "Areas for improvement" refers to areas where changes are needed to make the current way of doing things more effective.

[1853] "Advice" refers to advice or instructions for achieving a specific purpose.

[1854] "Profile" refers to information that compiles a customer's attributes, past transaction history, and other characteristics.

[1855] "Contextual data" refers to data related to current sales activities and customer status.

[1856] "Potential actions" refer to the next possible sales activity options that may be taken.

[1857] "Optimal action" refers to the sales activity that is considered to be the most effective among multiple options.

[1858] "Advice" refers to information or instructions that help you decide what to do next.

[1859] This invention is an integrated sales support system that automatically summarizes conversations with customers, provides advice on improving sales talks, advises on effective sales actions for the next time, and recommends appropriate products to propose. This system is provided in the form of SaaS (Software as a Service) and is expected to be used by a wide range of salespeople via the Internet.

[1860] The specific configuration and operation of the system will be described below.

[1861] Automatically summarize customer conversations

[1862] When a user starts a conversation with a customer, the device (such as a smartphone or PC) records the conversation. When the conversation ends, the device sends the recorded data to a server. The server uses a speech recognition engine such as Google Cloud Speech-to-Text to convert the voice data into text data. The server then analyzes the text data using a natural language processing algorithm such as NLTK (Natural Language Toolkit), extracts key points, and generates a summary. Finally, the generated summary is sent to the device and notified to the user.

[1863] As a concrete example, a salesperson discusses a new product with a customer and records the conversation. After the conversation ends, the system generates a summary such as "I'm interested in the new product's features" and "I'd like to know more about the price." Example prompts:

[1864] Generate a summary of a conversation with a customer. Here is the transcript of the conversation:

[1865] Sales talk skills training

[1866] The server maintains a database of past successful sales pitches, and when a user makes a new sales pitch, the device records the content and sends it to the server. The server uses Google Cloud Speech-to-Text to convert the audio data into text data. The server then compares it with past successes and analyzes areas for improvement. Advice is generated based on the analysis results and sent to the user's device.

[1867] For example, the system analyzes a salesperson's presentation and provides advice such as, "It would be more effective if you provided more specific examples." Example prompt:

[1868] Please offer some advice to improve the following sales pitch:...

[1869] Advice for next action

[1870] When a user enters their customer profile and current sales status into the device, the device sends this to the server. The server uses a database such as MongoDB to analyze the customer profile and sales status data, evaluate potential next sales actions, select the most appropriate next action, generate advice, and send it to the device.

[1871] As a concrete example, the generated advice is "It would be effective to demonstrate the new product at the next interview." Example prompt sentence:

[1872] Suggest the best next action based on the following customer profile and sales situation:...

[1873] Recommendation of proposed products

[1874] The user enters their customer profile and sales history into the terminal, which then sends it to the server. The server uses a database such as Salesforce to analyze the purchasing patterns of similar customers and extract related products. Recommended products are generated and the recommendation information is sent to the terminal.

[1875] For example, a "new printer" or a "high-performance computer" may be recommended. Example prompts:

[1876] Please recommend the appropriate products based on the following customer profile and sales history:

[1877] By linking the above functions, sales activities can be comprehensively supported and made more efficient.

[1878] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1879] Automatically summarize customer conversations

[1880] Step 1:

[1881] When the user starts a conversation with the customer, the terminal records the conversation.

[1882] Input: Voice of conversation with customer

[1883] Output: Recorded audio data

[1884] Specific operation: A salesperson uses a smartphone to record a discussion with a customer.

[1885] Step 2:

[1886] When the conversation is finished, the user presses the recording end button and the terminal sends the recorded data to the server.

[1887] Input: Recorded audio data

[1888] Output: Audio data sent to the server

[1889] Specific operation: The salesperson presses the end recording button, and the device uploads the audio data to the server.

[1890] Step 3:

[1891] The server uses a speech recognition engine such as Google Cloud Speech-to-Text to convert the voice data into text data.

[1892] Input: Audio data

[1893] Output: Character data

[1894] Specific operation: The server starts a speech recognition engine and converts the voice data into text.

[1895] Step 4:

[1896] The server uses natural language processing algorithms such as NLTK to analyze the text data, extract key points, and generate a summary.

[1897] Input: Character data

[1898] Output: Summary data

[1899] What it does: The server runs a natural language processing algorithm to extract customer points of interest and generate a summary.

[1900] Step 5:

[1901] The generated summary is sent from the server to the terminal, which notifies the user of it.

[1902] Input: Summary data

[1903] Output: Summary data reported to the user

[1904] Specific operation: The terminal uses the notification function to display the summary content to the user.

[1905] Sales talk skills training

[1906] Step 1:

[1907] When a user gives a sales pitch, the device records the content.

[1908] Input: Sales talk audio

[1909] Output: Recorded audio data

[1910] Specific operation: Start recording when the salesperson starts talking.

[1911] Step 2:

[1912] After the recording is completed, the device sends the recorded data to the server.

[1913] Input: Recorded audio data

[1914] Output: Audio data sent to the server

[1915] Specific operation: When the talk ends, the data is automatically sent to the server.

[1916] Step 3:

[1917] The server uses a voice recognition engine to convert the recorded data into text data.

[1918] Input: Audio data

[1919] Output: Character data

[1920] Specific operation: The server converts the voice data into text using Google Cloud Speech-to-Text.

[1921] Step 4:

[1922] The server compares the newly received text data with past successes and analyzes areas for improvement.

[1923] Input: Text data, past success data

[1924] Output: Improvements

[1925] How it works: The server compares new data with past databases and uses algorithms to identify areas for improvement.

[1926] Step 5:

[1927] The server transmits the generated advice to the terminal, which notifies the user of it.

[1928] Input: Advice data

[1929] Output: Advice given to the user

[1930] Specific action: The device displays a notification to the user and suggests specific improvements.

[1931] Advice for next action

[1932] Step 1:

[1933] The user enters a customer profile and current business status into the terminal.

[1934] Input: Customer profile, sales status

[1935] Output: Data entered into the terminal

[1936] Specific operation: A salesperson enters customer information and sales progress through the application.

[1937] Step 2:

[1938] The terminal transmits customer profile and sales status data to the server.

[1939] Input: Customer profile, sales status

[1940] Output: Data sent to the server

[1941] Specific operation: The input data is automatically sent to the server.

[1942] Step 3:

[1943] The server uses a database such as MongoDB to analyze the data and evaluate potential next sales actions.

[1944] Input: Customer profile, sales status data

[1945] Output: Evaluated action candidates

[1946] What happens: The server takes in the data and uses an algorithm to determine the next step.

[1947] Step 4:

[1948] The server selects the optimal next action, generates advice, and sends it to the terminal.

[1949] Input: Evaluated action candidates

[1950] Output: Generated advisory data

[1951] Specific operation: The server uses an algorithm to determine the most effective action for the customer and generates it as advice.

[1952] Step 5:

[1953] The terminal notifies the user of the advice content.

[1954] Input: Advisory data

[1955] Output: Advice given to the user

[1956] Specific operation: The terminal notifies the user of advice on the next action.

[1957] Recommendation of proposed products

[1958] Step 1:

[1959] The user enters their customer profile and sales history into the terminal.

[1960] Input: Customer profile, sales history

[1961] Output: Data entered into the terminal

[1962] What happens: A salesperson uses the application to enter customer and past purchase history.

[1963] Step 2:

[1964] The terminal transmits this data to the server.

[1965] Input: Customer profile, sales history

[1966] Output: Data sent to the server

[1967] Specific operation: The entered data is transferred from the terminal to the server.

[1968] Step 3:

[1969] The server uses databases such as Salesforce to analyze the purchasing patterns of similar customers and extract related products.

[1970] Input: Customer profile, sales history

[1971] Output: List of related products

[1972] What happens: The server queries the database and extracts the most relevant products.

[1973] Step 4:

[1974] The server generates recommended products and sends the recommendation information to the terminal.

[1975] Input: List of related products

[1976] Output: Recommended product data

[1977] Specific operation: The server uses a recommendation algorithm to select the most suitable product for the customer.

[1978] Step 5:

[1979] The terminal notifies the user of the recommended information.

[1980] Input: Recommended product data

[1981] Output: Recommendation information notified to the user

[1982] Specific operation: The terminal displays a list of suggested products to the user.

[1983] This detailed processing flow allows the functions of the present invention to work together, comprehensively supporting sales activities and making them more efficient.

[1984] (Application example 1)

[1985] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1986] In modern sales and marketing activities, it is important to effectively manage customer interactions, efficiently summarize their content, and improve sales pitches by referencing past success stories. However, performing these processes manually takes time and effort, placing a heavy burden on sales representatives. Furthermore, advice on next sales actions and proposals for appropriate products must also be provided quickly and accurately, but current systems are unable to meet these needs.

[1987] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1988] In this invention, the server includes a means for recording conversations with customers, a means for converting the recorded voice data into text data using speech recognition, and a natural language processing means for analyzing the converted text data and extracting key points. This enables efficient summaries of conversations with customers and delivering them to users. The server also includes a means for maintaining a database of past successful sales pitches, a means for collecting new sales pitches from users and converting the recorded data into text data, a means for comparing the converted text data with past successes and generating advice on areas for improvement, a means for providing advice in real time, and a means for utilizing a generative AI model. This allows users to receive effective improvement advice in real time based on comparisons with past successes. The server also includes a means for collecting and analyzing customer profiles and sales status data, a means for evaluating potential next sales actions and selecting the optimal action, a means for analyzing user input through prompts, and a means for recommending related products based on the customer profile. This enables quick and accurate recommendations for next sales actions and proposed products.

[1989] "Customer profile" refers to information about a customer, such as their attributes, purchasing history, and interests.

[1990] "Sales talk" refers to the way salespeople talk and the explanations they give to customers to promote sales.

[1991] "Speech recognition" refers to the technology of analyzing recorded voice data and converting it into text data.

[1992] "Natural language processing" refers to the technology of analyzing text data and understanding its meaning and structure.

[1993] A "summary" refers to information that has been extracted from analyzed text data and concisely summarized.

[1994] "Database" refers to a system for efficiently storing information and making it searchable and accessible.

[1995] "Real-time" refers to processing or response occurring immediately, without delay.

[1996] A "generative AI model" refers to an artificial intelligence model that uses machine learning to learn patterns from large amounts of data, and then generates and analyzes new data.

[1997] A "prompt sentence" refers to an instruction sentence or question sentence that a user inputs to a system.

[1998] "Recommendation" refers to the act of suggesting optimal products or services based on a user's preferences and behavioral history.

[1999] The present invention relates to a sales support system, and in particular to a system that summarizes conversations with customers, provides advice on improving sales talks, gives advice on the next sales action, and recommends appropriate products through an application that runs on a device such as a smartphone.

[2000] The details of the system embodiment will be described below.

[2001] 1. Automatic summarization of customer conversations

[2002] When a user starts a conversation with a customer, the device (e.g., a smartphone) records the conversation. When the conversation ends, the device sends the recorded data to a server. The server converts the voice data into text data using a speech recognition engine (e.g., Google Speech Recognition). The server then analyzes the converted text data using a natural language processing algorithm (e.g., the "T5" model from the Transformers library), extracts important points, and generates a summary. The generated summary is distributed to the device via an intranet or the Internet, and is notified to the user. As a concrete example, a sales representative may discuss the features of a new product with a customer, and a summary may be generated based on the recording.

[2003] 2. Sales pitch training

[2004] The server maintains a database of past successful sales pitches. When a user gives a new sales pitch, the device records the content and sends it to the server. The server converts the received data into text data using voice recognition, compares it with past successful examples, and analyzes areas for improvement. Based on the analysis results, the server generates advice and delivers it to the user's device. As a concrete example, the server analyzes a presentation a sales representative gave to a customer and provides advice such as "it would be good to include more specific examples."

[2005] 3. Advice on next steps

[2006] The user inputs the customer profile and current sales situation into the device. The device sends this data to the server, which uses a generative AI model to analyze the customer profile and sales situation data and evaluate potential next sales actions. The server selects the optimal next action, generates advice, and delivers it to the device. The user can use this advice to effectively decide on the next step. As a specific example, the advice provided to the user is, "It would be effective to demonstrate a new product at the next interview."

[2007] 4. Recommendation of proposed products

[2008] The user enters their customer profile and sales history into the device, and the device sends the data to the server. The server uses a generative AI model to analyze the purchasing patterns of similar customers based on the customer profile and extract related products from a database. The server analyzes the information through prompt text, generates recommended products, and delivers the recommended information to the device. Based on this information, the user proposes appropriate products to the customer. As a specific example, the server recommends related products such as "new printers" and "high-performance computers."

[2009] Prompt Sentence Examples

[2010] "After a five-minute conversation with a customer about new product features, you have the following transcript. Please summarize it:

[2011] "Hello, customer. Today I'd like to introduce you to our new X900 series printer. This printer is extremely high-performance... (omitted)"

[2012] This system enables users to efficiently manage conversations with customers and conduct effective sales activities.

[2013] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[2014] Step 1:

[2015] When a user starts a conversation with a customer, the user uses a terminal (smartphone) to record the conversation.

[2016] Input: Voice of conversation with customer

[2017] Output: Recorded audio file

[2018] What it does: Uses the device's recording feature to save a conversation with a customer as an audio file.

[2019] Step 2:

[2020] When the conversation ends, the terminal transmits the recorded voice data to the server.

[2021] Input: Recorded audio file

[2022] Output: Audio data uploaded to the server

[2023] How it works: Uploads recorded data from the device to a cloud server or database.

[2024] Step 3:

[2025] The server uses a speech recognition engine to convert the voice data into text data.

[2026] Input: Audio file

[2027] Output: Converted text data

[2028] What it does: Calls a speech recognition engine (e.g., Google Speech Recognition API) to convert an audio file into text.

[2029] Step 4:

[2030] The server analyzes the text data using natural language processing algorithms, extracts key points, and generates a summary.

[2031] Input: Text data

[2032] Output: Summary text

[2033] How it works: It uses natural language processing algorithms (e.g., Hugging Face Transformers' "T5" model) to extract key points and generate a summary.

[2034] Step 5:

[2035] The server delivers the generated summary to the terminal.

[2036] Input: Summary text

[2037] Output: Summary text delivered to the user's device

[2038] Action: Sends summary text to the terminal and notifies the user.

[2039] Step 6:

[2040] When a user makes a new sales pitch, the device records the content and sends it to the server.

[2041] Input: Sales pitch audio file

[2042] Output: Audio data uploaded to the server

[2043] Operation: Uploads the recorded audio data of the sales talk to the server.

[2044] Step 7:

[2045] The server converts the received voice data into text data using voice recognition.

[2046] Input: Audio file

[2047] Output: Converted text data

[2048] How it works: Uses a speech recognition engine to convert voice data into text data.

[2049] Step 8:

[2050] The server compares the text data with past success stories, analyzes areas for improvement, and generates advice.

[2051] Input: Text data, past success data

[2052] Output: Advice text

[2053] How it works: Text is vectorized using TfidfVectorizer, compared to past successes using cosine similarity, and a generative AI model is used to identify areas for improvement and generate advice.

[2054] Step 9:

[2055] The server distributes the generated advice to the user's terminal.

[2056] Input: Advice text

[2057] Output: Advice text delivered to the user's device

[2058] Action: Sends advice text to the user's device and notifies them.

[2059] Step 10:

[2060] The user enters a customer profile and current business status into the terminal.

[2061] Input: Customer profile, sales status data

[2062] Output: Profile and status data sent from the device to the server

[2063] How it works: A user enters their profile and status into a terminal and sends it to a database.

[2064] Step 11:

[2065] The server uses the generative AI model to analyze customer profiles and sales status data and select potential next sales actions.

[2066] Input: Customer profile, sales status data

[2067] Output: Text of advice on optimal sales actions

[2068] How it works: Analyzes data using a generative AI model to evaluate and select the next sales action.

[2069] Step 12:

[2070] The server delivers the selected action to the user as a recommendation.

[2071] Input: Sales action advice text

[2072] Output: Advice text delivered to the user's device

[2073] Operation: The advice is sent to the terminal and notified.

[2074] Step 13:

[2075] The user enters a customer profile and sales history into the terminal.

[2076] Input: Customer profile, sales history data

[2077] Output: Profile and sales history data sent from the device to the server

[2078] How it works: A user enters their profile and history into a terminal and sends it to a database.

[2079] Step 14:

[2080] The server uses a generative AI model to analyze the purchasing patterns of similar customers based on the customer profile and recommend related products.

[2081] Input: Customer profile, sales history data

[2082] Output: Recommended product list

[2083] How it works: Analyzes data using a generative AI model to recommend relevant products.

[2084] Step 15:

[2085] The server delivers a list of recommended products to the user's terminal.

[2086] Input: Recommended product list

[2087] Output: Product list delivered to the user's device

[2088] Operation: The recommended product list is sent to the device and notified.

[2089] For example, if you set the prompt text as follows:

[2090] "After a five-minute conversation with a customer about new product features, you have the following transcript. Please summarize it:

[2091] "Hello, customer. Today I'd like to introduce you to our new X900 series printer. This printer is extremely high-performance... (omitted)"

[2092] By using prompts in this way, users can efficiently issue instructions to the system and receive optimal summaries, advice, and recommendations.

[2093] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[2094] This embodiment of the present invention combines an emotion engine that recognizes the user's emotions with an integrated sales support system that automatically summarizes conversations with customers, provides advice on improving sales talks, advises on effective next sales actions, and recommends appropriate products to propose. The program and processing of this system are described below with specific examples.

[2095] 1. Automatically summarize customer conversations

[2096] When a user starts a conversation with a customer, the device activates the recording function and records the audio of the conversation. When the conversation ends, the device sends the recorded data to the server. The server converts the received audio data into text data using a speech recognition engine. The server then runs the text data through a natural language processing algorithm to extract key points and generate a summary. Furthermore, an emotion engine analyzes the user's emotions from the converted text and audio data and adjusts the summary content accordingly. Finally, the server sends the generated summary to the user's device, where the user can review it. As a concrete example, a salesperson discusses a new product with a customer, and a summary is generated based on the recording, reflecting the content in line with the emotions.

[2097] 2. Sales pitch training

[2098] The server maintains a database of past successful sales pitches. When a user gives a new sales pitch, the device records the content and sends it to the server. The server converts the received data into text data using a voice recognition engine and runs an analysis algorithm that compares the converted text data with past successful examples. The server generates advice on how to improve the sales pitch based on the user's emotional data. The advice generated by the server is sent to the user's device, where the user can confirm it. It is also possible to provide flexible advice based on the user's emotions. As a specific example, a salesperson's presentation to a customer is analyzed and emotionally sensitive advice such as "it would be more effective if you gave more specific examples" is provided.

[2099] 3. Advice on next steps

[2100] The user enters a customer profile and current sales status into the terminal. The terminal sends this data to the server. The server analyzes the customer profile and sales status data and runs it through an algorithm that evaluates potential next sales actions. The server selects the optimal next action based on the evaluation results and uses an emotion engine to generate advice that takes the user's emotions into consideration. The server sends the generated advice to the user's terminal, where the user can confirm it. As a specific example, advice such as "It would be effective to demonstrate a new product at the next interview" is provided in a form that is appropriate to the user's emotions.

[2101] 4. Recommendation of proposed products

[2102] The user enters their customer profile and sales history into the terminal, which then sends the data to the server. The server uses this data to analyze the purchasing patterns of similar customers and extracts related products from a database. The server uses an emotion engine to generate recommendation information that takes the emotional data into account. The server then sends the recommendation information to the user's terminal, where the user confirms it. As a specific example, the server provides related products such as "new printers" and "high-performance computers" as recommendation information tailored to the user's emotions.

[2103] By linking the above functions, this system comprehensively supports sales activities, improving efficiency and effectiveness. In addition, the combination of an emotion engine enables flexible responses that take user emotions into consideration, allowing for more personalized support.

[2104] The processing flow will be explained below.

[2105] Automatically summarize customer conversations

[2106] Step 1:

[2107] The user starts a conversation with the customer. The device activates the recording function and starts recording the audio of the conversation.

[2108] Step 2:

[2109] After the conversation ends, the device sends the recorded data to the server.

[2110] Step 3:

[2111] The server sends the received voice data to a voice recognition engine, which converts the voice data into text data.

[2112] Step 4:

[2113] The server sends the converted text data to a natural language processing algorithm, which analyzes the text data.

[2114] Step 5:

[2115] The server extracts important keywords and phrases and generates a summary.

[2116] Step 6:

[2117] The server sends the voice and text data to the emotion engine, which analyzes the user's emotions.

[2118] Step 7:

[2119] The server adjusts the summary content based on the analysis results of the emotion engine.

[2120] Step 8:

[2121] The server sends the generated summary to the user's terminal.

[2122] Step 9:

[2123] The terminal displays the summary and the user confirms it.

[2124] Sales talk skills training

[2125] Step 1:

[2126] A server maintains a database of past successful sales pitches.

[2127] Step 2:

[2128] The user makes a new sales pitch, and the device records the content.

[2129] Step 3:

[2130] After the sales talk ends, the device sends the recorded data to the server.

[2131] Step 4:

[2132] The server sends the received voice data to a voice recognition engine, which converts it into text data.

[2133] Step 5:

[2134] The server sends the converted text data to an analysis algorithm that compares it with past successes.

[2135] Step 6:

[2136] The server generates advice on how to improve the sales pitch based on the analysis results.

[2137] Step 7:

[2138] The server sends text data to the emotion engine, which analyzes the user's emotions.

[2139] Step 8:

[2140] The server adjusts the advice content based on the analysis results of the emotion engine.

[2141] Step 9:

[2142] The server transmits the generated advice to the user's terminal.

[2143] Step 10:

[2144] The terminal notifies the user of the advice, which is then confirmed by the user.

[2145] Advice for next action

[2146] Step 1:

[2147] The user enters a customer profile and current business status into the terminal.

[2148] Step 2:

[2149] The terminal transmits the input data to the server.

[2150] Step 3:

[2151] The server sends customer profile and sales data to an analytics algorithm.

[2152] Step 4:

[2153] The server generates the next sales action candidates based on the analysis results.

[2154] Step 5:

[2155] The server sends the user's emotional data to the emotion engine, which analyzes the emotions.

[2156] Step 6:

[2157] Based on the analysis results of the emotion engine, the server ranks and adjusts potential actions.

[2158] Step 7:

[2159] The server selects the best next action and generates advice.

[2160] Step 8:

[2161] The server sends the generated advice to the user's terminal.

[2162] Step 9:

[2163] The terminal notifies the user of the advice and the user confirms it.

[2164] Recommendation of proposed products

[2165] Step 1:

[2166] The user enters their customer profile and sales history into the terminal.

[2167] Step 2:

[2168] The terminal transmits the input data to the server.

[2169] Step 3:

[2170] The server sends customer profiles and sales histories to an analytics algorithm.

[2171] Step 4:

[2172] The server analyzes the purchasing patterns of similar customers and extracts related products from the database.

[2173] Step 5:

[2174] The server sends the user's emotional data to the emotion engine, which analyzes the emotions.

[2175] Step 6:

[2176] The server adjusts the recommendations based on the analysis results of the emotion engine.

[2177] Step 7:

[2178] The server transmits the generated recommendation information to the user's terminal.

[2179] Step 8:

[2180] The device notifies the user of the recommended information, which is then confirmed by the user.

[2181] As a result, the system of the present invention, which incorporates an emotion engine, can comprehensively support sales activities and respond flexibly while taking into account the user's emotions. As a specific example, the emotion engine can analyze the emotional state of a salesperson, and if the salesperson is feeling stressed, it can provide personalized support, such as advice on how to speak calmly.

[2182] Example 2

[2183] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2184] Conventional sales support systems have the problem of being unable to automatically and efficiently summarize conversations with customers, give advice on improving sales talks, or advise on next actions.In addition, these functions are not adjusted according to the user's emotions, and only standardized suggestions and advice are provided, which has led to issues that do not provide sufficient satisfaction to users.

[2185] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for recording a conversation with a customer, a means for converting the recorded voice data into text data using a voice recognition engine, a means for analyzing the converted text data using a natural language processing algorithm and extracting important points, a means for generating a summary from the extracted points and adjusting the summary content according to the user's emotions using an emotion engine, and a means for delivering the adjusted summary to the user. This makes it possible to provide summaries and advice that take the user's emotions into consideration, which is expected to improve the efficiency and effectiveness of sales activities.

[2186] The "means for recording conversations with customers" refers to a device or software for recording conversations as audio data when a user starts a conversation with a customer.

[2187] A "speech recognition engine" is software or algorithms that convert recorded voice data into text data.

[2188] A "natural language processing algorithm" is an algorithm for analyzing text data and extracting important information and patterns.

[2189] An "emotion engine" is software or an algorithm that analyzes emotions from text and voice data and makes adjustments based on those emotions.

[2190] A "summary generation means" is a device or software that extracts important points and creates a short summary of the text.

[2191] The "distribution means" refers to a device or software for transmitting the generated information or advice to the user's terminal and displaying it.

[2192] A "database" is an information accumulation system that stores data such as past successful sales pitches, customer profiles, and sales history, and can be searched and used as needed.

[2193] An "analysis algorithm" is an algorithm that analyzes collected data and finds patterns and trends.

[2194] "Potential sales actions" are suggestions for specific actions that can be taken in the next sales activity.

[2195] The "means for delivering advice" is a device or software that notifies the user of the optimal action to take in the next sales activity and provides it as a guideline for action.

[2196] The present invention is an integrated sales support system that automatically summarizes conversations with customers, provides advice on improving sales talks, advises on effective next sales actions, and recommends appropriate products. By combining this system with an emotion engine that recognizes the user's emotions, the present invention can provide more personalized support. The specific configuration and operation of this system are described below.

[2197] 1. Automatically summarize customer conversations

[2198] When a user starts a conversation with a customer, the device activates the recording function and records the audio of the conversation. A recording application (e.g., Recorder App) is installed on the device. When the conversation ends, the device sends the recorded data to the server. The recorded data is sent to the server using an HTTP request (e.g., in MP3 format). The server converts the received audio data into text data using a speech recognition engine (e.g., Google Cloud Speech-to-Text). The server then uses a natural language processing algorithm (e.g., BERT or GPT-3) to analyze the text data, extract key points, and generate a summary. After that, an emotion engine (e.g., Microsoft Azure Emotion API) analyzes the user's emotions from the converted text and audio data and adjusts the summary content. Finally, the server sends the generated summary to the user's device, where the user can review it.

[2199] For example, if a salesperson is discussing a new product with a customer, a summary is generated based on the recording, and the content reflects the customer's emotions.

[2200] Example prompt sentence:

[2201] "Summarize key points from recordings of customer conversations, analyze sentiment, and adjust accordingly."

[2202] 2. Sales pitch training

[2203] The server maintains a database of past successful sales pitches. When a user gives a new sales pitch, the device records the content and sends it to the server. The server converts the received data into text data using a voice recognition engine and runs an analysis algorithm (e.g., a similarity calculation algorithm) that compares the converted text data with past successful examples. The server generates advice on how to improve the sales pitch based on the user's emotional data. The generated advice is adjusted according to the user's emotions using an emotion engine. The server sends the adjusted advice to the user's device, where the user confirms it.

[2204] As a concrete example, the system analyzes a presentation a salesperson gives to a customer and provides emotionally sensitive advice such as "It would be more effective if you provided more specific examples."

[2205] Example prompt sentence:

[2206] "Please compare my current sales pitch with past successful ones and give me advice on how to improve it."

[2207] 3. Advice on next steps

[2208] The user inputs their customer profile and current sales situation into their device. The device then sends this data to the server. The server then runs this data through an analytical algorithm (e.g., a machine learning model) to evaluate potential next sales actions. The server then selects the optimal next action based on the evaluation results and uses an emotion engine to generate advice that takes the user's emotions into account. The server then sends the generated advice to the user's device, where the user confirms it.

[2209] As a specific example, advice such as "It would be effective to demonstrate a new product at the next interview" is provided in accordance with the user's emotions.

[2210] Example prompt sentence:

[2211] "Please advise me on my next action based on my customer profile and sales situation."

[2212] 4. Recommendation of proposed products

[2213] The user enters their customer profile and sales history into the terminal, which then sends the data to the server. The server uses this data to analyze the purchasing patterns of similar customers and extracts related products from the database. The server then uses an emotion engine to generate recommendation information that takes into account the emotion data. The generated recommendation information is then sent by the server to the user's terminal, where the user can confirm it.

[2214] As a specific example, the server provides related products such as "new printers" and "high-performance computers" as recommended information tailored to the user's emotions.

[2215] Example prompt sentence:

[2216] "Recommend relevant products based on customer profile and sales history."

[2217] By linking the above functions, this system comprehensively supports sales activities, improving efficiency and effectiveness. In addition, the combination of an emotion engine enables flexible responses that take into account the user's emotions, allowing for more personalized support.

[2218] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2219] Step 1:

[2220] When a user starts a conversation with a customer, the device automatically activates the recording function and collects the audio data of the conversation. Specifically, the recording application installed on the device starts running in the background. The recording application receives audio data as input and outputs a recorded audio file (e.g., MP3 format).

[2221] Step 2:

[2222] Once the conversation is finished, the device sends the recording to the server using an HTTP POST request to the server's API endpoint. The device receives the recorded audio file as input and outputs an HTTP request.

[2223] Step 3:

[2224] The server converts the received voice data into text data using a voice recognition engine. Specifically, the server calls the API of the voice recognition engine (for example, Google Cloud Speech-to-Text), provides the voice data as input, and receives the converted text data as output.

[2225] Step 4:

[2226] The server applies a natural language processing algorithm to the acquired text data to extract key points. Specifically, the server inputs the text data into a natural language processing algorithm (e.g., BERT or GPT-3) and outputs key points as the analysis results.

[2227] Step 5:

[2228] The server uses an emotion engine to analyze emotions from text data and voice data. Specifically, the server calls the API of the emotion engine (for example, Microsoft Azure Emotion API), provides text data and voice data as input, and receives the emotion analysis results as output.

[2229] Step 6:

[2230] The server adjusts the summary content based on the sentiment analysis results and generates the final summary. Specifically, the server receives the sentiment analysis results and key points as input and outputs the adjusted summary.

[2231] Step 7:

[2232] The server sends the generated summary to the user's device. Specifically, the server uses an HTTP response to send the summary data to the user's device. The user's device receives the summary data as input and outputs a custom UI to display to the user.

[2233] This allows the content of the conversation with the customer to be summarized and presented in a form that matches the user's emotions.

[2234] (Application example 2)

[2235] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2236] Conventional sales support systems are limited in their functionality, such as efficiently summarizing conversations with customers, providing advice on improving sales talks, advising on next actions, and recommending proposed products. Furthermore, few systems can take user emotions into account when it comes to these functions, and more personalized support is desired. Therefore, there is a need for a comprehensive sales support system that can summarize conversations with customers, provide guidance on sales talks, advise on next actions, recommend proposed products, and also consider user emotions.

[2237] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[2238] In this invention, the server includes means for recording conversations with customers, means for converting the recorded voice data into text data using voice recognition, natural language processing means for analyzing the converted text data and extracting important points, means for generating a summary of the extracted points and distributing it to the user, and means for recognizing the user's emotions and reflecting them in the summary content, thereby enabling efficient sales support that takes the user's emotions into consideration.

[2239] The server also includes means for maintaining a database of past successful sales talks, means for collecting new sales talks from users and converting the recorded data into text data, means for comparing the converted text data with past successful examples and generating advice on areas for improvement, means for delivering the generated advice to users, and means for generating and delivering flexible advice taking into account the user's emotions, thereby enabling the quality of sales talks to be improved while taking into account emotions.

[2240] Furthermore, the server includes means for collecting and analyzing customer profiles and sales status data, means for evaluating next sales action candidates and selecting the optimal action, means for delivering the selected action to the user as advice, and means for generating and delivering advice content that takes the user's emotions into consideration, thereby making the next sales activity more effective and personalized.

[2241] "Customer" refers to the person to whom goods or services are provided in a business transaction or transaction.

[2242] "Conversation recording means" refers to a device or function for recording a conversation with a customer as audio data.

[2243] "Speech recognition means" refers to technology or software that analyzes recorded voice data and converts it into text data.

[2244] "Character data" refers to text information converted by a speech recognition means.

[2245] "Natural language processing means" refers to algorithms and technologies for analyzing text data and extracting important points.

[2246] "Summary generation means" refers to a function that concisely summarizes a text based on extracted important points.

[2247] "Emotion recognition means" refers to the technology and algorithms that determine and analyze a user's emotions from text data and voice data.

[2248] A "sales pitch database" refers to a system or device that stores records and data of past successful sales pitches.

[2249] "Means for generating improvement advice" refers to a function that compares new sales pitches with existing successful examples and suggests more effective methods.

[2250] "Customer profile" refers to detailed data such as basic customer information and past transaction history.

[2251] "Sales status data" refers to info...

Claims

1. A means of recording conversations with customers; A means for converting the recorded voice data into character data by voice recognition; natural language processing means for analyzing the converted character data and extracting important points; means for generating a summary of the extracted points and delivering it to a user; A system including:

2. A means to maintain a database of past successful sales pitches, A means for collecting new sales talks from users and converting the recorded data into text data; A means for comparing the converted character data with past successes and generating advice on areas for improvement; means for delivering the generated advice to a user; The system of claim 1 , comprising:

3. A means of collecting and analyzing customer profile and sales data; A means to evaluate potential next sales actions and select the most appropriate action, means for delivering the selected action to the user as a recommendation; The system of claim 1 , comprising:

4. A means of collecting customer profiles and sales histories and analyzing the purchasing patterns of similar customers; A means for extracting related products from a database and generating recommendation information; A means for delivering the generated recommendation information to a user; The system of claim 1 , comprising:

Images