system

Abstract

We provide the system. [Solution] A user device means for collecting customer information, A means for generating solutions based on customer information using generative artificial intelligence, A means for outputting the generated solution as audio, A method for automatically generating replies to inquiry emails, A means for automatically generating network configuration proposals and advertising budget allocations, A method for collecting application information in a chat format and automatically generating application forms, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a character of the chatbot, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] ---

[0005] In conventional business activities, explanations and proposals regarding digital marketing and data products for corporations largely depend on the knowledge and skills of salespersons. Therefore, it is difficult to make an optimal proposal for each customer, and there are also problems that a lot of time and labor are required for clerical work such as responding to customer inquiries and creating application forms. For this reason, there is a need for a system that reduces the burden on salespersons and enables efficient and effective proposals and responses.

Means for Solving the Problems

[0006] To solve the above problems, the present invention provides a system that includes the following means: a user device means for collecting customer information; a means for generating solutions based on the customer information using generative artificial intelligence; a means for outputting the generated solutions as voice; a means for automatically generating replies to inquiry emails; a means for automatically generating network configuration plans and advertising budget allocations; and a means for collecting application information in talk format and automatically generating application forms. With this system, sales representatives can efficiently collect customer information, obtain optimal solution proposals and talk scripts generated by generative artificial intelligence, and significantly reduce the burden of handling inquiries and administrative tasks.

[0007] ---

[0008] ---

[0009] "Customer information" refers to information about customers in sales activities, as well as information about their needs and challenges.

[0010] A "user device" refers to a device used to collect customer information and to display requests and responses to generative artificial intelligence.

[0011] "Generative artificial intelligence" refers to an artificial intelligence system that generates optimal solutions and suggestions based on input data.

[0012] "Solution" refers to the solutions and countermeasures proposed by generative artificial intelligence based on customer information and issues.

[0013] "Voice output" refers to the function that plays back the generated solutions and proposals as audio.

[0014] An "inquiry email" refers to a question or request sent via email from a customer to a sales representative.

[0015] "Automatically generated reply" refers to a function that automatically creates an appropriate response to an inquiry email using generative artificial intelligence.

[0016] "Network configuration plan" refers to a document that proposes an optimal network configuration and design based on the customer's system requirements.

[0017] "Advertising budget allocation" refers to a plan for optimally allocating the customer's budget and formulating an effective advertising strategy.

[0018] "Application information" refers to the information required when a customer applies for a service or product.

[0019] "Dialogue format" refers to a method of collecting information where questions and answers proceed in a dialogue format.

[0020] "Application form" refers to the document required for a customer to formally apply for a service or product.

[0021] ---

[0022] The above are the definition texts of the important words included in the claims of the patent.

Brief Description of Drawings

[0023] [Figure 1] It is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] It is a conceptual diagram showing an example of the main functions of a data processing device and a smart device according to the first embodiment. [Figure 3] It is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] It is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] It is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6]This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] This is a sequence diagram showing the processing flow of the data processing system in Example 2, when an emotion engine is combined. [Figure 14] This is a sequence diagram showing the processing flow of the data processing system in Application Example 2, which combines an emotion engine. [Modes for carrying out the invention]

[0024] Hereinafter, an example of an embodiment of the system relating to the technology of this disclosure will be described with reference to the attached drawings.

[0025] First, let's explain the terminology used in the following explanation.

[0026] In the following embodiments, the signed processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Furthermore, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include CPU (Central Processing Unit), GPU (Graphics Processing Unit), GPGPU (General-Purpose computing on Graphics Processing Units), and APU (Accelerated Processing Unit).

[0027] In the following embodiments, signed RAM (Random Access Memory) is a memory that temporarily stores information and is used as work memory by the processor.

[0028] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0029] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0030] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0031] [First Embodiment]

[0032] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0033] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0034] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0035] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0036] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0037] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0038] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0039] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0040] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0041] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0042] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0043] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0044] ---

[0045] This invention is a system designed to streamline digital marketing and data product sales activities for corporate clients. The system comprehensively supports everything from customer information collection and proposal creation to inquiry handling and application processing.

[0046] This system primarily consists of the following elements: a user device, a generative artificial intelligence system, a server, and a voice output function. The specific functions and operation of each element are described below.

[0047] 1. Collection of customer information

[0048] The user (Sales representative) uses a user device such as an iPad® during the initial customer visit. The user device sends a request to the server to play a pre-prepared introductory video. This video contains basic information about the services and products being offered. After the video plays, a series of questions are displayed on the user device, and the user answers these questions while interacting with the customer. Each answer is sent to the server in real time, and the server stores this answer data in a database.

[0049] Specific example:

[0050] Question displayed by the user device: "What are your company's main business activities?"

[0051] When a user enters "e-commerce," the user's device sends this information to the server, which then updates the customer database as "e-commerce."

[0052] 2. Generating solutions and talk scripts

[0053] The server uses generative artificial intelligence to generate optimal solutions based on collected customer information and problem data. These solutions include specific suggestions and the reasoning behind them. The generated solutions are also created as talk scripts and used for voice output.

[0054] Specific example:

[0055] Server-generated solution example: "Implementation of an automated inventory management system"

[0056] Example of a server-generated talk script: "For companies like yours that operate e-commerce, implementing an automated inventory management system is highly recommended."

[0057] 3. Audio output of the talk script

[0058] The generated talk script is sent to the user's device and converted from text to audio data (TTS: Text-to-Speech). The converted audio data is played back on the user's device, allowing the user to explain the proposal to the customer verbally.

[0059] 4. Automatic reply to inquiry email

[0060] Later, when a customer sends an inquiry email to the server, the server analyzes its content and automatically generates an appropriate response using generative artificial intelligence. The generated reply email can then be sent to the customer after the user has reviewed it.

[0061] Specific example:

[0062] Customer inquiry: "How should we allocate our advertising budget?"

[0063] Server-generated response: "We recommend allocating 50% of your advertising budget to social media ads, 30% to search engine ads, and 20% to display ads."

[0064] 5. Automatic generation of network configuration proposals and advertising budget allocations.

[0065] The server automatically generates network configuration proposals and advertising budget allocation suggestions based on customer information. This allows for the rapid delivery of optimal proposals tailored to individual customer requirements.

[0066] 6. Input of application information in a chat format and automatic generation of application forms.

[0067] The user device prompts the user to input application information in a chat format. Once information is entered for each field, the server automatically generates an application form based on this information and sends the completed application form to the user device or the user's email address.

[0068] Specific example:

[0069] User device question: "Please tell me the applicant's name."

[0070] When a user enters "Taro Yamada," the server generates an application form that includes "Taro Yamada." Other necessary information is automatically added to the application form.

[0071] The above describes the details of the embodiments for carrying out the present invention. This system automates many processes in sales activities, enabling effective and efficient responses.

[0072] The following describes the processing flow.

[0073] ---

[0074] Step 1:

[0075] The user (Sales representative) activates a user device such as an iPad and opens the dedicated app. When the user instructs the system to play the initial explanation video, the device sends a request to the server.

[0076] Step 2:

[0077] The server receives the request and searches for the initial instructional video data. It prepares the corresponding video file for streaming and sends it to the terminal.

[0078] Step 3:

[0079] The device plays the received video data, and the user provides an initial explanation to the customer. After the video plays, a question-selection input screen is displayed on the device.

[0080] Step 4:

[0081] The user enters their answer to a question displayed on their device. The entered answer is sent to the server in real time.

[0082] Step 5:

[0083] The server saves the received response data to a database. After saving, it automatically generates the next question and sends it to the terminal.

[0084] Step 6:

[0085] This process is repeated to collect detailed customer information and issue data, and the server stores this information in a database.

[0086] Step 7:

[0087] Based on the collected customer information and issue data, the server invokes generative artificial intelligence to generate optimal solutions and talk scripts.

[0088] Step 8:

[0089] The server sends the generated talk script to the terminal. The terminal calls a TTS engine to convert the received talk script from text to speech.

[0090] Step 9:

[0091] The TTS engine converts the talk script into audio data and returns it to the terminal. The terminal plays the generated audio data, and the user explains the proposal to the customer.

[0092] Step 10:

[0093] Later, customer inquiry emails are sent to the server. The server analyzes the content of the inquiry email and automatically generates an appropriate response using generative artificial intelligence.

[0094] Step 11:

[0095] The server provides the generated reply email to the user for confirmation. After the user confirms, the reply email is sent to the customer.

[0096] Step 12:

[0097] The server automatically generates optimal proposal documents for network configuration and advertising budget allocation based on customer information. The generated documents are sent to the user's device or email address.

[0098] Step 13:

[0099] The user device prompts the user to enter application information in a chat format. Once the user enters an answer to each question, the input data is sent to the server.

[0100] Step 14:

[0101] The server automatically generates an application form using a standard format based on the application information it receives. The generated application form is then sent to the terminal or the user's email address.

[0102] Step 15:

[0103] The terminal notifies the user of the generated application form data, which the user then reviews and downloads.

[0104] ---

[0105] The above is an explanation of the program's processing broken down into specific steps.

[0106] (Example 1)

[0107] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0108] In B2B digital marketing and data product sales activities, the processes of collecting customer information, proposing solutions, responding to inquiries, and processing applications are complex and time-consuming, posing a challenge. This can lead to decreased sales efficiency and a decline in the quality of customer service.

[0109] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0110] In this invention, the server includes electronic means for collecting customer information, means for generating solutions based on the customer information using generative artificial intelligence, means for outputting the generated solutions as voice, means for automatically generating replies to inquiry messages, means for automatically generating communication network configuration proposals and budget allocation proposals, and means for collecting application information in a dialogue format and automatically generating application forms. This automates each process necessary for sales activities, such as information gathering, proposals, responses, and application processing, enabling efficient and effective customer service.

[0111] "Electronic devices" refer to electronic devices used for collecting, displaying, and inputting customer information. Specifically, this includes tablets and smartphones.

[0112] "Generative artificial intelligence" refers to artificial intelligence technology that generates solutions and reply messages based on collected customer information. A typical example is a model that performs natural language processing.

[0113] A "solution" refers to specific improvement measures or countermeasures proposed by generative artificial intelligence in response to customer problems and needs.

[0114] "Voice output means" refers to technologies for conveying solutions and other information to users as audio. Specifically, text-to-speech (TTS) functions fall under this category.

[0115] An "inquiry message" refers to an email or message sent by a customer to convey questions or requests.

[0116] A "communication network configuration proposal" refers to a proposal that suggests the optimal communication network configuration tailored to the customer's business needs.

[0117] A "budget allocation proposal" refers to a plan that suggests the most optimal way to allocate a limited budget in advertising activities, etc.

[0118] "Dialogue format" refers to a format in which the user and the system exchange information through a question-and-answer exchange.

[0119] An "application form" refers to a document containing the information necessary for a customer to apply for a service or product.

[0120] This invention is a system designed to streamline digital marketing and data product sales activities for corporate clients. This system comprehensively supports everything from customer information collection and proposal creation to inquiry handling and application processing.

[0121] This system consists of the following elements:

[0122] Electronic devices for collecting customer information

[0123] Generative artificial intelligence

[0124] server

[0125] Audio output function

[0126] Collection of customer information

[0127] The user uses an electronic device such as an iPad during their initial customer visit. The user device sends a request to the server to play a pre-prepared introductory video. This video contains basic information about the services and products being offered. After the video plays, a series of questions are displayed on the user device, and the user answers these questions while interacting with the customer. Each answer is sent to the server in real time, and the server stores this answer data in a database.

[0128] Specific example

[0129] Question displayed by the user device: "What are your company's main business activities?"

[0130] When a user enters "e-commerce," the user's device sends this information to the server, which then updates the customer database as "e-commerce."

[0131] Solution and Talk Script Generation

[0132] The server uses generative artificial intelligence (e.g., OpenAI's GPT-4) to generate optimal solutions based on collected customer information and problem data. These solutions include specific suggestions and the reasons behind them. The generated solutions are also created as talk scripts and used for voice output.

[0133] Specific example

[0134] Server-generated solution example: "Implementation of an automated inventory management system"

[0135] Example prompt: "Please propose the best IT solution for a new customer. The customer is an e-commerce business."

[0136] Audio output of talk script

[0137] The generated talk script is sent to the user's device and converted from text to audio data (TTS: Text-to-Speech). The converted audio data is played back on the user's device, allowing the user to explain the proposal to the customer verbally.

[0138] Specific example

[0139] Example of a server-generated talk script: "For companies like yours that operate e-commerce, implementing an automated inventory management system is highly recommended."

[0140] Automatic reply to inquiry email

[0141] When a customer sends an inquiry email to the server, the server analyzes its content and automatically generates an appropriate response using generative artificial intelligence. The generated reply email can then be sent to the customer after the user has reviewed it.

[0142] Specific example

[0143] Customer inquiry: "How should we allocate our advertising budget?"

[0144] Server-generated response: "We recommend allocating 50% of your advertising budget to social media ads, 30% to search engine ads, and 20% to display ads."

[0145] Automatic generation of network configuration proposals and advertising budget allocations.

[0146] The server automatically generates network configuration proposals and advertising budget allocation suggestions based on customer information. This allows for the rapid delivery of optimal proposals tailored to individual customer requirements.

[0147] Input of application information in a chat format and automatic generation of application forms.

[0148] The user device prompts the user to input application information in a chat format. Once information is entered for each field, the server automatically generates an application form based on this information and sends the completed application form to the user device or the user's email address.

[0149] Specific example

[0150] User device question: "Please tell me the applicant's name."

[0151] When a user enters "Taro Yamada," the server generates an application form that includes "Taro Yamada." Other necessary information is automatically added to the application form.

[0152] The above describes the details of embodiments for carrying out the present invention. This system automates many processes in sales activities, enabling effective and efficient responses.

[0153] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0154] Step 1: Collecting customer information

[0155] Input: Questions displayed on the user device during the first visit, and customer responses.

[0156] Process: The user uses an electronic device such as an iPad to play an initial explanatory video. After the video plays, a series of questions are displayed on the user's device, and the user answers these questions while interacting with the customer. The input data is sent to the server in real time.

[0157] Output: Customer information stored in the database on the server

[0158] Specific operation: An initial explanatory video is played, followed by the question, "What are your company's main business activities?" If the customer answers "E-commerce," it is recorded as "E-commerce" in the database.

[0159] Step 2: Data analysis and solution generation

[0160] Input: Customer information and issue data collected by the server

[0161] Processing: The server analyzes the collected customer information and generates the optimal solution using generative artificial intelligence (e.g., OpenAI's GPT-4). The generated solution is also created as a talk script.

[0162] Output: Generated solutions and talk scripts

[0163] Specific operation: The server analyzes data related to "e-commerce" and generates the implementation of an automated inventory management system as a solution. Generated talk script: "For companies like yours that operate e-commerce, implementing an automated inventory management system is effective."

[0164] Step 3: Outputting the solution via audio

[0165] Input: Generated talk script

[0166] Processing: The user device converts the talk script received from the server from text to audio data (TTS: Text-to-Speech). It then plays the converted audio data.

[0167] Output: Playback of audio data

[0168] Specific operation: The user device converts the generated talk script into audio data and plays the audio saying, "For companies like yours that operate e-commerce, implementing an automated inventory management system is effective."

[0169] Step 4: Automatic reply to inquiry email

[0170] Input: Customer inquiry email

[0171] Processing: The server receives the inquiry email and analyzes its contents. Using generative artificial intelligence, it automatically generates an appropriate reply. After the user confirms the generated reply email, it is sent to the customer.

[0172] Output: Generated reply email

[0173] Specific operation: A customer sends an email to the server asking, "How should I allocate my advertising budget?" The server automatically generates a reply saying, "We recommend allocating 50% of your advertising budget to social media ads, 30% to search engine ads, and 20% to display ads." The user then reviews and sends the reply.

[0174] Step 5: Automatic generation of network configuration and advertising budget allocation.

[0175] Input: Customer information stored on the server

[0176] Processing: The server analyzes customer information and automatically generates optimal communication network configurations and budget allocation plans for specific requirements.

[0177] Output: Generated network configuration and advertising budget allocation proposal

[0178] Specific operation: The server plans a network configuration based on the customer's business needs and proposes an effective allocation of the advertising budget.

[0179] Step 6: Interactive input of application information and automatic generation of application form.

[0180] Input: Application information entered on the user device

[0181] Processing: The user enters application information interactively through their device. The entered information is sent to the server, which automatically generates an application form based on it. The generated application form is sent to the user's device or to the user's email address.

[0182] Output: Generated application form

[0183] Specific operation: The user device inputs the question "Please tell me the applicant's name," and the user answers "Taro Yamada." The server generates an application form including "Taro Yamada" and sends the final application form to the user device.

[0184] The above describes the specific operations and inputs / outputs at each processing step. This system efficiently manages the processes of information gathering, proposals, responses, and application processing for sales activities.

[0185] (Application Example 1)

[0186] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0187] Traditional digital marketing and data product sales activities have faced problems such as the cumbersome process of collecting customer information, hindering efficient proposal creation, inquiry handling, and application processing. Furthermore, real-time customer support is difficult, resulting in numerous inefficient processes in users' sales activities. In addition, systems designed to automate these processes have difficulty meeting individual requirements and have failed to alleviate the burden on sales representatives.

[0188] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0189] In this invention, the server includes a user device means for collecting customer information, means for generating solutions based on the customer information using generative artificial intelligence, means for outputting the generated solutions as voice, means for automatically generating replies to inquiry emails, means for generating an optimal talk script and outputting it as voice, means for collecting information in real time through voice dialogue with customers, and means for collecting application information in talk format and automatically generating application forms. This makes it possible to efficiently and effectively automate a series of sales processes from collecting customer information to proposing solutions, responding to inquiries, and processing applications. Furthermore, by making real-time customer support easier, it is possible to reduce the burden on sales representatives and contribute to improving customer satisfaction.

[0190] "User device means for collecting customer information" refers to hardware and software used to collect necessary information through interactions and operations with customers.

[0191] "Means for generating solutions based on customer information using generative artificial intelligence" refers to an artificial intelligence model and its operating environment for generating optimal solutions based on collected customer information.

[0192] "Means for outputting generated solutions as audio" refers to a device and program that provides the function of converting solutions generated by a generative artificial intelligence from text to audio and playing it back.

[0193] "Methods for automatically generating replies to inquiry emails" refers to technologies and systems that analyze the content of customer inquiries and automatically generate appropriate reply content.

[0194] "Means for generating optimal talk scripts and outputting them as audio" refers to devices and technologies that create effective talk scripts based on customer information and conversation content, and then convert these scripts into audio to communicate with customers.

[0195] "Means for collecting information in real time through voice interaction with customers" refers to devices and programs that provide functions for analyzing customer interactions in real time using speech recognition technology and collecting necessary information.

[0196] "Methods for collecting application information in a conversational format and automatically generating application forms" refers to technologies and systems for collecting application information from customers in a dialogue format and automatically creating application forms based on that information.

[0197] This invention is a system for streamlining the entire process from customer information collection to proposal creation, inquiry handling, and application processing. This system is particularly applicable to sales support for physical stores utilizing smart glasses. A specific embodiment of this system is described below.

[0198] System Configuration

[0199] This system consists of the following main elements:

[0200] 1. User device means: This is built as smart glasses and collects necessary information through interaction and operation with the customer.

[0201] 2. Generative Artificial Intelligence: Use OpenAI's generative AI models (e.g., the Davinci Codex engine) to generate solutions based on collected customer information.

[0202] 3. Audio output means: Equipped with Text-to-Speech (TTS) functionality to convert the generated solution from text to speech (e.g., pyttsx3 library).

[0203] 4. Inquiry Email Handling Method: Analyze the content of customer inquiry emails and automatically generate reply content.

[0204] 5. Talk script generation and audio output means: It has a function to create an effective talk script based on customer information and conversation content, and output it as audio.

[0205] 6. Real-time information gathering means: Information is collected through voice interactions with customers using speech recognition technology (e.g., SpeechRecognition library).

[0206] 7. Automatic application form generation method: An application form is automatically created based on application information collected from the customer through dialogue.

[0207] Program Processing Description

[0208] 1. Collection of customer information

[0209] The user wears smart glasses and interacts with customers. During the interaction, the microphone and voice recognition technology built into the smart glasses transcribe the customer's speech into text in real time. For example, if a customer says, "We run an online retail business," the voice recognition technology transcribes this information into text and collects it as customer information.

[0210] 2. Generating solutions and talk scripts

[0211] The server uses a generative AI model to generate the optimal solution based on the collected customer information. The generative AI model used here is OpenAI's Davinci-codex engine. For example, if "online retail" is collected as customer information, the generative AI model will generate a solution such as "Implementing a loyalty program is effective for online retailers."

[0212] 3. Audio output of the generated solution

[0213] The server sends the generated solution to the smart glasses, where it is converted into speech via Text-to-Speech technology. For example, the pyttsx3 library is used to voice the solution and play it back to the user. This allows the user to explain the proposal to the customer verbally through the smart glasses.

[0214] 4. Automatic reply to inquiry email

[0215] The server analyzes customer inquiry emails and automatically generates appropriate replies using an AI model based on their content. For example, in response to an inquiry such as "How should I allocate my advertising budget?", it generates a reply such as "We recommend allocating 50% of your advertising budget to social media ads, 30% to search engine ads, and 20% to display ads."

[0216] 5. Collection of application information and automatic generation of application forms

[0217] The user device (smart glasses) collects application information from the customer in an interactive manner, and the server automatically generates an application form based on that information. For example, if the user asks the customer, "What is the applicant's name?", and the customer replies, "Taro Yamada", that information is sent to the server, and "Taro Yamada" is automatically added to the application form.

[0218] Examples of specific cases and prompt statements

[0219] Customer information example:

[0220] 1. Question: "What are your company's main business activities?"

[0221] 2. Customer response: "We operate an online retail business."

[0222] Generated solution:

[0223] 1. Proposal: "In online retail, implementing a loyalty program is effective. This program is expected to stimulate customer purchasing intent and increase repeat customers."

[0224] Example of a prompt:

[0225] Customer Information: We operate an online retail business.

[0226] Please propose the most suitable solution.

[0227] With the above configuration, the present invention can achieve efficient collection of customer information, automatic generation of optimal solutions, rapid and accurate response to inquiries, and automation of application processing. This makes it possible to significantly improve the productivity of sales activities and customer satisfaction.

[0228] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0229] Step 1:

[0230] The user wears smart glasses and interacts with the customer. Using a microphone and speech recognition technology built into the smart glasses, the customer's speech is converted into text data in real time. The input here is the customer's voice information, and the output is customer information in text format. Specifically, if the customer says, "We run an online retail business," that voice is converted into the text data "We run an online retail business."

[0231] Step 2:

[0232] The terminal sends the collected customer information to the server. The server receives this information and uses generative artificial intelligence (e.g., OpenAI's Davinci-codex engine) to generate the optimal solution based on the customer information. The input here is customer information in text format, and the output is the text of the generated solution. For example, if the input is customer information that says "We operate an online retail business," the server will generate the solution that says "Implementing a loyalty program is effective for online retail businesses."

[0233] Step 3:

[0234] The server sends the generated solution to the smart glasses, which then convert it into audio data using Text-to-Speech (TTS) technology (e.g., the pyttsx3 library) and present it to the user. The input here is the text data of the generated solution, and the output is audio data. Specifically, the text data "Implementing a loyalty program is effective in online retail" is converted into audio, which the user can then hear.

[0235] Step 4:

[0236] If a customer makes a follow-up inquiry, the user collects the information through smart glasses and sends it to the server. The server uses generative artificial intelligence to analyze the inquiry and automatically generate an appropriate response. The input here is the text data of the customer's follow-up inquiry, and the output is the automatically generated response text. For example, in response to the inquiry, "How should I allocate my advertising budget?", the system would generate a response such as, "We recommend allocating 50% of your advertising budget to social media ads, 30% to search engine ads, and 20% to display ads."

[0237] Step 5:

[0238] The user collects customer application information in a chat format via smart glasses and sends this information to the server. The server automatically generates an application form based on the received application information. The input here is the text data of the collected application information, and the output is the automatically generated application form. Specifically, if the user asks the customer, "Please tell me the applicant's name," and the customer replies, "Taro Yamada," the name "Taro Yamada" is automatically added to the application form based on that information.

[0239] The above describes the specific processing flow of the system program of the present invention. In each processing step, data processing and data calculations are performed based on the input data, and output data is generated accordingly. This enables efficient collection of customer information, rapid generation of optimal solutions, real-time customer support, and automation of application processing.

[0240] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0241] ---

[0242] This invention is a system for streamlining digital marketing and data product sales activities for corporate clients, and in particular, it comprehensively supports everything from customer information collection and proposal creation to inquiry handling and application processing. Furthermore, by incorporating an emotion engine that recognizes user emotions, it enables more effective dialogue and customer service.

[0243] This system primarily consists of the following elements: user device, generative artificial intelligence, server, voice output function, and emotion engine. The specific functions and operation of each element are described below.

[0244] 1. Collection of customer information

[0245] The user (Sales representative) uses a user device such as an iPad during the initial customer visit. The user device sends a request to the server to play a pre-prepared introductory video. This video contains basic information about the services and products being offered. After the video plays, a series of questions are displayed on the user device, and the user answers these questions while interacting with the customer. Each answer is sent to the server in real time, and the server stores this answer data in a database.

[0246] Specific example:

[0247] Question displayed by the user device: "What are your company's main business activities?"

[0248] When a user enters "e-commerce," the user's device sends this information to the server, which then updates the customer database as "e-commerce."

[0249] 2. Generating solutions and talk scripts

[0250] The server uses generative artificial intelligence to generate optimal solutions based on collected customer information and problem data. These solutions include specific suggestions and the reasons behind them. The generated solutions are also created as talk scripts and used for voice output. At this stage, the emotion engine analyzes the user's emotions and incorporates them into the generated solutions and talk scripts.

[0251] Specific example:

[0252] Server-generated solution example: "Implementation of an automated inventory management system"

[0253] Example of a server-generated talk script: "For companies like yours that operate e-commerce, implementing an automated inventory management system is highly recommended."

[0254] 3. Audio output of the talk script

[0255] The generated talk script is sent to the user's device and converted from text to audio data. During this process, the emotion engine adjusts the tone and speed of the voice according to the user's recognized emotions. This ensures that the suggestions are conveyed in a way that is best suited to the user's current situation.

[0256] Specific example:

[0257] The generated talk script proposes the "implementation of an automated inventory management system," and if the user is nervous, the voice tone is adjusted to be calmer.

[0258] 4. Automatic reply to inquiry email

[0259] Later, when a customer inquiry email is sent to the server, the server analyzes its content and automatically generates an appropriate response using generative artificial intelligence. The generated reply email can then be sent to the customer after the user has reviewed it.

[0260] Specific example:

[0261] Customer inquiry: "How should we allocate our advertising budget?"

[0262] Server-generated response: "We recommend allocating 50% of your advertising budget to social media ads, 30% to search engine ads, and 20% to display ads."

[0263] 5. Automatic generation of network configuration proposals and advertising budget allocations.

[0264] The server automatically generates network configuration proposals and advertising budget allocation suggestions based on customer information. This allows for the rapid delivery of optimal proposals tailored to individual customer requirements.

[0265] 6. Input of application information in a chat format and automatic generation of application forms.

[0266] The user device prompts the user to input application information in a chat format. Once information is entered for each field, the server automatically generates an application form based on this information and sends the completed application form to the user device or the user's email address.

[0267] Specific example:

[0268] User device question: "Please tell me the applicant's name."

[0269] When a user enters "Taro Yamada," the server generates an application form that includes "Taro Yamada." Other necessary information is automatically added to the application form.

[0270] The above describes the details of the embodiments for carrying out the present invention. This system automates many processes in sales activities, enabling effective and efficient responses. Furthermore, the introduction of an emotion engine enables responses that take into account the customer's emotions, contributing to improved customer satisfaction.

[0271] The following describes the processing flow.

[0272] ---

[0273] Step 1:

[0274] The user (Sales representative) activates a user device such as an iPad and opens the dedicated app. When the user instructs the device to play the initial explanation video, the device sends a request to the server.

[0275] Step 2:

[0276] The server receives the request and searches for the initial instructional video data. It prepares the corresponding video file for streaming and sends it to the terminal.

[0277] Step 3:

[0278] The device plays the received video data, and the user provides an initial explanation to the customer. After the video plays, a question-selection input screen is displayed on the device.

[0279] Step 4:

[0280] The user enters their answers to questions displayed on the device. The entered answers are sent to the server in real time. The emotion engine analyzes the user's emotions from facial expressions and voice data, and sends the analysis to the server.

[0281] Step 5:

[0282] The server saves the received response data and sentiment data in the database. After saving, it automatically generates the next question and sends it to the terminal.

[0283] Step 6:

[0284] This process is repeated to collect customer information and problem data in detail, and the server accumulates the information in the database.

[0285] Step 7:

[0286] Based on the collected customer information and problem data, the server calls the generative artificial intelligence to generate the optimal solution and the conversation script. Based on the sentiment data analyzed by the sentiment engine, the generated solution and conversation script are adjusted.

[0287] Step 8:

[0288] The server sends the generated conversation script to the terminal. The terminal calls a TTS engine to convert the received conversation script from text to voice.

[0289] Step 9:

[0290] The TTS engine converts the conversation script into voice data and returns it to the terminal. The terminal plays the generated voice data, and the user explains the proposed content to the customer. The tone and speed of the voice are adjusted based on the data of the sentiment engine.

[0291] Step 10:

[0292] At a later date, an inquiry email from the customer is sent to the server. The server analyzes the content of the inquiry email and automatically generates an appropriate response using the generative artificial intelligence.

[0293] Step 11:

[0294] The server provides the generated reply email to the user for confirmation. After the user confirms, the reply email is sent to the customer.

[0295] Step 12:

[0296] The server automatically generates optimal proposal documents for network configuration and advertising budget allocation based on customer information. The generated documents are sent to the user's device or email address.

[0297] Step 13:

[0298] The user device prompts the user to enter application information in a chat format. Once the user enters an answer to each question, the input data is sent to the server.

[0299] Step 14:

[0300] The server automatically generates an application form using a standard format based on the application information it receives. The generated application form is then sent to the terminal or the user's email address.

[0301] Step 15:

[0302] The terminal notifies the user of the generated application form data, which the user then reviews and downloads.

[0303] ---

[0304] The above is a detailed explanation of the program processing of a system incorporating an emotion engine, broken down into specific steps.

[0305] (Example 2)

[0306] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0307] In conventional business activities, many processes such as customer information collection, proposal creation, inquiry response, and application processing relied on manual work, which required time and effort. Furthermore, it was difficult to respond considering the customer's feelings, and there was a risk of a decrease in customer satisfaction. To solve such problems, improve the efficiency of business activities, and enhance customer satisfaction, the provision of an integrated system has been demanded.

[0308] The specific processing by the specific processing unit 290 of the data processing apparatus 12 in the second embodiment is realized by the following respective means.

[0309] In this invention, the server includes means for generating a solution based on customer information and sentiment data, means for converting the generated solution into voice output and adjusting it based on the sentiment data, and means for automatically generating a reply to an inquiry email. As a result, many processes are automated, enabling efficient and effective business activities. Also, by using voice output with an emotion engine, a response that takes into account the customer's feelings is realized, and an improvement in customer satisfaction can be expected.

[0310] "Customer information" refers to information such as basic data, needs, and requests regarding users of services or products.

[0311] "User device" is a hardware device used by a user and has a function for collecting customer information.

[0312] "Generative artificial intelligence" is software for generating various solutions and proposals using machine learning or deep learning based on the input data.

[0313] "Sentiment data" refers to information regarding the feelings and psychological states shown by users or customers, and is data analyzed by an emotion engine.

[0314] "Solution" refers to the specific proposal content created by generative artificial intelligence based on the collected customer information and the reasons therefor.

[0315] "Voice output" is a method of converting text data into audio data and communicating information to users or customers via voice.

[0316] An "inquiry email" is an email sent by a customer containing questions or requests regarding a service or product.

[0317] A "network configuration proposal" is a specific suggestion for optimally designing information systems and communication infrastructure.

[0318] "Advertising budget allocation" refers to a proposal outlining how the advertising budget will be allocated.

[0319] "Application information" refers to the information that a customer provides in order to officially use a service or product.

[0320] An "application form" is an official document automatically generated based on application information and is used to conclude a service or product usage agreement.

[0321] An "emotion engine" is software that analyzes the emotions and psychological state of users and customers and adjusts the system's output based on that analysis.

[0322] A "talk script" is a written document created by a generative artificial intelligence to effectively convey the proposed content, and is used for voice output.

[0323] This invention is a system designed to streamline digital marketing and data product sales activities for corporate clients. In particular, it provides comprehensive support from customer information collection and proposal creation to inquiry handling and application processing. Furthermore, by incorporating an emotion engine that recognizes user and customer emotions, it enables more effective dialogue and customer service.

[0324] System Configuration

[0325] This system mainly consists of the following elements:

[0326] 1. User equipment

[0327] Users use a user device such as a tablet during their initial customer visit. This user device displays a screen for collecting customer information and application information.

[0328] 2. Generative Artificial Intelligence

[0329] The server uses generative artificial intelligence to generate solutions based on collected customer information and sentiment data. This generative AI leverages machine learning and deep learning techniques.

[0330] 3. Emotional Engine

[0331] The emotion engine analyzes user and customer emotional data and adjusts the system's output based on that analysis. This enables more effective communication with customers.

[0332] 4. Server

[0333] The server handles tasks such as storing customer information, running generative artificial intelligence, analyzing emotion engines, automatically responding to inquiry emails, and generating application forms.

[0334] Function and operation of each element

[0335] 1. Collection of customer information

[0336] The user collects customer information using their device. The user device sends a request to the server to play an initial explanatory video. This video contains basic information about the services and products offered. After playing the video, the user enters answers to a series of questions, and these answers are sent to the server in real time. The server stores the answer data in a database.

[0337] Specific example: The user's device displays the question, "What are your company's main business activities?" The user enters "E-commerce," the information is sent to the server, and the customer database is updated.

[0338] 2. Generating solutions

[0339] The server uses generative artificial intelligence to generate optimal solutions based on collected customer information and emotional data. These solutions include specific suggestions and the reasoning behind them. The results of the emotional engine's analysis are also taken into consideration.

[0340] Specific example: Server-generated solution: "Implementation of an automated inventory management system." Proposal content: "Implementation of an automated inventory management system would be effective in improving the efficiency of your e-commerce business."

[0341] 3. Generate talk scripts and audio

[0342] The generated solutions are sent to the user's device and converted from text to audio data. An emotion engine adjusts the tone and speed of the audio, ensuring that the suggestions are delivered in a way that resonates with the customer's emotions.

[0343] Specific example: If the generated talk script proposes "implementing an automated inventory management system" and the user is relaxed, the voice tone will be adjusted to a calmer tone.

[0344] 4. Automatic reply to inquiry email

[0345] When a customer sends an inquiry email to the server, the server analyzes the content and automatically generates an appropriate response using generative artificial intelligence. The generated reply email is then reviewed by the user and sent to the customer.

[0346] Example: Customer inquiry: "How should I allocate my advertising budget?" Server-generated response: "We recommend allocating 50% of your advertising budget to social media ads, 30% to search engine ads, and 20% to display ads."

[0347] 5. Collection of application information and generation of application forms

[0348] The user terminal collects application information in a chat format, and once information is entered for each field, the server automatically generates the application form. The completed application form is sent to the user terminal or email address.

[0349] Specific example: The user terminal asks "Please tell us the applicant's name." The user enters "Taro Yamada," and the server generates an application form that includes "Taro Yamada."

[0350] Example of a prompt

[0351] "Generate proposals for effective automation solutions for e-commerce companies."

[0352] "Please create advice on the optimal allocation of our advertising budget."

[0353] "Please create a talk script that takes customer emotions into consideration."

[0354] As a result, this system automates many processes in sales activities, enabling effective and efficient responses. Furthermore, the introduction of an emotion engine allows for responses that take customer emotions into consideration, contributing to improved customer satisfaction.

[0355] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0356] Step 1:

[0357] Collection of customer information

[0358] The user uses a user device, such as a tablet, during their initial customer visit. The user sends a request from their device to the server, initiating the playback of an initial explanatory video. This video contains basic information about the services and products offered. After the video finishes, a series of questions are displayed on the user's device. The user answers these questions while interacting with the customer. Each answer is sent to the server in real time, and the server stores the answer data in a database.

[0359] Input: Customer information (collected through a questionnaire)

[0360] Data processing: Collection and transmission of response data.

[0361] Output: Update of customer information database

[0362] Specific example:

[0363] Question: "What are your company's main business activities?"

[0364] The user enters "e-commerce," and the information is sent to the server. The server updates the customer database with "e-commerce."

[0365] Step 2:

[0366] Processing customer information

[0367] The server analyzes the received customer response data and extracts the necessary information. Simultaneously, the emotion engine analyzes the emotional data collected during the user-customer interaction. This allows for an understanding of the customer's emotional state.

[0368] Input: Customer response data, sentiment data

[0369] Data processing: Data analysis and extraction

[0370] Output: Updated customer profile, saved sentiment information

[0371] Specific example:

[0372] The server stores the entered "e-commerce" data in the customer profile, and the emotion engine records the customer's emotions detected during the interaction (e.g., relaxed, excited, etc.).

[0373] Step 3:

[0374] Solution generation

[0375] The server uses generative artificial intelligence to generate the optimal solution based on customer information and sentiment data. This solution includes specific suggestions and reasoning. The generated solution is also used as a talk script, and the results of the sentiment engine's analysis are also reflected.

[0376] Input: Customer information, sentiment data

[0377] Data processing: Generating solutions using generative artificial intelligence.

[0378] Output: Solution data

[0379] Specific example:

[0380] Solution: "Implement an automated inventory management system"

[0381] Proposal: "Implementing an automated inventory management system would be an effective way to improve the efficiency of your e-commerce business."

[0382] Step 4:

[0383] Talk script and voice generation

[0384] The generated solution is sent to the user's device, which converts it from text to audio data. The emotion engine then adjusts the tone and speed of the audio based on the user's emotions, which it has analyzed.

[0385] Input: Solution data, sentiment data

[0386] Data processing: Text-to-speech conversion, tone and speed adjustment.

[0387] Output: Audio data

[0388] Specific example:

[0389] The generated talk script proposes the "implementation of an automated inventory management system," and the voice tone is adjusted to a calmer tone if the user is relaxed.

[0390] Step 5:

[0391] Automatic reply to inquiry email

[0392] After a customer inquiry email is sent to the server, the server analyzes its content and automatically generates an appropriate response using generative artificial intelligence. The generated reply email is then reviewed by the user and sent to the customer.

[0393] Input: Inquiry email

[0394] Data processing: Analysis of email content, generation of reply emails.

[0395] Output: Reply email

[0396] Specific example:

[0397] Customer inquiry: "How should we allocate our advertising budget?"

[0398] Generated reply: "We recommend allocating 50% of your advertising budget to social media ads, 30% to search engine ads, and 20% to display ads."

[0399] Step 6:

[0400] Automatic generation of network configuration proposals and advertising budget allocations.

[0401] The server automatically generates network configuration proposals and advertising budget allocation suggestions based on customer information. This allows for the rapid provision of optimal proposals tailored to each customer's requirements.

[0402] Input: Customer Information

[0403] Data processing: Generating proposal content

[0404] Output: Network configuration proposal, advertising budget allocation data

[0405] Specific example:

[0406] Proposal: "The optimal network configuration is to adopt a cloud-based solution and place an independent server at each store."

[0407] Step 7:

[0408] Input of application information in a chat format and automatic generation of application forms.

[0409] The user terminal prompts the user to input application information in a chat format. Once each element is entered, the server automatically generates an application form based on this information and sends the completed application form to the user terminal or email address.

[0410] Input: Application Information

[0411] Data processing: Application form generation

[0412] Output: Application form

[0413] Specific example:

[0414] Question: "Please tell me the applicant's name."

[0415] The user enters "Taro Yamada," and the server generates an application form that includes "Taro Yamada."

[0416] (Application Example 2)

[0417] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0418] Traditional digital marketing and sales support systems could collect customer information and generate proposals, but they had the challenge of not being able to respond in a way that took into account the customer's emotions during the conversation. Furthermore, the inability to adjust voice tone according to the customer's emotions made it difficult to improve customer satisfaction. This, in turn, made it difficult for sales staff in physical stores to provide effective customer service, resulting in challenges in improving sales and customer satisfaction.

[0419] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a user device means for collecting customer information, means for generating solutions based on the customer information using generative artificial intelligence, and means for analyzing customer emotions using an emotion engine and adjusting the content and voice tone generated based on the analysis results. This enables appropriate responses in accordance with customer emotions, realizing effective customer service by sales staff in physical stores, and as a result, an increase in customer satisfaction and sales can be expected.

[0420] "Customer information" refers to all data about a customer, including their name, needs, and emotional state.

[0421] A "user device" is a device used to collect customer information and transmit it to a server, and specifically includes smartphones and smart glasses.

[0422] "Generative artificial intelligence" refers to artificial intelligence that has the ability to generate optimal solutions or talk scripts based on input data.

[0423] "Solutions" refer to specific suggestions or countermeasures generated by generative artificial intelligence based on customer information, aimed at meeting customer needs.

[0424] "Audio output" refers to playing back generated solutions or talk scripts as audio, and includes technologies that convert text data into speech.

[0425] An "inquiry email" is an email sent by a customer to the system containing questions or requests.

[0426] "Automatic reply" refers to the process of analyzing the content of an inquiry email and automatically generating an appropriate response.

[0427] A "network configuration proposal" refers to a proposed layout and design of the network infrastructure that is optimal for the customer's business operations.

[0428] "Advertising budget allocation" refers to a specific plan that shows how the budget used for advertising activities will be distributed.

[0429] "Application information" refers to the information a customer uses to formally apply for a service or product.

[0430] An "application form" is an official document that is automatically generated based on the collected application information.

[0431] An "emotion engine" is a technology that analyzes emotions from customer voice and text and applies the results.

[0432] "Analysis results" refers to the data obtained by the emotion engine from analyzing the customer's emotional state.

[0433] "Voice tone" refers to the pitch, speed, and volume of a voice, and is a factor that greatly influences the impression of the information presented.

[0434] Modes for carrying out the invention

[0435] This invention is a system for supporting sales staff in physical stores, and in particular, it can analyze customer emotions using an emotion engine to effectively engage in dialogue and make suggestions. The system consists of a user device, generative artificial intelligence, an emotion engine, a server voice output function, and related software.

[0436] System Configuration

[0437] 1. User device

[0438] These are devices used for collecting customer information, receiving suggestions, and displaying the results of sentiment analysis. Specifically, this includes smartphones and smart glasses.

[0439] 2. Generative Artificial Intelligence

[0440] This artificial intelligence generates optimal solutions and talk scripts based on customer information. Specifically, it is implemented in Python and can utilize Google® Cloud Natural Language and IBM Watson® as external APIs.

[0441] 3. Emotional Engine

[0442] This technology analyzes customer emotions from their voice and text, and adjusts the generated content and voice tone based on the analysis results. The emotion analysis utilizes the Google Cloud Natural Language API and the IBM Watson API.

[0443] 4. Server

[0444] This system plays a central role in collecting and storing customer information transmitted from user devices, analyzing it using generative artificial intelligence and emotion engines, and sending the results back to the user devices. Cloud services (e.g., AWS®, Google Cloud) are used for the servers.

[0445] 5. Audio output function

[0446] This feature converts server-generated suggestions and talk scripts into audio for playback on the user's device. Specifically, it uses the Google Cloud Text-to-Speech API.

[0447] System operation

[0448] 1. Collection of customer information

[0449] Sales staff use smartphones or smart glasses to input the customer's name and needs. For example, the format might be: "Please enter the customer's name: Taro Yamada" and "Please enter the customer's needs: I'm looking for a large refrigerator for my family."

[0450] The entered information is sent to the server in real time and stored in the database.

[0451] 2. Proposal generation and sentiment analysis

[0452] The server uses generative artificial intelligence to generate the optimal solution based on the collected customer information. Simultaneously, the customer's emotional state is analyzed using an emotion engine.

[0453] For example, if a customer enters "I'm looking for a large refrigerator for my family," the emotion engine will analyze the input and, if it detects a "positive" emotion, it will suggest new products.

[0454] 3. Generating the talk script and outputting the audio.

[0455] The generated solutions are also created as talk scripts and converted into speech using the voice output function. This speech is output in an appropriate tone and speed based on the results of the sentiment analysis.

[0456] For example, if the generated solution is "Special sale information on large refrigerators," the voice tone will be bright and persuasive.

[0457] 4. Automatic reply function

[0458] The server also analyzes customer inquiry emails and uses generative artificial intelligence to generate appropriate automated replies.

[0459] For example, if a customer's email includes a question like, "How should I allocate my advertising budget?", an automated reply will be generated that says, "We recommend allocating 50% of your advertising budget to social media ads, 30% to search engine ads, and 20% to display ads."

[0460] 5. Automatic generation of application information and application form

[0461] Sales staff use user devices to input application information in a chat format, and the server automatically generates an application form based on that information.

[0462] For example, if you enter "Please tell us the applicant's name" and "Taro Yamada," an application form containing "Taro Yamada" will be generated and sent to the user's email address.

[0463] Examples of specific prompt messages

[0464] "Please enter the customer's name: Taro Yamada"

[0465] "Please enter your customer needs: I'm looking for a large refrigerator for my family."

[0466] "My recommended product is a large refrigerator made by LG."

[0467] Thus, this system enables appropriate responses that take customer emotions into consideration, providing effective customer service in physical stores. As a result, improved customer satisfaction and increased sales can be expected.

[0468] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0469] Step 1:

[0470] Users collect customer information using user devices such as smartphones and smart glasses. When a customer visits a store, sales staff enter the customer's name and needs. This information is sent to the server in real time. Specifically, the customer's name "Taro Yamada" and needs "Looking for a large refrigerator for family" are entered. Customer information is collected as input data and stored in the server's database.

[0471] Step 2:

[0472] The server stores customer information sent from the user device in a database. Based on the entered customer information, it calls upon generative artificial intelligence to generate the optimal solution. For example, if the customer needs entered are "I'm looking for a large refrigerator for my family," it will select products and services that meet those needs. The generative AI performs data calculations and outputs "special sale information for large refrigerators" as a solution.

[0473] Step 3:

[0474] The server simultaneously uses an emotion engine to analyze the customer's emotions. It analyzes the customer's emotional state using text and voice data sent from the user's device. Using the customer's response, "I'm looking for a large refrigerator for my family," as input, it uses an emotion analysis API (e.g., Google Cloud Natural Language API) to determine that the emotion is "positive." The analysis results are reflected in the generated solution.

[0475] Step 4:

[0476] The server generates an appropriate talk script based on the generated solutions and the results of sentiment analysis. For example, if the customer is judged to be "positive," a talk script will be created that suggests "sale information on large refrigerators" as a solution. The talk script will be written in a cheerful tone to match the "positive" emotion. Generative artificial intelligence converts the suggestions into a specific talk script and outputs its content.

[0477] Step 5:

[0478] The server invokes a voice output function to convert the generated talk script into audio data. For example, the talk script "Special Sale Information on Large Refrigerators" is converted into audio data using the Google Cloud Text-to-Speech API. The voice tone is adjusted according to the results of sentiment analysis. The talk script is converted into audio data and sent to the user's device.

[0479] Step 6:

[0480] The user device plays audio data transmitted from the server. Sales staff explain proposals to customers via audio using smartphones or smart glasses. Specifically, an audio message about a special sale on large refrigerators is transmitted to the customer, and the customer responds to the proposal. The audio data is then played back as output to provide information to the customer.

[0481] Step 7:

[0482] When a customer sends an inquiry email, the server analyzes its content and generates an automated reply. For example, if a customer asks, "How should I allocate my advertising budget?", the generative AI will automatically generate a specific proposal for budget allocation. It takes the customer's inquiry as input and generates a proposal as output: "We recommend allocating 50% of your advertising budget to social media ads, 30% to search engine ads, and 20% to display ads."

[0483] Step 8:

[0484] The user reviews the automated response sent from the server and makes corrections as needed. Then, the reviewed response is sent to the customer. For example, the user reviews an "advertising budget allocation proposal" and sends it back to the customer if appropriate. The system receives the automated response from the server as input and sends either the corrected or uncorrected response to the customer as output.

[0485] Step 9:

[0486] The user device collects application information from the customer in a chat format and sends it to the server. For example, in response to the question "Please tell me the applicant's name," the user might input "Taro Yamada." The entered application information is sent to the server, which automatically generates an appropriate application form. Finally, the completed application form is sent to the user's email address. Application information is collected as input, and an application form is generated and sent as output.

[0487] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0488] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0489] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0490] [Second Embodiment]

[0491] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0492] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0493] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0494] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0495] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0496] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0497] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0498] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0499] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0500] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0501] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0502] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0503] ---

[0504] This invention is a system designed to streamline digital marketing and data product sales activities for corporate clients. The system comprehensively supports everything from customer information collection and proposal creation to inquiry handling and application processing.

[0505] This system primarily consists of the following elements: a user device, a generative artificial intelligence system, a server, and a voice output function. The specific functions and operation of each element are described below.

[0506] 1. Collection of customer information

[0507] The user (Sales representative) uses a user device such as an iPad during the initial customer visit. The user device sends a request to the server to play a pre-prepared introductory video. This video contains basic information about the services and products being offered. After the video plays, a series of questions are displayed on the user device, and the user answers these questions while interacting with the customer. Each answer is sent to the server in real time, and the server stores this answer data in a database.

[0508] Specific example:

[0509] Question displayed by the user device: "What are your company's main business activities?"

[0510] When a user enters "e-commerce," the user's device sends this information to the server, which then updates the customer database as "e-commerce."

[0511] 2. Generating solutions and talk scripts

[0512] The server uses generative artificial intelligence to generate optimal solutions based on collected customer information and problem data. These solutions include specific suggestions and the reasoning behind them. The generated solutions are also created as talk scripts and used for voice output.

[0513] Specific example:

[0514] Server-generated solution example: "Implementation of an automated inventory management system"

[0515] Example of a server-generated talk script: "For companies like yours that operate e-commerce, implementing an automated inventory management system is highly recommended."

[0516] 3. Audio output of the talk script

[0517] The generated talk script is sent to the user's device and converted from text to audio data (TTS: Text-to-Speech). The converted audio data is played back on the user's device, allowing the user to explain the proposal to the customer verbally.

[0518] 4. Automatic reply to inquiry email

[0519] Later, when a customer sends an inquiry email to the server, the server analyzes its content and automatically generates an appropriate response using generative artificial intelligence. The generated reply email can then be sent to the customer after the user has reviewed it.

[0520] Specific example:

[0521] Customer inquiry: "How should we allocate our advertising budget?"

[0522] Server-generated response: "We recommend allocating 50% of your advertising budget to social media ads, 30% to search engine ads, and 20% to display ads."

[0523] 5. Automatic generation of network configuration proposals and advertising budget allocations.

[0524] The server automatically generates network configuration proposals and advertising budget allocation suggestions based on customer information. This allows for the rapid delivery of optimal proposals tailored to individual customer requirements.

[0525] 6. Input of application information in a chat format and automatic generation of application forms.

[0526] The user device prompts the user to input application information in a chat format. Once information is entered for each field, the server automatically generates an application form based on this information and sends the completed application form to the user device or the user's email address.

[0527] Specific example:

[0528] User device question: "Please tell me the applicant's name."

[0529] When a user enters "Taro Yamada," the server generates an application form that includes "Taro Yamada." Other necessary information is automatically added to the application form.

[0530] The above describes the details of the embodiments for carrying out the present invention. This system automates many processes in sales activities, enabling effective and efficient responses.

[0531] The following describes the processing flow.

[0532] ---

[0533] Step 1:

[0534] The user (Sales representative) activates a user device such as an iPad and opens the dedicated app. When the user instructs the device to play the initial explanation video, the device sends a request to the server.

[0535] Step 2:

[0536] The server receives the request and searches for the initial instructional video data. It prepares the corresponding video file for streaming and sends it to the terminal.

[0537] Step 3:

[0538] The device plays the received video data, and the user provides an initial explanation to the customer. After the video plays, a question-selection input screen is displayed on the device.

[0539] Step 4:

[0540] The user enters their answer to a question displayed on their device. The entered answer is sent to the server in real time.

[0541] Step 5:

[0542] The server saves the received response data to a database. After saving, it automatically generates the next question and sends it to the terminal.

[0543] Step 6:

[0544] This process is repeated to collect detailed customer information and issue data, and the server stores this information in a database.

[0545] Step 7:

[0546] Based on the collected customer information and issue data, the server invokes generative artificial intelligence to generate optimal solutions and talk scripts.

[0547] Step 8:

[0548] The server sends the generated talk script to the terminal. The terminal calls a TTS engine to convert the received talk script from text to speech.

[0549] Step 9:

[0550] The TTS engine converts the talk script into audio data and returns it to the terminal. The terminal plays the generated audio data, and the user explains the proposal to the customer.

[0551] Step 10:

[0552] Later, customer inquiry emails are sent to the server. The server analyzes the content of the inquiry email and automatically generates an appropriate response using generative artificial intelligence.

[0553] Step 11:

[0554] The server provides the generated reply email to the user for confirmation. After the user confirms, the reply email is sent to the customer.

[0555] Step 12:

[0556] The server automatically generates optimal proposal documents for network configuration and advertising budget allocation based on customer information. The generated documents are sent to the user's device or email address.

[0557] Step 13:

[0558] The user device prompts the user to enter application information in a chat format. Once the user enters an answer to each question, the input data is sent to the server.

[0559] Step 14:

[0560] The server automatically generates an application form using a standard format based on the application information it receives. The generated application form is then sent to the terminal or the user's email address.

[0561] Step 15:

[0562] The terminal notifies the user of the generated application form data, which the user then reviews and downloads.

[0563] ---

[0564] The above is an explanation of the program's processing broken down into specific steps.

[0565] (Example 1)

[0566] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0567] In B2B digital marketing and data product sales activities, the processes of collecting customer information, proposing solutions, responding to inquiries, and processing applications are complex and time-consuming, posing a challenge. This can lead to decreased sales efficiency and a decline in the quality of customer service.

[0568] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0569] In this invention, the server includes electronic means for collecting customer information, means for generating solutions based on the customer information using generative artificial intelligence, means for outputting the generated solutions as voice, means for automatically generating replies to inquiry messages, means for automatically generating communication network configuration proposals and budget allocation proposals, and means for collecting application information in a dialogue format and automatically generating application forms. This automates each process necessary for sales activities, such as information gathering, proposals, responses, and application processing, enabling efficient and effective customer service.

[0570] "Electronic devices" refer to electronic devices used for collecting, displaying, and inputting customer information. Specifically, this includes tablets and smartphones.

[0571] "Generative artificial intelligence" refers to artificial intelligence technology that generates solutions and reply messages based on collected customer information. A typical example is a model that performs natural language processing.

[0572] A "solution" refers to specific improvement measures or countermeasures proposed by generative artificial intelligence in response to customer problems and needs.

[0573] "Voice output means" refers to technologies for conveying solutions and other information to users as audio. Specifically, text-to-speech (TTS) functions fall under this category.

[0574] An "inquiry message" refers to an email or message sent by a customer to convey questions or requests.

[0575] A "communication network configuration proposal" refers to a proposal that suggests the optimal communication network configuration tailored to the customer's business needs.

[0576] A "budget allocation proposal" refers to a plan that suggests the most optimal way to allocate a limited budget in advertising activities, etc.

[0577] "Dialogue format" refers to a format in which the user and the system exchange information through a question-and-answer exchange.

[0578] An "application form" refers to a document containing the information necessary for a customer to apply for a service or product.

[0579] This invention is a system designed to streamline digital marketing and data product sales activities for corporate clients. This system comprehensively supports everything from customer information collection and proposal creation to inquiry handling and application processing.

[0580] This system consists of the following elements:

[0581] Electronic devices for collecting customer information

[0582] Generative artificial intelligence

[0583] server

[0584] Audio output function

[0585] Collection of customer information

[0586] The user uses an electronic device such as an iPad during their initial customer visit. The user device sends a request to the server to play a pre-prepared introductory video. This video contains basic information about the services and products being offered. After the video plays, a series of questions are displayed on the user device, and the user answers these questions while interacting with the customer. Each answer is sent to the server in real time, and the server stores this answer data in a database.

[0587] Specific example

[0588] Question displayed by the user device: "What are your company's main business activities?"

[0589] When a user enters "e-commerce," the user's device sends this information to the server, which then updates the customer database as "e-commerce."

[0590] Solution and Talk Script Generation

[0591] The server uses generative artificial intelligence (e.g., OpenAI's GPT-4) to generate optimal solutions based on collected customer information and problem data. These solutions include specific suggestions and the reasoning behind them. The generated solutions are also created as talk scripts and used for voice output.

[0592] Specific example

[0593] Server-generated solution example: "Implementation of an automated inventory management system"

[0594] Example prompt: "Please propose the best IT solution for a new customer. The customer is an e-commerce business."

[0595] Audio output of talk script

[0596] The generated talk script is sent to the user's device and converted from text to audio data (TTS: Text-to-Speech). The converted audio data is played back on the user's device, allowing the user to explain the proposal to the customer verbally.

[0597] Specific example

[0598] Example of a server-generated talk script: "For companies like yours that operate e-commerce, implementing an automated inventory management system is highly recommended."

[0599] Automatic reply to inquiry email

[0600] When a customer sends an inquiry email to the server, the server analyzes its content and automatically generates an appropriate response using generative artificial intelligence. The generated reply email can then be sent to the customer after the user has reviewed it.

[0601] Specific example

[0602] Customer inquiry: "How should we allocate our advertising budget?"

[0603] Server-generated response: "We recommend allocating 50% of your advertising budget to social media ads, 30% to search engine ads, and 20% to display ads."

[0604] Automatic generation of network configuration proposals and advertising budget allocations.

[0605] The server automatically generates network configuration proposals and advertising budget allocation suggestions based on customer information. This allows for the rapid delivery of optimal proposals tailored to individual customer requirements.

[0606] Input of application information in a chat format and automatic generation of application forms.

[0607] The user device prompts the user to input application information in a chat format. Once information is entered for each field, the server automatically generates an application form based on this information and sends the completed application form to the user device or the user's email address.

[0608] Specific example

[0609] User device question: "Please tell me the applicant's name."

[0610] When a user enters "Taro Yamada," the server generates an application form that includes "Taro Yamada." Other necessary information is automatically added to the application form.

[0611] The above describes the details of embodiments for carrying out the present invention. This system automates many processes in sales activities, enabling effective and efficient responses.

[0612] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0613] Step 1: Collecting customer information

[0614] Input: Questions displayed on the user device during the first visit, and customer responses.

[0615] Process: The user uses an electronic device such as an iPad to play an initial explanatory video. After the video plays, a series of questions are displayed on the user's device, and the user answers these questions while interacting with the customer. The input data is sent to the server in real time.

[0616] Output: Customer information stored in the database on the server

[0617] Specific operation: An initial explanatory video is played, followed by the question, "What are your company's main business activities?" If the customer answers "E-commerce," it is recorded as "E-commerce" in the database.

[0618] Step 2: Data analysis and solution generation

[0619] Input: Customer information and issue data collected by the server

[0620] Processing: The server analyzes the collected customer information and generates the optimal solution using generative artificial intelligence (e.g., OpenAI's GPT-4). The generated solution is also created as a talk script.

[0621] Output: Generated solutions and talk scripts

[0622] Specific operation: The server analyzes data related to "e-commerce" and generates the implementation of an automated inventory management system as a solution. Generated talk script: "For companies like yours that operate e-commerce, implementing an automated inventory management system is effective."

[0623] Step 3: Outputting the solution via audio

[0624] Input: Generated talk script

[0625] Processing: The user device converts the talk script received from the server from text to audio data (TTS: Text-to-Speech). It then plays the converted audio data.

[0626] Output: Playback of audio data

[0627] Specific operation: The user device converts the generated talk script into audio data and plays the audio saying, "For companies like yours that operate e-commerce, implementing an automated inventory management system is effective."

[0628] Step 4: Automatic reply to inquiry email

[0629] Input: Customer inquiry email

[0630] Processing: The server receives the inquiry email and analyzes its contents. Using generative artificial intelligence, it automatically generates an appropriate reply. After the user confirms the generated reply email, it is sent to the customer.

[0631] Output: Generated reply email

[0632] Specific operation: A customer sends an email to the server asking, "How should I allocate my advertising budget?" The server automatically generates a reply saying, "We recommend allocating 50% of your advertising budget to social media ads, 30% to search engine ads, and 20% to display ads." The user then reviews and sends the reply.

[0633] Step 5: Automatic generation of network configuration and advertising budget allocation.

[0634] Input: Customer information stored on the server

[0635] Processing: The server analyzes customer information and automatically generates optimal communication network configurations and budget allocation plans for specific requirements.

[0636] Output: Generated network configuration and advertising budget allocation proposal

[0637] Specific operation: The server plans a network configuration based on the customer's business needs and proposes an effective allocation of the advertising budget.

[0638] Step 6: Interactive input of application information and automatic generation of application form.

[0639] Input: Application information entered on the user device

[0640] Processing: The user enters application information interactively through their device. The entered information is sent to the server, which automatically generates an application form based on it. The generated application form is sent to the user's device or to the user's email address.

[0641] Output: Generated application form

[0642] Specific operation: The user device inputs the question "Please tell me the applicant's name," and the user answers "Taro Yamada." The server generates an application form including "Taro Yamada" and sends the final application form to the user device.

[0643] The above describes the specific operations and inputs / outputs at each processing step. This system efficiently manages the processes of information gathering, proposals, responses, and application processing for sales activities.

[0644] (Application Example 1)

[0645] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0646] Traditional digital marketing and data product sales activities have faced problems such as the cumbersome process of collecting customer information, hindering efficient proposal creation, inquiry handling, and application processing. Furthermore, real-time customer support is difficult, resulting in numerous inefficient processes in users' sales activities. In addition, systems designed to automate these processes have difficulty meeting individual requirements and have failed to alleviate the burden on sales representatives.

[0647] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0648] In this invention, the server includes a user device means for collecting customer information, means for generating solutions based on the customer information using generative artificial intelligence, means for outputting the generated solutions as voice, means for automatically generating replies to inquiry emails, means for generating an optimal talk script and outputting it as voice, means for collecting information in real time through voice dialogue with customers, and means for collecting application information in talk format and automatically generating application forms. This makes it possible to efficiently and effectively automate a series of sales processes from collecting customer information to proposing solutions, responding to inquiries, and processing applications. Furthermore, by making real-time customer support easier, it is possible to reduce the burden on sales representatives and contribute to improving customer satisfaction.

[0649] "User device means for collecting customer information" refers to hardware and software used to collect necessary information through interactions and operations with customers.

[0650] "Means for generating solutions based on customer information using generative artificial intelligence" refers to an artificial intelligence model and its operating environment for generating optimal solutions based on collected customer information.

[0651] "Means for outputting generated solutions as audio" refers to a device and program that provides the function of converting solutions generated by a generative artificial intelligence from text to audio and playing it back.

[0652] "Methods for automatically generating replies to inquiry emails" refers to technologies and systems that analyze the content of customer inquiries and automatically generate appropriate reply content.

[0653] "Means for generating optimal talk scripts and outputting them as audio" refers to devices and technologies that create effective talk scripts based on customer information and conversation content, and then convert these scripts into audio to communicate with customers.

[0654] "Means for collecting information in real time through voice interaction with customers" refers to devices and programs that provide functions for analyzing customer interactions in real time using speech recognition technology and collecting necessary information.

[0655] "Methods for collecting application information in a conversational format and automatically generating application forms" refers to technologies and systems for collecting application information from customers in a dialogue format and automatically creating application forms based on that information.

[0656] This invention is a system for streamlining the entire process from customer information collection to proposal creation, inquiry handling, and application processing. This system is particularly applicable to sales support for physical stores utilizing smart glasses. A specific embodiment of this system is described below.

[0657] System Configuration

[0658] This system consists of the following main elements:

[0659] 1. User device means: This is built as smart glasses and collects necessary information through interaction and operation with the customer.

[0660] 2. Generative Artificial Intelligence: Use OpenAI's generative AI models (e.g., the Davinci Codex engine) to generate solutions based on collected customer information.

[0661] 3. Audio output means: Equipped with Text-to-Speech (TTS) functionality to convert the generated solution from text to speech (e.g., pyttsx3 library).

[0662] 4. Inquiry Email Handling Method: Analyze the content of customer inquiry emails and automatically generate reply content.

[0663] 5. Talk script generation and audio output means: It has a function to create an effective talk script based on customer information and conversation content, and output it as audio.

[0664] 6. Real-time information gathering means: Information is collected through voice interactions with customers using speech recognition technology (e.g., SpeechRecognition library).

[0665] 7. Automatic application form generation method: An application form is automatically created based on application information collected from the customer through dialogue.

[0666] Program Processing Description

[0667] 1. Collection of customer information

[0668] The user wears smart glasses and interacts with customers. During the interaction, the microphone and voice recognition technology built into the smart glasses transcribe the customer's speech into text in real time. For example, if a customer says, "We run an online retail business," the voice recognition technology transcribes this information into text and collects it as customer information.

[0669] 2. Generating solutions and talk scripts

[0670] The server uses a generative AI model to generate the optimal solution based on the collected customer information. The generative AI model used here is OpenAI's Davinci-codex engine. For example, if "online retail" is collected as customer information, the generative AI model will generate a solution such as "Implementing a loyalty program is effective for online retailers."

[0671] 3. Audio output of the generated solution

[0672] The server sends the generated solution to the smart glasses, where it is converted into speech via Text-to-Speech technology. For example, the pyttsx3 library is used to voice the solution and play it back to the user. This allows the user to explain the proposal to the customer verbally through the smart glasses.

[0673] 4. Automatic reply to inquiry email

[0674] The server analyzes customer inquiry emails and automatically generates appropriate replies using an AI model based on their content. For example, in response to an inquiry such as "How should I allocate my advertising budget?", it generates a reply such as "We recommend allocating 50% of your advertising budget to social media ads, 30% to search engine ads, and 20% to display ads."

[0675] 5. Collection of application information and automatic generation of application forms

[0676] The user device (smart glasses) collects application information from the customer in an interactive manner, and the server automatically generates an application form based on that information. For example, if the user asks the customer, "What is the applicant's name?", and the customer replies, "Taro Yamada", that information is sent to the server, and "Taro Yamada" is automatically added to the application form.

[0677] Examples of specific cases and prompt statements

[0678] Customer information example:

[0679] 1. Question: "What are your company's main business activities?"

[0680] 2. Customer response: "We operate an online retail business."

[0681] Generated solution:

[0682] 1. Proposal: "In online retail, implementing a loyalty program is effective. This program is expected to stimulate customer purchasing intent and increase repeat customers."

[0683] Example of a prompt:

[0684] Customer Information: We operate an online retail business.

[0685] Please propose the most suitable solution.

[0686] With the above configuration, the present invention can achieve efficient collection of customer information, automatic generation of optimal solutions, rapid and accurate response to inquiries, and automation of application processing. This makes it possible to significantly improve the productivity of sales activities and customer satisfaction.

[0687] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0688] Step 1:

[0689] The user wears smart glasses and interacts with the customer. Using a microphone and speech recognition technology built into the smart glasses, the customer's speech is converted into text data in real time. The input here is the customer's voice information, and the output is customer information in text format. Specifically, if the customer says, "We run an online retail business," that voice is converted into the text data "We run an online retail business."

[0690] Step 2:

[0691] The terminal sends the collected customer information to the server. The server receives this information and uses generative artificial intelligence (e.g., OpenAI's Davinci-codex engine) to generate the optimal solution based on the customer information. The input here is customer information in text format, and the output is the text of the generated solution. For example, if the input is customer information that says "We operate an online retail business," the server will generate the solution that says "Implementing a loyalty program is effective for online retail businesses."

[0692] Step 3:

[0693] The server sends the generated solution to the smart glasses, which then convert it into audio data using Text-to-Speech (TTS) technology (e.g., the pyttsx3 library) and present it to the user. The input here is the text data of the generated solution, and the output is audio data. Specifically, the text data "Implementing a loyalty program is effective in online retail" is converted into audio, which the user can then hear.

[0694] Step 4:

[0695] If a customer makes a follow-up inquiry, the user collects the information through smart glasses and sends it to the server. The server uses generative artificial intelligence to analyze the inquiry and automatically generate an appropriate response. The input here is the text data of the customer's follow-up inquiry, and the output is the automatically generated response text. For example, in response to the inquiry, "How should I allocate my advertising budget?", the system would generate a response such as, "We recommend allocating 50% of your advertising budget to social media ads, 30% to search engine ads, and 20% to display ads."

[0696] Step 5:

[0697] The user collects customer application information in a chat format via smart glasses and sends this information to the server. The server automatically generates an application form based on the received application information. The input here is the text data of the collected application information, and the output is the automatically generated application form. Specifically, if the user asks the customer, "Please tell me the applicant's name," and the customer replies, "Taro Yamada," the name "Taro Yamada" is automatically added to the application form based on that information.

[0698] The above describes the specific processing flow of the system program of the present invention. In each processing step, data processing and data calculations are performed based on the input data, and output data is generated accordingly. This enables efficient collection of customer information, rapid generation of optimal solutions, real-time customer support, and automation of application processing.

[0699] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0700] ---

[0701] This invention is a system for streamlining digital marketing and data product sales activities for corporate clients, and in particular, it comprehensively supports everything from customer information collection and proposal creation to inquiry handling and application processing. Furthermore, by incorporating an emotion engine that recognizes user emotions, it enables more effective dialogue and customer service.

[0702] This system primarily consists of the following elements: user device, generative artificial intelligence, server, voice output function, and emotion engine. The specific functions and operation of each element are described below.

[0703] 1. Collection of customer information

[0704] The user (Sales representative) uses a user device such as an iPad during the initial customer visit. The user device sends a request to the server to play a pre-prepared introductory video. This video contains basic information about the services and products being offered. After the video plays, a series of questions are displayed on the user device, and the user answers these questions while interacting with the customer. Each answer is sent to the server in real time, and the server stores this answer data in a database.

[0705] Specific example:

[0706] Question displayed by the user device: "What are your company's main business activities?"

[0707] When a user enters "e-commerce," the user's device sends this information to the server, which then updates the customer database as "e-commerce."

[0708] 2. Generating solutions and talk scripts

[0709] The server uses generative artificial intelligence to generate optimal solutions based on collected customer information and problem data. These solutions include specific suggestions and the reasons behind them. The generated solutions are also created as talk scripts and used for voice output. At this stage, the emotion engine analyzes the user's emotions and incorporates them into the generated solutions and talk scripts.

[0710] Specific example:

[0711] Server-generated solution example: "Implementation of an automated inventory management system"

[0712] Example of a server-generated talk script: "For companies like yours that operate e-commerce, implementing an automated inventory management system is highly recommended."

[0713] 3. Audio output of the talk script

[0714] The generated talk script is sent to the user's device and converted from text to audio data. During this process, the emotion engine adjusts the tone and speed of the voice according to the user's recognized emotions. This ensures that the suggestions are conveyed in a way that is best suited to the user's current situation.

[0715] Specific example:

[0716] The generated talk script proposes the "implementation of an automated inventory management system," and if the user is nervous, the voice tone is adjusted to be calmer.

[0717] 4. Automatic reply to inquiry email

[0718] Later, when a customer inquiry email is sent to the server, the server analyzes its content and automatically generates an appropriate response using generative artificial intelligence. The generated reply email can then be sent to the customer after the user has reviewed it.

[0719] Specific example:

[0720] Customer inquiry: "How should we allocate our advertising budget?"

[0721] Server-generated response: "We recommend allocating 50% of your advertising budget to social media ads, 30% to search engine ads, and 20% to display ads."

[0722] 5. Automatic generation of network configuration proposals and advertising budget allocations.

[0723] The server automatically generates network configuration proposals and advertising budget allocation suggestions based on customer information. This allows for the rapid delivery of optimal proposals tailored to individual customer requirements.

[0724] 6. Input of application information in a chat format and automatic generation of application forms.

[0725] The user device prompts the user to input application information in a chat format. Once information is entered for each field, the server automatically generates an application form based on this information and sends the completed application form to the user device or the user's email address.

[0726] Specific example:

[0727] User device question: "Please tell me the applicant's name."

[0728] When a user enters "Taro Yamada," the server generates an application form that includes "Taro Yamada." Other necessary information is automatically added to the application form.

[0729] The above describes the details of the embodiments for carrying out the present invention. This system automates many processes in sales activities, enabling effective and efficient responses. Furthermore, the introduction of an emotion engine enables responses that take into account the customer's emotions, contributing to improved customer satisfaction.

[0730] The following describes the processing flow.

[0731] ---

[0732] Step 1:

[0733] The user (Sales representative) activates a user device such as an iPad and opens the dedicated app. When the user instructs the device to play the initial explanation video, the device sends a request to the server.

[0734] Step 2:

[0735] The server receives the request and searches for the initial instructional video data. It prepares the corresponding video file for streaming and sends it to the terminal.

[0736] Step 3:

[0737] The device plays the received video data, and the user provides an initial explanation to the customer. After the video plays, a question-selection input screen is displayed on the device.

[0738] Step 4:

[0739] The user enters their answers to questions displayed on the device. The entered answers are sent to the server in real time. The emotion engine analyzes the user's emotions from facial expressions and voice data, and sends the analysis to the server.

[0740] Step 5:

[0741] The server saves the received response data and sentiment data to a database. After saving, it automatically generates the next question and sends it to the terminal.

[0742] Step 6:

[0743] This process is repeated to collect detailed customer information and issue data, and the server stores this information in a database.

[0744] Step 7:

[0745] The server uses collected customer information and issue data to invoke generative artificial intelligence to generate optimal solutions and talk scripts. The emotion engine then adjusts the generated solutions and talk scripts based on the emotional data it analyzes.

[0746] Step 8:

[0747] The server sends the generated talk script to the terminal. The terminal calls a TTS engine to convert the received talk script from text to speech.

[0748] Step 9:

[0749] The TTS engine converts the talk script into audio data and returns it to the terminal. The terminal plays the generated audio data, and the user explains the proposal to the customer. The tone and speed of the voice are adjusted based on data from the emotion engine.

[0750] Step 10:

[0751] Later, customer inquiry emails are sent to the server. The server analyzes the content of the inquiry email and automatically generates an appropriate response using generative artificial intelligence.

[0752] Step 11:

[0753] The server provides the generated reply email to the user for confirmation. After the user confirms, the reply email is sent to the customer.

[0754] Step 12:

[0755] The server automatically generates optimal proposal documents for network configuration and advertising budget allocation based on customer information. The generated documents are sent to the user's device or email address.

[0756] Step 13:

[0757] The user device prompts the user to enter application information in a chat format. Once the user enters an answer to each question, the input data is sent to the server.

[0758] Step 14:

[0759] The server automatically generates an application form using a standard format based on the application information it receives. The generated application form is then sent to the terminal or the user's email address.

[0760] Step 15:

[0761] The terminal notifies the user of the generated application form data, which the user then reviews and downloads.

[0762] ---

[0763] The above is a detailed explanation of the program processing of a system incorporating an emotion engine, broken down into specific steps.

[0764] (Example 2)

[0765] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0766] In traditional sales activities, many processes, such as collecting customer information, creating proposals, responding to inquiries, and processing applications, relied on manual labor, resulting in time-consuming and labor-intensive tasks. Furthermore, it was difficult to respond in a way that was sensitive to customer emotions, leading to a risk of decreased customer satisfaction. There was a need for a comprehensive system that could solve these problems, streamline sales activities, and improve customer satisfaction.

[0767] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0768] In this invention, the server includes means for generating solutions based on customer information and emotional data, means for converting the generated solutions into voice output and adjusting them based on emotional data, and means for automatically generating replies to inquiry emails. This automates many processes, enabling efficient and effective sales activities. Furthermore, voice output using an emotional engine enables responses that take customer emotions into consideration, which is expected to improve customer satisfaction.

[0769] "Customer information" refers to basic data, needs, and requests regarding users of a service or product.

[0770] "User equipment" refers to hardware devices used by users that have the function of collecting customer information.

[0771] "Generative artificial intelligence" refers to software that generates various solutions and suggestions based on input data, using machine learning and deep learning.

[0772] "Emotional data" refers to information about the emotions and psychological states exhibited by users and customers, and is data that is analyzed by an emotion engine.

[0773] A "solution" refers to the specific proposals and their reasons generated by a generative artificial intelligence system based on the collected customer information.

[0774] "Voice output" is a method of converting text data into audio data and communicating information to users or customers via voice.

[0775] An "inquiry email" is an email sent by a customer containing questions or requests regarding a service or product.

[0776] A "network configuration proposal" is a specific suggestion for optimally designing information systems and communication infrastructure.

[0777] "Advertising budget allocation" refers to a proposal outlining how the advertising budget will be allocated.

[0778] "Application information" refers to the information that a customer provides in order to officially use a service or product.

[0779] An "application form" is an official document automatically generated based on application information and is used to conclude a service or product usage agreement.

[0780] An "emotion engine" is software that analyzes the emotions and psychological state of users and customers and adjusts the system's output based on that analysis.

[0781] A "talk script" is a written document created by a generative artificial intelligence to effectively convey the proposed content, and is used for voice output.

[0782] This invention is a system designed to streamline digital marketing and data product sales activities for corporate clients. In particular, it provides comprehensive support from customer information collection and proposal creation to inquiry handling and application processing. Furthermore, by incorporating an emotion engine that recognizes user and customer emotions, it enables more effective dialogue and customer service.

[0783] System Configuration

[0784] This system mainly consists of the following elements:

[0785] 1. User equipment

[0786] Users use a user device such as a tablet during their initial customer visit. This user device displays a screen for collecting customer information and application information.

[0787] 2. Generative Artificial Intelligence

[0788] The server uses generative artificial intelligence to generate solutions based on collected customer information and sentiment data. This generative AI leverages machine learning and deep learning techniques.

[0789] 3. Emotional Engine

[0790] The emotion engine analyzes user and customer emotional data and adjusts the system's output based on that analysis. This enables more effective communication with customers.

[0791] 4. Server

[0792] The server handles tasks such as storing customer information, running generative artificial intelligence, analyzing emotion engines, automatically responding to inquiry emails, and generating application forms.

[0793] Function and operation of each element

[0794] 1. Collection of customer information

[0795] The user collects customer information using their device. The user device sends a request to the server to play an initial explanatory video. This video contains basic information about the services and products offered. After playing the video, the user enters answers to a series of questions, and these answers are sent to the server in real time. The server stores the answer data in a database.

[0796] Specific example: The user's device displays the question, "What are your company's main business activities?" The user enters "E-commerce," the information is sent to the server, and the customer database is updated.

[0797] 2. Generating solutions

[0798] The server uses generative artificial intelligence to generate optimal solutions based on collected customer information and emotional data. These solutions include specific suggestions and the reasoning behind them. The results of the emotional engine's analysis are also taken into consideration.

[0799] Specific example: Server-generated solution: "Implementation of an automated inventory management system." Proposal content: "Implementation of an automated inventory management system would be effective in improving the efficiency of your e-commerce business."

[0800] 3. Generate talk scripts and audio

[0801] The generated solutions are sent to the user's device and converted from text to audio data. An emotion engine adjusts the tone and speed of the audio, ensuring that the suggestions are delivered in a way that resonates with the customer's emotions.

[0802] Specific example: If the generated talk script proposes "implementing an automated inventory management system" and the user is relaxed, the voice tone will be adjusted to a calmer tone.

[0803] 4. Automatic reply to inquiry email

[0804] When a customer sends an inquiry email to the server, the server analyzes the content and automatically generates an appropriate response using generative artificial intelligence. The generated reply email is then reviewed by the user and sent to the customer.

[0805] Example: Customer inquiry: "How should I allocate my advertising budget?" Server-generated response: "We recommend allocating 50% of your advertising budget to social media ads, 30% to search engine ads, and 20% to display ads."

[0806] 5. Collection of application information and generation of application forms

[0807] The user terminal collects application information in a chat format, and once information is entered for each field, the server automatically generates the application form. The completed application form is sent to the user terminal or email address.

[0808] Specific example: The user terminal asks "Please tell us the applicant's name." The user enters "Taro Yamada," and the server generates an application form that includes "Taro Yamada."

[0809] Example of a prompt

[0810] "Generate proposals for effective automation solutions for e-commerce companies."

[0811] "Please create advice on the optimal allocation of our advertising budget."

[0812] "Please create a talk script that takes customer emotions into consideration."

[0813] As a result, this system automates many processes in sales activities, enabling effective and efficient responses. Furthermore, the introduction of an emotion engine allows for responses that take customer emotions into consideration, contributing to improved customer satisfaction.

[0814] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0815] Step 1:

[0816] Collection of customer information

[0817] The user uses a user device, such as a tablet, during their initial customer visit. The user sends a request from their device to the server, initiating the playback of an initial explanatory video. This video contains basic information about the services and products offered. After the video finishes, a series of questions are displayed on the user's device. The user answers these questions while interacting with the customer. Each answer is sent to the server in real time, and the server stores the answer data in a database.

[0818] Input: Customer information (collected through a questionnaire)

[0819] Data processing: Collection and transmission of response data.

[0820] Output: Update of customer information database

[0821] Specific example:

[0822] Question: "What are your company's main business activities?"

[0823] The user enters "e-commerce," and the information is sent to the server. The server updates the customer database with "e-commerce."

[0824] Step 2:

[0825] Processing customer information

[0826] The server analyzes the received customer response data and extracts the necessary information. Simultaneously, the emotion engine analyzes the emotional data collected during the user-customer interaction. This allows for an understanding of the customer's emotional state.

[0827] Input: Customer response data, sentiment data

[0828] Data processing: Data analysis and extraction

[0829] Output: Updated customer profile, saved sentiment information

[0830] Specific example:

[0831] The server stores the entered "e-commerce" data in the customer profile, and the emotion engine records the customer's emotions detected during the interaction (e.g., relaxed, excited, etc.).

[0832] Step 3:

[0833] Solution generation

[0834] The server uses generative artificial intelligence to generate the optimal solution based on customer information and sentiment data. This solution includes specific suggestions and reasoning. The generated solution is also used as a talk script, and the results of the sentiment engine's analysis are also reflected.

[0835] Input: Customer information, sentiment data

[0836] Data processing: Generating solutions using generative artificial intelligence.

[0837] Output: Solution data

[0838] Specific example:

[0839] Solution: "Implement an automated inventory management system"

[0840] Proposal: "Implementing an automated inventory management system would be an effective way to improve the efficiency of your e-commerce business."

[0841] Step 4:

[0842] Talk script and voice generation

[0843] The generated solution is sent to the user's device, which converts it from text to audio data. The emotion engine then adjusts the tone and speed of the audio based on the user's emotions, which it has analyzed.

[0844] Input: Solution data, sentiment data

[0845] Data processing: Text-to-speech conversion, tone and speed adjustment.

[0846] Output: Audio data

[0847] Specific example:

[0848] The generated talk script proposes the "implementation of an automated inventory management system," and the voice tone is adjusted to a calmer tone if the user is relaxed.

[0849] Step 5:

[0850] Automatic reply to inquiry email

[0851] After a customer inquiry email is sent to the server, the server analyzes its content and automatically generates an appropriate response using generative artificial intelligence. The generated reply email is then reviewed by the user and sent to the customer.

[0852] Input: Inquiry email

[0853] Data processing: Analysis of email content, generation of reply emails.

[0854] Output: Reply email

[0855] Specific example:

[0856] Customer inquiry: "How should we allocate our advertising budget?"

[0857] Generated reply: "We recommend allocating 50% of your advertising budget to social media ads, 30% to search engine ads, and 20% to display ads."

[0858] Step 6:

[0859] Automatic generation of network configuration proposals and advertising budget allocations.

[0860] The server automatically generates network configuration proposals and advertising budget allocation suggestions based on customer information. This allows for the rapid provision of optimal proposals tailored to each customer's requirements.

[0861] Input: Customer Information

[0862] Data processing: Generating proposal content

[0863] Output: Network configuration proposal, advertising budget allocation data

[0864] Specific example:

[0865] Proposal: "The optimal network configuration is to adopt a cloud-based solution and place an independent server at each store."

[0866] Step 7:

[0867] Input of application information in a chat format and automatic generation of application forms.

[0868] The user terminal prompts the user to input application information in a chat format. Once each element is entered, the server automatically generates an application form based on this information and sends the completed application form to the user terminal or email address.

[0869] Input: Application Information

[0870] Data processing: Application form generation

[0871] Output: Application form

[0872] Specific example:

[0873] Question: "Please tell me the applicant's name."

[0874] The user enters "Taro Yamada," and the server generates an application form that includes "Taro Yamada."

[0875] (Application Example 2)

[0876] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0877] Traditional digital marketing and sales support systems could collect customer information and generate proposals, but they had the challenge of not being able to respond in a way that took into account the customer's emotions during the conversation. Furthermore, the inability to adjust voice tone according to the customer's emotions made it difficult to improve customer satisfaction. This, in turn, made it difficult for sales staff in physical stores to provide effective customer service, resulting in challenges in improving sales and customer satisfaction.

[0878] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a user device means for collecting customer information, means for generating solutions based on the customer information using generative artificial intelligence, and means for analyzing customer emotions using an emotion engine and adjusting the content and voice tone generated based on the analysis results. This enables appropriate responses in accordance with customer emotions, realizing effective customer service by sales staff in physical stores, and as a result, an increase in customer satisfaction and sales can be expected.

[0879] "Customer information" refers to all data about a customer, including their name, needs, and emotional state.

[0880] A "user device" is a device used to collect customer information and transmit it to a server, and specifically includes smartphones and smart glasses.

[0881] "Generative artificial intelligence" refers to artificial intelligence that has the ability to generate optimal solutions or talk scripts based on input data.

[0882] "Solutions" refer to specific suggestions or countermeasures generated by generative artificial intelligence based on customer information, aimed at meeting customer needs.

[0883] "Audio output" refers to playing back generated solutions or talk scripts as audio, and includes technologies that convert text data into speech.

[0884] An "inquiry email" is an email sent by a customer to the system containing questions or requests.

[0885] "Automatic reply" refers to the process of analyzing the content of an inquiry email and automatically generating an appropriate response.

[0886] A "network configuration proposal" refers to a proposed layout and design of the network infrastructure that is optimal for the customer's business operations.

[0887] "Advertising budget allocation" refers to a specific plan that shows how the budget used for advertising activities will be distributed.

[0888] "Application information" refers to the information a customer uses to formally apply for a service or product.

[0889] An "application form" is an official document that is automatically generated based on the collected application information.

[0890] An "emotion engine" is a technology that analyzes emotions from customer voice and text and applies the results.

[0891] "Analysis results" refers to the data obtained by the emotion engine from analyzing the customer's emotional state.

[0892] "Voice tone" refers to the pitch, speed, and volume of a voice, and is a factor that greatly influences the impression of the information presented.

[0893] Modes for carrying out the invention

[0894] This invention is a system for supporting sales staff in physical stores, and in particular, it can analyze customer emotions using an emotion engine to effectively engage in dialogue and make suggestions. The system consists of a user device, generative artificial intelligence, an emotion engine, a server voice output function, and related software.

[0895] System Configuration

[0896] 1. User device

[0897] These are devices used for collecting customer information, receiving suggestions, and displaying the results of sentiment analysis. Specifically, this includes smartphones and smart glasses.

[0898] 2. Generative Artificial Intelligence

[0899] This artificial intelligence generates optimal solutions and talk scripts based on customer information. Specifically, it is implemented in Python and can utilize Google Cloud Natural Language and IBM Watson as external APIs.

[0900] 3. Emotional Engine

[0901] This technology analyzes customer emotions from their voice and text, and adjusts the generated content and voice tone based on the analysis results. The emotion analysis utilizes the Google Cloud Natural Language API and the IBM Watson API.

[0902] 4. Server

[0903] This system plays a central role in collecting and storing customer information transmitted from user devices, analyzing it using generative artificial intelligence and emotion engines, and sending the results back to the user devices. Cloud services (e.g., AWS, Google Cloud) are used for the servers.

[0904] 5. Audio output function

[0905] This feature converts server-generated suggestions and talk scripts into audio for playback on the user's device. Specifically, it uses the Google Cloud Text-to-Speech API.

[0906] System operation

[0907] 1. Collection of customer information

[0908] Sales staff use smartphones or smart glasses to input the customer's name and needs. For example, the format might be: "Please enter the customer's name: Taro Yamada" and "Please enter the customer's needs: I'm looking for a large refrigerator for my family."

[0909] The entered information is sent to the server in real time and stored in the database.

[0910] 2. Proposal generation and sentiment analysis

[0911] The server uses generative artificial intelligence to generate the optimal solution based on the collected customer information. Simultaneously, the customer's emotional state is analyzed using an emotion engine.

[0912] For example, if a customer enters "I'm looking for a large refrigerator for my family," the emotion engine will analyze the input and, if it detects a "positive" emotion, it will suggest new products.

[0913] 3. Generating the talk script and outputting the audio.

[0914] The generated solutions are also created as talk scripts and converted into speech using the voice output function. This speech is output in an appropriate tone and speed based on the results of the sentiment analysis.

[0915] For example, if the generated solution is "Special sale information on large refrigerators," the voice tone will be bright and persuasive.

[0916] 4. Automatic reply function

[0917] The server also analyzes customer inquiry emails and uses generative artificial intelligence to generate appropriate automated replies.

[0918] For example, if a customer's email includes a question like, "How should I allocate my advertising budget?", an automated reply will be generated that says, "We recommend allocating 50% of your advertising budget to social media ads, 30% to search engine ads, and 20% to display ads."

[0919] 5. Automatic generation of application information and application form

[0920] Sales staff use user devices to input application information in a chat format, and the server automatically generates an application form based on that information.

[0921] For example, if you enter "Please tell us the applicant's name" and "Taro Yamada," an application form containing "Taro Yamada" will be generated and sent to the user's email address.

[0922] Examples of specific prompt messages

[0923] "Please enter the customer's name: Taro Yamada"

[0924] "Please enter your customer needs: I'm looking for a large refrigerator for my family."

[0925] "My recommended product is a large refrigerator made by LG."

[0926] Thus, this system enables appropriate responses that take customer emotions into consideration, providing effective customer service in physical stores. As a result, improved customer satisfaction and increased sales can be expected.

[0927] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0928] Step 1:

[0929] Users collect customer information using user devices such as smartphones and smart glasses. When a customer visits a store, sales staff enter the customer's name and needs. This information is sent to the server in real time. Specifically, the customer's name "Taro Yamada" and needs "Looking for a large refrigerator for family" are entered. Customer information is collected as input data and stored in the server's database.

[0930] Step 2:

[0931] The server stores customer information sent from the user device in a database. Based on the entered customer information, it calls upon generative artificial intelligence to generate the optimal solution. For example, if the customer needs entered are "I'm looking for a large refrigerator for my family," it will select products and services that meet those needs. The generative AI performs data calculations and outputs "special sale information for large refrigerators" as a solution.

[0932] Step 3:

[0933] The server simultaneously uses an emotion engine to analyze the customer's emotions. It analyzes the customer's emotional state using text and voice data sent from the user's device. Using the customer's response, "I'm looking for a large refrigerator for my family," as input, it uses an emotion analysis API (e.g., Google Cloud Natural Language API) to determine that the emotion is "positive." The analysis results are reflected in the generated solution.

[0934] Step 4:

[0935] The server generates an appropriate talk script based on the generated solutions and the results of sentiment analysis. For example, if the customer is judged to be "positive," a talk script will be created that suggests "sale information on large refrigerators" as a solution. The talk script will be written in a cheerful tone to match the "positive" emotion. Generative artificial intelligence converts the suggestions into a specific talk script and outputs its content.

[0936] Step 5:

[0937] The server invokes a voice output function to convert the generated talk script into audio data. For example, the talk script "Special Sale Information on Large Refrigerators" is converted into audio data using the Google Cloud Text-to-Speech API. The voice tone is adjusted according to the results of sentiment analysis. The talk script is converted into audio data and sent to the user's device.

[0938] Step 6:

[0939] The user device plays audio data transmitted from the server. Sales staff explain proposals to customers via audio using smartphones or smart glasses. Specifically, an audio message about a special sale on large refrigerators is transmitted to the customer, and the customer responds to the proposal. The audio data is then played back as output to provide information to the customer.

[0940] Step 7:

[0941] When a customer sends an inquiry email, the server analyzes its content and generates an automated reply. For example, if a customer asks, "How should I allocate my advertising budget?", the generative AI will automatically generate a specific proposal for budget allocation. It takes the customer's inquiry as input and generates a proposal as output: "We recommend allocating 50% of your advertising budget to social media ads, 30% to search engine ads, and 20% to display ads."

[0942] Step 8:

[0943] The user reviews the automated response sent from the server and makes corrections as needed. Then, the reviewed response is sent to the customer. For example, the user reviews an "advertising budget allocation proposal" and sends it back to the customer if appropriate. The system receives the automated response from the server as input and sends either the corrected or uncorrected response to the customer as output.

[0944] Step 9:

[0945] The user device collects application information from the customer in a chat format and sends it to the server. For example, in response to the question "Please tell me the applicant's name," the user might input "Taro Yamada." The entered application information is sent to the server, which automatically generates an appropriate application form. Finally, the completed application form is sent to the user's email address. Application information is collected as input, and an application form is generated and sent as output.

[0946] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0947] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0948] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0949] [Third Embodiment]

[0950] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0951] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0952] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0953] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0954] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0955] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0956] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0957] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0958] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0959] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0960] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0961] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0962] ---

[0963] This invention is a system designed to streamline digital marketing and data product sales activities for corporate clients. The system comprehensively supports everything from customer information collection and proposal creation to inquiry handling and application processing.

[0964] This system primarily consists of the following elements: a user device, a generative artificial intelligence system, a server, and a voice output function. The specific functions and operation of each element are described below.

[0965] 1. Collection of customer information

[0966] The user (Sales representative) uses a user device such as an iPad during the initial customer visit. The user device sends a request to the server to play a pre-prepared introductory video. This video contains basic information about the services and products being offered. After the video plays, a series of questions are displayed on the user device, and the user answers these questions while interacting with the customer. Each answer is sent to the server in real time, and the server stores this answer data in a database.

[0967] Specific example:

[0968] Question displayed by the user device: "What are your company's main business activities?"

[0969] When a user enters "e-commerce," the user's device sends this information to the server, which then updates the customer database as "e-commerce."

[0970] 2. Generating solutions and talk scripts

[0971] The server uses generative artificial intelligence to generate optimal solutions based on collected customer information and problem data. These solutions include specific suggestions and the reasoning behind them. The generated solutions are also created as talk scripts and used for voice output.

[0972] Specific example:

[0973] Server-generated solution example: "Implementation of an automated inventory management system"

[0974] Example of a server-generated talk script: "For companies like yours that operate e-commerce, implementing an automated inventory management system is highly recommended."

[0975] 3. Audio output of the talk script

[0976] The generated talk script is sent to the user's device and converted from text to audio data (TTS: Text-to-Speech). The converted audio data is played back on the user's device, allowing the user to explain the proposal to the customer verbally.

[0977] 4. Automatic reply to inquiry email

[0978] Later, when a customer sends an inquiry email to the server, the server analyzes its content and automatically generates an appropriate response using generative artificial intelligence. The generated reply email can then be sent to the customer after the user has reviewed it.

[0979] Specific example:

[0980] Customer inquiry: "How should we allocate our advertising budget?"

[0981] Server-generated response: "We recommend allocating 50% of your advertising budget to social media ads, 30% to search engine ads, and 20% to display ads."

[0982] 5. Automatic generation of network configuration proposals and advertising budget allocations.

[0983] The server automatically generates network configuration proposals and advertising budget allocation suggestions based on customer information. This allows for the rapid delivery of optimal proposals tailored to individual customer requirements.

[0984] 6. Input of application information in a chat format and automatic generation of application forms.

[0985] The user device prompts the user to input application information in a chat format. Once information is entered for each field, the server automatically generates an application form based on this information and sends the completed application form to the user device or the user's email address.

[0986] Specific example:

[0987] User device question: "Please tell me the applicant's name."

[0988] When a user enters "Taro Yamada," the server generates an application form that includes "Taro Yamada." Other necessary information is automatically added to the application form.

[0989] The above describes the details of the embodiments for carrying out the present invention. This system automates many processes in sales activities, enabling effective and efficient responses.

[0990] The following describes the processing flow.

[0991] ---

[0992] Step 1:

[0993] The user (Sales representative) activates a user device such as an iPad and opens the dedicated app. When the user instructs the device to play the initial explanation video, the device sends a request to the server.

[0994] Step 2:

[0995] The server receives the request and searches for the initial instructional video data. It prepares the corresponding video file for streaming and sends it to the terminal.

[0996] Step 3:

[0997] The device plays the received video data, and the user provides an initial explanation to the customer. After the video plays, a question-selection input screen is displayed on the device.

[0998] Step 4:

[0999] The user enters their answer to a question displayed on their device. The entered answer is sent to the server in real time.

[1000] Step 5:

[1001] The server saves the received response data to a database. After saving, it automatically generates the next question and sends it to the terminal.

[1002] Step 6:

[1003] This process is repeated to collect detailed customer information and issue data, and the server stores this information in a database.

[1004] Step 7:

[1005] Based on the collected customer information and issue data, the server invokes generative artificial intelligence to generate optimal solutions and talk scripts.

[1006] Step 8:

[1007] The server sends the generated talk script to the terminal. The terminal calls a TTS engine to convert the received talk script from text to speech.

[1008] Step 9:

[1009] The TTS engine converts the talk script into audio data and returns it to the terminal. The terminal plays the generated audio data, and the user explains the proposal to the customer.

[1010] Step 10:

[1011] Later, customer inquiry emails are sent to the server. The server analyzes the content of the inquiry email and automatically generates an appropriate response using generative artificial intelligence.

[1012] Step 11:

[1013] The server provides the generated reply email to the user for confirmation. After the user confirms, the reply email is sent to the customer.

[1014] Step 12:

[1015] The server automatically generates optimal proposal documents for network configuration and advertising budget allocation based on customer information. The generated documents are sent to the user's device or email address.

[1016] Step 13:

[1017] The user device prompts the user to enter application information in a chat format. Once the user enters an answer to each question, the input data is sent to the server.

[1018] Step 14:

[1019] The server automatically generates an application form using a standard format based on the application information it receives. The generated application form is then sent to the terminal or the user's email address.

[1020] Step 15:

[1021] The terminal notifies the user of the generated application form data, which the user then reviews and downloads.

[1022] ---

[1023] The above is an explanation of the program's processing broken down into specific steps.

[1024] (Example 1)

[1025] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1026] In B2B digital marketing and data product sales activities, the processes of collecting customer information, proposing solutions, responding to inquiries, and processing applications are complex and time-consuming, posing a challenge. This can lead to decreased sales efficiency and a decline in the quality of customer service.

[1027] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1028] In this invention, the server includes electronic means for collecting customer information, means for generating solutions based on the customer information using generative artificial intelligence, means for outputting the generated solutions as voice, means for automatically generating replies to inquiry messages, means for automatically generating communication network configuration proposals and budget allocation proposals, and means for collecting application information in a dialogue format and automatically generating application forms. This automates each process necessary for sales activities, such as information gathering, proposals, responses, and application processing, enabling efficient and effective customer service.

[1029] "Electronic devices" refer to electronic devices used for collecting, displaying, and inputting customer information. Specifically, this includes tablets and smartphones.

[1030] "Generative artificial intelligence" refers to artificial intelligence technology that generates solutions and reply messages based on collected customer information. A typical example is a model that performs natural language processing.

[1031] A "solution" refers to specific improvement measures or countermeasures proposed by generative artificial intelligence in response to customer problems and needs.

[1032] "Voice output means" refers to technologies for conveying solutions and other information to users as audio. Specifically, text-to-speech (TTS) functions fall under this category.

[1033] An "inquiry message" refers to an email or message sent by a customer to convey questions or requests.

[1034] A "communication network configuration proposal" refers to a proposal that suggests the optimal communication network configuration tailored to the customer's business needs.

[1035] A "budget allocation proposal" refers to a plan that suggests the most optimal way to allocate a limited budget in advertising activities, etc.

[1036] "Dialogue format" refers to a format in which the user and the system exchange information through a question-and-answer exchange.

[1037] An "application form" refers to a document containing the information necessary for a customer to apply for a service or product.

[1038] This invention is a system designed to streamline digital marketing and data product sales activities for corporate clients. This system comprehensively supports everything from customer information collection and proposal creation to inquiry handling and application processing.

[1039] This system consists of the following elements:

[1040] Electronic devices for collecting customer information

[1041] Generative artificial intelligence

[1042] server

[1043] Audio output function

[1044] Collection of customer information

[1045] The user uses an electronic device such as an iPad during their initial customer visit. The user device sends a request to the server to play a pre-prepared introductory video. This video contains basic information about the services and products being offered. After the video plays, a series of questions are displayed on the user device, and the user answers these questions while interacting with the customer. Each answer is sent to the server in real time, and the server stores this answer data in a database.

[1046] Specific example

[1047] Question displayed by the user device: "What are your company's main business activities?"

[1048] When a user enters "e-commerce," the user's device sends this information to the server, which then updates the customer database as "e-commerce."

[1049] Solution and Talk Script Generation

[1050] The server uses generative artificial intelligence (e.g., OpenAI's GPT-4) to generate optimal solutions based on collected customer information and problem data. These solutions include specific suggestions and the reasoning behind them. The generated solutions are also created as talk scripts and used for voice output.

[1051] Specific example

[1052] Server-generated solution example: "Implementation of an automated inventory management system"

[1053] Example prompt: "Please propose the best IT solution for a new customer. The customer is an e-commerce business."

[1054] Audio output of talk script

[1055] The generated talk script is sent to the user's device and converted from text to audio data (TTS: Text-to-Speech). The converted audio data is played back on the user's device, allowing the user to explain the proposal to the customer verbally.

[1056] Specific example

[1057] Example of a server-generated talk script: "For companies like yours that operate e-commerce, implementing an automated inventory management system is highly recommended."

[1058] Automatic reply to inquiry email

[1059] When a customer sends an inquiry email to the server, the server analyzes its content and automatically generates an appropriate response using generative artificial intelligence. The generated reply email can then be sent to the customer after the user has reviewed it.

[1060] Specific example

[1061] Customer inquiry: "How should we allocate our advertising budget?"

[1062] Server-generated response: "We recommend allocating 50% of your advertising budget to social media ads, 30% to search engine ads, and 20% to display ads."

[1063] Automatic generation of network configuration proposals and advertising budget allocations.

[1064] The server automatically generates network configuration proposals and advertising budget allocation suggestions based on customer information. This allows for the rapid delivery of optimal proposals tailored to individual customer requirements.

[1065] Input of application information in a chat format and automatic generation of application forms.

[1066] The user device prompts the user to input application information in a chat format. Once information is entered for each field, the server automatically generates an application form based on this information and sends the completed application form to the user device or the user's email address.

[1067] Specific example

[1068] User device question: "Please tell me the applicant's name."

[1069] When a user enters "Taro Yamada," the server generates an application form that includes "Taro Yamada." Other necessary information is automatically added to the application form.

[1070] The above describes the details of embodiments for carrying out the present invention. This system automates many processes in sales activities, enabling effective and efficient responses.

[1071] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1072] Step 1: Collecting customer information

[1073] Input: Questions displayed on the user device during the first visit, and customer responses.

[1074] Process: The user uses an electronic device such as an iPad to play an initial explanatory video. After the video plays, a series of questions are displayed on the user's device, and the user answers these questions while interacting with the customer. The input data is sent to the server in real time.

[1075] Output: Customer information stored in the database on the server

[1076] Specific operation: An initial explanatory video is played, followed by the question, "What are your company's main business activities?" If the customer answers "E-commerce," it is recorded as "E-commerce" in the database.

[1077] Step 2: Data analysis and solution generation

[1078] Input: Customer information and issue data collected by the server

[1079] Processing: The server analyzes the collected customer information and generates the optimal solution using generative artificial intelligence (e.g., OpenAI's GPT-4). The generated solution is also created as a talk script.

[1080] Output: Generated solutions and talk scripts

[1081] Specific operation: The server analyzes data related to "e-commerce" and generates the implementation of an automated inventory management system as a solution. Generated talk script: "For companies like yours that operate e-commerce, implementing an automated inventory management system is effective."

[1082] Step 3: Outputting the solution via audio

[1083] Input: Generated talk script

[1084] Processing: The user device converts the talk script received from the server from text to audio data (TTS: Text-to-Speech). It then plays the converted audio data.

[1085] Output: Playback of audio data

[1086] Specific operation: The user device converts the generated talk script into audio data and plays the audio saying, "For companies like yours that operate e-commerce, implementing an automated inventory management system is effective."

[1087] Step 4: Automatic reply to inquiry email

[1088] Input: Customer inquiry email

[1089] Processing: The server receives the inquiry email and analyzes its contents. Using generative artificial intelligence, it automatically generates an appropriate reply. After the user confirms the generated reply email, it is sent to the customer.

[1090] Output: Generated reply email

[1091] Specific operation: A customer sends an email to the server asking, "How should I allocate my advertising budget?" The server automatically generates a reply saying, "We recommend allocating 50% of your advertising budget to social media ads, 30% to search engine ads, and 20% to display ads." The user then reviews and sends the reply.

[1092] Step 5: Automatic generation of network configuration and advertising budget allocation.

[1093] Input: Customer information stored on the server

[1094] Processing: The server analyzes customer information and automatically generates optimal communication network configurations and budget allocation plans for specific requirements.

[1095] Output: Generated network configuration and advertising budget allocation proposal

[1096] Specific operation: The server plans a network configuration based on the customer's business needs and proposes an effective allocation of the advertising budget.

[1097] Step 6: Interactive input of application information and automatic generation of application form.

[1098] Input: Application information entered on the user device

[1099] Processing: The user enters application information interactively through their device. The entered information is sent to the server, which automatically generates an application form based on it. The generated application form is sent to the user's device or to the user's email address.

[1100] Output: Generated application form

[1101] Specific operation: The user device inputs the question "Please tell me the applicant's name," and the user answers "Taro Yamada." The server generates an application form including "Taro Yamada" and sends the final application form to the user device.

[1102] The above describes the specific operations and inputs / outputs at each processing step. This system efficiently manages the processes of information gathering, proposals, responses, and application processing for sales activities.

[1103] (Application Example 1)

[1104] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1105] Traditional digital marketing and data product sales activities have faced problems such as the cumbersome process of collecting customer information, hindering efficient proposal creation, inquiry handling, and application processing. Furthermore, real-time customer support is difficult, resulting in numerous inefficient processes in users' sales activities. In addition, systems designed to automate these processes have difficulty meeting individual requirements and have failed to alleviate the burden on sales representatives.

[1106] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1107] In this invention, the server includes a user device means for collecting customer information, means for generating solutions based on the customer information using generative artificial intelligence, means for outputting the generated solutions as voice, means for automatically generating replies to inquiry emails, means for generating an optimal talk script and outputting it as voice, means for collecting information in real time through voice dialogue with customers, and means for collecting application information in talk format and automatically generating application forms. This makes it possible to efficiently and effectively automate a series of sales processes from collecting customer information to proposing solutions, responding to inquiries, and processing applications. Furthermore, by making real-time customer support easier, it is possible to reduce the burden on sales representatives and contribute to improving customer satisfaction.

[1108] "User device means for collecting customer information" refers to hardware and software used to collect necessary information through interactions and operations with customers.

[1109] "Means for generating solutions based on customer information using generative artificial intelligence" refers to an artificial intelligence model and its operating environment for generating optimal solutions based on collected customer information.

[1110] "Means for outputting generated solutions as audio" refers to a device and program that provides the function of converting solutions generated by a generative artificial intelligence from text to audio and playing it back.

[1111] "Methods for automatically generating replies to inquiry emails" refers to technologies and systems that analyze the content of customer inquiries and automatically generate appropriate reply content.

[1112] "Means for generating optimal talk scripts and outputting them as audio" refers to devices and technologies that create effective talk scripts based on customer information and conversation content, and then convert these scripts into audio to communicate with customers.

[1113] "Means for collecting information in real time through voice interaction with customers" refers to devices and programs that provide functions for analyzing customer interactions in real time using speech recognition technology and collecting necessary information.

[1114] "Methods for collecting application information in a conversational format and automatically generating application forms" refers to technologies and systems for collecting application information from customers in a dialogue format and automatically creating application forms based on that information.

[1115] This invention is a system for streamlining the entire process from customer information collection to proposal creation, inquiry handling, and application processing. This system is particularly applicable to sales support for physical stores utilizing smart glasses. A specific embodiment of this system is described below.

[1116] System Configuration

[1117] This system consists of the following main elements:

[1118] 1. User device means: This is built as smart glasses and collects necessary information through interaction and operation with the customer.

[1119] 2. Generative Artificial Intelligence: Use OpenAI's generative AI models (e.g., the Davinci Codex engine) to generate solutions based on collected customer information.

[1120] 3. Audio output means: Equipped with Text-to-Speech (TTS) functionality to convert the generated solution from text to speech (e.g., pyttsx3 library).

[1121] 4. Inquiry Email Handling Method: Analyze the content of customer inquiry emails and automatically generate reply content.

[1122] 5. Talk script generation and audio output means: It has a function to create an effective talk script based on customer information and conversation content, and output it as audio.

[1123] 6. Real-time information gathering means: Information is collected through voice interactions with customers using speech recognition technology (e.g., SpeechRecognition library).

[1124] 7. Automatic application form generation method: An application form is automatically created based on application information collected from the customer through dialogue.

[1125] Program Processing Description

[1126] 1. Collection of customer information

[1127] The user wears smart glasses and interacts with customers. During the interaction, the microphone and voice recognition technology built into the smart glasses transcribe the customer's speech into text in real time. For example, if a customer says, "We run an online retail business," the voice recognition technology transcribes this information into text and collects it as customer information.

[1128] 2. Generating solutions and talk scripts

[1129] The server uses a generative AI model to generate the optimal solution based on the collected customer information. The generative AI model used here is OpenAI's Davinci-codex engine. For example, if "online retail" is collected as customer information, the generative AI model will generate a solution such as "Implementing a loyalty program is effective for online retailers."

[1130] 3. Audio output of the generated solution

[1131] The server sends the generated solution to the smart glasses, where it is converted into speech via Text-to-Speech technology. For example, the pyttsx3 library is used to voice the solution and play it back to the user. This allows the user to explain the proposal to the customer verbally through the smart glasses.

[1132] 4. Automatic reply to inquiry email

[1133] The server analyzes customer inquiry emails and automatically generates appropriate replies using an AI model based on their content. For example, in response to an inquiry such as "How should I allocate my advertising budget?", it generates a reply such as "We recommend allocating 50% of your advertising budget to social media ads, 30% to search engine ads, and 20% to display ads."

[1134] 5. Collection of application information and automatic generation of application forms

[1135] The user device (smart glasses) collects application information from the customer in an interactive manner, and the server automatically generates an application form based on that information. For example, if the user asks the customer, "What is the applicant's name?", and the customer replies, "Taro Yamada", that information is sent to the server, and "Taro Yamada" is automatically added to the application form.

[1136] Examples of specific cases and prompt statements

[1137] Customer information example:

[1138] 1. Question: "What are your company's main business activities?"

[1139] 2. Customer response: "We operate an online retail business."

[1140] Generated solution:

[1141] 1. Proposal: "In online retail, implementing a loyalty program is effective. This program is expected to stimulate customer purchasing intent and increase repeat customers."

[1142] Example of a prompt:

[1143] Customer Information: We operate an online retail business.

[1144] Please propose the most suitable solution.

[1145] With the above configuration, the present invention can achieve efficient collection of customer information, automatic generation of optimal solutions, rapid and accurate response to inquiries, and automation of application processing. This makes it possible to significantly improve the productivity of sales activities and customer satisfaction.

[1146] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1147] Step 1:

[1148] The user wears smart glasses and interacts with the customer. Using a microphone and speech recognition technology built into the smart glasses, the customer's speech is converted into text data in real time. The input here is the customer's voice information, and the output is customer information in text format. Specifically, if the customer says, "We run an online retail business," that voice is converted into the text data "We run an online retail business."

[1149] Step 2:

[1150] The terminal sends the collected customer information to the server. The server receives this information and uses generative artificial intelligence (e.g., OpenAI's Davinci-codex engine) to generate the optimal solution based on the customer information. The input here is customer information in text format, and the output is the text of the generated solution. For example, if the input is customer information that says "We operate an online retail business," the server will generate the solution that says "Implementing a loyalty program is effective for online retail businesses."

[1151] Step 3:

[1152] The server sends the generated solution to the smart glasses, which then convert it into audio data using Text-to-Speech (TTS) technology (e.g., the pyttsx3 library) and present it to the user. The input here is the text data of the generated solution, and the output is audio data. Specifically, the text data "Implementing a loyalty program is effective in online retail" is converted into audio, which the user can then hear.

[1153] Step 4:

[1154] If a customer makes a follow-up inquiry, the user collects the information through smart glasses and sends it to the server. The server uses generative artificial intelligence to analyze the inquiry and automatically generate an appropriate response. The input here is the text data of the customer's follow-up inquiry, and the output is the automatically generated response text. For example, in response to the inquiry, "How should I allocate my advertising budget?", the system would generate a response such as, "We recommend allocating 50% of your advertising budget to social media ads, 30% to search engine ads, and 20% to display ads."

[1155] Step 5:

[1156] The user collects customer application information in a chat format via smart glasses and sends this information to the server. The server automatically generates an application form based on the received application information. The input here is the text data of the collected application information, and the output is the automatically generated application form. Specifically, if the user asks the customer, "Please tell me the applicant's name," and the customer replies, "Taro Yamada," the name "Taro Yamada" is automatically added to the application form based on that information.

[1157] The above describes the specific processing flow of the system program of the present invention. In each processing step, data processing and data calculations are performed based on the input data, and output data is generated accordingly. This enables efficient collection of customer information, rapid generation of optimal solutions, real-time customer support, and automation of application processing.

[1158] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1159] ---

[1160] This invention is a system for streamlining digital marketing and data product sales activities for corporate clients, and in particular, it comprehensively supports everything from customer information collection and proposal creation to inquiry handling and application processing. Furthermore, by incorporating an emotion engine that recognizes user emotions, it enables more effective dialogue and customer service.

[1161] This system primarily consists of the following elements: user device, generative artificial intelligence, server, voice output function, and emotion engine. The specific functions and operation of each element are described below.

[1162] 1. Collection of customer information

[1163] The user (Sales representative) uses a user device such as an iPad during the initial customer visit. The user device sends a request to the server to play a pre-prepared introductory video. This video contains basic information about the services and products being offered. After the video plays, a series of questions are displayed on the user device, and the user answers these questions while interacting with the customer. Each answer is sent to the server in real time, and the server stores this answer data in a database.

[1164] Specific example:

[1165] Question displayed by the user device: "What are your company's main business activities?"

[1166] When a user enters "e-commerce," the user's device sends this information to the server, which then updates the customer database as "e-commerce."

[1167] 2. Generating solutions and talk scripts

[1168] The server uses generative artificial intelligence to generate optimal solutions based on collected customer information and problem data. These solutions include specific suggestions and the reasons behind them. The generated solutions are also created as talk scripts and used for voice output. At this stage, the emotion engine analyzes the user's emotions and incorporates them into the generated solutions and talk scripts.

[1169] Specific example:

[1170] Server-generated solution example: "Implementation of an automated inventory management system"

[1171] Example of a server-generated talk script: "For companies like yours that operate e-commerce, implementing an automated inventory management system is highly recommended."

[1172] 3. Audio output of the talk script

[1173] The generated talk script is sent to the user's device and converted from text to audio data. During this process, the emotion engine adjusts the tone and speed of the voice according to the user's recognized emotions. This ensures that the suggestions are conveyed in a way that is best suited to the user's current situation.

[1174] Specific example:

[1175] The generated talk script proposes the "implementation of an automated inventory management system," and if the user is nervous, the voice tone is adjusted to be calmer.

[1176] 4. Automatic reply to inquiry email

[1177] Later, when a customer inquiry email is sent to the server, the server analyzes its content and automatically generates an appropriate response using generative artificial intelligence. The generated reply email can then be sent to the customer after the user has reviewed it.

[1178] Specific example:

[1179] Customer inquiry: "How should we allocate our advertising budget?"

[1180] Server-generated response: "We recommend allocating 50% of your advertising budget to social media ads, 30% to search engine ads, and 20% to display ads."

[1181] 5. Automatic generation of network configuration proposals and advertising budget allocations.

[1182] The server automatically generates network configuration proposals and advertising budget allocation suggestions based on customer information. This allows for the rapid delivery of optimal proposals tailored to individual customer requirements.

[1183] 6. Input of application information in a chat format and automatic generation of application forms.

[1184] The user device prompts the user to input application information in a chat format. Once information is entered for each field, the server automatically generates an application form based on this information and sends the completed application form to the user device or the user's email address.

[1185] Specific example:

[1186] User device question: "Please tell me the applicant's name."

[1187] When a user enters "Taro Yamada," the server generates an application form that includes "Taro Yamada." Other necessary information is automatically added to the application form.

[1188] The above describes the details of the embodiments for carrying out the present invention. This system automates many processes in sales activities, enabling effective and efficient responses. Furthermore, the introduction of an emotion engine enables responses that take into account the customer's emotions, contributing to improved customer satisfaction.

[1189] The following describes the processing flow.

[1190] ---

[1191] Step 1:

[1192] The user (Sales representative) activates a user device such as an iPad and opens the dedicated app. When the user instructs the device to play the initial explanation video, the device sends a request to the server.

[1193] Step 2:

[1194] The server receives the request and searches for the initial instructional video data. It prepares the corresponding video file for streaming and sends it to the terminal.

[1195] Step 3:

[1196] The device plays the received video data, and the user provides an initial explanation to the customer. After the video plays, a question-selection input screen is displayed on the device.

[1197] Step 4:

[1198] The user enters their answers to questions displayed on the device. The entered answers are sent to the server in real time. The emotion engine analyzes the user's emotions from facial expressions and voice data, and sends the analysis to the server.

[1199] Step 5:

[1200] The server saves the received response data and sentiment data to a database. After saving, it automatically generates the next question and sends it to the terminal.

[1201] Step 6:

[1202] This process is repeated to collect detailed customer information and issue data, and the server stores this information in a database.

[1203] Step 7:

[1204] The server uses collected customer information and issue data to invoke generative artificial intelligence to generate optimal solutions and talk scripts. The emotion engine then adjusts the generated solutions and talk scripts based on the emotional data it analyzes.

[1205] Step 8:

[1206] The server sends the generated talk script to the terminal. The terminal calls a TTS engine to convert the received talk script from text to speech.

[1207] Step 9:

[1208] The TTS engine converts the talk script into audio data and returns it to the terminal. The terminal plays the generated audio data, and the user explains the proposal to the customer. The tone and speed of the voice are adjusted based on data from the emotion engine.

[1209] Step 10:

[1210] Later, customer inquiry emails are sent to the server. The server analyzes the content of the inquiry email and automatically generates an appropriate response using generative artificial intelligence.

[1211] Step 11:

[1212] The server provides the generated reply email to the user for confirmation. After the user confirms, the reply email is sent to the customer.

[1213] Step 12:

[1214] The server automatically generates optimal proposal documents for network configuration and advertising budget allocation based on customer information. The generated documents are sent to the user's device or email address.

[1215] Step 13:

[1216] The user device prompts the user to enter application information in a chat format. Once the user enters an answer to each question, the input data is sent to the server.

[1217] Step 14:

[1218] The server automatically generates an application form using a standard format based on the application information it receives. The generated application form is then sent to the terminal or the user's email address.

[1219] Step 15:

[1220] The terminal notifies the user of the generated application form data, which the user then reviews and downloads.

[1221] ---

[1222] The above is a detailed explanation of the program processing of a system incorporating an emotion engine, broken down into specific steps.

[1223] (Example 2)

[1224] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1225] In traditional sales activities, many processes, such as collecting customer information, creating proposals, responding to inquiries, and processing applications, relied on manual labor, resulting in time-consuming and labor-intensive tasks. Furthermore, it was difficult to respond in a way that was sensitive to customer emotions, leading to a risk of decreased customer satisfaction. There was a need for a comprehensive system that could solve these problems, streamline sales activities, and improve customer satisfaction.

[1226] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1227] In this invention, the server includes means for generating solutions based on customer information and emotional data, means for converting the generated solutions into voice output and adjusting them based on emotional data, and means for automatically generating replies to inquiry emails. This automates many processes, enabling efficient and effective sales activities. Furthermore, voice output using an emotional engine enables responses that take customer emotions into consideration, which is expected to improve customer satisfaction.

[1228] "Customer information" refers to basic data, needs, and requests regarding users of a service or product.

[1229] "User equipment" refers to hardware devices used by users that have the function of collecting customer information.

[1230] "Generative artificial intelligence" refers to software that generates various solutions and suggestions based on input data, using machine learning and deep learning.

[1231] "Emotional data" refers to information about the emotions and psychological states exhibited by users and customers, and is data that is analyzed by an emotion engine.

[1232] A "solution" refers to the specific proposals and their reasons generated by a generative artificial intelligence system based on the collected customer information.

[1233] "Voice output" is a method of converting text data into audio data and communicating information to users or customers via voice.

[1234] An "inquiry email" is an email sent by a customer containing questions or requests regarding a service or product.

[1235] A "network configuration proposal" is a specific suggestion for optimally designing information systems and communication infrastructure.

[1236] "Advertising budget allocation" refers to a proposal outlining how the advertising budget will be allocated.

[1237] "Application information" refers to the information that a customer provides in order to officially use a service or product.

[1238] An "application form" is an official document automatically generated based on application information and is used to conclude a service or product usage agreement.

[1239] An "emotion engine" is software that analyzes the emotions and psychological state of users and customers and adjusts the system's output based on that analysis.

[1240] A "talk script" is a written document created by a generative artificial intelligence to effectively convey the proposed content, and is used for voice output.

[1241] This invention is a system designed to streamline digital marketing and data product sales activities for corporate clients. In particular, it provides comprehensive support from customer information collection and proposal creation to inquiry handling and application processing. Furthermore, by incorporating an emotion engine that recognizes user and customer emotions, it enables more effective dialogue and customer service.

[1242] System Configuration

[1243] This system mainly consists of the following elements:

[1244] 1. User equipment

[1245] Users use a user device such as a tablet during their initial customer visit. This user device displays a screen for collecting customer information and application information.

[1246] 2. Generative Artificial Intelligence

[1247] The server uses generative artificial intelligence to generate solutions based on collected customer information and sentiment data. This generative AI leverages machine learning and deep learning techniques.

[1248] 3. Emotional Engine

[1249] The emotion engine analyzes user and customer emotional data and adjusts the system's output based on that analysis. This enables more effective communication with customers.

[1250] 4. Server

[1251] The server handles tasks such as storing customer information, running generative artificial intelligence, analyzing emotion engines, automatically responding to inquiry emails, and generating application forms.

[1252] Function and operation of each element

[1253] 1. Collection of customer information

[1254] The user collects customer information using their device. The user device sends a request to the server to play an initial explanatory video. This video contains basic information about the services and products offered. After playing the video, the user enters answers to a series of questions, and these answers are sent to the server in real time. The server stores the answer data in a database.

[1255] Specific example: The user's device displays the question, "What are your company's main business activities?" The user enters "E-commerce," the information is sent to the server, and the customer database is updated.

[1256] 2. Generating solutions

[1257] The server uses generative artificial intelligence to generate optimal solutions based on collected customer information and emotional data. These solutions include specific suggestions and the reasoning behind them. The results of the emotional engine's analysis are also taken into consideration.

[1258] Specific example: Server-generated solution: "Implementation of an automated inventory management system." Proposal content: "Implementation of an automated inventory management system would be effective in improving the efficiency of your e-commerce business."

[1259] 3. Generate talk scripts and audio

[1260] The generated solutions are sent to the user's device and converted from text to audio data. An emotion engine adjusts the tone and speed of the audio, ensuring that the suggestions are delivered in a way that resonates with the customer's emotions.

[1261] Specific example: If the generated talk script proposes "implementing an automated inventory management system" and the user is relaxed, the voice tone will be adjusted to a calmer tone.

[1262] 4. Automatic reply to inquiry email

[1263] When a customer sends an inquiry email to the server, the server analyzes the content and automatically generates an appropriate response using generative artificial intelligence. The generated reply email is then reviewed by the user and sent to the customer.

[1264] Example: Customer inquiry: "How should I allocate my advertising budget?" Server-generated response: "We recommend allocating 50% of your advertising budget to social media ads, 30% to search engine ads, and 20% to display ads."

[1265] 5. Collection of application information and generation of application forms

[1266] The user terminal collects application information in a chat format, and once information is entered for each field, the server automatically generates the application form. The completed application form is sent to the user terminal or email address.

[1267] Specific example: The user terminal asks "Please tell us the applicant's name." The user enters "Taro Yamada," and the server generates an application form that includes "Taro Yamada."

[1268] Example of a prompt

[1269] "Generate proposals for effective automation solutions for e-commerce companies."

[1270] "Please create advice on the optimal allocation of our advertising budget."

[1271] "Please create a talk script that takes customer emotions into consideration."

[1272] As a result, this system automates many processes in sales activities, enabling effective and efficient responses. Furthermore, the introduction of an emotion engine allows for responses that take customer emotions into consideration, contributing to improved customer satisfaction.

[1273] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1274] Step 1:

[1275] Collection of customer information

[1276] The user uses a user device, such as a tablet, during their initial customer visit. The user sends a request from their device to the server, initiating the playback of an initial explanatory video. This video contains basic information about the services and products offered. After the video finishes, a series of questions are displayed on the user's device. The user answers these questions while interacting with the customer. Each answer is sent to the server in real time, and the server stores the answer data in a database.

[1277] Input: Customer information (collected through a questionnaire)

[1278] Data processing: Collection and transmission of response data.

[1279] Output: Update of customer information database

[1280] Specific example:

[1281] Question: "What are your company's main business activities?"

[1282] The user enters "e-commerce," and the information is sent to the server. The server updates the customer database with "e-commerce."

[1283] Step 2:

[1284] Processing customer information

[1285] The server analyzes the received customer response data and extracts the necessary information. Simultaneously, the emotion engine analyzes the emotional data collected during the user-customer interaction. This allows for an understanding of the customer's emotional state.

[1286] Input: Customer response data, sentiment data

[1287] Data processing: Data analysis and extraction

[1288] Output: Updated customer profile, saved sentiment information

[1289] Specific example:

[1290] The server stores the entered "e-commerce" data in the customer profile, and the emotion engine records the customer's emotions detected during the interaction (e.g., relaxed, excited, etc.).

[1291] Step 3:

[1292] Solution generation

[1293] The server uses generative artificial intelligence to generate the optimal solution based on customer information and sentiment data. This solution includes specific suggestions and reasoning. The generated solution is also used as a talk script, and the results of the sentiment engine's analysis are also reflected.

[1294] Input: Customer information, sentiment data

[1295] Data processing: Generating solutions using generative artificial intelligence.

[1296] Output: Solution data

[1297] Specific example:

[1298] Solution: "Implement an automated inventory management system"

[1299] Proposal: "Implementing an automated inventory management system would be an effective way to improve the efficiency of your e-commerce business."

[1300] Step 4:

[1301] Talk script and voice generation

[1302] The generated solution is sent to the user's device, which converts it from text to audio data. The emotion engine then adjusts the tone and speed of the audio based on the user's emotions, which it has analyzed.

[1303] Input: Solution data, sentiment data

[1304] Data processing: Text-to-speech conversion, tone and speed adjustment.

[1305] Output: Audio data

[1306] Specific example:

[1307] The generated talk script proposes the "implementation of an automated inventory management system," and the voice tone is adjusted to a calmer tone if the user is relaxed.

[1308] Step 5:

[1309] Automatic reply to inquiry email

[1310] After a customer inquiry email is sent to the server, the server analyzes its content and automatically generates an appropriate response using generative artificial intelligence. The generated reply email is then reviewed by the user and sent to the customer.

[1311] Input: Inquiry email

[1312] Data processing: Analysis of email content, generation of reply emails.

[1313] Output: Reply email

[1314] Specific example:

[1315] Customer inquiry: "How should we allocate our advertising budget?"

[1316] Generated reply: "We recommend allocating 50% of your advertising budget to social media ads, 30% to search engine ads, and 20% to display ads."

[1317] Step 6:

[1318] Automatic generation of network configuration proposals and advertising budget allocations.

[1319] The server automatically generates network configuration proposals and advertising budget allocation suggestions based on customer information. This allows for the rapid provision of optimal proposals tailored to each customer's requirements.

[1320] Input: Customer Information

[1321] Data processing: Generating proposal content

[1322] Output: Network configuration proposal, advertising budget allocation data

[1323] Specific example:

[1324] Proposal: "The optimal network configuration is to adopt a cloud-based solution and place an independent server at each store."

[1325] Step 7:

[1326] Input of application information in a chat format and automatic generation of application forms.

[1327] The user terminal prompts the user to input application information in a chat format. Once each element is entered, the server automatically generates an application form based on this information and sends the completed application form to the user terminal or email address.

[1328] Input: Application Information

[1329] Data processing: Application form generation

[1330] Output: Application form

[1331] Specific example:

[1332] Question: "Please tell me the applicant's name."

[1333] The user enters "Taro Yamada," and the server generates an application form that includes "Taro Yamada."

[1334] (Application Example 2)

[1335] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1336] Traditional digital marketing and sales support systems could collect customer information and generate proposals, but they had the challenge of not being able to respond in a way that took into account the customer's emotions during the conversation. Furthermore, the inability to adjust voice tone according to the customer's emotions made it difficult to improve customer satisfaction. This, in turn, made it difficult for sales staff in physical stores to provide effective customer service, resulting in challenges in improving sales and customer satisfaction.

[1337] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a user device means for collecting customer information, means for generating solutions based on the customer information using generative artificial intelligence, and means for analyzing customer emotions using an emotion engine and adjusting the content and voice tone generated based on the analysis results. This enables appropriate responses in accordance with customer emotions, realizing effective customer service by sales staff in physical stores, and as a result, an increase in customer satisfaction and sales can be expected.

[1338] "Customer information" refers to all data about a customer, including their name, needs, and emotional state.

[1339] A "user device" is a device used to collect customer information and transmit it to a server, and specifically includes smartphones and smart glasses.

[1340] "Generative artificial intelligence" refers to artificial intelligence that has the ability to generate optimal solutions or talk scripts based on input data.

[1341] "Solutions" refer to specific suggestions or countermeasures generated by generative artificial intelligence based on customer information, aimed at meeting customer needs.

[1342] "Audio output" refers to playing back generated solutions or talk scripts as audio, and includes technologies that convert text data into speech.

[1343] An "inquiry email" is an email sent by a customer to the system containing questions or requests.

[1344] "Automatic reply" refers to the process of analyzing the content of an inquiry email and automatically generating an appropriate response.

[1345] A "network configuration proposal" refers to a proposed layout and design of the network infrastructure that is optimal for the customer's business operations.

[1346] "Advertising budget allocation" refers to a specific plan that shows how the budget used for advertising activities will be distributed.

[1347] "Application information" refers to the information a customer uses to formally apply for a service or product.

[1348] An "application form" is an official document that is automatically generated based on the collected application information.

[1349] An "emotion engine" is a technology that analyzes emotions from customer voice and text and applies the results.

[1350] "Analysis results" refers to the data obtained by the emotion engine from analyzing the customer's emotional state.

[1351] "Voice tone" refers to the pitch, speed, and volume of a voice, and is a factor that greatly influences the impression of the information presented.

[1352] Modes for carrying out the invention

[1353] This invention is a system for supporting sales staff in physical stores, and in particular, it can analyze customer emotions using an emotion engine to effectively engage in dialogue and make suggestions. The system consists of a user device, generative artificial intelligence, an emotion engine, a server voice output function, and related software.

[1354] System Configuration

[1355] 1. User device

[1356] These are devices used for collecting customer information, receiving suggestions, and displaying the results of sentiment analysis. Specifically, this includes smartphones and smart glasses.

[1357] 2. Generative Artificial Intelligence

[1358] This artificial intelligence generates optimal solutions and talk scripts based on customer information. Specifically, it is implemented in Python and can utilize Google Cloud Natural Language and IBM Watson as external APIs.

[1359] 3. Emotional Engine

[1360] This technology analyzes customer emotions from their voice and text, and adjusts the generated content and voice tone based on the analysis results. The emotion analysis utilizes the Google Cloud Natural Language API and the IBM Watson API.

[1361] 4. Server

[1362] This system plays a central role in collecting and storing customer information transmitted from user devices, analyzing it using generative artificial intelligence and emotion engines, and sending the results back to the user devices. Cloud services (e.g., AWS, Google Cloud) are used for the servers.

[1363] 5. Audio output function

[1364] This feature converts server-generated suggestions and talk scripts into audio for playback on the user's device. Specifically, it uses the Google Cloud Text-to-Speech API.

[1365] System operation

[1366] 1. Collection of customer information

[1367] Sales staff use smartphones or smart glasses to input the customer's name and needs. For example, the format might be: "Please enter the customer's name: Taro Yamada" and "Please enter the customer's needs: I'm looking for a large refrigerator for my family."

[1368] The entered information is sent to the server in real time and stored in the database.

[1369] 2. Proposal generation and sentiment analysis

[1370] The server uses generative artificial intelligence to generate the optimal solution based on the collected customer information. Simultaneously, the customer's emotional state is analyzed using an emotion engine.

[1371] For example, if a customer enters "I'm looking for a large refrigerator for my family," the emotion engine will analyze the input and, if it detects a "positive" emotion, it will suggest new products.

[1372] 3. Generating the talk script and outputting the audio.

[1373] The generated solutions are also created as talk scripts and converted into speech using the voice output function. This speech is output in an appropriate tone and speed based on the results of the sentiment analysis.

[1374] For example, if the generated solution is "Special sale information on large refrigerators," the voice tone will be bright and persuasive.

[1375] 4. Automatic reply function

[1376] The server also analyzes customer inquiry emails and uses generative artificial intelligence to generate appropriate automated replies.

[1377] For example, if a customer's email includes a question like, "How should I allocate my advertising budget?", an automated reply will be generated that says, "We recommend allocating 50% of your advertising budget to social media ads, 30% to search engine ads, and 20% to display ads."

[1378] 5. Automatic generation of application information and application form

[1379] Sales staff use user devices to input application information in a chat format, and the server automatically generates an application form based on that information.

[1380] For example, if you enter "Please tell us the applicant's name" and "Taro Yamada," an application form containing "Taro Yamada" will be generated and sent to the user's email address.

[1381] Examples of specific prompt messages

[1382] "Please enter the customer's name: Taro Yamada"

[1383] "Please enter your customer needs: I'm looking for a large refrigerator for my family."

[1384] "My recommended product is a large refrigerator made by LG."

[1385] Thus, this system enables appropriate responses that take customer emotions into consideration, providing effective customer service in physical stores. As a result, improved customer satisfaction and increased sales can be expected.

[1386] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1387] Step 1:

[1388] Users collect customer information using user devices such as smartphones and smart glasses. When a customer visits a store, sales staff enter the customer's name and needs. This information is sent to the server in real time. Specifically, the customer's name "Taro Yamada" and needs "Looking for a large refrigerator for family" are entered. Customer information is collected as input data and stored in the server's database.

[1389] Step 2:

[1390] The server stores customer information sent from the user device in a database. Based on the entered customer information, it calls upon generative artificial intelligence to generate the optimal solution. For example, if the customer needs entered are "I'm looking for a large refrigerator for my family," it will select products and services that meet those needs. The generative AI performs data calculations and outputs "special sale information for large refrigerators" as a solution.

[1391] Step 3:

[1392] The server simultaneously uses an emotion engine to analyze the customer's emotions. It analyzes the customer's emotional state using text and voice data sent from the user's device. Using the customer's response, "I'm looking for a large refrigerator for my family," as input, it uses an emotion analysis API (e.g., Google Cloud Natural Language API) to determine that the emotion is "positive." The analysis results are reflected in the generated solution.

[1393] Step 4:

[1394] The server generates an appropriate talk script based on the generated solutions and the results of sentiment analysis. For example, if the customer is judged to be "positive," a talk script will be created that suggests "sale information on large refrigerators" as a solution. The talk script will be written in a cheerful tone to match the "positive" emotion. Generative artificial intelligence converts the suggestions into a specific talk script and outputs its content.

[1395] Step 5:

[1396] The server invokes a voice output function to convert the generated talk script into audio data. For example, the talk script "Special Sale Information on Large Refrigerators" is converted into audio data using the Google Cloud Text-to-Speech API. The voice tone is adjusted according to the results of sentiment analysis. The talk script is converted into audio data and sent to the user's device.

[1397] Step 6:

[1398] The user device plays audio data transmitted from the server. Sales staff explain proposals to customers via audio using smartphones or smart glasses. Specifically, an audio message about a special sale on large refrigerators is transmitted to the customer, and the customer responds to the proposal. The audio data is then played back as output to provide information to the customer.

[1399] Step 7:

[1400] When a customer sends an inquiry email, the server analyzes its content and generates an automated reply. For example, if a customer asks, "How should I allocate my advertising budget?", the generative AI will automatically generate a specific proposal for budget allocation. It takes the customer's inquiry as input and generates a proposal as output: "We recommend allocating 50% of your advertising budget to social media ads, 30% to search engine ads, and 20% to display ads."

[1401] Step 8:

[1402] The user reviews the automated response sent from the server and makes corrections as needed. Then, the reviewed response is sent to the customer. For example, the user reviews an "advertising budget allocation proposal" and sends it back to the customer if appropriate. The system receives the automated response from the server as input and sends either the corrected or uncorrected response to the customer as output.

[1403] Step 9:

[1404] The user device collects application information from the customer in a chat format and sends it to the server. For example, in response to the question "Please tell me the applicant's name," the user might input "Taro Yamada." The entered application information is sent to the server, which automatically generates an appropriate application form. Finally, the completed application form is sent to the user's email address. Application information is collected as input, and an application form is generated and sent as output.

[1405] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1406] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1407] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[1408] [Fourth Embodiment]

[1409] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[1410] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1411] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1412] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[1413] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1414] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1415] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1416] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[1417] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1418] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1419] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1420] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1421] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1422] ---

[1423] This invention is a system designed to streamline digital marketing and data product sales activities for corporate clients. The system comprehensively supports everything from customer information collection and proposal creation to inquiry handling and application processing.

[1424] This system primarily consists of the following elements: a user device, a generative artificial intelligence system, a server, and a voice output function. The specific functions and operation of each element are described below.

[1425] 1. Collection of customer information

[1426] The user (Sales representative) uses a user device such as an iPad during the initial customer visit. The user device sends a request to the server to play a pre-prepared introductory video. This video contains basic information about the services and products being offered. After the video plays, a series of questions are displayed on the user device, and the user answers these questions while interacting with the customer. Each answer is sent to the server in real time, and the server stores this answer data in a database.

[1427] Specific example:

[1428] Question displayed by the user device: "What are your company's main business activities?"

[1429] When a user enters "e-commerce," the user's device sends this information to the server, which then updates the customer database as "e-commerce."

[1430] 2. Generating solutions and talk scripts

[1431] The server uses generative artificial intelligence to generate optimal solutions based on collected customer information and problem data. These solutions include specific suggestions and the reasoning behind them. The generated solutions are also created as talk scripts and used for voice output.

[1432] Specific example:

[1433] Server-generated solution example: "Implementation of an automated inventory management system"

[1434] Example of a server-generated talk script: "For companies like yours that operate e-commerce, implementing an automated inventory management system is highly recommended."

[1435] 3. Audio output of the talk script

[1436] The generated talk script is sent to the user's device and converted from text to audio data (TTS: Text-to-Speech). The converted audio data is played back on the user's device, allowing the user to explain the proposal to the customer verbally.

[1437] 4. Automatic reply to inquiry email

[1438] Later, when a customer sends an inquiry email to the server, the server analyzes its content and automatically generates an appropriate response using generative artificial intelligence. The generated reply email can then be sent to the customer after the user has reviewed it.

[1439] Specific example:

[1440] Customer inquiry: "How should we allocate our advertising budget?"

[1441] Server-generated response: "We recommend allocating 50% of your advertising budget to social media ads, 30% to search engine ads, and 20% to display ads."

[1442] 5. Automatic generation of network configuration proposals and advertising budget allocations.

[1443] The server automatically generates network configuration proposals and advertising budget allocation suggestions based on customer information. This allows for the rapid delivery of optimal proposals tailored to individual customer requirements.

[1444] 6. Input of application information in a chat format and automatic generation of application forms.

[1445] The user device prompts the user to input application information in a chat format. Once information is entered for each field, the server automatically generates an application form based on this information and sends the completed application form to the user device or the user's email address.

[1446] Specific example:

[1447] User device question: "Please tell me the applicant's name."

[1448] When a user enters "Taro Yamada," the server generates an application form that includes "Taro Yamada." Other necessary information is automatically added to the application form.

[1449] The above describes the details of the embodiments for carrying out the present invention. This system automates many processes in sales activities, enabling effective and efficient responses.

[1450] The following describes the processing flow.

[1451] ---

[1452] Step 1:

[1453] The user (Sales representative) activates a user device such as an iPad and opens the dedicated app. When the user instructs the device to play the initial explanation video, the device sends a request to the server.

[1454] Step 2:

[1455] The server receives the request and searches for the initial instructional video data. It prepares the corresponding video file for streaming and sends it to the terminal.

[1456] Step 3:

[1457] The device plays the received video data, and the user provides an initial explanation to the customer. After the video plays, a question-selection input screen is displayed on the device.

[1458] Step 4:

[1459] The user enters their answer to a question displayed on their device. The entered answer is sent to the server in real time.

[1460] Step 5:

[1461] The server saves the received response data to a database. After saving, it automatically generates the next question and sends it to the terminal.

[1462] Step 6:

[1463] This process is repeated to collect detailed customer information and issue data, and the server stores this information in a database.

[1464] Step 7:

[1465] Based on the collected customer information and issue data, the server invokes generative artificial intelligence to generate optimal solutions and talk scripts.

[1466] Step 8:

[1467] The server sends the generated talk script to the terminal. The terminal calls a TTS engine to convert the received talk script from text to speech.

[1468] Step 9:

[1469] The TTS engine converts the talk script into audio data and returns it to the terminal. The terminal plays the generated audio data, and the user explains the proposal to the customer.

[1470] Step 10:

[1471] Later, customer inquiry emails are sent to the server. The server analyzes the content of the inquiry email and automatically generates an appropriate response using generative artificial intelligence.

[1472] Step 11:

[1473] The server provides the generated reply email to the user for confirmation. After the user confirms, the reply email is sent to the customer.

[1474] Step 12:

[1475] The server automatically generates optimal proposal documents for network configuration and advertising budget allocation based on customer information. The generated documents are sent to the user's device or email address.

[1476] Step 13:

[1477] The user device prompts the user to enter application information in a chat format. Once the user enters an answer to each question, the input data is sent to the server.

[1478] Step 14:

[1479] The server automatically generates an application form using a standard format based on the application information it receives. The generated application form is then sent to the terminal or the user's email address.

[1480] Step 15:

[1481] The terminal notifies the user of the generated application form data, which the user then reviews and downloads.

[1482] ---

[1483] The above is an explanation of the program's processing broken down into specific steps.

[1484] (Example 1)

[1485] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1486] In B2B digital marketing and data product sales activities, the processes of collecting customer information, proposing solutions, responding to inquiries, and processing applications are complex and time-consuming, posing a challenge. This can lead to decreased sales efficiency and a decline in the quality of customer service.

[1487] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1488] In this invention, the server includes electronic means for collecting customer information, means for generating solutions based on the customer information using generative artificial intelligence, means for outputting the generated solutions as voice, means for automatically generating replies to inquiry messages, means for automatically generating communication network configuration proposals and budget allocation proposals, and means for collecting application information in a dialogue format and automatically generating application forms. This automates each process necessary for sales activities, such as information gathering, proposals, responses, and application processing, enabling efficient and effective customer service.

[1489] "Electronic devices" refer to electronic devices used for collecting, displaying, and inputting customer information. Specifically, this includes tablets and smartphones.

[1490] "Generative artificial intelligence" refers to artificial intelligence technology that generates solutions and reply messages based on collected customer information. A typical example is a model that performs natural language processing.

[1491] A "solution" refers to specific improvement measures or countermeasures proposed by generative artificial intelligence in response to customer problems and needs.

[1492] "Voice output means" refers to technologies for conveying solutions and other information to users as audio. Specifically, text-to-speech (TTS) functions fall under this category.

[1493] An "inquiry message" refers to an email or message sent by a customer to convey questions or requests.

[1494] A "communication network configuration proposal" refers to a proposal that suggests the optimal communication network configuration tailored to the customer's business needs.

[1495] A "budget allocation proposal" refers to a plan that suggests the most optimal way to allocate a limited budget in advertising activities, etc.

[1496] "Dialogue format" refers to a format in which the user and the system exchange information through a question-and-answer exchange.

[1497] An "application form" refers to a document containing the information necessary for a customer to apply for a service or product.

[1498] This invention is a system designed to streamline digital marketing and data product sales activities for corporate clients. The system comprehensively supports everything from customer information collection and proposal creation to inquiry handling and application processing.

[1499] This system consists of the following elements:

[1500] Electronic devices for collecting customer information

[1501] Generative artificial intelligence

[1502] server

[1503] Audio output function

[1504] Collection of customer information

[1505] The user uses an electronic device such as an iPad during their initial customer visit. The user device sends a request to the server to play a pre-prepared introductory video. This video contains basic information about the services and products being offered. After the video plays, a series of questions are displayed on the user device, and the user answers these questions while interacting with the customer. Each answer is sent to the server in real time, and the server stores this answer data in a database.

[1506] Specific example

[1507] Question displayed by the user device: "What are your company's main business activities?"

[1508] When a user enters "e-commerce," the user's device sends this information to the server, which then updates the customer database as "e-commerce."

[1509] Solution and Talk Script Generation

[1510] The server uses generative artificial intelligence (e.g., OpenAI's GPT-4) to generate optimal solutions based on collected customer information and problem data. These solutions include specific suggestions and the reasoning behind them. The generated solutions are also created as talk scripts and used for voice output.

[1511] Specific example

[1512] Server-generated solution example: "Implementation of an automated inventory management system"

[1513] Example prompt: "Please propose the best IT solution for a new customer. The customer is an e-commerce business."

[1514] Audio output of talk script

[1515] The generated talk script is sent to the user's device and converted from text to audio data (TTS: Text-to-Speech). The converted audio data is played back on the user's device, allowing the user to explain the proposal to the customer verbally.

[1516] Specific example

[1517] Example of a server-generated talk script: "For companies like yours that operate e-commerce, implementing an automated inventory management system is highly recommended."

[1518] Automatic reply to inquiry email

[1519] When a customer sends an inquiry email to the server, the server analyzes its content and automatically generates an appropriate response using generative artificial intelligence. The generated reply email can then be sent to the customer after the user has reviewed it.

[1520] Specific example

[1521] Customer inquiry: "How should we allocate our advertising budget?"

[1522] Server-generated response: "We recommend allocating 50% of your advertising budget to social media ads, 30% to search engine ads, and 20% to display ads."

[1523] Automatic generation of network configuration proposals and advertising budget allocations.

[1524] The server automatically generates network configuration proposals and advertising budget allocation suggestions based on customer information. This allows for the rapid delivery of optimal proposals tailored to individual customer requirements.

[1525] Input of application information in a chat format and automatic generation of application forms.

[1526] The user device prompts the user to input application information in a chat format. Once information is entered for each field, the server automatically generates an application form based on this information and sends the completed application form to the user device or the user's email address.

[1527] Specific example

[1528] User device question: "Please tell me the applicant's name."

[1529] When a user enters "Taro Yamada," the server generates an application form that includes "Taro Yamada." Other necessary information is automatically added to the application form.

[1530] The above describes the details of embodiments for carrying out the present invention. This system automates many processes in sales activities, enabling effective and efficient responses.

[1531] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1532] Step 1: Collecting customer information

[1533] Input: Questions displayed on the user device during the first visit, and customer responses.

[1534] Process: The user uses an electronic device such as an iPad to play an initial explanatory video. After the video plays, a series of questions are displayed on the user's device, and the user answers these questions while interacting with the customer. The input data is sent to the server in real time.

[1535] Output: Customer information stored in the database on the server

[1536] Specific operation: An initial explanatory video is played, followed by the question, "What are your company's main business activities?" If the customer answers "E-commerce," it is recorded as "E-commerce" in the database.

[1537] Step 2: Data analysis and solution generation

[1538] Input: Customer information and issue data collected by the server

[1539] Processing: The server analyzes the collected customer information and generates the optimal solution using generative artificial intelligence (e.g., OpenAI's GPT-4). The generated solution is also created as a talk script.

[1540] Output: Generated solutions and talk scripts

[1541] Specific operation: The server analyzes data related to "e-commerce" and generates the implementation of an automated inventory management system as a solution. Generated talk script: "For companies like yours that operate e-commerce, implementing an automated inventory management system is effective."

[1542] Step 3: Outputting the solution via audio

[1543] Input: Generated talk script

[1544] Processing: The user device converts the talk script received from the server from text to audio data (TTS: Text-to-Speech). It then plays the converted audio data.

[1545] Output: Playback of audio data

[1546] Specific operation: The user device converts the generated talk script into audio data and plays the audio saying, "For companies like yours that operate e-commerce, implementing an automated inventory management system is effective."

[1547] Step 4: Automatic reply to inquiry email

[1548] Input: Customer inquiry email

[1549] Processing: The server receives the inquiry email and analyzes its contents. Using generative artificial intelligence, it automatically generates an appropriate reply. After the user confirms the generated reply email, it is sent to the customer.

[1550] Output: Generated reply email

[1551] Specific operation: A customer sends an email to the server asking, "How should I allocate my advertising budget?" The server automatically generates a reply saying, "We recommend allocating 50% of your advertising budget to social media ads, 30% to search engine ads, and 20% to display ads." The user then reviews and sends the reply.

[1552] Step 5: Automatic generation of network configuration and advertising budget allocation.

[1553] Input: Customer information stored on the server

[1554] Processing: The server analyzes customer information and automatically generates optimal communication network configurations and budget allocation plans for specific requirements.

[1555] Output: Generated network configuration and advertising budget allocation proposal

[1556] Specific operation: The server plans a network configuration based on the customer's business needs and proposes an effective allocation of the advertising budget.

[1557] Step 6: Interactive input of application information and automatic generation of application form.

[1558] Input: Application information entered on the user device

[1559] Processing: The user enters application information interactively through their device. The entered information is sent to the server, which automatically generates an application form based on it. The generated application form is sent to the user's device or to the user's email address.

[1560] Output: Generated application form

[1561] Specific operation: The user device inputs the question "Please tell me the applicant's name," and the user answers "Taro Yamada." The server generates an application form including "Taro Yamada" and sends the final application form to the user device.

[1562] The above describes the specific operations and inputs / outputs at each processing step. This system efficiently manages the processes of information gathering, proposals, responses, and application processing for sales activities.

[1563] (Application Example 1)

[1564] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1565] Traditional digital marketing and data product sales activities have faced problems such as the cumbersome process of collecting customer information, hindering efficient proposal creation, inquiry handling, and application processing. Furthermore, real-time customer support is difficult, resulting in numerous inefficient processes in users' sales activities. In addition, systems designed to automate these processes have difficulty meeting individual requirements and have failed to alleviate the burden on sales representatives.

[1566] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1567] In this invention, the server includes a user device means for collecting customer information, means for generating solutions based on the customer information using generative artificial intelligence, means for outputting the generated solutions as voice, means for automatically generating replies to inquiry emails, means for generating an optimal talk script and outputting it as voice, means for collecting information in real time through voice dialogue with customers, and means for collecting application information in talk format and automatically generating application forms. This makes it possible to efficiently and effectively automate a series of sales processes from collecting customer information to proposing solutions, responding to inquiries, and processing applications. Furthermore, by making real-time customer support easier, it is possible to reduce the burden on sales representatives and contribute to improving customer satisfaction.

[1568] "User device means for collecting customer information" refers to hardware and software used to collect necessary information through interactions and operations with customers.

[1569] "Means for generating solutions based on customer information using generative artificial intelligence" refers to an artificial intelligence model and its operating environment for generating optimal solutions based on collected customer information.

[1570] "Means for outputting generated solutions as audio" refers to a device and program that provides the function of converting solutions generated by a generative artificial intelligence from text to audio and playing it back.

[1571] "Methods for automatically generating replies to inquiry emails" refers to technologies and systems that analyze the content of customer inquiries and automatically generate appropriate reply content.

[1572] "Means for generating optimal talk scripts and outputting them as audio" refers to devices and technologies that create effective talk scripts based on customer information and conversation content, and then convert these scripts into audio to communicate with customers.

[1573] "Means for collecting information in real time through voice interaction with customers" refers to devices and programs that provide functions for analyzing customer interactions in real time using speech recognition technology and collecting necessary information.

[1574] "Methods for collecting application information in a conversational format and automatically generating application forms" refers to technologies and systems for collecting application information from customers in a dialogue format and automatically creating application forms based on that information.

[1575] This invention is a system for streamlining the entire process from customer information collection to proposal creation, inquiry handling, and application processing. This system is particularly applicable to sales support for physical stores utilizing smart glasses. A specific embodiment of this system is described below.

[1576] System Configuration

[1577] This system consists of the following main elements:

[1578] 1. User device means: This is built as smart glasses and collects necessary information through interaction and operation with the customer.

[1579] 2. Generative Artificial Intelligence: Use OpenAI's generative AI models (e.g., the Davinci Codex engine) to generate solutions based on collected customer information.

[1580] 3. Audio output means: Equipped with Text-to-Speech (TTS) functionality to convert the generated solution from text to speech (e.g., pyttsx3 library).

[1581] 4. Inquiry Email Handling Method: Analyze the content of customer inquiry emails and automatically generate reply content.

[1582] 5. Talk script generation and audio output means: It has a function to create an effective talk script based on customer information and conversation content, and output it as audio.

[1583] 6. Real-time information gathering means: Information is collected through voice interactions with customers using speech recognition technology (e.g., SpeechRecognition library).

[1584] 7. Automatic application form generation method: An application form is automatically created based on application information collected from the customer through dialogue.

[1585] Program Processing Description

[1586] 1. Collection of customer information

[1587] The user wears smart glasses and interacts with customers. During the interaction, the microphone and voice recognition technology built into the smart glasses transcribe the customer's speech into text in real time. For example, if a customer says, "We run an online retail business," the voice recognition technology transcribes this information into text and collects it as customer information.

[1588] 2. Generating solutions and talk scripts

[1589] The server uses a generative AI model to generate the optimal solution based on the collected customer information. The generative AI model used here is OpenAI's Davinci-codex engine. For example, if "online retail" is collected as customer information, the generative AI model will generate a solution such as "Implementing a loyalty program is effective for online retailers."

[1590] 3. Audio output of the generated solution

[1591] The server sends the generated solution to the smart glasses, where it is converted into speech via Text-to-Speech technology. For example, the pyttsx3 library is used to voice the solution and play it back to the user. This allows the user to explain the proposal to the customer verbally through the smart glasses.

[1592] 4. Automatic reply to inquiry email

[1593] The server analyzes customer inquiry emails and automatically generates appropriate replies using an AI model based on their content. For example, in response to an inquiry such as "How should I allocate my advertising budget?", it generates a reply such as "We recommend allocating 50% of your advertising budget to social media ads, 30% to search engine ads, and 20% to display ads."

[1594] 5. Collection of application information and automatic generation of application forms

[1595] The user device (smart glasses) collects application information from the customer in an interactive manner, and the server automatically generates an application form based on that information. For example, if the user asks the customer, "What is the applicant's name?", and the customer replies, "Taro Yamada", that information is sent to the server, and "Taro Yamada" is automatically added to the application form.

[1596] Examples of specific cases and prompt statements

[1597] Customer information example:

[1598] 1. Question: "What are your company's main business activities?"

[1599] 2. Customer response: "We operate an online retail business."

[1600] Generated solution:

[1601] 1. Proposal: "In online retail, implementing a loyalty program is effective. This program is expected to stimulate customer purchasing intent and increase repeat customers."

[1602] Example of a prompt:

[1603] Customer Information: We operate an online retail business.

[1604] Please propose the most suitable solution.

[1605] With the above configuration, the present invention can achieve efficient collection of customer information, automatic generation of optimal solutions, rapid and accurate response to inquiries, and automation of application processing. This makes it possible to significantly improve the productivity of sales activities and customer satisfaction.

[1606] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1607] Step 1:

[1608] The user wears smart glasses and interacts with the customer. Using a microphone and speech recognition technology built into the smart glasses, the customer's speech is converted into text data in real time. The input here is the customer's voice information, and the output is customer information in text format. Specifically, if the customer says, "We run an online retail business," that voice is converted into the text data "We run an online retail business."

[1609] Step 2:

[1610] The terminal sends the collected customer information to the server. The server receives this information and uses generative artificial intelligence (e.g., OpenAI's Davinci-codex engine) to generate the optimal solution based on the customer information. The input here is customer information in text format, and the output is the text of the generated solution. For example, if the input is customer information that says "We operate an online retail business," the server will generate the solution that says "Implementing a loyalty program is effective for online retail businesses."

[1611] Step 3:

[1612] The server sends the generated solution to the smart glasses, which then convert it into audio data using Text-to-Speech (TTS) technology (e.g., the pyttsx3 library) and present it to the user. The input here is the text data of the generated solution, and the output is audio data. Specifically, the text data "Implementing a loyalty program is effective in online retail" is converted into audio, which the user can then hear.

[1613] Step 4:

[1614] If a customer makes a follow-up inquiry, the user collects the information through smart glasses and sends it to the server. The server uses generative artificial intelligence to analyze the inquiry and automatically generate an appropriate response. The input here is the text data of the customer's follow-up inquiry, and the output is the automatically generated response text. For example, in response to the inquiry, "How should I allocate my advertising budget?", the system would generate a response such as, "We recommend allocating 50% of your advertising budget to social media ads, 30% to search engine ads, and 20% to display ads."

[1615] Step 5:

[1616] The user collects customer application information in a chat format via smart glasses and sends this information to the server. The server automatically generates an application form based on the received application information. The input here is the text data of the collected application information, and the output is the automatically generated application form. Specifically, if the user asks the customer, "Please tell me the applicant's name," and the customer replies, "Taro Yamada," the name "Taro Yamada" is automatically added to the application form based on that information.

[1617] The above describes the specific processing flow of the system program of the present invention. In each processing step, data processing and data calculations are performed based on the input data, and output data is generated accordingly. This enables efficient collection of customer information, rapid generation of optimal solutions, real-time customer support, and automation of application processing.

[1618] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1619] ---

[1620] This invention is a system for streamlining digital marketing and data product sales activities for corporate clients, and in particular, it comprehensively supports everything from customer information collection and proposal creation to inquiry handling and application processing. Furthermore, by incorporating an emotion engine that recognizes user emotions, it enables more effective dialogue and customer service.

[1621] This system primarily consists of the following elements: user device, generative artificial intelligence, server, voice output function, and emotion engine. The specific functions and operation of each element are described below.

[1622] 1. Collection of customer information

[1623] The user (Sales representative) uses a user device such as an iPad during the initial customer visit. The user device sends a request to the server to play a pre-prepared introductory video. This video contains basic information about the services and products being offered. After the video plays, a series of questions are displayed on the user device, and the user answers these questions while interacting with the customer. Each answer is sent to the server in real time, and the server stores this answer data in a database.

[1624] Specific example:

[1625] Question displayed by the user device: "What are your company's main business activities?"

[1626] When a user enters "e-commerce," the user's device sends this information to the server, which then updates the customer database as "e-commerce."

[1627] 2. Generating solutions and talk scripts

[1628] The server uses generative artificial intelligence to generate optimal solutions based on collected customer information and problem data. These solutions include specific suggestions and the reasons behind them. The generated solutions are also created as talk scripts and used for voice output. At this stage, the emotion engine analyzes the user's emotions and incorporates them into the generated solutions and talk scripts.

[1629] Specific example:

[1630] Server-generated solution example: "Implementation of an automated inventory management system"

[1631] Example of a server-generated talk script: "For companies like yours that operate e-commerce, implementing an automated inventory management system is highly recommended."

[1632] 3. Audio output of the talk script

[1633] The generated talk script is sent to the user's device and converted from text to audio data. During this process, the emotion engine adjusts the tone and speed of the voice according to the user's recognized emotions. This ensures that the suggestions are conveyed in a way that is best suited to the user's current situation.

[1634] Specific example:

[1635] The generated talk script proposes the "implementation of an automated inventory management system," and if the user is nervous, the voice tone is adjusted to be calmer.

[1636] 4. Automatic reply to inquiry email

[1637] Later, when a customer inquiry email is sent to the server, the server analyzes its content and automatically generates an appropriate response using generative artificial intelligence. The generated reply email can then be sent to the customer after the user has reviewed it.

[1638] Specific example:

[1639] Customer inquiry: "How should we allocate our advertising budget?"

[1640] Server-generated response: "We recommend allocating 50% of your advertising budget to social media ads, 30% to search engine ads, and 20% to display ads."

[1641] 5. Automatic generation of network configuration proposals and advertising budget allocations.

[1642] The server automatically generates network configuration proposals and advertising budget allocation suggestions based on customer information. This allows for the rapid delivery of optimal proposals tailored to individual customer requirements.

[1643] 6. Input of application information in a chat format and automatic generation of application forms.

[1644] The user device prompts the user to input application information in a chat format. Once information is entered for each field, the server automatically generates an application form based on this information and sends the completed application form to the user device or the user's email address.

[1645] Specific example:

[1646] User device question: "Please tell me the applicant's name."

[1647] When a user enters "Taro Yamada," the server generates an application form that includes "Taro Yamada." Other necessary information is automatically added to the application form.

[1648] The above describes the details of the embodiments for carrying out the present invention. This system automates many processes in sales activities, enabling effective and efficient responses. Furthermore, the introduction of an emotion engine enables responses that take into account the customer's emotions, contributing to improved customer satisfaction.

[1649] The following describes the processing flow.

[1650] ---

[1651] Step 1:

[1652] The user (Sales representative) activates a user device such as an iPad and opens the dedicated app. When the user instructs the device to play the initial explanation video, the device sends a request to the server.

[1653] Step 2:

[1654] The server receives the request and searches for the initial instructional video data. It prepares the corresponding video file for streaming and sends it to the terminal.

[1655] Step 3:

[1656] The device plays the received video data, and the user provides an initial explanation to the customer. After the video plays, a question-selection input screen is displayed on the device.

[1657] Step 4:

[1658] The user enters their answers to questions displayed on the device. The entered answers are sent to the server in real time. The emotion engine analyzes the user's emotions from facial expressions and voice data, and sends the analysis to the server.

[1659] Step 5:

[1660] The server saves the received response data and sentiment data to a database. After saving, it automatically generates the next question and sends it to the terminal.

[1661] Step 6:

[1662] This process is repeated to collect detailed customer information and issue data, and the server stores this information in a database.

[1663] Step 7:

[1664] The server uses collected customer information and issue data to invoke generative artificial intelligence to generate optimal solutions and talk scripts. The emotion engine then adjusts the generated solutions and talk scripts based on the emotional data it analyzes.

[1665] Step 8:

[1666] The server sends the generated talk script to the terminal. The terminal calls a TTS engine to convert the received talk script from text to speech.

[1667] Step 9:

[1668] The TTS engine converts the talk script into audio data and returns it to the terminal. The terminal plays the generated audio data, and the user explains the proposal to the customer. The tone and speed of the voice are adjusted based on data from the emotion engine.

[1669] Step 10:

[1670] Later, customer inquiry emails are sent to the server. The server analyzes the content of the inquiry email and automatically generates an appropriate response using generative artificial intelligence.

[1671] Step 11:

[1672] The server provides the generated reply email to the user for confirmation. After the user confirms, the reply email is sent to the customer.

[1673] Step 12:

[1674] The server automatically generates optimal proposal documents for network configuration and advertising budget allocation based on customer information. The generated documents are sent to the user's device or email address.

[1675] Step 13:

[1676] The user device prompts the user to enter application information in a chat format. Once the user enters an answer to each question, the input data is sent to the server.

[1677] Step 14:

[1678] The server automatically generates an application form using a standard format based on the application information it receives. The generated application form is then sent to the terminal or the user's email address.

[1679] Step 15:

[1680] The terminal notifies the user of the generated application form data, which the user then reviews and downloads.

[1681] ---

[1682] The above is a detailed explanation of the program processing of a system incorporating an emotion engine, broken down into specific steps.

[1683] (Example 2)

[1684] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1685] In traditional sales activities, many processes, such as collecting customer information, creating proposals, responding to inquiries, and processing applications, relied on manual labor, resulting in time-consuming and labor-intensive tasks. Furthermore, it was difficult to respond in a way that was sensitive to customer emotions, leading to a risk of decreased customer satisfaction. There was a need for a comprehensive system that could solve these problems, streamline sales activities, and improve customer satisfaction.

[1686] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1687] In this invention, the server includes means for generating solutions based on customer information and emotional data, means for converting the generated solutions into voice output and adjusting them based on emotional data, and means for automatically generating replies to inquiry emails. This automates many processes, enabling efficient and effective sales activities. Furthermore, voice output using an emotional engine enables responses that take customer emotions into consideration, which is expected to improve customer satisfaction.

[1688] "Customer information" refers to basic data, needs, and requests regarding users of a service or product.

[1689] "User equipment" refers to hardware devices used by users that have the function of collecting customer information.

[1690] "Generative artificial intelligence" refers to software that generates various solutions and suggestions based on input data, using machine learning and deep learning.

[1691] "Emotional data" refers to information about the emotions and psychological states exhibited by users and customers, and is data that is analyzed by an emotion engine.

[1692] A "solution" refers to the specific proposals and their reasons generated by a generative artificial intelligence system based on the collected customer information.

[1693] "Voice output" is a method of converting text data into audio data and communicating information to users or customers via voice.

[1694] An "inquiry email" is an email sent by a customer containing questions or requests regarding a service or product.

[1695] A "network configuration proposal" is a specific suggestion for optimally designing information systems and communication infrastructure.

[1696] "Advertising budget allocation" refers to a proposal outlining how the advertising budget will be allocated.

[1697] "Application information" refers to the information that a customer provides in order to officially use a service or product.

[1698] An "application form" is an official document automatically generated based on application information and is used to conclude a service or product usage agreement.

[1699] An "emotion engine" is software that analyzes the emotions and psychological state of users and customers and adjusts the system's output based on that analysis.

[1700] A "talk script" is a written document created by a generative artificial intelligence to effectively convey the proposed content, and is used for voice output.

[1701] This invention is a system designed to streamline digital marketing and data product sales activities for corporate clients. In particular, it provides comprehensive support from customer information collection and proposal creation to inquiry handling and application processing. Furthermore, by incorporating an emotion engine that recognizes user and customer emotions, it enables more effective dialogue and customer service.

[1702] System Configuration

[1703] This system mainly consists of the following elements:

[1704] 1. User equipment

[1705] Users use a user device such as a tablet during their initial customer visit. This user device displays a screen for collecting customer information and application information.

[1706] 2. Generative Artificial Intelligence

[1707] The server uses generative artificial intelligence to generate solutions based on collected customer information and sentiment data. This generative AI leverages machine learning and deep learning techniques.

[1708] 3. Emotional Engine

[1709] The emotion engine analyzes user and customer emotional data and adjusts the system's output based on that analysis. This enables more effective communication with customers.

[1710] 4. Server

[1711] The server handles tasks such as storing customer information, running generative artificial intelligence, analyzing emotion engines, automatically responding to inquiry emails, and generating application forms.

[1712] Function and operation of each element

[1713] 1. Collection of customer information

[1714] The user collects customer information using their device. The user device sends a request to the server to play an initial explanatory video. This video contains basic information about the services and products offered. After playing the video, the user enters answers to a series of questions, and these answers are sent to the server in real time. The server stores the answer data in a database.

[1715] Specific example: The user's device displays the question, "What are your company's main business activities?" The user enters "E-commerce," the information is sent to the server, and the customer database is updated.

[1716] 2. Generating solutions

[1717] The server uses generative artificial intelligence to generate optimal solutions based on collected customer information and emotional data. These solutions include specific suggestions and the reasoning behind them. The results of the emotional engine's analysis are also taken into consideration.

[1718] Specific example: Server-generated solution: "Implementation of an automated inventory management system." Proposal content: "Implementation of an automated inventory management system would be effective in improving the efficiency of your e-commerce business."

[1719] 3. Generate talk scripts and audio

[1720] The generated solutions are sent to the user's device and converted from text to audio data. An emotion engine adjusts the tone and speed of the audio, ensuring that the suggestions are delivered in a way that resonates with the customer's emotions.

[1721] Specific example: If the generated talk script proposes "implementing an automated inventory management system" and the user is relaxed, the voice tone will be adjusted to a calmer tone.

[1722] 4. Automatic reply to inquiry email

[1723] When a customer sends an inquiry email to the server, the server analyzes the content and automatically generates an appropriate response using generative artificial intelligence. The generated reply email is then reviewed by the user and sent to the customer.

[1724] Example: Customer inquiry: "How should I allocate my advertising budget?" Server-generated response: "We recommend allocating 50% of your advertising budget to social media ads, 30% to search engine ads, and 20% to display ads."

[1725] 5. Collection of application information and generation of application forms

[1726] The user terminal collects application information in a chat format, and once information is entered for each field, the server automatically generates the application form. The completed application form is sent to the user terminal or email address.

[1727] Specific example: The user terminal asks "Please tell us the applicant's name." The user enters "Taro Yamada," and the server generates an application form that includes "Taro Yamada."

[1728] Example of a prompt

[1729] "Generate proposals for effective automation solutions for e-commerce companies."

[1730] "Please create advice on the optimal allocation of our advertising budget."

[1731] "Please create a talk script that takes customer emotions into consideration."

[1732] As a result, this system automates many processes in sales activities, enabling effective and efficient responses. Furthermore, the introduction of an emotion engine allows for responses that take customer emotions into consideration, contributing to improved customer satisfaction.

[1733] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1734] Step 1:

[1735] Collection of customer information

[1736] The user uses a user device, such as a tablet, during their initial customer visit. The user sends a request from their device to the server, initiating the playback of an initial explanatory video. This video contains basic information about the services and products offered. After the video finishes, a series of questions are displayed on the user's device. The user answers these questions while interacting with the customer. Each answer is sent to the server in real time, and the server stores the answer data in a database.

[1737] Input: Customer information (collected through a questionnaire)

[1738] Data processing: Collection and transmission of response data.

[1739] Output: Update of customer information database

[1740] Specific example:

[1741] Question: "What are your company's main business activities?"

[1742] The user enters "e-commerce," and the information is sent to the server. The server updates the customer database with "e-commerce."

[1743] Step 2:

[1744] Processing customer information

[1745] The server analyzes the received customer response data and extracts the necessary information. Simultaneously, the emotion engine analyzes the emotional data collected during the user-customer interaction. This allows for an understanding of the customer's emotional state.

[1746] Input: Customer response data, sentiment data

[1747] Data processing: Data analysis and extraction

[1748] Output: Updated customer profile, saved sentiment information

[1749] Specific example:

[1750] The server stores the entered "e-commerce" data in the customer profile, and the emotion engine records the customer's emotions detected during the interaction (e.g., relaxed, excited, etc.).

[1751] Step 3:

[1752] Solution generation

[1753] The server uses generative artificial intelligence to generate the optimal solution based on customer information and sentiment data. This solution includes specific suggestions and reasoning. The generated solution is also used as a talk script, and the results of the sentiment engine's analysis are also reflected.

[1754] Input: Customer information, sentiment data

[1755] Data processing: Generating solutions using generative artificial intelligence.

[1756] Output: Solution data

[1757] Specific example:

[1758] Solution: "Implement an automated inventory management system"

[1759] Proposal: "Implementing an automated inventory management system would be an effective way to improve the efficiency of your e-commerce business."

[1760] Step 4:

[1761] Talk script and voice generation

[1762] The generated solution is sent to the user's device, which converts it from text to audio data. The emotion engine then adjusts the tone and speed of the audio based on the user's emotions, which it has analyzed.

[1763] Input: Solution data, sentiment data

[1764] Data processing: Text-to-speech conversion, tone and speed adjustment.

[1765] Output: Audio data

[1766] Specific example:

[1767] The generated talk script proposes the "implementation of an automated inventory management system," and the voice tone is adjusted to a calmer tone if the user is relaxed.

[1768] Step 5:

[1769] Automatic reply to inquiry email

[1770] After a customer inquiry email is sent to the server, the server analyzes its content and automatically generates an appropriate response using generative artificial intelligence. The generated reply email is then reviewed by the user and sent to the customer.

[1771] Input: Inquiry email

[1772] Data processing: Analysis of email content, generation of reply emails.

[1773] Output: Reply email

[1774] Specific example:

[1775] Customer inquiry: "How should we allocate our advertising budget?"

[1776] Generated reply: "We recommend allocating 50% of your advertising budget to social media ads, 30% to search engine ads, and 20% to display ads."

[1777] Step 6:

[1778] Automatic generation of network configuration proposals and advertising budget allocations.

[1779] The server automatically generates network configuration proposals and advertising budget allocation suggestions based on customer information. This allows for the rapid provision of optimal proposals tailored to each customer's requirements.

[1780] Input: Customer Information

[1781] Data processing: Generating proposal content

[1782] Output: Network configuration proposal, advertising budget allocation data

[1783] Specific example:

[1784] Proposal: "The optimal network configuration is to adopt a cloud-based solution and place an independent server at each store."

[1785] Step 7:

[1786] Input of application information in a chat format and automatic generation of application forms.

[1787] The user terminal prompts the user to input application information in a chat format. Once each element is entered, the server automatically generates an application form based on this information and sends the completed application form to the user terminal or email address.

[1788] Input: Application Information

[1789] Data processing: Application form generation

[1790] Output: Application form

[1791] Specific example:

[1792] Question: "Please tell me the applicant's name."

[1793] The user enters "Taro Yamada," and the server generates an application form that includes "Taro Yamada."

[1794] (Application Example 2)

[1795] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1796] Traditional digital marketing and sales support systems could collect customer information and generate proposals, but they had the challenge of not being able to respond in a way that took into account the customer's emotions during the conversation. Furthermore, the inability to adjust voice tone according to the customer's emotions made it difficult to improve customer satisfaction. This, in turn, made it difficult for sales staff in physical stores to provide effective customer service, resulting in challenges in improving sales and customer satisfaction.

[1797] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a user device means for collecting customer information, means for generating solutions based on the customer information using generative artificial intelligence, and means for analyzing customer emotions using an emotion engine and adjusting the content and voice tone generated based on the analysis results. This enables appropriate responses in accordance with customer emotions, realizing effective customer service by sales staff in physical stores, and as a result, an increase in customer satisfaction and sales can be expected.

[1798] "Customer information" refers to all data about a customer, including their name, needs, and emotional state.

[1799] A "user device" is a device used to collect customer information and transmit it to a server, and specifically includes smartphones and smart glasses.

[1800] "Generative artificial intelligence" refers to artificial intelligence that has the ability to generate optimal solutions or talk scripts based on input data.

[1801] "Solutions" refer to specific suggestions or countermeasures generated by generative artificial intelligence based on customer information, aimed at meeting customer needs.

[1802] "Audio output" refers to playing back generated solutions or talk scripts as audio, and includes technologies that convert text data into speech.

[1803] An "inquiry email" is an email sent by a customer to the system containing questions or requests.

[1804] "Automatic reply" refers to the process of analyzing the content of an inquiry email and automatically generating an appropriate response.

[1805] A "network configuration proposal" refers to a proposed layout and design of the network infrastructure that is optimal for the customer's business operations.

[1806] "Advertising budget allocation" refers to a specific plan that shows how the budget used for advertising activities will be distributed.

[1807] "Application information" refers to the information a customer uses to formally apply for a service or product.

[1808] An "application form" is an official document that is automatically generated based on the collected application information.

[1809] An "emotion engine" is a technology that analyzes emotions from customer voice and text and applies the results.

[1810] "Analysis results" refers to the data obtained by the emotion engine from analyzing the customer's emotional state.

[1811] "Voice tone" refers to the pitch, speed, and volume of a voice, and is a factor that greatly influences the impression of the information presented.

[1812] Modes for carrying out the invention

[1813] This invention is a system for supporting sales staff in physical stores, and in particular, it can analyze customer emotions using an emotion engine to effectively engage in dialogue and make suggestions. The system consists of a user device, generative artificial intelligence, an emotion engine, a server voice output function, and related software.

[1814] System Configuration

[1815] 1. User device

[1816] These are devices used for collecting customer information, receiving suggestions, and displaying the results of sentiment analysis. Specifically, this includes smartphones and smart glasses.

[1817] 2. Generative Artificial Intelligence

[1818] This artificial intelligence generates optimal solutions and talk scripts based on customer information. Specifically, it is implemented in Python and can utilize Google Cloud Natural Language and IBM Watson as external APIs.

[1819] 3. Emotional Engine

[1820] This technology analyzes customer emotions from their voice and text, and adjusts the generated content and voice tone based on the analysis results. The emotion analysis utilizes the Google Cloud Natural Language API and the IBM Watson API.

[1821] 4. Server

[1822] This system plays a central role in collecting and storing customer information transmitted from user devices, analyzing it using generative artificial intelligence and emotion engines, and sending the results back to the user devices. Cloud services (e.g., AWS, Google Cloud) are used for the servers.

[1823] 5. Audio output function

[1824] This feature converts server-generated suggestions and talk scripts into audio for playback on the user's device. Specifically, it uses the Google Cloud Text-to-Speech API.

[1825] System operation

[1826] 1. Collection of customer information

[1827] Sales staff use smartphones or smart glasses to input the customer's name and needs. For example, the format might be: "Please enter the customer's name: Taro Yamada" and "Please enter the customer's needs: I'm looking for a large refrigerator for my family."

[1828] The entered information is sent to the server in real time and stored in the database.

[1829] 2. Proposal generation and sentiment analysis

[1830] The server uses generative artificial intelligence to generate the optimal solution based on the collected customer information. Simultaneously, the customer's emotional state is analyzed using an emotion engine.

[1831] For example, if a customer enters "I'm looking for a large refrigerator for my family," the emotion engine will analyze the input and, if it detects a "positive" emotion, it will suggest new products.

[1832] 3. Generating the talk script and outputting the audio.

[1833] The generated solutions are also created as talk scripts and converted into speech using the voice output function. This speech is output in an appropriate tone and speed based on the results of the sentiment analysis.

[1834] For example, if the generated solution is "Special sale information on large refrigerators," the voice tone will be bright and persuasive.

[1835] 4. Automatic reply function

[1836] The server also analyzes customer inquiry emails and uses generative artificial intelligence to generate appropriate automated replies. ...

Claims

1. A user device means for collecting customer information, A means for generating solutions based on customer information using generative artificial intelligence, A means for outputting the generated solution as audio, A method for automatically generating replies to inquiry emails, A means for automatically generating network configuration proposals and advertising budget allocations, A method for collecting application information in a chat format and automatically generating application forms, A system that includes this.

2. The system according to claim 1, comprising means including video playback and input in the form of questions for collecting the aforementioned customer information.

3. The system according to claim 1, further comprising a text-to-speech function for converting the solution generated by the generative artificial intelligence into audio data.

Images