System

Abstract

To provide a system which allows optimal proposals to be made to customers, and which can realize quality improvement and efficient handling of customer services.SOLUTION: A system includes: means for enabling a user who is a salesperson to input the results of hearing customers about their life environment, by a voice, when providing customer services; means for generating a slide show, by using a prompt sentence instructing generation of the slide show showing a commercial material and a proposal method of the commercial material that are suitable for a customer, based on the hearing results, and by using a generative AI model; and means for displaying the generated slide show in a display unit of smart spectacles worn by the user.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] The current system used by sales staff when serving customers does not standardize the methods of making proposals based on customer environment interviews or the appeal of products, making it difficult to shorten customer service time or serve customers with multiple staff members. [Means for solving the problem]

[0005] We provide a system that combines a means for inputting customer environmental interviews during customer service, a generative AI means for generating a picture-story show-style slideshow based on the input interviews, and a means for using the generated slideshow to present proposal methods and product appeals that are tailored to the customer. This will enable standardization and efficiency in customer service. [Brief explanation of the drawings]

[0006] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 2 is a sequence diagram showing a flow of processing in the data processing system according to the first embodiment of the first form example. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1 of Embodiment 1. [Figure 13] FIG. 10 is a sequence diagram showing a processing flow of a data processing system in a second embodiment of the second form example. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 of Embodiment Example 2. [Figure 15] FIG. 10 is a sequence diagram showing the flow of processing in a data processing system according to a third embodiment of the third embodiment. [Figure 16]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 3 of Embodiment 3. [Figure 17] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the first embodiment of the first form example when an emotion engine is combined. [Figure 18] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1 of Form Example 1 when an emotion engine is combined. [Figure 19] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the second embodiment of the second form example when an emotion engine is combined. [Figure 20] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 of Form Example 2 when an emotion engine is combined. [Figure 21] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the third embodiment of the third form example when an emotion engine is combined. [Figure 22] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 3 of Form Example 3 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0007] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0008] First, the terms used in the following description will be explained.

[0009] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, the processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (TENSOR PROCESSING UNIT (registered trademark)).

[0010] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0011] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0012] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0013] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0014] [First embodiment]

[0015] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0016] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0017] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0018] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0019] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0020] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0021] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0022] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0023] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0024] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0025] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0026] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0027] "Example 1"

[0028] In this embodiment, the crew interviews the customer about their environment while serving them and inputs that information into the system. The input information is analyzed by generative AI, and a picture-story-style slideshow suited to the customer's situation is generated. The generated slideshow is used as a guide for the crew when presenting proposal methods and product appeals to the customer.

[0029] "Example 2"

[0030] As a specific example, if a crew member inputs information such as "I live in a newly built detached house" during a customer environment interview, the generative AI will generate a slideshow based on that information, including introductions to products and services suitable for newly built detached houses. The crew member will then use the generated slideshow to suggest products and services suitable for newly built detached houses.

[0031] "Example 3"

[0032] Generative AI also generates slideshows to shorten customer service time and enable multiple crew members to serve customers. For example, to shorten customer service time, the slideshow is designed to efficiently convey necessary information. To enable multiple crew members to serve customers, the slideshow is designed so that multiple crew members can use it simultaneously.

[0033] The processing flow of each embodiment will be described below.

[0034] "Example 1"

[0035] Step 1: When serving customers, the crew will ask about their environment and enter that information into the system.

[0036] Step 2: The input information is analyzed by generative AI.

[0037] Step 3: The generative AI generates a picture-story slideshow suited to the customer's situation.

[0038] Step 4: The generated slideshow will be used as a guide for the crew to present their proposals and products to the customer.

[0039] "Example 2"

[0040] Step 1: The crew will enter the information that "I live in a newly built detached house" during the customer's environmental interview.

[0041] Step 2: Based on this information, the generative AI generates a slideshow introducing products and services suitable for newly built single-family homes.

[0042] Step 3: The crew uses the generated slideshow to suggest products and services suitable for the newly built single-family home.

[0043] "Example 3"

[0044] Step 1: The generative AI generates a slideshow to shorten customer service time and enable multiple people to serve customers.

[0045] Step 2: To shorten customer service time, the slideshow is designed to efficiently convey the necessary information.

[0046] Step 3: To accommodate multiple guests, the slideshow is designed to be used by multiple crew members simultaneously.

[0047] Example 1

[0048] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0049] With the conventional customer service system, it was difficult to quickly and accurately provide suggestions that met the customer's needs, which tended to result in long service times. In addition, it was difficult to have multiple staff members provide service or standardize the service, and the system was heavily dependent on the skills of the crew. This resulted in the problem of inconsistent quality of customer service.

[0050] The identification process by the identification processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means. In this invention, the server includes a means for inputting customer environment interview information, a means for transmitting the input information to the server, a means for the server to transmit a prompt sentence to the generation AI model, a means for the generation AI model to perform analysis and generate a slideshow, a means for saving the generated slideshow and transmitting it to the terminal, a means for the terminal to display the slideshow, and a means for using the generated slideshow to present a proposal method and product appeal tailored to the customer. This reduces customer service time, enables multiple people to serve customers, and standardizes customer service.

[0051] "Customer environmental interview information" refers to information about customer needs and the environment collected by crew members while serving customers.

[0052] The "means of input" refers to an interface that allows crew members to input customer environmental hearing information using devices such as tablets or PCs.

[0053] The "means for transmitting to the server" is a communication means for encrypting information input from the terminal and transmitting the encrypted information to the server.

[0054] A "generative AI model" is an artificial intelligence model that analyzes input information and generates a slideshow that is appropriate for the customer's situation.

[0055] A "prompt statement" is an instruction statement created by the server to cause the generative AI model to perform analysis.

[0056] The "means for generating a slideshow" is a function that allows the generative AI model to analyze the prompt text and generate a picture-story show-style slideshow.

[0057] The "means for saving" is a function for saving the generated slide show in a presentation software format.

[0058] The "means for transmitting to the terminal" is a communication means for the server to transmit the generated slide show to the crew's terminal.

[0059] The "means for displaying a slide show" is a function for displaying a slide show received by a terminal.

[0060] The "method of proposal and means of appealing to merchandise" refers to a means by which the crew uses the generated slideshow to make specific proposals and appeal to the customer about merchandise.

[0061] This invention is a system that uses a generative AI model to generate a picture-story show-style slideshow based on environmental information about customers collected by the crew during customer service, and promotes proposal methods and products.

[0062] First, the user (crew member) uses a device such as a tablet or PC to input information about the customer's needs and environment. This information is input through a dedicated application. For example, specific information such as "The customer is considering a new office layout" is input.

[0063] The device then sends the entered information to the server in real time, using the HTTPS protocol to ensure security as the data is encrypted before transmission.

[0064] The server creates a prompt sentence based on the received information and sends it to a generative AI model (e.g., GPT-4 (registered trademark)). The prompt sentence will be in the form of, "The client is considering the layout of a new office. Please generate a slideshow about efficient desk arrangements and space utilization methods."

[0065] The generative AI model analyzes the prompt text and generates a picture-story-style slideshow that is appropriate for the customer's situation, including specific proposals and the selling points of the product.

[0066] The server saves the generated slideshow in a presentation software format (e.g., Microsoft® PowerPoint or Google® Slides), and then transmits the slideshow to the crew's terminal.

[0067] The device then displays the received slideshow, which the crew can use to provide specific suggestions to the customer. For example, a slideshow on optimizing office layout includes ideas for desk placement and efficient space utilization.

[0068] This system reduces customer service time, allows multiple people to serve customers, and standardizes customer service. An example of a specific prompt sentence is, "The customer is considering a new office layout. Please generate a slideshow about efficient desk arrangements and space utilization methods."

[0069] In this way, the system provides the crew with the tools to make optimal recommendations to customers based on the information they collect.

[0070] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0071] Step 1:

[0072] The user (crew) inputs the environmental hearing information.

[0073] Specifically, the crew uses a dedicated application on a tablet or PC to input information about the customer's needs and environment. For example, they input specific information such as, "The customer is considering the layout of a new office." The input information is temporarily saved in the application.

[0074] Input: Information about your needs and circumstances

[0075] Output: Temporarily stored hearing information

[0076] Step 2:

[0077] The terminal sends the input information to the server.

[0078] Specifically, the device encrypts the input information and sends it to the server using the HTTPS protocol. The server receives the data and stores it in a database.

[0079] Input: Temporarily stored hearing information

[0080] Output: Hearing information sent to the server

[0081] Step 3:

[0082] The server sends a prompt to the generative AI model.

[0083] Specifically, the server creates a prompt based on the received information. For example, it generates a prompt such as, "The client is considering a new office layout. Please generate a slideshow about efficient desk arrangements and space utilization methods." This prompt is then sent to the generative AI model.

[0084] Input: Hearing information stored on the server

[0085] Output: The prompt sent to the generative AI model

[0086] Step 4:

[0087] A generative AI model performs the analysis and generates a slideshow.

[0088] Specifically, a generative AI model (e.g., GPT-4) analyzes the prompt and generates a picture-story-style slideshow appropriate to the customer's situation, including specific proposals and selling points of the product.

[0089] Input: The prompt sent to the generative AI model

[0090] Output: Generated slideshow

[0091] Step 5:

[0092] The server saves the generated slideshow and transmits it to the device.

[0093] Specifically, the server saves the generated slideshow in a presentation software format (e.g., Microsoft PowerPoint or Google Slides), and then transmits the slideshow to the crew's device.

[0094] Input: Generated slideshow

[0095] Output: Slideshow sent to device

[0096] Step 6:

[0097] Your device will display a slideshow.

[0098] Specifically, the device displays the received slideshow. The crew member uses this slideshow to provide specific suggestions to the customer. For example, a slideshow on optimizing office layout includes ideas for desk arrangement and efficient space utilization.

[0099] Input: Slideshow sent to device

[0100] Output: Rendered slideshow

[0101] In this way, the system provides the crew with the tools to make optimal recommendations to customers based on the information they collect.

[0102] (Application example 1)

[0103] Next, a description will be given of Application Example 1 of Embodiment Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0104] With the conventional customer service system, it was difficult to respond to customer needs quickly and accurately, which tended to result in long service times. Also, since the quality of customer service depended on the experience and skills of the crew, it was difficult to provide a consistent service. Furthermore, it was difficult to serve multiple customers at the same time, so efficient customer service was required.

[0105] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0106] In this invention, the server includes a means for inputting information about the customer's environment during the customer service session, a generation system AI means for generating a picture-story slideshow based on the input information, a means for using the generated slideshow to present a proposal method and product appeal tailored to the customer, a means for accepting voice input, a voice recognition means for converting voice into text, and a display means for displaying the generated slideshow. This allows for quick and accurate response to customer needs, shortening customer service time and enabling simultaneous response to multiple customers. It also standardizes the quality of customer service and realizes efficient service provision.

[0107] "Means for inputting environmental information from customers during customer service" refers to a means for collecting customer needs and environmental information through dialogue with customers and inputting that information into the system.

[0108] The "generative AI means for generating a picture-story show-style slideshow based on input hearing" refers to an artificial intelligence means for analyzing collected customer information and generating a picture-story show-style slideshow to make optimal proposals to customers.

[0109] "Means for making proposals and promoting products that suit customers using generated slideshows" refers to means for making appropriate proposals and promoting products to customers using generated slideshows.

[0110] The "means for accepting voice input" is a means for the crew to input the customer's needs by voice.

[0111] "Speech recognition means for converting speech into text" refers to means for analyzing input speech and converting it into text data.

[0112] The "display means for displaying the generated slide show" refers to a means for visually displaying the generated slide show.

[0113] A system for implementing this invention has the following configuration. First, it includes a voice input means for the crew to input customer needs and environmental information by voice while serving customers. This voice input means uses a microphone installed in a device such as smart glasses or a smartphone.

[0114] Next, the information input by voice is converted into text data by a voice recognition means. This voice recognition means uses a SpeechRecognition library or the like. The voice recognition means analyzes the voice data and converts it into text data.

[0115] The converted text data is sent to a generative AI means. A generative AI model such as OpenAI (registered trademark) or GPT-3 (registered trademark) is used as the generative AI means. This generative AI means analyzes the input text data and generates a picture-story show-style slideshow based on the customer's needs.

[0116] The generated slideshow is visually displayed by a display means, which may be a display of smart glasses or a smartphone screen. The display means provides the generated slideshow to the crew and serves as a guide for making effective proposals and promoting products to customers.

[0117] As a specific example, if a customer says, "I'm looking for a new smartphone, especially one with a good camera," the speech recognition means converts this speech into text, and the generative AI means generates the following prompt sentence:

[0118] Example prompt sentence:

[0119] "Based on the customer's needs, generate a Kamishibai-style slideshow with the following information: I'm looking for a new smartphone, especially one with a good camera."

[0120] Based on this prompt, the generative AI system generates a slideshow comparing the latest smartphone camera features, including the features and pricing information for each model. The generated slideshow is displayed on the smart glasses display and serves as a guide for the crew to make effective recommendations to the customer.

[0121] In this way, the present invention responds to customer needs quickly and accurately, shortens customer service time, and enables simultaneous service for multiple customers. It also standardizes the quality of customer service and provides efficient service.

[0122] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0123] Step 1:

[0124] The user uses the microphone on the smart glasses or smartphone to input customer needs and environmental information by voice.

[0125] Input: Customer voice information

[0126] Output: Audio data

[0127] Specific operation: The user interacts with the customer and inputs the customer's requests and questions by voice.

[0128] Step 2:

[0129] The device converts the voice input information into text data using a voice recognition method (e.g., SpeechRecognition library).

[0130] Input: Audio data

[0131] Output: Text data

[0132] Specific operation: The device analyzes the voice data and applies a voice recognition algorithm to convert it into text data.

[0133] Step 3:

[0134] The device sends the converted text data to a generative AI method (e.g., OpenAI GPT-3).

[0135] Input: Text data

[0136] Output: Request to generative AI

[0137] Specific operation: The device creates and sends a request to send text data to a generative AI model in the cloud.

[0138] Step 4:

[0139] The server uses generative AI means to generate a picture-story show-style slideshow based on the input text data.

[0140] Input: Text data

[0141] Output: Kamishibai style slideshow

[0142] Specific operation: The server inputs prompt text into the generative AI model and generates a slideshow based on the customer's needs.

[0143] Step 5:

[0144] The server transmits the generated slideshow to the terminal.

[0145] Input: Kamishibai style slideshow

[0146] Output: Slideshow data to the device

[0147] Specific operation: The server performs communication processing to transmit the generated slideshow data to the terminal.

[0148] Step 6:

[0149] The terminal displays the received slideshow on a display means (e.g., a display of smart glasses or a smartphone screen).

[0150] Input: Slideshow data

[0151] Output: Rendered slideshow

[0152] Specific operation: The terminal transmits the received slideshow data to the display device and displays it visually.

[0153] Step 7:

[0154] Based on the displayed slideshow, the user can make proposals and appeal to customers about products.

[0155] Input: Displayed slideshow

[0156] Output: Proposals and appeals to customers

[0157] Specific operation: The user watches the slideshow and makes appropriate suggestions and explains products to the customer.

[0158] Example 2

[0159] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0160] With conventional customer service systems, it took a lot of time and effort to propose appropriate products and services based on customer environment information. Furthermore, the quality of customer service varied depending on the staff member, making it difficult to provide consistent service. Furthermore, it was difficult to provide efficient service in a short amount of time with multiple staff members.

[0161] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0162] In this invention, the server includes means for inputting customer environment information, means for generating prompt sentences based on the input hearing information, means for calling a generative AI model using the generated prompt sentences to generate a slideshow, and means for proposing products and services suitable for the customer using the generated slideshow. This reduces customer service time, enables multiple people to serve customers, and further standardizes customer service.

[0163] "Customer environmental interview information" refers to information about the customer's current living environment and housing situation.

[0164] An "input means" is an interface or device that allows a user to input information into a system.

[0165] A "prompt" is an instruction that instructs a generative AI model to generate a specific output.

[0166] A "generative AI model" is an artificial intelligence model that generates a specific output based on an input prompt.

[0167] A "slide show" is a presentation material in which multiple slides are displayed in sequence.

[0168] The "means of proposal" refers to a method or tool for explaining and proposing products or services to customers using the generated slideshow.

[0169] A "server" is a computer system for processing and storing data.

[0170] This invention is a system that generates and proposes slideshows using a generative AI model based on customer environment interview information. Specific embodiments of this system are described below.

[0171] Server Roles

[0172] The server receives the customer's environmental interview information entered by the crew member on their device. This information includes the data that "you live in a newly built detached house." Based on this information, the server generates a prompt to be input into the generative AI model. Specifically, the server creates a prompt that reads, "Please generate a slideshow introducing products and services suitable for customers who live in newly built detached houses."

[0173] Next, the server uses the generated prompt to call a generative AI model (for example, OpenAI's GPT-4). The prompt is input into the AI ​​model to generate the content for the slideshow. Based on the content returned by the AI ​​model, the server generates the slideshow. The slideshow is created in PowerPoint or PDF format, for example. Finally, the server sends the generated slideshow to the crew's device.

[0174] Device Role

[0175] The terminal receives the customer's environmental interview information entered by the crew. For example, the information entered is "I live in a newly built detached house." The terminal then sends the entered information to the server. The terminal then receives the slideshow sent from the server. To display the received slideshow, the terminal uses software such as Microsoft PowerPoint or Google Slides. The crew then uses this software to show the slideshow to the customer.

[0176] User (Crew) Roles

[0177] The user (crew member) interviews the customer about their living environment and inputs the information that they "live in a newly built detached house" into the terminal. Next, the user checks the slideshow sent from the server on the terminal and proposes products and services suitable for the newly built detached house. The crew member uses the slideshow to explain specific products and services to the customer.

[0178] Specific examples

[0179] Example of information entered by crew

[0180] "You live in a newly built detached house."

[0181] Example prompts to input to the generative AI model

[0182] "Generate a slideshow showcasing products and services suitable for customers living in newly constructed single-family homes."

[0183] Example of generated slideshow content

[0184] 1. Slide 1: Title: "Best Products and Services for Newly Built Single-Family Homes"

[0185] 2. Slide 2: Product Introduction "Latest Security System"

[0186] 3. Slide 3: Service Introduction "Garden Design and Maintenance Service"

[0187] 4. Slide 4: Product Introduction "Energy-efficient Home Appliances"

[0188] 5. Slide 5: Service Introduction "Home Automation Installation Service"

[0189] The crew uses this slideshow to suggest products and services suitable for newly built single-family homes to customers.

[0190] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0191] Step 1:

[0192] The user (crew) interviews the customer about their living environment and collects information such as "I live in a newly built detached house." The user then enters this information into the device. The entered information is temporarily stored in the device's database.

[0193] Input: Customer's environmental information (e.g., "I live in a newly built detached house")

[0194] Output: Hearing information stored on the device

[0195] Step 2:

[0196] The terminal transmits the input hearing information to the server. The terminal uses a communication module to transmit data to the server via a network.

[0197] Input: Hearing information stored on the device

[0198] Output: Hearing information sent to the server

[0199] Step 3:

[0200] The server generates a prompt based on the received information. Specifically, it creates the prompt, "Please generate a slideshow introducing products and services suitable for customers living in newly built detached houses."

[0201] Input: Hearing information sent to the server

[0202] Output: Generated prompt statement

[0203] Step 4:

[0204] The server calls the generative AI model using the generated prompt sentence. The server inputs the prompt sentence into the generative AI model (e.g., GPT-4) to generate the content for the slideshow.

[0205] Input: Generated prompt text

[0206] Output: The slideshow content returned by the generative AI model

[0207] Step 5:

[0208] The server generates a slideshow based on the content returned by the generative AI model, for example in PowerPoint or PDF format.

[0209] Input: Slideshow content returned by a generative AI model

[0210] Output: Generated slideshow (PowerPoint and PDF formats)

[0211] Step 6:

[0212] The server transmits the generated slideshow to the crew's terminals. The server uses a communication module to transmit data to the terminals via a network.

[0213] Input: Generated slideshow

[0214] Output: Slideshow sent to device

[0215] Step 7:

[0216] The device receives the slideshow sent from the server and uses software such as Microsoft PowerPoint or Google Slides to display the received slideshow.

[0217] Input: Slideshow sent from server

[0218] Output: Slideshow displayed on device

[0219] Step 8:

[0220] The user (crew member) checks the slideshow received on the terminal and proposes products and services suitable for the newly built detached house. The crew member uses the slideshow to explain specific products and services to the customer.

[0221] Input: Slideshow displayed on device

[0222] Output: Products and services proposed to the customer

[0223] (Application example 2)

[0224] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0225] With conventional customer service systems, it was difficult to propose optimal products and services based on customer needs, which often resulted in long customer service times. Furthermore, because the quality of customer service depended on the skills and experience of the salesperson, there was also the problem of inconsistent quality. Furthermore, it was difficult to serve multiple customers at the same time.

[0226] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for inputting environmental interviews with customers during customer service, generative AI means for generating a picture-story show-style slideshow based on the input interviews, means for using the generated slideshow to present a proposal method and product appeal suited to the customer, means installed on the smart device for allowing the store clerk to propose optimal products and services based on the customer's needs, and means for displaying the generated slideshow. This makes it possible to efficiently propose optimal products and services based on the customer's needs, as well as shorten the time required for customer service and handle multiple customers simultaneously.

[0227] The "means for inputting information about the customer's environment during customer service" is an interface that allows store staff to collect information about the customer's living environment, needs, etc., and input it into the system.

[0228] "Generative AI means that generates a picture-story show-style slideshow based on input interview information" is an artificial intelligence technology that automatically generates a slideshow introducing optimal products and services based on collected customer information.

[0229] "Means of using generated slideshows to present proposals and products that suit customers" refers to a method of using generated slideshows to propose appropriate products and services to customers and effectively communicate their appeal.

[0230] "A means installed on a smart device that enables store clerks to suggest the most suitable products and services based on the customer's needs" refers to an application that is installed on a device such as a smartphone or tablet, and enables store clerks to input customer information, display the generated slideshow, and make suggestions.

[0231] The "means for displaying the generated slideshow" is a function for displaying the slideshow created by the generative AI means on the screen of the smart device.

[0232] A system for implementing this invention includes a means for inputting environmental information about a customer when serving them, a generative AI means for generating a picture-story slideshow in the form of a picture book based on the input information, a means for using the generated slideshow to present a proposal method and product appeal that is suited to the customer, a means that is installed on a smart device and enables a store clerk to propose optimal products and services based on the customer's needs, and a means for displaying the generated slideshow.

[0233] System program processing explanation

[0234] Hardware and Software

[0235] Hardware: smartphones, tablets, servers

[0236] Software: Python, OpenAI API, device application

[0237] Data processing and calculation

[0238] 1. Entering customer information: A salesperson uses a smart device to enter information about the customer's living environment and needs. For example, they enter information such as "I live in a newly built detached house."

[0239] 2. Utilizing generative AI models: Based on the customer information entered, a generative AI model (e.g., GPT-3) generates a slideshow introducing the most suitable products and services.

[0240] 3. Displaying the slideshow: The generated slideshow is displayed on the smart device screen, and the store clerk makes suggestions to the customer.

[0241] Specific examples

[0242] For example, if a customer enters "I live in a newly built detached house," the AI ​​will generate a slideshow like the one below.

[0243] Slide 1: Solar power generation system that supports eco-friendly living

[0244] Slide 2: The best smart home devices for new homes

[0245] Slide 3: Energy-efficient appliances

[0246] Prompt Sentence Examples

[0247] "Generate a slideshow showcasing products and services suitable for customers living in new homes. Customer Information: New home, eco-friendly lifestyle."

[0248] In this way, it is possible to efficiently propose optimal products and services based on customer needs, shorten customer service time, and handle multiple customers simultaneously.

[0249] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0250] Step 1:

[0251] The user uses a smart device to input customer information, specifically the customer's living environment and needs (e.g., "I live in a newly built detached house"). The input information is saved in the application on the device.

[0252] Input: Customer's living environment and needs

[0253] Output: Stored customer information

[0254] Step 2:

[0255] The device sends the stored customer information to the server, which analyzes the received customer information and generates a prompt to be passed to the generative AI model.

[0256] Input: Saved customer information

[0257] Output: Generated prompt statement

[0258] Step 3:

[0259] The server sends a request to a generative AI model (e.g., GPT-3) using the generated prompt, which then generates slideshow content to showcase the most suitable products and services.

[0260] Input: Generated prompt text

[0261] Output: The generated slideshow content

[0262] Step 4:

[0263] The server receives the generated slideshow content and transmits it to the terminal, which converts the received slideshow content into data for display.

[0264] Input: Generated slideshow content

[0265] Output: Slideshow data for display

[0266] Step 5:

[0267] The terminal uses the slideshow data for display to display a slideshow on the smart device screen, and the store clerk uses this slideshow to suggest the most suitable products and services to the customer.

[0268] Input: Slideshow data to display

[0269] Output: Slideshow displayed on smart device

[0270] In this way, it is possible to efficiently propose optimal products and services based on customer needs, shorten customer service time, and handle multiple customers simultaneously.

[0271] Example 3

[0272] Next, a description will be given of a third embodiment of the third embodiment. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0273] With the conventional customer service system, customer service times tended to be long and it was difficult for multiple staff to serve customers. In addition, the content of customer service often varied depending on the staff member, making it difficult to provide a consistent service. This led to problems such as a decline in customer satisfaction and a deterioration in efficiency.

[0274] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[0275] In this invention, the server includes means for a user to input a prompt text, means for a terminal to send the prompt text to the server, means for the server to receive the prompt text and input it to a generative AI model, means for the generative AI model to generate slideshow content based on the prompt text, means for the server to input the generated content to slideshow generation software, means for the slideshow generation software to generate a visually formatted slideshow, means for the server to send the generated slideshow to the terminal, and means for the terminal to display the slideshow to the user. This reduces customer service time, enables multiple people to serve customers, and further standardizes customer service.

[0276] A "user" is an entity that uses the system to input prompt sentences and generate a slide show.

[0277] A "prompt sentence" is an instruction sentence entered by the user, and is the information that the generative AI model uses to generate the content of the slideshow.

[0278] A "terminal" is a device through which a user enters prompts and communicates with a server.

[0279] The "server" is a computer system that receives prompt text, generates a slideshow using a generative AI model, and sends it to the device.

[0280] A "generative AI model" is an artificial intelligence algorithm that generates slideshow content based on prompt text.

[0281] "Slideshow generation software" is software that visually formats the content generated by a generative AI model and outputs it as a slideshow.

[0282] A "slideshow" is a visually stylized series of slides generated by generative AI models and slideshow generation software.

[0283] This invention is a system that generates a slideshow using a generative AI model, shortening the time it takes to serve customers and enabling multiple people to serve customers. Specific embodiments of this system are described below.

[0284] First, the user inputs a prompt using the device. The prompt is the information that the generative AI model uses to generate the content of the slideshow. For example, the user might input a prompt such as, "I would like you to create a slideshow that explains the features of our new product."

[0285] The device then sends the prompt to a server, which receives it and inputs it into a generative AI model, such as an advanced natural language processing model like GPT-4.

[0286] The generative AI model generates slideshow content based on prompts, which are presented in text format and include information such as the features, benefits, and usage of new products.

[0287] The server then inputs the generated text into slideshow generation software, such as the Microsoft PowerPoint API or Google Slides API, which visually formats the generated text and outputs it as a slideshow.

[0288] The generated slideshow is sent from the server to the terminal, which then displays it to the user. The user can then review the generated slideshow and make any necessary corrections or additions.

[0289] For example, if a user inputs the prompt "Please create a slideshow that explains the features of a new product," the generative AI model generates text describing the new product's features, benefits, and usage. The model then uses the Microsoft PowerPoint API to generate a visually formatted slideshow and sends it to the device. The user can then review the generated slideshow and make any necessary corrections or additions.

[0290] In this way, the slideshow generation system using the generative AI model functions as a tool for shortening customer service time and efficiently serving customers with multiple people. It is also possible to standardize customer service. The flow of the identification process in Example 3 will be described with reference to FIG. 15.

[0291] Step 1:

[0292] The user enters a prompt statement.

[0293] The user inputs a prompt into the input field of the device, for example, "I want you to create a slideshow that explains the features of our new product." The input prompt is temporarily stored in the device's memory.

[0294] Step 2:

[0295] The terminal sends the prompt to the server.

[0296] The terminal sends the prompt text entered by the user to the server. HTTP or HTTPS is used as the communication protocol. The input is the prompt text, and the output is a notification to the server that transmission has been completed.

[0297] Step 3:

[0298] The server receives the prompt sentence and inputs it into the generative AI model.

[0299] The server receives the prompt sentence sent from the terminal and inputs the received prompt sentence into the generative AI model. The input is the prompt sentence, and the output is a notification to the generative AI model that the input has been completed.

[0300] Step 4:

[0301] A generative AI model generates slideshow content based on prompts.

[0302] A generative AI model generates content for each slide in a slideshow based on a prompt, such as text about the features, benefits, and usage of a new product. The input is the prompt, and the output is the generated text.

[0303] Step 5:

[0304] The server inputs the generated content into the slideshow generation software.

[0305] The server inputs the text obtained from the generative AI model into the slideshow generation software. The input is the generated text, and the output is a notification to the slideshow generation software that the input has been completed.

[0306] Step 6:

[0307] Slideshow generation software generates a visually formatted slideshow.

[0308] Slideshow generation software generates a visually formatted slideshow from input text, for example by applying an appropriate layout or design to each slide. The input is the generated text and the output is a visually formatted slideshow.

[0309] Step 7:

[0310] The server transmits the generated slideshow to the terminal.

[0311] The server sends the generated slideshow to the device using HTTP or HTTPS as the communication protocol. The input is the visually formatted slideshow, and the output is a notification of completion of transmission to the device.

[0312] Step 8:

[0313] The terminal displays the slideshow to the user.

[0314] The terminal displays the slideshow received from the server to the user. The user can review the generated slideshow and make corrections or additions as needed. The input is a visually formatted slideshow, and the output is the display to the user.

[0315] (Application example 3)

[0316] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0317] With conventional customer service systems, each customer service representative must deal with each customer individually, which results in long service times and makes it difficult to serve multiple customers at the same time. Another problem is that the quality of customer service varies depending on the representative. Furthermore, while there is a demand for efficient information provision using smart devices, the current system is unable to adequately address this.

[0318] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[0319] In this invention, the server includes means for inputting information about the customer's environment during customer service, a generation system AI means for generating a picture-story show-style slideshow based on the input information, means for using the generated slideshow to present a proposal method and product appeal tailored to the customer, means for displaying the slideshow on a smart device, and means for generating and displaying individual slideshows so that multiple customer service representatives can simultaneously serve different customers. This reduces customer service time and enables simultaneous service to multiple customers, resulting in standardized customer service quality and efficient information provision using smart devices.

[0320] "Means for inputting environmental information about customers during customer service" refers to devices or methods for collecting information about customers' needs, interests, and environment and inputting it into the system.

[0321] A "picture-story show slideshow" is a presentation format that conveys information visually by displaying a series of images and text in sequence.

[0322] "Generative AI means" refers to technologies or devices that use artificial intelligence to automatically generate content based on input data.

[0323] "Methods of proposing and promoting products that suit customers" refers to methods and devices for proposing optimal products and services based on customers' needs and interests, and effectively communicating their appeal.

[0324] "Means for displaying a slideshow on a smart device" refers to techniques or devices for displaying the generated slideshow on a device such as smart glasses or a smartphone.

[0325] "Means for generating and displaying individual slideshows so that multiple customer service representatives can simultaneously serve different customers" refers to technology or devices that allow multiple customer service representatives to simultaneously generate and display individually customized slideshows for different customers.

[0326] The system for implementing this invention includes a means for inputting information about the customer's environment when serving customers, a generative AI means, a means for displaying a slideshow on a smart device, and a means for generating and displaying individual slideshows so that multiple staff members can serve different customers simultaneously.

[0327] System configuration

[0328] 1. A way to input information about the customer's environment during customer service

[0329] This is a device that uses input devices such as tablets and smartphones to collect information about customers' needs, interests, and environment and input it into the system.

[0330] 2. Generative AI means

[0331] The generative AI method uses generative AI models such as the OpenAI API to automatically generate a picture-story show based on the input interview data. The generated slideshow is used to present optimal proposal methods and product appeal to customers.

[0332] 3. How to display slideshows on smart devices

[0333] This technology and equipment allows for the display of generated slideshows on devices such as smart glasses (e.g., Google Glass®, Vuzix Blade) and smartphones, enabling customer service staff to provide information visually to customers.

[0334] 4. A way to generate and display individual slideshows for multiple salespeople to simultaneously serve different customers

[0335] The generative AI method allows multiple customer service representatives to simultaneously generate and display individually customized slideshows for different customers, making it possible to serve multiple customers at the same time.

[0336] Program processing explanation

[0337] The server first receives data collected through a means for inputting information about the customer's environment during service. Next, it uses a generative AI means to generate a picture-story show-style slideshow based on the input data. The generated slideshow is displayed on a smart device, which may include smart glasses or a smartphone. Furthermore, the generative AI means can generate and display individual slideshows so that multiple service representatives can simultaneously serve different customers.

[0338] Specific examples

[0339] For example, if a customer asks, "What are the features of my new smartphone?", the salesperson uses a tablet to input this question into the system. The generative AI means generates a slideshow based on this input and displays it on the smart glasses. If another salesperson is asked by another customer, "What are the latest fashion items?", a separate slideshow is similarly generated and displayed.

[0340] Prompt Sentence Examples

[0341] "Please tell me the features of your new smartphone."

[0342] In this way, the system can efficiently and effectively handle customer inquiries.

[0343] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[0344] Step 1:

[0345] Users use input devices such as tablets and smartphones to input information about their needs, interests, and surroundings. The input data is then sent to the server.

[0346] Input: Information about your needs, interests, and circumstances

[0347] Output: Hearing data sent to the server

[0348] Step 2:

[0349] The server passes the received hearing data to the generative AI means, which uses a generative AI model such as the OpenAI API to generate a picture-story show-style slideshow based on the input data.

[0350] Input: Hearing data

[0351] Output: Generated slideshow

[0352] Step 3:

[0353] The server transmits the generated slideshow to a smart device, which may include smart glasses or a smartphone.

[0354] Input: Generated slideshow

[0355] Output: Slideshow sent to smart device

[0356] Step 4:

[0357] The smart device displays the received slideshow, and the customer service representative provides the information visually to the customer through the smart device.

[0358] Input: Slideshow sent to smart device

[0359] Output: Slideshow displayed on smart device

[0360] Step 5:

[0361] When multiple customer service representatives are simultaneously serving different customers, the server passes each representative's interview data to the generative AI means to generate individual slideshows, which are then sent to and displayed on each smart device.

[0362] Input: Multiple hearing data

[0363] Output: Individually generated slideshow

[0364] In this way, the system can efficiently and effectively handle customer inquiries.

[0365] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0366] "Example 1"

[0367] One embodiment of the present invention is a system that incorporates an emotion engine. In this system, the environment of the customer is interviewed during the customer service session, and that information is input into the system. The input information is analyzed by a generative AI, which generates a picture-story slideshow suited to the customer's situation. The generated slideshow is used as a guide for the crew when presenting proposal methods and product appeals to the customer. Furthermore, the emotion engine recognizes the customer's emotions and adjusts the content of the slideshow and the customer service style based on those emotions.

[0368] "Example 2"

[0369] As a specific example, consider the case where a crew member inputs information about a customer's living environment, such as "I live in a newly built detached house," and the customer shows a pleased expression. Based on this information and the emotion recognition results of the emotion engine, the generative AI generates a slideshow introducing products and services suitable for a newly built detached house. The crew member then uses the generated slideshow to suggest products and services that the customer would enjoy.

[0370] "Example 3"

[0371] The emotion engine also adjusts the customer service style according to the customer's emotions. For example, if a customer shows a confused expression, the emotion engine conveys that information to the generative AI, which then generates a slideshow with more detailed explanations. This makes it possible to provide customer service that is easy for the customer to understand.

[0372] The processing flow of each embodiment will be described below.

[0373] "Example 1"

[0374] Step 1: Interview the customer about their environment during the service and enter that information into the system. Step 2: The input information is analyzed by generative AI, and a picture-story-style slideshow appropriate to the customer's situation is generated.

[0375] Step 3: The generated slideshow will be used as a guide for the crew to present their proposals and products to the customer.

[0376] Step 4: The emotion engine recognizes the customer's emotions and adjusts the slideshow content and customer service style based on those emotions.

[0377] "Example 2"

[0378] Step 1: The crew will enter the information that "I live in a newly built detached house" during the customer's environmental interview.

[0379] Step 2: The emotion engine recognizes facial expressions that the customer is likely to be happy with.

[0380] Step 3: Based on this information and the emotion recognition results from the emotion engine, the generative AI generates a slideshow introducing products and services suitable for newly built single-family homes.

[0381] Step 4: The crew uses the generated slideshow to suggest products and services that the customer might enjoy.

[0382] "Example 3"

[0383] Step 1: The emotion engine adjusts the customer service style according to the customer's emotions.

[0384] Step 2: For example, if the customer shows a confused expression, the emotion engine conveys that information to the generative AI.

[0385] Step 3: The generative AI generates a slideshow with more detailed explanations.

[0386] Step 4: This allows for customer service that is easy for customers to understand.

[0387] Example 1

[0388] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0389] With conventional customer service systems, it was difficult to quickly and effectively make proposals based on the customer's needs and situation. It was also difficult to standardize the quality of customer service and efficiently handle customer service with multiple staff. Furthermore, it was not possible to grasp the customer's emotions in real time and respond accordingly, making it difficult to improve customer satisfaction.

[0390] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0391] In this invention, the server includes: a means for inputting information about the customer's environment during customer service; a means for transmitting the input information to the server; a means for passing the information received by the server to a generative AI model for analysis; a means for transmitting the generated picture-story slideshow from the server to a terminal; a means for using the generated slideshow to present a proposal method and product appeal tailored to the customer; and a means for analyzing the customer's emotions using an emotion engine and adjusting the content of the slideshow and the customer service style based on the emotions. This enables prompt and effective proposals tailored to customer needs, standardizes the quality of customer service, and enables efficient customer service across multiple staff. Furthermore, by understanding customer emotions in real time and responding accordingly, customer satisfaction can be improved.

[0392] The "means for inputting information about the customer's environment when serving customers" is a function that provides an interface for users to input information about the customer's needs and situation using a terminal.

[0393] The "means for transmitting input hearing information to a server" is a communication function that enables the terminal to transmit information input by a user to a server.

[0394] "Means for passing the information received by the server to the generative AI model for analysis" refers to the processing function for passing the customer information received by the server to the generative AI model for analysis.

[0395] "Means for transmitting the generated picture-story show-style slideshow from the server to the terminal" refers to a communication function for transmitting the slideshow generated by the generative AI model from the server to the terminal.

[0396] "Means of using the generated slideshow to present a proposal method and product appeal that is suited to the customer" is a function that allows the user to present a generated slideshow to the customer using the terminal and appeal to the customer with a proposal method or product appeal.

[0397] "A means of analyzing customer emotions using an emotion engine and adjusting the content of the slideshow and customer service style based on those emotions" is a function that analyzes the customer's facial expressions using the device's camera and adjusts the content of the slideshow and customer service style based on that emotional data.

[0398] This invention is a system that listens to the customer's environment while serving them, generates a picture-story show-style slideshow using a generative AI model based on that information, and then uses an emotion engine to adjust the content of the slideshow and the serving style based on the customer's emotions.

[0399] The user uses a device (e.g., a tablet or PC) to conduct an environment interview with the customer. Specifically, the user uses a dedicated application on the device (e.g., "Customer Insight App") to input information about the customer's needs and situation. The input information is sent from the device to the server. The HTTPS protocol is used for transmission to ensure data security.

[0400] The server passes the received information to a generative AI model (e.g., "OpenAI GPT-4") for analysis. Based on the input information, the generative AI model generates a picture-story show-style slideshow suited to the customer's situation. The generated slideshow is then sent from the server to the device.

[0401] The user presents the generated slideshow to the customer using the terminal, which is used as a guide for presenting proposals and promoting products.

[0402] Furthermore, if an emotion engine (such as "Affectiva Emotion AI") is built in, it analyzes the customer's facial expressions in real time through the device's camera. The analyzed emotion data is sent from the device to a server. The server adjusts the content of the slideshow and customer service style based on the received emotion data. For example, if the customer shows no interest, it sends another prompt to the generative AI model, instructing it to generate a more appropriate slideshow.

[0403] As a concrete example, a user uses the "Customer Insight App" to input that they have just purchased a new home. This information is sent to a server and analyzed by a generative AI model (OpenAI GPT-4). The generative AI model generates a picture-story show-style slideshow containing relevant suggestions for the customer's new home (e.g., furniture and interior design suggestions).

[0404] An example of a prompt is as follows:

[0405] "A client just purchased a new home and is interested in furniture and interior design. Generate a Kamishibai-style slideshow with suggestions tailored to their needs."

[0406] Based on this prompt, the generative AI model generates a slideshow that is appropriate for the customer's situation. If an emotion engine is built in, it analyzes the customer's facial expressions via the camera while they are viewing the slideshow, and if the customer shows no interest, it sends that data to the server. The server then sends the prompt to the generative AI model again, instructing it to generate a more appropriate slideshow.

[0407] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0408] Step 1:

[0409] The user uses the terminal to interview the customer about their environment.

[0410] Users launch the "Customer Insight App" on their device and input information about their needs and circumstances, such as that they have just purchased a new home or are interested in a particular interior design.

[0411] Input: Information about your needs and circumstances

[0412] Output: Input hearing information

[0413] Step 2:

[0414] The terminal transmits the input hearing information to the server.

[0415] The device transmits the information entered by the user to the server in real time using the HTTPS protocol to ensure data security.

[0416] Input: Hearing information entered

[0417] Output: Hearing information sent to the server

[0418] Step 3:

[0419] The information received by the server is passed to the generative AI model for analysis.

[0420] The server passes the received information to a generative AI model (such as "OpenAI GPT-4") for analysis. Based on the input information, the generative AI model generates a picture-story show-style slideshow suited to the customer's situation.

[0421] Input: Hearing information sent to the server

[0422] Output: A generated Kamishibai-style slideshow

[0423] Step 4:

[0424] The server transmits the generated picture-story show format slide show to the terminal.

[0425] The server sends the slideshow generated by the generative AI model to the device, also using the HTTPS protocol.

[0426] Input: A generated Kamishibai-style slideshow

[0427] Output: Slideshow sent to device

[0428] Step 5:

[0429] The user uses the device to present the slideshow to the customer.

[0430] The user presents the received slideshow to the customer on the terminal. The slideshow is used as a guide for presenting proposals and promoting products.

[0431] Input: Slideshow sent to device

[0432] Output: Slideshow presented to the customer

[0433] Step 6:

[0434] If an emotion engine is built in, the device will analyze the customer's emotions through the camera and send them to the server.

[0435] If an emotion engine (such as "Affectiva Emotion AI") is installed, the device's camera will analyze the customer's facial expressions in real time. The analyzed emotion data will be sent from the device to a server.

[0436] Input: Customer's facial expression data

[0437] Output: Emotion data sent to the server

[0438] Step 7:

[0439] The server adjusts the content of the slideshow and customer service style based on the emotional data.

[0440] The server adjusts the content of the slideshow and customer service style based on the received emotional data. For example, if the customer shows no interest, it sends another prompt to the generative AI model to instruct it to generate a more appropriate slideshow.

[0441] Input: Emotion data sent to the server

[0442] Output: Adjusted slideshow content and customer service style

[0443] (Application example 1)

[0444] Next, a description will be given of Application Example 1 of Embodiment Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0445] Conventional customer service systems made uniform suggestions based on information gathered from customer environment interviews, making it difficult to respond flexibly to individual customer emotions and situations. Furthermore, crew members were unable to grasp customers' emotions in real time while serving them and adjust their suggestions accordingly, resulting in inconsistent customer service quality. Furthermore, there was a lack of tools for crew members to analyze their gaze and facial expressions, making it difficult to improve customer service efficiency.

[0446] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0447] In this invention, the server includes: a means for inputting information about the customer's environment during customer service; a generative AI means for generating a picture-story slideshow based on the input information; a means for using the generated slideshow to present a proposal method and product appeal tailored to the customer; an emotion recognition means for recognizing the customer's emotions and adjusting the content of the slideshow and customer service style based on the emotions; a means for displaying the slideshow on smart glasses; and a means for capturing and analyzing the crew's gaze and the customer's facial expressions in real time. This enables flexible proposals based on the customer's emotions and situation, improving the quality of customer service. Furthermore, the crew's analysis of gaze and facial expressions can improve the efficiency of customer service.

[0448] "Means for inputting environmental information about customers during customer service" refers to devices or software that collect information such as the customer's age, gender, and interests and input it into the system.

[0449] "Generative AI means" is an artificial intelligence technology that analyzes input hearing information and generates a picture-story show-style slideshow based on that information.

[0450] "Means of using the generated slideshow to present proposals and merchandise that are tailored to the customer" refers to devices and software that utilize the generated slideshow to present optimal proposals and product introductions to customers.

[0451] An "emotion recognition means" is a device or software that recognizes emotions from customers' facial expressions, tone of voice, etc., and adjusts the content of the slideshow and customer service style based on that information.

[0452] The "means for displaying a slideshow on smart glasses" refers to a device or software for displaying the generated slideshow on the display of the smart glasses.

[0453] "Means for capturing the gaze of crew members and facial expressions of customers and analyzing them in real time" refers to devices and software that capture the gaze of crew members and facial expressions of customers using cameras or other devices and analyze the data in real time.

[0454] The system for implementing this invention includes a means for inputting information about the customer's environment during service, a generative AI means, an emotion recognition means, a means for displaying a slideshow on smart glasses, and a means for capturing the crew's line of sight and the customer's facial expressions and analyzing them in real time.

[0455] System program generation

[0456] The system operates using the following hardware and software:

[0457] Hardware: Smart glasses (e.g., Google Glass)

[0458] Software: OpenCV (image processing library), EmotionRecognizer (emotion recognition library), SlideGenerator (slide generation library), SmartGlassesDisplay (smart glasses display control library)

[0459] Explanation of program processing

[0460] The server first inputs the customer's environmental information collected by the staff during the service. This information includes the customer's age, gender, interests, etc. The input information is then analyzed by a generative AI system, and a picture-story show-style slideshow is generated.

[0461] The generated slideshow is displayed on the smart glasses. The crew wears the glasses and makes optimal suggestions to the customer while watching the slideshow. Furthermore, an emotion recognition system captures the crew's gaze and the customer's facial expressions and analyzes them in real time. This allows the content of the slideshow and the customer service style to be adjusted based on the customer's emotions.

[0462] Adding specific examples

[0463] As a concrete example, the following prompt sentence is input into the generative AI model:

[0464] The customer is 30 years old and female. Her areas of interest are fashion and technology. Her facial expressions indicate her interest. Based on this, generate a slideshow showcasing products related to fashion and technology.

[0465] By inputting this prompt into a generative AI model, a slideshow tailored to the customer is generated and displayed on the smart glasses. The crew member can then use this slideshow as a guide to make effective suggestions to the customer.

[0466] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0467] Step 1:

[0468] The crew will wear smart glasses and conduct an interview with the customer about their environment.

[0469] Input: Information such as your age, gender, and interests

[0470] Output: Hearing information is entered into the system

[0471] How it works: The crew member uses the microphone and touchpad on the smart glasses to enter the customer's information.

[0472] Step 2:

[0473] The server receives the input hearing information and analyzes it using generative AI means.

[0474] Input: Hearing information

[0475] Output: A picture-story show based on the analysis results

[0476] Specific operation: The server inputs prompt sentences into the generative AI model and generates a slideshow.

[0477] Step 3:

[0478] The generated slideshow is displayed on the smart glasses.

[0479] Input: Generated slideshow

[0480] Output: A slideshow displayed on the smart glasses display

[0481] Specific operation: The server sends the slideshow data to the smart glasses and displays it on the display.

[0482] Step 4:

[0483] The crew will make suggestions to customers while watching a slideshow through smart glasses.

[0484] Input: Slideshow displayed on smart glasses

[0485] Output: Proposal to the customer

[0486] Specific actions: The crew will use the slideshow to recommend products and services to customers.

[0487] Step 5:

[0488] Emotion recognition means captures the crew's gaze and passengers' facial expressions and analyzes them in real time.

[0489] Input: Crew gaze data, customer facial expression data

[0490] Output: Customer's emotional state

[0491] Specific operation: The camera installed in the smart glasses captures gaze and facial expressions, and the emotion recognition engine analyzes them.

[0492] Step 6:

[0493] The server adjusts the content of the slideshow and the customer service style based on the emotion recognition results.

[0494] Input: Customer's emotional state

[0495] Output: Coordinated slideshows and presentation styles

[0496] Specific operation: Based on the emotion recognition results, the server re-enters the prompt text into the generative AI model and adjusts the slideshow and customer service style.

[0497] Step 7:

[0498] The adjusted slideshow is displayed again on the smart glasses.

[0499] Input: Adjusted slideshow

[0500] Output: A slideshow replayed on the smart glasses display

[0501] Specific operation: The server sends the adjusted slideshow data to the smart glasses and re-displays it on the display.

[0502] Example 2

[0503] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0504] Conventional customer service systems have had the problem of making it difficult to quickly and effectively provide proposals tailored to customer needs. It was also difficult to standardize the quality of customer service and efficiently handle multiple customer service staff. This could result in longer customer service times and lower customer satisfaction.

[0505] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0506] In this invention, the server includes means for inputting information about the customer's environment during customer service, means for transmitting the input information to the server, means for transmitting prompts to the generative AI model based on the information received by the server, means for the generative AI model to generate a slideshow based on the prompts, means for transmitting the generated slideshow from the server to the terminal, and means for using the generated slideshow to present a proposal method and product appeal tailored to the customer. This enables shortening customer service time, improving the efficiency of customer service across multiple staff, and standardizing the quality of customer service.

[0507] "Hearing about the customer's environment during customer service" is the process of collecting information about the customer's living environment and needs.

[0508] The "means for inputting" is an interface that allows a user to input information using a terminal.

[0509] A "server" is a central processing unit that receives and processes input information.

[0510] A "generative AI model" is an artificial intelligence algorithm that generates a slideshow based on an input prompt.

[0511] A "prompt" is text that contains instructions for the generative AI model to generate a slideshow.

[0512] A "slide show" is a presentation format in which multiple slides are displayed in succession.

[0513] A "terminal" is a device operated by a user, and includes tablets, PCs, etc.

[0514] A "proposal method" is a technique for effectively introducing products and services to customers.

[0515] "Product promotion" is an activity to communicate the appeal of a product or service to customers.

[0516] This invention is a system for quickly and effectively making proposals tailored to customer needs during customer service. The system involves a series of processes in which a user uses a terminal to interview the customer about their environment, sends that information to a server, and generates a slideshow using a generative AI model.

[0517] Hardware and software used

[0518] Hardware: Servers, devices (tablets, PCs, etc.)

[0519] Software: Generative AI model, emotion engine

[0520] Specific operation of the system

[0521] 1. The user uses the device to interview the customer about their environment.

[0522] Users use devices such as tablets and PCs to input information about their living environment and needs.

[0523] Example: A user enters information such as "I live in a newly built detached house."

[0524] 2. The device sends the entered information to the server

[0525] The terminal transmits the input information to the server in real time.

[0526] Example: A device sends information to a server that it "lives in a newly built detached house."

[0527] 3. The server sends a prompt to the generative AI model based on the received information.

[0528] The server analyzes the received information and generates a prompt sentence appropriate for the generative AI model.

[0529] Example: The server sends a prompt to the generative AI model: "Generate a slideshow showcasing products and services suitable for customers living in newly built single-family homes."

[0530] 4. The generative AI model generates a slideshow based on the prompt.

[0531] The generative AI model receives a prompt and generates a slideshow based on it.

[0532] Example: A generative AI model generates a slideshow showcasing furniture and interior design, home security systems, energy-efficient appliances, and more suitable for a newly built single-family home.

[0533] 5. The server sends the generated slideshow to the device.

[0534] The server transmits the generated slide show to the terminal.

[0535] Example: A server sends a generated slideshow to a user's device.

[0536] 6. The user reviews the slideshow generated on their device and makes suggestions to the customer.

[0537] The user checks the slideshow generated on the terminal and makes suggestions to the customer.

[0538] Example: A user watches a slideshow that suggests products and services suitable for a new home.

[0539] Prompt Sentence Examples

[0540] "Generate a slideshow showcasing products and services suitable for customers living in newly constructed single-family homes."

[0541] "Generate a slideshow showcasing products and services that would appeal to customers living in new homes."

[0542] This system allows users to effectively propose products and services that meet customer needs, shorten customer service time, improve the efficiency of customer service across multiple staff, and standardize the quality of customer service.

[0543] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0544] Step 1:

[0545] The user uses the terminal to interview the customer about their environment.

[0546] Input: Information about your living situation and needs (e.g., "I live in a newly built detached house").

[0547] Specific operation: The user uses a device such as a tablet or PC to enter customer information into a dedicated input form.

[0548] Output: The entered customer information is saved on the device.

[0549] Step 2:

[0550] The terminal transmits the input information to the server.

[0551] Input: Your information stored on your device.

[0552] Specific operation: The terminal uses a communication protocol to transmit input information to the server in real time.

[0553] Output: Your information is sent to and stored on our servers.

[0554] Step 3:

[0555] Based on the information received by the server, a prompt is sent to the generative AI model.

[0556] Input: Your information stored on our server.

[0557] Specific operation: The server analyzes the received information and generates a prompt sentence appropriate for the generative AI model.

[0558] Output: The generated prompt sentence is sent to the generative AI model.

[0559] Example: A prompt might read, "Generate a slideshow showcasing products and services suitable for customers living in newly constructed homes."

[0560] Step 4:

[0561] A generative AI model generates a slideshow based on the prompt.

[0562] Input: The prompt sent to the generative AI model.

[0563] How it works: The generative AI model analyzes the prompt and generates a slideshow based on it.

[0564] Output: The generated slideshow is sent to the server.

[0565] Examples include: a slideshow showcasing furniture and interior design for new homes, home security systems, and energy-efficient appliances.

[0566] Step 5:

[0567] The server transmits the generated slideshow to the terminal.

[0568] Input: A slideshow sent from a generative AI model.

[0569] Specific operation: The server uses a communication protocol to send the generated slideshow to the terminal.

[0570] Output: The slideshow is sent to your device and saved on your device.

[0571] Step 6:

[0572] The user checks the slideshow generated on the device and makes suggestions to the customer.

[0573] Input: A slideshow stored on your device.

[0574] Specific operations: The user reviews the slideshow generated on the device and makes suggestions to the customer.

[0575] Output: Effective proposals are made to customers.

[0576] Example: A user watches a slideshow that suggests products and services suitable for a new home.

[0577] (Application example 2)

[0578] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0579] In conventional customer service systems, it was difficult to reflect the emotions and reactions of individual customers in real time when proposing products and services based on the customer's environmental information. Also, experience and skill are required for salespeople to make optimal proposals to customers, which can lead to variations in the quality of customer service. Furthermore, there is a demand for shortening customer service time and efficiently handling multiple customers.

[0580] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0581] In this invention, the server includes: a means for a salesperson user, while serving a customer, to input the results of a conversation about the customer's living environment by voice; a prompt statement instructing the server to generate a slideshow showing products and a method for proposing the products that are suitable for the customer based on the results of the conversation; a means for generating the slideshow using a generative AI model; and a means for displaying the generated slideshow on a display unit of smart glasses worn by the user. The server also includes: a means for recognizing the customer's emotion from the customer's facial expression using an emotion engine; a prompt statement instructing the server to modify the generated slideshow based on the recognition result of the customer's emotion; and a means for modifying the generated slideshow using the generative AI model. The displaying means displays the modified slideshow on a display unit of smart glasses worn by the user. This enables optimal suggestions that reflect the customer's emotions and reactions in real time, improving the quality of customer service and achieving efficient responses.

[0582] "Means for inputting information about the customer's environment during customer service" refers to the means by which salespeople collect information about the customer's living environment and living situation, and input it into the system.

[0583] "Generative AI methods" are artificial intelligence methods that automatically generate slideshows introducing appropriate products and services based on input customer information.

[0584] "A means of using the generated slideshow to present proposals and products that suit the customer" is a means of utilizing the generated slideshow to propose optimal products and services to customers and convey their appeal.

[0585] "Emotion recognition means for recognizing customer facial expressions in real time" is a means for capturing customer facial expressions in real time and analyzing their emotions.

[0586] "Means for the generative AI means to optimize the slideshow based on the emotion recognition results" refers to means for the generative AI to adjust the content of the slideshow and make more appropriate suggestions based on the customer's emotion data obtained by the emotion recognition means.

[0587] "A means for displaying a slideshow using smart glasses and allowing a salesperson to make suggestions" is a means for displaying a generated slideshow on the display of smart glasses and allowing a salesperson to make suggestions to a customer while viewing it.

[0588] A system for implementing the present invention uses the following hardware and software.

[0589] Hardware used

[0590] Smart glasses (e.g. Google Glass)

[0591] Facial recognition camera (built into smart glasses)

[0592] Microphone (for voice input)

[0593] Software used

[0594] Generative AI models (e.g., OpenAI GPT-4)

[0595] Emotion recognition engine (e.g., Microsoft Azure (registered trademark) Emotion API)

[0596] Database (stores customer information, product information)

[0597] System Operation

[0598] 1. Enter your customer information:

[0599] The salesperson listens to environmental information about the customer's living environment and inputs it into the smart glasses' microphone.

[0600] Example: "I live in a newly built detached house."

[0601] 2. Emotion recognition:

[0602] The camera in the smart glasses captures the customer's facial expressions in real time, and an emotion recognition engine analyzes them to recognize the customer's emotions.

[0603] Example: When a customer looks happy

[0604] 3. Slideshow generation using generative AI:

[0605] The generative AI model introduces suitable products and services to customers based on the input customer information and emotion recognition results, and generates a slideshow in accordance with prompts that instruct it to generate a slideshow showing how to propose the products and services.

[0606] Example: Proposing furniture and interior design suitable for a newly built detached house

[0607] 4. View Slideshow:

[0608] The generated slideshow is displayed on the smart glasses display, and the salesperson makes suggestions to the customer.

[0609] Specific examples

[0610] If a customer inputs that they "live in a newly built detached house," the generative AI will introduce "the perfect furniture set for a newly built detached house" or "the latest smart home devices," and generate a slideshow that shows how to propose these (key points and phrases to use when introducing the products). The system also recognizes customer emotions in real time and modifies the generated slideshow based on the results of the customer emotion recognition using prompts and the generative AI model. The modified slideshow is then displayed on the display of the smart glasses worn by the user. For example, if the customer shows a happy expression, the emotion recognition engine feeds that information back to the generative AI model, which then modifies the slideshow to suggest more attractive products and services.

[0611] Prompt Sentence Examples

[0612] Customer information: Living in a newly built detached house

[0613] Emotion recognition result: Happy facial expression

[0614] Generated slideshows: Furniture sets for new homes, latest smart home devices

[0615] In this way, sales staff in brick-and-mortar stores can use smart glasses to recommend the best products and services to customers.

[0616] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0617] Step 1:

[0618] A salesperson uses the smart glasses to input information about the customer's environment. The input information is captured as voice data through the smart glasses' microphone and converted into text data by voice recognition software. The input data is the customer's environmental information, such as "I live in a newly built detached house." The output is sent to the server as text data.

[0619] Step 2:

[0620] The server captures the customer's facial expressions in real time using the camera in the smart glasses. The captured image data is sent to an emotion recognition engine to analyze the customer's emotions. The input data is the captured image data, and the output data is the analyzed emotion data (e.g., a happy expression).

[0621] Step 3:

[0622] The server generates a slideshow using a generative AI model based on the input customer information and emotion recognition results. The generative AI model receives prompts containing customer information and emotion data, and generates a slideshow that introduces suitable products and services and shows the recommendation method. The input data is customer information and emotion data, and the output data is the generated slideshow.

[0623] Step 4:

[0624] The server transmits the generated slideshow to the display of the smart glasses. The smart glasses display the received slideshow, and the salesperson makes suggestions to the customer while viewing the slideshow. The input data is the generated slideshow, and the output data is the slideshow displayed on the display of the smart glasses.

[0625] Step 5:

[0626] Salespeople recommend optimal products and services to customers based on a slideshow displayed on the smart glasses. The input data is the slideshow displayed on the smart glasses, and the output data is the proposal to the customer. In addition, while the salesperson is making a proposal based on the slideshow displayed on the smart glasses, the emotion engine recognizes the customer's emotions in real time. The server then modifies the generated slideshow based on the customer's emotion recognition results, using a prompt statement instructing the salesperson to modify the generated slideshow and the generative AI model. The modified slideshow is then displayed on the display of the smart glasses worn by the user. This allows the salesperson to adjust the proposal content while observing the customer's reaction.

[0627] In this way, the system can suggest optimal products and services based on the customer's environmental information and emotional data. Furthermore, when a user who is a salesperson is serving multiple customers simultaneously, a slide show may be generated for each customer, and the displayed slide show may be switched each time the user switches between customers. In this case, in the server, the input means, when the user is serving multiple customers simultaneously, inputs the results of the interview for each of the multiple customers by voice. The generating means generates the slide show for each of the multiple customers. The displaying means switches to display the generated slide show for each customer each time the user switches between customers.

[0628] Example 3

[0629] Next, a description will be given of a third embodiment of the third embodiment. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0630] With the conventional customer service system, it was difficult to respond flexibly to the customer's emotions, shorten the service time, and handle multiple customer service sessions. In addition, there was a lack of means to standardize customer service, which resulted in inconsistent quality of customer service.

[0631] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[0632] In this invention, the server includes a means for inputting information about the customer's environment during the customer service session, a generative AI means for generating a picture-story slideshow based on the input information, a means for using the generated slideshow to present a proposal method and product appeal tailored to the customer, a means for capturing the customer's facial expression, an emotion analysis means for analyzing the captured facial expression data and identifying the customer's emotion, and a means for adjusting the content of the slideshow based on the identified emotion. This enables flexible customer service according to the customer's emotion, shortening the customer service time and enabling multiple staff members to serve the customer. It also enables standardization of customer service, making it possible to maintain a consistent quality of customer service.

[0633] "Means for inputting information about the customer's environment when serving customers" refers to devices or software for inputting the customer's needs and situation when serving customers.

[0634] A "paper theater-style slideshow" is a presentation format that conveys information visually by displaying multiple slides in sequence.

[0635] "Generative AI means" refers to technologies and devices that use artificial intelligence to automatically generate content for specific purposes.

[0636] "Means of making proposals and promoting products that suit the customer" refers to devices and software that allow you to introduce the best proposals and products according to the customer's needs and situation.

[0637] "Means for capturing customer facial expressions" refers to devices or technologies that use cameras or sensors to capture customer facial expressions and acquire them as data.

[0638] "Emotion analysis means for analyzing captured facial expression data to identify a customer's emotions" refers to technology or equipment for analyzing acquired facial expression data to identify a customer's emotional state (e.g., confusion, joy, interest, etc.).

[0639] "Means for adjusting the content of a slideshow based on the identified emotion" refers to a technique or device for dynamically changing the content of a slideshow based on the analyzed emotion information.

[0640] This invention is a system that enables flexible responses according to customer emotions during customer service. Specifically, it combines generative AI and an emotion analysis engine to generate slideshows that shorten customer service time and enable multiple people to serve customers. The emotion analysis engine also adjusts the customer service style according to the customer's emotions.

[0641] Hardware and software used

[0642] Hardware: Servers, devices (PCs, tablets, etc.)

[0643] Software: Generative AI models (e.g., GPT-4), sentiment analysis engines (e.g., Affectiva)

[0644] Specific operation of the system

[0645] 1. The user starts serving customers

[0646] The user turns on the device (PC or tablet) and opens the customer service application.

[0647] The user clicks the button to start serving customers.

[0648] 2. The device captures the customer's facial expression through the camera

[0649] The device's camera will automatically activate and recognize the customer's face.

[0650] The camera continues to capture your facial expressions in real time.

[0651] 3. The device sends the captured facial expression data to the server.

[0652] The facial expression data captured by the device is sent to the server at regular intervals.

[0653] Data is encrypted before transmission to ensure security.

[0654] 4. The server analyzes the facial expression data using an emotion analysis engine.

[0655] The server inputs the received facial expression data into an emotion analysis engine (e.g., Affectiva).

[0656] An emotion analysis engine analyzes facial expression data to identify customer emotions (e.g., confusion, joy, interest, etc.).

[0657] 5. The server sends the analysis results to the generative AI model

[0658] The server sends the analysis results to a generative AI model (e.g., GPT-4).

[0659] The analysis results include emotional information about the customer.

[0660] 6. Generative AI models generate slideshows

[0661] The generative AI model generates an appropriate slideshow based on the emotional information it receives.

[0662] For example, if a customer is confused, generate a slideshow with detailed explanations.

[0663] 7. The server sends the generated slideshow to the device.

[0664] The server transmits the generated slideshow to the terminal.

[0665] Slideshows will be sent in an appropriate format (e.g. PDF, PPT).

[0666] 8. The device displays a slideshow

[0667] The device will automatically display the received slideshow.

[0668] The user can continue serving customers while checking the slideshow.

[0669] 9. Users serve customers using slideshows

[0670] A user uses a slideshow to explain products and services to customers.

[0671] The slideshow can also accommodate multiple attendees, so multiple crew members can use it at the same time.

[0672] Specific examples

[0673] For example, when a user is explaining a new product, they input the following prompt sentence into the generative AI model:

[0674] Example prompt sentence:

[0675] "Generate a slideshow that succinctly explains the features of your new product. If your customers are confused, add a detailed explanation."

[0676] Based on this prompt, the generative AI model generates the following slideshow:

[0677] 1. Slide 1: New Product Overview

[0678] 2. Slide 2: Key Features and Benefits

[0679] 3. Slide 3: A brief explanation of how to use it

[0680] 4. Slide 4: When Confused Expressions are Detected, Detailed Instructions and FAQs

[0681] In this way, the user can provide appropriate service according to the customer's emotions. The flow of the identification process in the third embodiment will be described with reference to FIG.

[0682] Step 1:

[0683] The user starts serving customers.

[0684] The user starts up the device (PC or tablet) and opens the customer service application. The user clicks the button to start customer service. This switches the system into customer service mode, and the camera and sensors are ready to start up.

[0685] Step 2:

[0686] The device captures the customer's facial expression through the camera.

[0687] The device's camera automatically activates and recognizes the customer's face. The camera continues to capture the customer's facial expressions in real time. The input is video data of the customer's face, and the output is the captured facial expression data.

[0688] Step 3:

[0689] The facial expression data captured by the device is sent to the server.

[0690] The facial expression data captured by the device is sent to the server at regular intervals. The data is encrypted before transmission, ensuring security. The input is the captured facial expression data, and the output is the encrypted data sent to the server.

[0691] Step 4:

[0692] The server analyzes the facial expression data using an emotion analysis engine.

[0693] The server inputs the received facial expression data into an emotion analysis engine. The emotion analysis engine analyzes the facial expression data and identifies the customer's emotion (e.g., confusion, joy, interest, etc.). The input is encrypted facial expression data, and the output is analyzed emotional information.

[0694] Step 5:

[0695] The server sends the analysis results to the generative AI model.

[0696] The server sends the analysis results to the generative AI model. The analysis results include the customer's emotional information. The input is the analyzed emotional information, and the output is the data sent to the generative AI model.

[0697] Step 6:

[0698] A generative AI model generates the slideshow.

[0699] The generative AI model generates an appropriate slideshow based on the emotional information it receives. For example, if a customer is confused, it generates a slideshow with detailed explanations. The input is the emotional information and prompt text, and the output is the generated slideshow.

[0700] Step 7:

[0701] The server transmits the generated slideshow to the terminal.

[0702] The server sends the generated slideshow to the device. The slideshow is sent in the appropriate format (e.g. PDF, PPT). The input is the generated slideshow and the output is the slideshow sent to the device.

[0703] Step 8:

[0704] Your device will display a slideshow.

[0705] The terminal automatically displays the received slideshow. The user can continue serving customers while checking the slideshow. The input is the slideshow sent to the terminal, and the output is the displayed slideshow.

[0706] Step 9:

[0707] The user serves customers using a slide show.

[0708] The user explains products and services to customers using a slideshow. The slideshow can also accommodate multiple attendees, allowing multiple crew members to use it simultaneously. The input is the displayed slideshow, and the output is the explanation given to the customer.

[0709] (Application example 3)

[0710] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0711] With conventional customer service systems, it was difficult to respond flexibly to customer emotions, and it was difficult to have multiple staff members serving customers at the same time or to shorten the service time. This led to problems such as a decrease in customer satisfaction and a deterioration in customer service efficiency. Furthermore, it was not possible to standardize customer service, and each staff member had their own different service style, which led to the issue of inconsistent service quality.

[0712] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[0713] In this invention, the server includes a means for inputting information about the customer's environment during service, a generative AI means for generating a picture-story show based on the input information, a means for using the generated slideshow to present a proposal method and product appeal tailored to the customer, an emotion engine means for analyzing the customer's emotions, a generative AI means for adjusting the slideshow based on information from the emotion engine means, and a means designed for simultaneous use by multiple users. This allows for flexible responses based on customer emotions, enabling simultaneous service by multiple people and shortening service time. It also allows for standardization of service and maintains consistent service quality.

[0714] "Means for inputting information about the customer's environment during customer service" refers to devices or methods for collecting information to understand the customer's needs and situation and inputting it into the system.

[0715] "Generative AI means" refers to artificial intelligence technology that automatically generates content or materials for specific purposes based on input information.

[0716] A "picture-story show slideshow" is a format that uses a sequence of images or slides to tell a story or explain something visually.

[0717] "Methods of proposing and promoting products that suit customers" refers to methods and devices for proposing optimal products and services according to the customer's needs and situation, and effectively communicating their appeal.

[0718] "Emotion engine means" is a technology that analyzes emotions from the customer's facial expressions, tone of voice, etc., and provides that information to the system.

[0719] The "generative AI means for adjusting the slideshow based on information from the emotion engine means" is an artificial intelligence technology that dynamically changes the content and order of the slideshow based on the emotion information obtained from the emotion engine means.

[0720] "Means designed to enable simultaneous use by multiple users" refers to designs or technologies that allow multiple users to use the system simultaneously.

[0721] The following system configuration will be described as an embodiment of the present invention.

[0722] System Configuration

[0723] This system includes a means for inputting environmental information about customers during customer service, a generative AI means, an emotion engine means, and a means designed to be usable by multiple users simultaneously.

[0724] Hardware and Software

[0725] Hardware: Smart glasses (with camera and display)

[0726] software:

[0727] OpenCV: Acquiring and displaying camera images

[0728] EmotionEngine: A library for analyzing customer emotions

[0729] SlideGenerator: A library for generating emotion-sensitive slideshows

[0730] Processing flow

[0731] 1. Environmental information input: Using the smart glasses' camera, environmental information such as the customer's facial expression and tone of voice is collected in real time, allowing us to understand the customer's needs and situation.

[0732] 2. Emotion analysis: The acquired environmental information is analyzed by the Emotion Engine to identify the customer's emotions. For example, if the customer shows a confused expression, the emotion engine will determine that the emotion is "confused."

[0733] 3. Slideshow generation: Based on the emotional information obtained from the emotion engine, the generative AI generates an appropriate slideshow. For example, if a confused expression is detected, a slideshow with more detailed explanations will be generated.

[0734] 4. Displaying the slideshow: The generated slideshow is displayed on the smart glasses display, allowing the salesperson to make appropriate suggestions and appeal to the customer.

[0735] 5. Multi-user support: The system is designed to be used by multiple store associates simultaneously, allowing each associate to individually assist customers.

[0736] Specific examples

[0737] For example, imagine a customer in a store is considering purchasing a new smartphone. If the customer shows a confused expression, the camera in the smart glasses captures that expression and the Emotion Engine determines that the expression is "confused." Based on this information, the generative AI generates a slideshow that details the smartphone's functions and features, and displays it on the smart glasses' display. The store clerk can then provide an appropriate explanation to the customer by viewing the slideshow.

[0738] Prompt Sentence Examples

[0739] "The customer has a confused look on their face. Please generate a slideshow with detailed product descriptions."

[0740] In this way, this invention enables flexible responses according to customer emotions, allows multiple staff to serve customers simultaneously, and shortens the time it takes to serve them.It also standardizes customer service and maintains a consistent quality of service.

[0741] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[0742] Step 1:

[0743] The server uses the smart glasses' camera to capture environmental information such as the customer's facial expressions and tone of voice in real time. The input is the camera image, and the output is environmental information data. This data is used for subsequent emotion analysis. Specifically, the camera captures the image and sends the image data to the server.

[0744] Step 2:

[0745] The server analyzes the acquired environmental information data using the Emotion Engine to identify the customer's emotions. The input is environmental information data, and the output is emotional information. Specifically, the Emotion Engine analyzes the video data using a facial expression recognition algorithm and classifies emotions into categories such as "confusion," "happiness," and "interest."

[0746] Step 3:

[0747] The server sends a prompt to the generative AI based on the emotional information obtained from the emotion engine, and generates an appropriate slideshow. The input is emotional information, and the output is slideshow data. Specifically, the generative AI receives the prompt, "The customer has a confused expression. Please generate a slideshow that includes a detailed description of the product," and generates a slideshow based on that.

[0748] Step 4:

[0749] The server displays the generated slideshow on the display of the smart glasses. The input is the slideshow data, and the output is the slideshow displayed on the display. In specific operations, the slideshow data is sent to the smart glasses and displayed on the display.

[0750] Step 5:

[0751] The user (store clerk) provides an appropriate explanation to the customer while watching the slideshow displayed on the smart glasses display. The input is the slideshow displayed on the display, and the output is the explanation to the customer. Specifically, the store clerk checks the contents of the slideshow and explains the product details to the customer based on that.

[0752] Step 6:

[0753] The server manages the system so that multiple users can use it simultaneously. The input is the customer service status data for each user, and the output is the system's resource allocation data. Specifically, the server monitors the customer service status of each user and dynamically allocates resources as needed.

[0754] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0755] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0756] Another example of generative AI is Gemini (registered trademark) (Internet search engine). <url: https: gemini.google.com ?hl="ja">) are listed.

[0757] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0758] [Second embodiment]

[0759] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0760] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0761] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0762] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0763] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0764] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0765] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0766] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0767] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0768] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0769] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0770] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0771] "Example 1"

[0772] In this embodiment, the crew interviews the customer about their environment while serving them and inputs that information into the system. The input information is analyzed by generative AI, and a picture-story-style slideshow suited to the customer's situation is generated. The generated slideshow is used as a guide for the crew when presenting proposal methods and product appeals to the customer.

[0773] "Example 2"

[0774] As a specific example, if a crew member inputs information such as "I live in a newly built detached house" during a customer environment interview, the generative AI will generate a slideshow based on that information, including introductions to products and services suitable for newly built detached houses. The crew member will then use the generated slideshow to suggest products and services suitable for newly built detached houses.

[0775] "Example 3"

[0776] Generative AI also generates slideshows to shorten customer service time and enable multiple crew members to serve customers. For example, to shorten customer service time, the slideshow is designed to efficiently convey necessary information. To enable multiple crew members to serve customers, the slideshow is designed so that multiple crew members can use it simultaneously.

[0777] The processing flow of each embodiment will be described below.

[0778] "Example 1"

[0779] Step 1: When serving customers, the crew will ask about their environment and enter that information into the system.

[0780] Step 2: The input information is analyzed by generative AI.

[0781] Step 3: The generative AI generates a picture-story slideshow suited to the customer's situation.

[0782] Step 4: The generated slideshow will be used as a guide for the crew to present their proposals and products to the customer.

[0783] "Example 2"

[0784] Step 1: The crew will enter the information that "I live in a newly built detached house" during the customer's environmental interview.

[0785] Step 2: Based on this information, the generative AI generates a slideshow introducing products and services suitable for newly built single-family homes.

[0786] Step 3: The crew uses the generated slideshow to suggest products and services suitable for the newly built single-family home.

[0787] "Example 3"

[0788] Step 1: The generative AI generates a slideshow to shorten customer service time and enable multiple people to serve customers.

[0789] Step 2: To shorten customer service time, the slideshow is designed to efficiently convey the necessary information.

[0790] Step 3: To accommodate multiple guests, the slideshow is designed to be used by multiple crew members simultaneously.

[0791] Example 1

[0792] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0793] With the conventional customer service system, it was difficult to quickly and accurately provide suggestions that met the customer's needs, which tended to result in long service times. In addition, it was difficult to have multiple staff members provide service or standardize the service, and the system was heavily dependent on the skills of the crew. This resulted in the problem of inconsistent quality of customer service.

[0794] The identification process by the identification processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means. In this invention, the server includes a means for inputting customer environment interview information, a means for transmitting the input information to the server, a means for the server to transmit a prompt sentence to the generation AI model, a means for the generation AI model to perform analysis and generate a slideshow, a means for saving the generated slideshow and transmitting it to the terminal, a means for the terminal to display the slideshow, and a means for using the generated slideshow to present a proposal method and product appeal tailored to the customer. This reduces customer service time, enables multiple people to serve customers, and standardizes customer service.

[0795] "Customer environmental interview information" refers to information about customer needs and the environment collected by crew members while serving customers.

[0796] The "means of input" refers to an interface that allows crew members to input customer environmental hearing information using devices such as tablets or PCs.

[0797] The "means for transmitting to the server" is a communication means for encrypting information input from the terminal and transmitting the encrypted information to the server.

[0798] A "generative AI model" is an artificial intelligence model that analyzes input information and generates a slideshow that is appropriate for the customer's situation.

[0799] A "prompt statement" is an instruction statement created by the server to cause the generative AI model to perform analysis.

[0800] The "means for generating a slideshow" is a function that allows the generative AI model to analyze the prompt text and generate a picture-story show-style slideshow.

[0801] The "means for saving" is a function for saving the generated slide show in a presentation software format.

[0802] The "means for transmitting to the terminal" is a communication means for the server to transmit the generated slide show to the crew's terminal.

[0803] The "means for displaying a slide show" is a function for displaying a slide show received by a terminal.

[0804] The "method of proposal and means of appealing to merchandise" refers to a means by which the crew uses the generated slideshow to make specific proposals and appeal to the customer about merchandise.

[0805] This invention is a system that uses a generative AI model to generate a picture-story show-style slideshow based on environmental information about customers collected by the crew during customer service, and promotes proposal methods and products.

[0806] First, the user (crew member) uses a device such as a tablet or PC to input information about the customer's needs and environment. This information is input through a dedicated application. For example, specific information such as "The customer is considering a new office layout" is input.

[0807] The device then sends the entered information to the server in real time, using the HTTPS protocol to ensure security as the data is encrypted before transmission.

[0808] The server creates a prompt based on the received information and sends it to the generative AI model (e.g., GPT-4). The prompt might be something like, "The client is considering a new office layout. Please generate a slideshow about efficient desk arrangements and space utilization methods."

[0809] The generative AI model analyzes the prompt text and generates a picture-story-style slideshow that is appropriate for the customer's situation, including specific proposals and the selling points of the product.

[0810] The server saves the generated slideshow in a presentation software format (e.g., Microsoft PowerPoint or Google Slides), and then transmits the slideshow to the crew's device.

[0811] The device then displays the received slideshow, which the crew can use to provide specific suggestions to the customer. For example, a slideshow on optimizing office layout includes ideas for desk placement and efficient space utilization.

[0812] This system reduces customer service time, allows multiple people to serve customers, and standardizes customer service. An example of a specific prompt sentence is, "The customer is considering a new office layout. Please generate a slideshow about efficient desk arrangements and space utilization methods."

[0813] In this way, the system provides the crew with the tools to make optimal recommendations to customers based on the information they collect.

[0814] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0815] Step 1:

[0816] The user (crew) inputs the environmental hearing information.

[0817] Specifically, the crew uses a dedicated application on a tablet or PC to input information about the customer's needs and environment. For example, they input specific information such as, "The customer is considering the layout of a new office." The input information is temporarily saved in the application.

[0818] Input: Information about your needs and circumstances

[0819] Output: Temporarily stored hearing information

[0820] Step 2:

[0821] The terminal sends the input information to the server.

[0822] Specifically, the device encrypts the input information and sends it to the server using the HTTPS protocol. The server receives the data and stores it in a database.

[0823] Input: Temporarily stored hearing information

[0824] Output: Hearing information sent to the server

[0825] Step 3:

[0826] The server sends a prompt to the generative AI model.

[0827] Specifically, the server creates a prompt based on the received information. For example, it generates a prompt such as, "The client is considering a new office layout. Please generate a slideshow about efficient desk arrangements and space utilization methods." This prompt is then sent to the generative AI model.

[0828] Input: Hearing information stored on the server

[0829] Output: The prompt sent to the generative AI model

[0830] Step 4:

[0831] A generative AI model performs the analysis and generates a slideshow.

[0832] Specifically, a generative AI model (e.g., GPT-4) analyzes the prompt and generates a picture-story-style slideshow appropriate to the customer's situation, including specific proposals and selling points of the product.

[0833] Input: The prompt sent to the generative AI model

[0834] Output: Generated slideshow

[0835] Step 5:

[0836] The server saves the generated slideshow and transmits it to the device.

[0837] Specifically, the server saves the generated slideshow in a presentation software format (e.g., Microsoft PowerPoint or Google Slides), and then transmits the slideshow to the crew's device.

[0838] Input: Generated slideshow

[0839] Output: Slideshow sent to device

[0840] Step 6:

[0841] Your device will display a slideshow.

[0842] Specifically, the device displays the received slideshow. The crew member uses this slideshow to provide specific suggestions to the customer. For example, a slideshow on optimizing office layout includes ideas for desk arrangement and efficient space utilization.

[0843] Input: Slideshow sent to device

[0844] Output: Rendered slideshow

[0845] In this way, the system provides the crew with the tools to make optimal recommendations to customers based on the information they collect.

[0846] (Application example 1)

[0847] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0848] With the conventional customer service system, it was difficult to respond to customer needs quickly and accurately, which tended to result in long service times. Also, since the quality of customer service depended on the experience and skills of the crew, it was difficult to provide a consistent service. Furthermore, it was difficult to serve multiple customers at the same time, so efficient customer service was required.

[0849] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0850] In this invention, the server includes a means for inputting information about the customer's environment during the customer service session, a generation system AI means for generating a picture-story slideshow based on the input information, a means for using the generated slideshow to present a proposal method and product appeal tailored to the customer, a means for accepting voice input, a voice recognition means for converting voice into text, and a display means for displaying the generated slideshow. This allows for quick and accurate response to customer needs, shortening customer service time and enabling simultaneous response to multiple customers. It also standardizes the quality of customer service and realizes efficient service provision.

[0851] "Means for inputting environmental information from customers during customer service" refers to a means for collecting customer needs and environmental information through dialogue with customers and inputting that information into the system.

[0852] The "generative AI means for generating a picture-story show-style slideshow based on input hearing" refers to an artificial intelligence means for analyzing collected customer information and generating a picture-story show-style slideshow to make optimal proposals to customers.

[0853] "Means for making proposals and promoting products that suit customers using generated slideshows" refers to means for making appropriate proposals and promoting products to customers using generated slideshows.

[0854] The "means for accepting voice input" is a means for the crew to input the customer's needs by voice.

[0855] "Speech recognition means for converting speech into text" refers to means for analyzing input speech and converting it into text data.

[0856] The "display means for displaying the generated slide show" refers to a means for visually displaying the generated slide show.

[0857] A system for implementing this invention has the following configuration. First, it includes a voice input means for the crew to input customer needs and environmental information by voice while serving customers. This voice input means uses a microphone installed in a device such as smart glasses or a smartphone.

[0858] Next, the information input by voice is converted into text data by a voice recognition means. This voice recognition means uses a SpeechRecognition library or the like. The voice recognition means analyzes the voice data and converts it into text data.

[0859] The converted text data is sent to a generative AI means, which uses a generative AI model such as OpenAI GPT-3. This generative AI means analyzes the input text data and generates a picture-story show-style slideshow based on the customer's needs.

[0860] The generated slideshow is visually displayed by a display means, which may be a display of smart glasses or a smartphone screen. The display means provides the generated slideshow to the crew and serves as a guide for making effective proposals and promoting products to customers.

[0861] As a specific example, if a customer says, "I'm looking for a new smartphone, especially one with a good camera," the speech recognition means converts this speech into text, and the generative AI means generates the following prompt sentence:

[0862] Example prompt sentence:

[0863] "Based on the customer's needs, generate a Kamishibai-style slideshow with the following information: I'm looking for a new smartphone, especially one with a good camera."

[0864] Based on this prompt, the generative AI system generates a slideshow comparing the latest smartphone camera features, including the features and pricing information for each model. The generated slideshow is displayed on the smart glasses display and serves as a guide for the crew to make effective recommendations to the customer.

[0865] In this way, the present invention responds to customer needs quickly and accurately, shortens customer service time, and enables simultaneous service for multiple customers. It also standardizes the quality of customer service and provides efficient service.

[0866] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0867] Step 1:

[0868] The user uses the microphone on the smart glasses or smartphone to input customer needs and environmental information by voice.

[0869] Input: Customer voice information

[0870] Output: Audio data

[0871] Specific operation: The user interacts with the customer and inputs the customer's requests and questions by voice.

[0872] Step 2:

[0873] The device converts the voice input information into text data using a voice recognition method (e.g., SpeechRecognition library).

[0874] Input: Audio data

[0875] Output: Text data

[0876] Specific operation: The device analyzes the voice data and applies a voice recognition algorithm to convert it into text data.

[0877] Step 3:

[0878] The device sends the converted text data to a generative AI method (e.g., OpenAI GPT-3).

[0879] Input: Text data

[0880] Output: Request to generative AI

[0881] Specific operation: The device creates and sends a request to send text data to a generative AI model in the cloud.

[0882] Step 4:

[0883] The server uses generative AI means to generate a picture-story show-style slideshow based on the input text data.

[0884] Input: Text data

[0885] Output: Kamishibai style slideshow

[0886] Specific operation: The server inputs prompt text into the generative AI model and generates a slideshow based on the customer's needs.

[0887] Step 5:

[0888] The server transmits the generated slideshow to the terminal.

[0889] Input: Kamishibai style slideshow

[0890] Output: Slideshow data to the device

[0891] Specific operation: The server performs communication processing to transmit the generated slideshow data to the terminal.

[0892] Step 6:

[0893] The terminal displays the received slideshow on a display means (e.g., a display of smart glasses or a smartphone screen).

[0894] Input: Slideshow data

[0895] Output: Rendered slideshow

[0896] Specific operation: The terminal transmits the received slideshow data to the display device and displays it visually.

[0897] Step 7:

[0898] Based on the displayed slideshow, the user can make proposals and appeal to customers about products.

[0899] Input: Displayed slideshow

[0900] Output: Proposals and appeals to customers

[0901] Specific operation: The user watches the slideshow and makes appropriate suggestions and explains products to the customer.

[0902] Example 2

[0903] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0904] With conventional customer service systems, it took a lot of time and effort to propose appropriate products and services based on customer environment information. Furthermore, the quality of customer service varied depending on the staff member, making it difficult to provide consistent service. Furthermore, it was difficult to provide efficient service in a short amount of time with multiple staff members.

[0905] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0906] In this invention, the server includes means for inputting customer environment information, means for generating prompt sentences based on the input hearing information, means for calling a generative AI model using the generated prompt sentences to generate a slideshow, and means for proposing products and services suitable for the customer using the generated slideshow. This reduces customer service time, enables multiple people to serve customers, and further standardizes customer service.

[0907] "Customer environmental interview information" refers to information about the customer's current living environment and housing situation.

[0908] An "input means" is an interface or device that allows a user to input information into a system.

[0909] A "prompt" is an instruction that instructs a generative AI model to generate a specific output.

[0910] A "generative AI model" is an artificial intelligence model that generates a specific output based on an input prompt.

[0911] A "slide show" is a presentation material in which multiple slides are displayed in sequence.

[0912] The "means of proposal" refers to a method or tool for explaining and proposing products or services to customers using the generated slideshow.

[0913] A "server" is a computer system for processing and storing data.

[0914] This invention is a system that generates and proposes slideshows using a generative AI model based on customer environment interview information. Specific embodiments of this system are described below.

[0915] Server Roles

[0916] The server receives the customer's environmental interview information entered by the crew member on their device. This information includes the data that "you live in a newly built detached house." Based on this information, the server generates a prompt to be input into the generative AI model. Specifically, the server creates a prompt that reads, "Please generate a slideshow introducing products and services suitable for customers who live in newly built detached houses."

[0917] Next, the server uses the generated prompt to call a generative AI model (for example, OpenAI's GPT-4). The prompt is input into the AI ​​model to generate the content for the slideshow. Based on the content returned by the AI ​​model, the server generates the slideshow. The slideshow is created in PowerPoint or PDF format, for example. Finally, the server sends the generated slideshow to the crew's device.

[0918] Device Role

[0919] The terminal receives the customer's environmental interview information entered by the crew. For example, the information entered is "I live in a newly built detached house." The terminal then sends the entered information to the server. The terminal then receives the slideshow sent from the server. To display the received slideshow, the terminal uses software such as Microsoft PowerPoint or Google Slides. The crew then uses this software to show the slideshow to the customer.

[0920] User (Crew) Roles

[0921] The user (crew member) interviews the customer about their living environment and inputs the information that they "live in a newly built detached house" into the terminal. Next, the user checks the slideshow sent from the server on the terminal and proposes products and services suitable for the newly built detached house. The crew member uses the slideshow to explain specific products and services to the customer.

[0922] Specific examples

[0923] Example of information entered by crew

[0924] "You live in a newly built detached house."

[0925] Example prompts to input to the generative AI model

[0926] "Generate a slideshow showcasing products and services suitable for customers living in newly constructed single-family homes."

[0927] Example of generated slideshow content

[0928] 1. Slide 1: Title: "Best Products and Services for Newly Built Single-Family Homes"

[0929] 2. Slide 2: Product Introduction "Latest Security System"

[0930] 3. Slide 3: Service Introduction "Garden Design and Maintenance Service"

[0931] 4. Slide 4: Product Introduction "Energy-efficient Home Appliances"

[0932] 5. Slide 5: Service Introduction "Home Automation Installation Service"

[0933] The crew uses this slideshow to suggest products and services suitable for newly built single-family homes to customers.

[0934] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0935] Step 1:

[0936] The user (crew) interviews the customer about their living environment and collects information such as "I live in a newly built detached house." The user then enters this information into the device. The entered information is temporarily stored in the device's database.

[0937] Input: Customer's environmental information (e.g., "I live in a newly built detached house")

[0938] Output: Hearing information stored on the device

[0939] Step 2:

[0940] The terminal transmits the input hearing information to the server. The terminal uses a communication module to transmit data to the server via a network.

[0941] Input: Hearing information stored on the device

[0942] Output: Hearing information sent to the server

[0943] Step 3:

[0944] The server generates a prompt based on the received information. Specifically, it creates the prompt, "Please generate a slideshow introducing products and services suitable for customers living in newly built detached houses."

[0945] Input: Hearing information sent to the server

[0946] Output: Generated prompt statement

[0947] Step 4:

[0948] The server calls the generative AI model using the generated prompt sentence. The server inputs the prompt sentence into the generative AI model (e.g., GPT-4) to generate the content for the slideshow.

[0949] Input: Generated prompt text

[0950] Output: The slideshow content returned by the generative AI model

[0951] Step 5:

[0952] The server generates a slideshow based on the content returned by the generative AI model, for example in PowerPoint or PDF format.

[0953] Input: Slideshow content returned by a generative AI model

[0954] Output: Generated slideshow (PowerPoint and PDF formats)

[0955] Step 6:

[0956] The server transmits the generated slideshow to the crew's terminals. The server uses a communication module to transmit data to the terminals via a network.

[0957] Input: Generated slideshow

[0958] Output: Slideshow sent to device

[0959] Step 7:

[0960] The device receives the slideshow sent from the server and uses software such as Microsoft PowerPoint or Google Slides to display the received slideshow.

[0961] Input: Slideshow sent from server

[0962] Output: Slideshow displayed on device

[0963] Step 8:

[0964] The user (crew member) checks the slideshow received on the terminal and proposes products and services suitable for the newly built detached house. The crew member uses the slideshow to explain specific products and services to the customer.

[0965] Input: Slideshow displayed on device

[0966] Output: Products and services proposed to the customer

[0967] (Application example 2)

[0968] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0969] With conventional customer service systems, it was difficult to propose optimal products and services based on customer needs, which often resulted in long customer service times. Furthermore, because the quality of customer service depended on the skills and experience of the salesperson, there was also the problem of inconsistent quality. Furthermore, it was difficult to serve multiple customers at the same time.

[0970] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for inputting environmental interviews with customers during customer service, generative AI means for generating a picture-story show-style slideshow based on the input interviews, means for using the generated slideshow to present a proposal method and product appeal suited to the customer, means installed on the smart device for allowing the store clerk to propose optimal products and services based on the customer's needs, and means for displaying the generated slideshow. This makes it possible to efficiently propose optimal products and services based on the customer's needs, as well as shorten the time required for customer service and handle multiple customers simultaneously.

[0971] The "means for inputting information about the customer's environment during customer service" is an interface that allows store staff to collect information about the customer's living environment, needs, etc., and input it into the system.

[0972] "Generative AI means that generates a picture-story show-style slideshow based on input interview information" is an artificial intelligence technology that automatically generates a slideshow introducing optimal products and services based on collected customer information.

[0973] "Means of using generated slideshows to present proposals and products that suit customers" refers to a method of using generated slideshows to propose appropriate products and services to customers and effectively communicate their appeal.

[0974] "A means installed on a smart device that enables store clerks to suggest the most suitable products and services based on the customer's needs" refers to an application that is installed on a device such as a smartphone or tablet, and enables store clerks to input customer information, display the generated slideshow, and make suggestions.

[0975] The "means for displaying the generated slideshow" is a function for displaying the slideshow created by the generative AI means on the screen of the smart device.

[0976] A system for implementing this invention includes a means for inputting environmental information about a customer when serving them, a generative AI means for generating a picture-story slideshow in the form of a picture book based on the input information, a means for using the generated slideshow to present a proposal method and product appeal that is suited to the customer, a means that is installed on a smart device and enables a store clerk to propose optimal products and services based on the customer's needs, and a means for displaying the generated slideshow.

[0977] System program processing explanation

[0978] Hardware and Software

[0979] Hardware: smartphones, tablets, servers

[0980] Software: Python, OpenAI API, device application

[0981] Data processing and calculation

[0982] 1. Entering customer information: A salesperson uses a smart device to enter information about the customer's living environment and needs. For example, they enter information such as "I live in a newly built detached house."

[0983] 2. Utilizing generative AI models: Based on the customer information entered, a generative AI model (e.g., GPT-3) generates a slideshow introducing the most suitable products and services.

[0984] 3. Displaying the slideshow: The generated slideshow is displayed on the smart device screen, and the store clerk makes suggestions to the customer.

[0985] Specific examples

[0986] For example, if a customer enters "I live in a newly built detached house," the AI ​​will generate a slideshow like the one below.

[0987] Slide 1: Solar power generation system that supports eco-friendly living

[0988] Slide 2: The best smart home devices for new homes

[0989] Slide 3: Energy-efficient appliances

[0990] Prompt Sentence Examples

[0991] "Generate a slideshow showcasing products and services suitable for customers living in new homes. Customer Information: New home, eco-friendly lifestyle."

[0992] In this way, it is possible to efficiently propose optimal products and services based on customer needs, shorten customer service time, and handle multiple customers simultaneously.

[0993] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0994] Step 1:

[0995] The user uses a smart device to input customer information, specifically the customer's living environment and needs (e.g., "I live in a newly built detached house"). The input information is saved in the application on the device.

[0996] Input: Customer's living environment and needs

[0997] Output: Stored customer information

[0998] Step 2:

[0999] The device sends the stored customer information to the server, which analyzes the received customer information and generates a prompt to be passed to the generative AI model.

[1000] Input: Saved customer information

[1001] Output: Generated prompt statement

[1002] Step 3:

[1003] The server sends a request to a generative AI model (e.g., GPT-3) using the generated prompt, which then generates slideshow content to showcase the most suitable products and services.

[1004] Input: Generated prompt text

[1005] Output: The generated slideshow content

[1006] Step 4:

[1007] The server receives the generated slideshow content and transmits it to the terminal, which converts the received slideshow content into data for display.

[1008] Input: Generated slideshow content

[1009] Output: Slideshow data for display

[1010] Step 5:

[1011] The terminal uses the slideshow data for display to display a slideshow on the smart device screen, and the store clerk uses this slideshow to suggest the most suitable products and services to the customer.

[1012] Input: Slideshow data to display

[1013] Output: Slideshow displayed on smart device

[1014] In this way, it is possible to efficiently propose optimal products and services based on customer needs, shorten customer service time, and handle multiple customers simultaneously.

[1015] Example 3

[1016] Next, a description will be given of Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1017] With the conventional customer service system, customer service times tended to be long and it was difficult for multiple staff to serve customers. In addition, the content of customer service often varied depending on the staff member, making it difficult to provide a consistent service. This led to problems such as a decline in customer satisfaction and a deterioration in efficiency.

[1018] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[1019] In this invention, the server includes means for a user to input a prompt text, means for a terminal to send the prompt text to the server, means for the server to receive the prompt text and input it to a generative AI model, means for the generative AI model to generate slideshow content based on the prompt text, means for the server to input the generated content to slideshow generation software, means for the slideshow generation software to generate a visually formatted slideshow, means for the server to send the generated slideshow to the terminal, and means for the terminal to display the slideshow to the user. This reduces customer service time, enables multiple people to serve customers, and further standardizes customer service.

[1020] A "user" is an entity that uses the system to input prompt sentences and generate a slide show.

[1021] A "prompt sentence" is an instruction sentence entered by the user, and is the information that the generative AI model uses to generate the content of the slideshow.

[1022] A "terminal" is a device through which a user enters prompts and communicates with a server.

[1023] The "server" is a computer system that receives prompt text, generates a slideshow using a generative AI model, and sends it to the device.

[1024] A "generative AI model" is an artificial intelligence algorithm that generates slideshow content based on prompt text.

[1025] "Slideshow generation software" is software that visually formats the content generated by a generative AI model and outputs it as a slideshow.

[1026] A "slideshow" is a visually stylized series of slides generated by generative AI models and slideshow generation software.

[1027] This invention is a system that generates a slideshow using a generative AI model, shortening the time it takes to serve customers and enabling multiple people to serve customers. Specific embodiments of this system are described below.

[1028] First, the user inputs a prompt using the device. The prompt is the information that the generative AI model uses to generate the content of the slideshow. For example, the user might input a prompt such as, "I would like you to create a slideshow that explains the features of our new product."

[1029] The device then sends the prompt to a server, which receives it and inputs it into a generative AI model, such as an advanced natural language processing model like GPT-4.

[1030] The generative AI model generates slideshow content based on prompts, which are presented in text format and include information such as the features, benefits, and usage of new products.

[1031] The server then inputs the generated text into slideshow generation software, such as the Microsoft PowerPoint API or Google Slides API, which visually formats the generated text and outputs it as a slideshow.

[1032] The generated slideshow is sent from the server to the terminal, which then displays it to the user. The user can then review the generated slideshow and make any necessary corrections or additions.

[1033] For example, if a user inputs the prompt "Please create a slideshow that explains the features of a new product," the generative AI model generates text describing the new product's features, benefits, and usage. The model then uses the Microsoft PowerPoint API to generate a visually formatted slideshow and sends it to the device. The user can then review the generated slideshow and make any necessary corrections or additions.

[1034] In this way, the slideshow generation system using the generative AI model functions as a tool for shortening customer service time and efficiently serving customers with multiple people. It is also possible to standardize customer service. The flow of the identification process in Example 3 will be described with reference to FIG. 15.

[1035] Step 1:

[1036] The user enters a prompt statement.

[1037] The user inputs a prompt into the input field of the device, for example, "I want you to create a slideshow that explains the features of our new product." The input prompt is temporarily stored in the device's memory.

[1038] Step 2:

[1039] The terminal sends the prompt to the server.

[1040] The terminal sends the prompt text entered by the user to the server. HTTP or HTTPS is used as the communication protocol. The input is the prompt text, and the output is a notification to the server that transmission has been completed.

[1041] Step 3:

[1042] The server receives the prompt sentence and inputs it into the generative AI model.

[1043] The server receives the prompt sentence sent from the terminal and inputs the received prompt sentence into the generative AI model. The input is the prompt sentence, and the output is a notification to the generative AI model that the input has been completed.

[1044] Step 4:

[1045] A generative AI model generates slideshow content based on prompts.

[1046] A generative AI model generates content for each slide in a slideshow based on a prompt, such as text about the features, benefits, and usage of a new product. The input is the prompt, and the output is the generated text.

[1047] Step 5:

[1048] The server inputs the generated content into the slideshow generation software.

[1049] The server inputs the text obtained from the generative AI model into the slideshow generation software. The input is the generated text, and the output is a notification to the slideshow generation software that the input has been completed.

[1050] Step 6:

[1051] Slideshow generation software generates a visually formatted slideshow.

[1052] Slideshow generation software generates a visually formatted slideshow from input text, for example by applying an appropriate layout or design to each slide. The input is the generated text and the output is a visually formatted slideshow.

[1053] Step 7:

[1054] The server transmits the generated slideshow to the terminal.

[1055] The server sends the generated slideshow to the device using HTTP or HTTPS as the communication protocol. The input is the visually formatted slideshow, and the output is a notification of completion of transmission to the device.

[1056] Step 8:

[1057] The terminal displays the slideshow to the user.

[1058] The terminal displays the slideshow received from the server to the user. The user can review the generated slideshow and make corrections or additions as needed. The input is a visually formatted slideshow, and the output is the display to the user.

[1059] (Application example 3)

[1060] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1061] With conventional customer service systems, each customer service representative must deal with each customer individually, which results in long service times and makes it difficult to serve multiple customers at the same time. Another problem is that the quality of customer service varies depending on the representative. Furthermore, while there is a demand for efficient information provision using smart devices, the current system is unable to adequately address this.

[1062] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[1063] In this invention, the server includes means for inputting information about the customer's environment during customer service, a generation system AI means for generating a picture-story show-style slideshow based on the input information, means for using the generated slideshow to present a proposal method and product appeal tailored to the customer, means for displaying the slideshow on a smart device, and means for generating and displaying individual slideshows so that multiple customer service representatives can simultaneously serve different customers. This reduces customer service time and enables simultaneous service to multiple customers, resulting in standardized customer service quality and efficient information provision using smart devices.

[1064] "Means for inputting environmental information about customers during customer service" refers to devices or methods for collecting information about customers' needs, interests, and environment and inputting it into the system.

[1065] A "picture-story show slideshow" is a presentation format that conveys information visually by displaying a series of images and text in sequence.

[1066] "Generative AI means" refers to technologies or devices that use artificial intelligence to automatically generate content based on input data.

[1067] "Methods of proposing and promoting products that suit customers" refers to methods and devices for proposing optimal products and services based on customers' needs and interests, and effectively communicating their appeal.

[1068] "Means for displaying a slideshow on a smart device" refers to techniques or devices for displaying the generated slideshow on a device such as smart glasses or a smartphone.

[1069] "Means for generating and displaying individual slideshows so that multiple customer service representatives can simultaneously serve different customers" refers to technology or devices that allow multiple customer service representatives to simultaneously generate and display individually customized slideshows for different customers.

[1070] The system for implementing this invention includes a means for inputting information about the customer's environment when serving customers, a generative AI means, a means for displaying a slideshow on a smart device, and a means for generating and displaying individual slideshows so that multiple staff members can serve different customers simultaneously.

[1071] System configuration

[1072] 1. A way to input information about the customer's environment during customer service

[1073] This is a device that uses input devices such as tablets and smartphones to collect information about customers' needs, interests, and environment and input it into the system.

[1074] 2. Generative AI means

[1075] The generative AI method uses generative AI models such as the OpenAI API to automatically generate a picture-story show based on the input interview data. The generated slideshow is used to present optimal proposal methods and product appeal to customers.

[1076] 3. How to display slideshows on smart devices

[1077] A technology or device that displays generated slideshows on devices such as smart glasses (e.g., Google Glass, Vuzix Blade) or smartphones, allowing customer service staff to provide information visually to customers.

[1078] 4. A way to generate and display individual slideshows for multiple salespeople to simultaneously serve different customers

[1079] The generative AI method allows multiple customer service representatives to simultaneously generate and display individually customized slideshows for different customers, making it possible to serve multiple customers at the same time.

[1080] Program processing explanation

[1081] The server first receives data collected through a means for inputting information about the customer's environment during service. Next, it uses a generative AI means to generate a picture-story show-style slideshow based on the input data. The generated slideshow is displayed on a smart device, which may include smart glasses or a smartphone. Furthermore, the generative AI means can generate and display individual slideshows so that multiple service representatives can simultaneously serve different customers.

[1082] Specific examples

[1083] For example, if a customer asks, "What are the features of my new smartphone?", the salesperson uses a tablet to input this question into the system. The generative AI means generates a slideshow based on this input and displays it on the smart glasses. If another salesperson is asked by another customer, "What are the latest fashion items?", a separate slideshow is similarly generated and displayed.

[1084] Prompt Sentence Examples

[1085] "Please tell me the features of your new smartphone."

[1086] In this way, the system can efficiently and effectively handle customer inquiries.

[1087] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[1088] Step 1:

[1089] Users use input devices such as tablets and smartphones to input information about their needs, interests, and surroundings. The input data is then sent to the server.

[1090] Input: Information about your needs, interests, and circumstances

[1091] Output: Hearing data sent to the server

[1092] Step 2:

[1093] The server passes the received hearing data to the generative AI means, which uses a generative AI model such as the OpenAI API to generate a picture-story show-style slideshow based on the input data.

[1094] Input: Hearing data

[1095] Output: Generated slideshow

[1096] Step 3:

[1097] The server transmits the generated slideshow to a smart device, which may include smart glasses or a smartphone.

[1098] Input: Generated slideshow

[1099] Output: Slideshow sent to smart device

[1100] Step 4:

[1101] The smart device displays the received slideshow, and the customer service representative provides the information visually to the customer through the smart device.

[1102] Input: Slideshow sent to smart device

[1103] Output: Slideshow displayed on smart device

[1104] Step 5:

[1105] When multiple customer service representatives are simultaneously serving different customers, the server passes each representative's interview data to the generative AI means to generate individual slideshows, which are then sent to and displayed on each smart device.

[1106] Input: Multiple hearing data

[1107] Output: Individually generated slideshow

[1108] In this way, the system can efficiently and effectively handle customer inquiries.

[1109] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1110] "Example 1"

[1111] One embodiment of the present invention is a system that incorporates an emotion engine. In this system, the environment of the customer is interviewed during the customer service session, and that information is input into the system. The input information is analyzed by a generative AI, which generates a picture-story slideshow suited to the customer's situation. The generated slideshow is used as a guide for the crew when presenting proposal methods and product appeals to the customer. Furthermore, the emotion engine recognizes the customer's emotions and adjusts the content of the slideshow and the customer service style based on those emotions.

[1112] "Example 2"

[1113] As a specific example, consider the case where a crew member inputs information about a customer's living environment, such as "I live in a newly built detached house," and the customer shows a pleased expression. Based on this information and the emotion recognition results of the emotion engine, the generative AI generates a slideshow introducing products and services suitable for a newly built detached house. The crew member then uses the generated slideshow to suggest products and services that the customer would enjoy.

[1114] "Example 3"

[1115] The emotion engine also adjusts the customer service style according to the customer's emotions. For example, if a customer shows a confused expression, the emotion engine conveys that information to the generative AI, which then generates a slideshow with more detailed explanations. This makes it possible to provide customer service that is easy for the customer to understand.

[1116] The processing flow of each embodiment will be described below.

[1117] "Example 1"

[1118] Step 1: Interview the customer about their environment during the service and enter that information into the system. Step 2: The input information is analyzed by generative AI, and a picture-story-style slideshow appropriate to the customer's situation is generated.

[1119] Step 3: The generated slideshow will be used as a guide for the crew to present their proposals and products to the customer.

[1120] Step 4: The emotion engine recognizes the customer's emotions and adjusts the slideshow content and customer service style based on those emotions.

[1121] "Example 2"

[1122] Step 1: The crew will enter the information that "I live in a newly built detached house" during the customer's environmental interview.

[1123] Step 2: The emotion engine recognizes facial expressions that the customer is likely to be happy with.

[1124] Step 3: Based on this information and the emotion recognition results from the emotion engine, the generative AI generates a slideshow introducing products and services suitable for newly built single-family homes.

[1125] Step 4: The crew uses the generated slideshow to suggest products and services that the customer might enjoy.

[1126] "Example 3"

[1127] Step 1: The emotion engine adjusts the customer service style according to the customer's emotions.

[1128] Step 2: For example, if the customer shows a confused expression, the emotion engine conveys that information to the generative AI.

[1129] Step 3: The generative AI generates a slideshow with more detailed explanations.

[1130] Step 4: This allows for customer service that is easy for customers to understand.

[1131] Example 1

[1132] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1133] With conventional customer service systems, it was difficult to quickly and effectively make proposals based on the customer's needs and situation. It was also difficult to standardize the quality of customer service and efficiently handle customer service with multiple staff. Furthermore, it was not possible to grasp the customer's emotions in real time and respond accordingly, making it difficult to improve customer satisfaction.

[1134] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1135] In this invention, the server includes: a means for inputting information about the customer's environment during customer service; a means for transmitting the input information to the server; a means for passing the information received by the server to a generative AI model for analysis; a means for transmitting the generated picture-story slideshow from the server to a terminal; a means for using the generated slideshow to present a proposal method and product appeal tailored to the customer; and a means for analyzing the customer's emotions using an emotion engine and adjusting the content of the slideshow and the customer service style based on the emotions. This enables prompt and effective proposals tailored to customer needs, standardizes the quality of customer service, and enables efficient customer service across multiple staff. Furthermore, by understanding customer emotions in real time and responding accordingly, customer satisfaction can be improved.

[1136] The "means for inputting information about the customer's environment when serving customers" is a function that provides an interface for users to input information about the customer's needs and situation using a terminal.

[1137] The "means for transmitting input hearing information to a server" is a communication function that enables the terminal to transmit information input by a user to a server.

[1138] "Means for passing the information received by the server to the generative AI model for analysis" refers to the processing function for passing the customer information received by the server to the generative AI model for analysis.

[1139] "Means for transmitting the generated picture-story show-style slideshow from the server to the terminal" refers to a communication function for transmitting the slideshow generated by the generative AI model from the server to the terminal.

[1140] "Means of using the generated slideshow to present a proposal method and product appeal that is suited to the customer" is a function that allows the user to present a generated slideshow to the customer using the terminal and appeal to the customer with a proposal method or product appeal.

[1141] "A means of analyzing customer emotions using an emotion engine and adjusting the content of the slideshow and customer service style based on those emotions" is a function that analyzes the customer's facial expressions using the device's camera and adjusts the content of the slideshow and customer service style based on that emotional data.

[1142] This invention is a system that listens to the customer's environment while serving them, generates a picture-story show-style slideshow using a generative AI model based on that information, and then uses an emotion engine to adjust the content of the slideshow and the serving style based on the customer's emotions.

[1143] The user uses a device (e.g., a tablet or PC) to conduct an environment interview with the customer. Specifically, the user uses a dedicated application on the device (e.g., "Customer Insight App") to input information about the customer's needs and situation. The input information is sent from the device to the server. The HTTPS protocol is used for transmission to ensure data security.

[1144] The server passes the received information to a generative AI model (e.g., "OpenAI GPT-4") for analysis. Based on the input information, the generative AI model generates a picture-story show-style slideshow suited to the customer's situation. The generated slideshow is then sent from the server to the device.

[1145] The user presents the generated slideshow to the customer using the terminal, which is used as a guide for presenting proposals and promoting products.

[1146] Furthermore, if an emotion engine (such as "Affectiva Emotion AI") is built in, it analyzes the customer's facial expressions in real time through the device's camera. The analyzed emotion data is sent from the device to a server. The server adjusts the content of the slideshow and customer service style based on the received emotion data. For example, if the customer shows no interest, it sends another prompt to the generative AI model, instructing it to generate a more appropriate slideshow.

[1147] As a concrete example, a user uses the "Customer Insight App" to input that they have just purchased a new home. This information is sent to a server and analyzed by a generative AI model (OpenAI GPT-4). The generative AI model generates a picture-story show-style slideshow containing relevant suggestions for the customer's new home (e.g., furniture and interior design suggestions).

[1148] An example of a prompt is as follows:

[1149] "A client just purchased a new home and is interested in furniture and interior design. Generate a Kamishibai-style slideshow with suggestions tailored to their needs."

[1150] Based on this prompt, the generative AI model generates a slideshow that is appropriate for the customer's situation. If an emotion engine is built in, it analyzes the customer's facial expressions via the camera while they are viewing the slideshow, and if the customer shows no interest, it sends that data to the server. The server then sends the prompt to the generative AI model again, instructing it to generate a more appropriate slideshow.

[1151] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1152] Step 1:

[1153] The user uses the terminal to interview the customer about their environment.

[1154] Users launch the "Customer Insight App" on their device and input information about their needs and circumstances, such as that they have just purchased a new home or are interested in a particular interior design.

[1155] Input: Information about your needs and circumstances

[1156] Output: Input hearing information

[1157] Step 2:

[1158] The terminal transmits the input hearing information to the server.

[1159] The device transmits the information entered by the user to the server in real time using the HTTPS protocol to ensure data security.

[1160] Input: Hearing information entered

[1161] Output: Hearing information sent to the server

[1162] Step 3:

[1163] The information received by the server is passed to the generative AI model for analysis.

[1164] The server passes the received information to a generative AI model (such as "OpenAI GPT-4") for analysis. Based on the input information, the generative AI model generates a picture-story show-style slideshow suited to the customer's situation.

[1165] Input: Hearing information sent to the server

[1166] Output: A generated Kamishibai-style slideshow

[1167] Step 4:

[1168] The server transmits the generated picture-story show format slide show to the terminal.

[1169] The server sends the slideshow generated by the generative AI model to the device, also using the HTTPS protocol.

[1170] Input: A generated Kamishibai-style slideshow

[1171] Output: Slideshow sent to device

[1172] Step 5:

[1173] The user uses the device to present the slideshow to the customer.

[1174] The user presents the received slideshow to the customer on the terminal. The slideshow is used as a guide for presenting proposals and promoting products.

[1175] Input: Slideshow sent to device

[1176] Output: Slideshow presented to the customer

[1177] Step 6:

[1178] If an emotion engine is built in, the device will analyze the customer's emotions through the camera and send them to the server.

[1179] If an emotion engine (such as "Affectiva Emotion AI") is installed, the device's camera will analyze the customer's facial expressions in real time. The analyzed emotion data will be sent from the device to a server.

[1180] Input: Customer's facial expression data

[1181] Output: Emotion data sent to the server

[1182] Step 7:

[1183] The server adjusts the content of the slideshow and customer service style based on the emotional data.

[1184] The server adjusts the content of the slideshow and customer service style based on the received emotional data. For example, if the customer shows no interest, it sends another prompt to the generative AI model to instruct it to generate a more appropriate slideshow.

[1185] Input: Emotion data sent to the server

[1186] Output: Adjusted slideshow content and customer service style

[1187] (Application example 1)

[1188] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1189] Conventional customer service systems made uniform suggestions based on information gathered from customer environment interviews, making it difficult to respond flexibly to individual customer emotions and situations. Furthermore, crew members were unable to grasp customers' emotions in real time while serving them and adjust their suggestions accordingly, resulting in inconsistent customer service quality. Furthermore, there was a lack of tools for crew members to analyze their gaze and facial expressions, making it difficult to improve customer service efficiency.

[1190] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1191] In this invention, the server includes: a means for inputting information about the customer's environment during customer service; a generative AI means for generating a picture-story slideshow based on the input information; a means for using the generated slideshow to present a proposal method and product appeal tailored to the customer; an emotion recognition means for recognizing the customer's emotions and adjusting the content of the slideshow and customer service style based on the emotions; a means for displaying the slideshow on smart glasses; and a means for capturing and analyzing the crew's gaze and the customer's facial expressions in real time. This enables flexible proposals based on the customer's emotions and situation, improving the quality of customer service. Furthermore, the crew's analysis of gaze and facial expressions can improve the efficiency of customer service.

[1192] "Means for inputting environmental information about customers during customer service" refers to devices or software that collect information such as the customer's age, gender, and interests and input it into the system.

[1193] "Generative AI means" is an artificial intelligence technology that analyzes input hearing information and generates a picture-story show-style slideshow based on that information.

[1194] "Means of using the generated slideshow to present proposals and merchandise that are tailored to the customer" refers to devices and software that utilize the generated slideshow to present optimal proposals and product introductions to customers.

[1195] An "emotion recognition means" is a device or software that recognizes emotions from customers' facial expressions, tone of voice, etc., and adjusts the content of the slideshow and customer service style based on that information.

[1196] The "means for displaying a slideshow on smart glasses" refers to a device or software for displaying the generated slideshow on the display of the smart glasses.

[1197] "Means for capturing the gaze of crew members and facial expressions of customers and analyzing them in real time" refers to devices and software that capture the gaze of crew members and facial expressions of customers using cameras or other devices and analyze the data in real time.

[1198] The system for implementing this invention includes a means for inputting information about the customer's environment during service, a generative AI means, an emotion recognition means, a means for displaying a slideshow on smart glasses, and a means for capturing the crew's line of sight and the customer's facial expressions and analyzing them in real time.

[1199] System program generation

[1200] The system operates using the following hardware and software:

[1201] Hardware: Smart glasses (e.g., Google Glass)

[1202] Software: OpenCV (image processing library), EmotionRecognizer (emotion recognition library), SlideGenerator (slide generation library), SmartGlassesDisplay (smart glasses display control library)

[1203] Explanation of program processing

[1204] The server first inputs the customer's environmental information collected by the staff during the service. This information includes the customer's age, gender, interests, etc. The input information is then analyzed by a generative AI system, and a picture-story show-style slideshow is generated.

[1205] The generated slideshow is displayed on the smart glasses. The crew wears the glasses and makes optimal suggestions to the customer while watching the slideshow. Furthermore, an emotion recognition system captures the crew's gaze and the customer's facial expressions and analyzes them in real time. This allows the content of the slideshow and the customer service style to be adjusted based on the customer's emotions.

[1206] Adding specific examples

[1207] As a concrete example, the following prompt sentence is input into the generative AI model:

[1208] The customer is 30 years old and female. Her areas of interest are fashion and technology. Her facial expressions indicate her interest. Based on this, generate a slideshow showcasing products related to fashion and technology.

[1209] By inputting this prompt into a generative AI model, a slideshow tailored to the customer is generated and displayed on the smart glasses. The crew member can then use this slideshow as a guide to make effective suggestions to the customer.

[1210] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1211] Step 1:

[1212] The crew will wear smart glasses and conduct an interview with the customer about their environment.

[1213] Input: Information such as your age, gender, and interests

[1214] Output: Hearing information is entered into the system

[1215] How it works: The crew member uses the microphone and touchpad on the smart glasses to enter the customer's information.

[1216] Step 2:

[1217] The server receives the input hearing information and analyzes it using generative AI means.

[1218] Input: Hearing information

[1219] Output: A picture-story show based on the analysis results

[1220] Specific operation: The server inputs prompt sentences into the generative AI model and generates a slideshow.

[1221] Step 3:

[1222] The generated slideshow is displayed on the smart glasses.

[1223] Input: Generated slideshow

[1224] Output: A slideshow displayed on the smart glasses display

[1225] Specific operation: The server sends the slideshow data to the smart glasses and displays it on the display.

[1226] Step 4:

[1227] The crew will make suggestions to customers while watching a slideshow through smart glasses.

[1228] Input: Slideshow displayed on smart glasses

[1229] Output: Proposal to the customer

[1230] Specific actions: The crew will use the slideshow to recommend products and services to customers.

[1231] Step 5:

[1232] Emotion recognition means captures the crew's gaze and passengers' facial expressions and analyzes them in real time.

[1233] Input: Crew gaze data, customer facial expression data

[1234] Output: Customer's emotional state

[1235] Specific operation: The camera installed in the smart glasses captures gaze and facial expressions, and the emotion recognition engine analyzes them.

[1236] Step 6:

[1237] The server adjusts the content of the slideshow and the customer service style based on the emotion recognition results.

[1238] Input: Customer's emotional state

[1239] Output: Coordinated slideshows and presentation styles

[1240] Specific operation: Based on the emotion recognition results, the server re-enters the prompt text into the generative AI model and adjusts the slideshow and customer service style.

[1241] Step 7:

[1242] The adjusted slideshow is displayed again on the smart glasses.

[1243] Input: Adjusted slideshow

[1244] Output: A slideshow replayed on the smart glasses display

[1245] Specific operation: The server sends the adjusted slideshow data to the smart glasses and re-displays it on the display.

[1246] Example 2

[1247] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1248] Conventional customer service systems have had the problem of making it difficult to quickly and effectively provide proposals tailored to customer needs. It was also difficult to standardize the quality of customer service and efficiently handle multiple customer service staff. This could result in longer customer service times and lower customer satisfaction.

[1249] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1250] In this invention, the server includes means for inputting information about the customer's environment during customer service, means for transmitting the input information to the server, means for transmitting prompts to the generative AI model based on the information received by the server, means for the generative AI model to generate a slideshow based on the prompts, means for transmitting the generated slideshow from the server to the terminal, and means for using the generated slideshow to present a proposal method and product appeal tailored to the customer. This enables shortening customer service time, improving the efficiency of customer service across multiple staff, and standardizing the quality of customer service.

[1251] "Hearing about the customer's environment during customer service" is the process of collecting information about the customer's living environment and needs.

[1252] The "means for inputting" is an interface that allows a user to input information using a terminal.

[1253] A "server" is a central processing unit that receives and processes input information.

[1254] A "generative AI model" is an artificial intelligence algorithm that generates a slideshow based on an input prompt.

[1255] A "prompt" is text that contains instructions for the generative AI model to generate a slideshow.

[1256] A "slide show" is a presentation format in which multiple slides are displayed in succession.

[1257] A "terminal" is a device operated by a user, and includes tablets, PCs, etc.

[1258] A "proposal method" is a technique for effectively introducing products and services to customers.

[1259] "Product promotion" is an activity to communicate the appeal of a product or service to customers.

[1260] This invention is a system for quickly and effectively making proposals tailored to customer needs during customer service. The system involves a series of processes in which a user uses a terminal to interview the customer about their environment, sends that information to a server, and generates a slideshow using a generative AI model.

[1261] Hardware and software used

[1262] Hardware: Servers, devices (tablets, PCs, etc.)

[1263] Software: Generative AI model, emotion engine

[1264] Specific operation of the system

[1265] 1. The user uses the device to interview the customer about their environment.

[1266] Users use devices such as tablets and PCs to input information about their living environment and needs.

[1267] Example: A user enters information such as "I live in a newly built detached house."

[1268] 2. The device sends the entered information to the server

[1269] The terminal transmits the input information to the server in real time.

[1270] Example: A device sends information to a server that it "lives in a newly built detached house."

[1271] 3. The server sends a prompt to the generative AI model based on the received information.

[1272] The server analyzes the received information and generates a prompt sentence appropriate for the generative AI model.

[1273] Example: The server sends a prompt to the generative AI model: "Generate a slideshow showcasing products and services suitable for customers living in newly built single-family homes."

[1274] 4. The generative AI model generates a slideshow based on the prompt.

[1275] The generative AI model receives a prompt and generates a slideshow based on it.

[1276] Example: A generative AI model generates a slideshow showcasing furniture and interior design, home security systems, energy-efficient appliances, and more suitable for a newly built single-family home.

[1277] 5. The server sends the generated slideshow to the device.

[1278] The server transmits the generated slide show to the terminal.

[1279] Example: A server sends a generated slideshow to a user's device.

[1280] 6. The user reviews the slideshow generated on their device and makes suggestions to the customer.

[1281] The user checks the slideshow generated on the terminal and makes suggestions to the customer.

[1282] Example: A user watches a slideshow that suggests products and services suitable for a new home.

[1283] Prompt Sentence Examples

[1284] "Generate a slideshow showcasing products and services suitable for customers living in newly constructed single-family homes."

[1285] "Generate a slideshow showcasing products and services that would appeal to customers living in new homes."

[1286] This system allows users to effectively propose products and services that meet customer needs, shorten customer service time, improve the efficiency of customer service across multiple staff, and standardize the quality of customer service.

[1287] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1288] Step 1:

[1289] The user uses the terminal to interview the customer about their environment.

[1290] Input: Information about your living situation and needs (e.g., "I live in a newly built detached house").

[1291] Specific operation: The user uses a device such as a tablet or PC to enter customer information into a dedicated input form.

[1292] Output: The entered customer information is saved on the device.

[1293] Step 2:

[1294] The terminal transmits the input information to the server.

[1295] Input: Your information stored on your device.

[1296] Specific operation: The terminal uses a communication protocol to transmit input information to the server in real time.

[1297] Output: Your information is sent to and stored on our servers.

[1298] Step 3:

[1299] Based on the information received by the server, a prompt is sent to the generative AI model.

[1300] Input: Your information stored on our server.

[1301] Specific operation: The server analyzes the received information and generates a prompt sentence appropriate for the generative AI model.

[1302] Output: The generated prompt sentence is sent to the generative AI model.

[1303] Example: A prompt might read, "Generate a slideshow showcasing products and services suitable for customers living in newly constructed homes."

[1304] Step 4:

[1305] A generative AI model generates a slideshow based on the prompt.

[1306] Input: The prompt sent to the generative AI model.

[1307] How it works: The generative AI model analyzes the prompt and generates a slideshow based on it.

[1308] Output: The generated slideshow is sent to the server.

[1309] Examples include: a slideshow showcasing furniture and interior design for new homes, home security systems, and energy-efficient appliances.

[1310] Step 5:

[1311] The server transmits the generated slideshow to the terminal.

[1312] Input: A slideshow sent from a generative AI model.

[1313] Specific operation: The server uses a communication protocol to send the generated slideshow to the terminal.

[1314] Output: The slideshow is sent to your device and saved on your device.

[1315] Step 6:

[1316] The user checks the slideshow generated on the device and makes suggestions to the customer.

[1317] Input: A slideshow stored on your device.

[1318] Specific operations: The user reviews the slideshow generated on the device and makes suggestions to the customer.

[1319] Output: Effective proposals are made to customers.

[1320] Example: A user watches a slideshow that suggests products and services suitable for a new home.

[1321] (Application example 2)

[1322] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1323] In conventional customer service systems, it was difficult to reflect the emotions and reactions of individual customers in real time when proposing products and services based on the customer's environmental information. Also, experience and skill are required for salespeople to make optimal proposals to customers, which can lead to variations in the quality of customer service. Furthermore, there is a demand for shortening customer service time and efficiently handling multiple customers.

[1324] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1325] In this invention, the server includes: a means for a salesperson user, while serving a customer, to input the results of a conversation about the customer's living environment by voice; a prompt statement instructing the server to generate a slideshow showing products and a method for proposing the products that are suitable for the customer based on the results of the conversation; a means for generating the slideshow using a generative AI model; and a means for displaying the generated slideshow on a display unit of smart glasses worn by the user. The server also includes: a means for recognizing the customer's emotion from the customer's facial expression using an emotion engine; a prompt statement instructing the server to modify the generated slideshow based on the recognition result of the customer's emotion; and a means for modifying the generated slideshow using the generative AI model. The displaying means displays the modified slideshow on a display unit of smart glasses worn by the user. This enables optimal suggestions that reflect the customer's emotions and reactions in real time, improving the quality of customer service and achieving efficient responses.

[1326] "Means for inputting information about the customer's environment during customer service" refers to the means by which salespeople collect information about the customer's living environment and living situation, and input it into the system.

[1327] "Generative AI methods" are artificial intelligence methods that automatically generate slideshows introducing appropriate products and services based on input customer information.

[1328] "A means of using the generated slideshow to present proposals and products that suit the customer" is a means of utilizing the generated slideshow to propose optimal products and services to customers and convey their appeal.

[1329] "Emotion recognition means for recognizing customer facial expressions in real time" is a means for capturing customer facial expressions in real time and analyzing their emotions.

[1330] "Means for the generative AI means to optimize the slideshow based on the emotion recognition results" refers to means for the generative AI to adjust the content of the slideshow and make more appropriate suggestions based on the customer's emotion data obtained by the emotion recognition means.

[1331] "A means for displaying a slideshow using smart glasses and allowing a salesperson to make suggestions" is a means for displaying a generated slideshow on the display of smart glasses and allowing a salesperson to make suggestions to a customer while viewing it.

[1332] A system for implementing the present invention uses the following hardware and software.

[1333] Hardware used

[1334] Smart glasses (e.g. Google Glass)

[1335] Facial recognition camera (built into smart glasses)

[1336] Microphone (for voice input)

[1337] Software used

[1338] Generative AI models (e.g., OpenAI GPT-4)

[1339] Emotion recognition engine (e.g., Microsoft Azure (registered trademark) Emotion API)

[1340] Database (stores customer information, product information)

[1341] System Operation

[1342] 1. Enter your customer information:

[1343] The salesperson listens to environmental information about the customer's living environment and inputs it into the smart glasses' microphone.

[1344] Example: "I live in a newly built detached house."

[1345] 2. Emotion recognition:

[1346] The camera in the smart glasses captures the customer's facial expressions in real time, and an emotion recognition engine analyzes them to recognize the customer's emotions.

[1347] Example: When a customer looks happy

[1348] 3. Slideshow generation using generative AI:

[1349] The generative AI model introduces suitable products and services to customers based on the input customer information and emotion recognition results, and generates a slideshow in accordance with prompts that instruct it to generate a slideshow showing how to propose the products and services.

[1350] Example: Proposing furniture and interior design suitable for a newly built detached house

[1351] 4. View Slideshow:

[1352] The generated slideshow is displayed on the smart glasses display, and the salesperson makes suggestions to the customer.

[1353] Specific examples

[1354] If a customer inputs that they "live in a newly built detached house," the generative AI will introduce "the perfect furniture set for a newly built detached house" or "the latest smart home devices," and generate a slideshow that shows how to propose these (key points and phrases to use when introducing the products). The system also recognizes customer emotions in real time and modifies the generated slideshow based on the results of the customer emotion recognition using prompts and the generative AI model. The modified slideshow is then displayed on the display of the smart glasses worn by the user. For example, if the customer shows a happy expression, the emotion recognition engine feeds that information back to the generative AI model, which then modifies the slideshow to suggest more attractive products and services.

[1355] Prompt Sentence Examples

[1356] Customer information: Living in a newly built detached house

[1357] Emotion recognition result: Happy facial expression

[1358] Generated slideshows: Furniture sets for new homes, latest smart home devices

[1359] In this way, sales staff in brick-and-mortar stores can use smart glasses to recommend the best products and services to customers.

[1360] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1361] Step 1:

[1362] A salesperson uses the smart glasses to input information about the customer's environment. The input information is captured as voice data through the smart glasses' microphone and converted into text data by voice recognition software. The input data is the customer's environmental information, such as "I live in a newly built detached house." The output is sent to the server as text data.

[1363] Step 2:

[1364] The server captures the customer's facial expressions in real time using the camera in the smart glasses. The captured image data is sent to an emotion recognition engine to analyze the customer's emotions. The input data is the captured image data, and the output data is the analyzed emotion data (e.g., a happy expression).

[1365] Step 3:

[1366] The server generates a slideshow using a generative AI model based on the input customer information and emotion recognition results. The generative AI model receives prompts containing customer information and emotion data, and generates a slideshow that introduces suitable products and services and shows the recommendation method. The input data is customer information and emotion data, and the output data is the generated slideshow.

[1367] Step 4:

[1368] The server transmits the generated slideshow to the display of the smart glasses. The smart glasses display the received slideshow, and the salesperson makes suggestions to the customer while viewing the slideshow. The input data is the generated slideshow, and the output data is the slideshow displayed on the display of the smart glasses.

[1369] Step 5:

[1370] Salespeople recommend optimal products and services to customers based on a slideshow displayed on the smart glasses. The input data is the slideshow displayed on the smart glasses, and the output data is the proposal to the customer. In addition, while the salesperson is making a proposal based on the slideshow displayed on the smart glasses, the emotion engine recognizes the customer's emotions in real time. The server then modifies the generated slideshow based on the customer's emotion recognition results, using a prompt statement instructing the salesperson to modify the generated slideshow and the generative AI model. The modified slideshow is then displayed on the display of the smart glasses worn by the user. This allows the salesperson to adjust the proposal content while observing the customer's reaction.

[1371] In this way, the system can suggest optimal products and services based on the customer's environmental information and emotional data. Furthermore, when a user who is a salesperson is serving multiple customers simultaneously, a slide show may be generated for each customer, and the displayed slide show may be switched each time the user switches between customers. In this case, in the server, the input means, when the user is serving multiple customers simultaneously, inputs the results of the interview for each of the multiple customers by voice. The generating means generates the slide show for each of the multiple customers. The displaying means switches to display the generated slide show for each customer each time the user switches between customers.

[1372] Example 3

[1373] Next, a description will be given of Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1374] With the conventional customer service system, it was difficult to respond flexibly to the customer's emotions, shorten the service time, and handle multiple customer service sessions. In addition, there was a lack of means to standardize customer service, which resulted in inconsistent quality of customer service.

[1375] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[1376] In this invention, the server includes a means for inputting information about the customer's environment during the customer service session, a generative AI means for generating a picture-story slideshow based on the input information, a means for using the generated slideshow to present a proposal method and product appeal tailored to the customer, a means for capturing the customer's facial expression, an emotion analysis means for analyzing the captured facial expression data and identifying the customer's emotion, and a means for adjusting the content of the slideshow based on the identified emotion. This enables flexible customer service according to the customer's emotion, shortening the customer service time and enabling multiple staff members to serve the customer. It also enables standardization of customer service, making it possible to maintain a consistent quality of customer service.

[1377] "Means for inputting information about the customer's environment when serving customers" refers to devices or software for inputting the customer's needs and situation when serving customers.

[1378] A "paper theater-style slideshow" is a presentation format that conveys information visually by displaying multiple slides in sequence.

[1379] "Generative AI means" refers to technologies and devices that use artificial intelligence to automatically generate content for specific purposes.

[1380] "Means of making proposals and promoting products that suit the customer" refers to devices and software that allow you to introduce the best proposals and products according to the customer's needs and situation.

[1381] "Means for capturing customer facial expressions" refers to devices or technologies that use cameras or sensors to capture customer facial expressions and acquire them as data.

[1382] "Emotion analysis means for analyzing captured facial expression data to identify a customer's emotions" refers to technology or equipment for analyzing acquired facial expression data to identify a customer's emotional state (e.g., confusion, joy, interest, etc.).

[1383] "Means for adjusting the content of a slideshow based on the identified emotion" refers to a technique or device for dynamically changing the content of a slideshow based on the analyzed emotion information.

[1384] This invention is a system that enables flexible responses according to customer emotions during customer service. Specifically, it combines generative AI and an emotion analysis engine to generate slideshows that shorten customer service time and enable multiple people to serve customers. The emotion analysis engine also adjusts the customer service style according to the customer's emotions.

[1385] Hardware and software used

[1386] Hardware: Servers, devices (PCs, tablets, etc.)

[1387] Software: Generative AI models (e.g., GPT-4), sentiment analysis engines (e.g., Affectiva)

[1388] Specific operation of the system

[1389] 1. The user starts serving customers

[1390] The user turns on the device (PC or tablet) and opens the customer service application.

[1391] The user clicks the button to start serving customers.

[1392] 2. The device captures the customer's facial expression through the camera

[1393] The device's camera will automatically activate and recognize the customer's face.

[1394] The camera continues to capture your facial expressions in real time.

[1395] 3. The device sends the captured facial expression data to the server.

[1396] The facial expression data captured by the device is sent to the server at regular intervals.

[1397] Data is encrypted before transmission to ensure security.

[1398] 4. The server analyzes the facial expression data using an emotion analysis engine.

[1399] The server inputs the received facial expression data into an emotion analysis engine (e.g., Affectiva).

[1400] An emotion analysis engine analyzes facial expression data to identify customer emotions (e.g., confusion, joy, interest, etc.).

[1401] 5. The server sends the analysis results to the generative AI model

[1402] The server sends the analysis results to a generative AI model (e.g., GPT-4).

[1403] The analysis results include emotional information about the customer.

[1404] 6. Generative AI models generate slideshows

[1405] The generative AI model generates an appropriate slideshow based on the emotional information it receives.

[1406] For example, if a customer is confused, generate a slideshow with detailed explanations.

[1407] 7. The server sends the generated slideshow to the device.

[1408] The server transmits the generated slideshow to the terminal.

[1409] Slideshows will be sent in an appropriate format (e.g. PDF, PPT).

[1410] 8. The device displays a slideshow

[1411] The device will automatically display the received slideshow.

[1412] The user can continue serving customers while checking the slideshow.

[1413] 9. Users serve customers using slideshows

[1414] A user uses a slideshow to explain products and services to customers.

[1415] The slideshow can also accommodate multiple attendees, so multiple crew members can use it at the same time.

[1416] Specific examples

[1417] For example, when a user is explaining a new product, they input the following prompt sentence into the generative AI model:

[1418] Example prompt sentence:

[1419] "Generate a slideshow that succinctly explains the features of your new product. If your customers are confused, add a detailed explanation."

[1420] Based on this prompt, the generative AI model generates the following slideshow:

[1421] 1. Slide 1: New Product Overview

[1422] 2. Slide 2: Key Features and Benefits

[1423] 3. Slide 3: A brief explanation of how to use it

[1424] 4. Slide 4: When Confused Expressions are Detected, Detailed Instructions and FAQs

[1425] In this way, the user can provide appropriate service according to the customer's emotions. The flow of the identification process in the third embodiment will be described with reference to FIG.

[1426] Step 1:

[1427] The user starts serving customers.

[1428] The user starts up the device (PC or tablet) and opens the customer service application. The user clicks the button to start customer service. This switches the system into customer service mode, and the camera and sensors are ready to start up.

[1429] Step 2:

[1430] The device captures the customer's facial expression through the camera.

[1431] The device's camera automatically activates and recognizes the customer's face. The camera continues to capture the customer's facial expressions in real time. The input is video data of the customer's face, and the output is the captured facial expression data.

[1432] Step 3:

[1433] The facial expression data captured by the device is sent to the server.

[1434] The facial expression data captured by the device is sent to the server at regular intervals. The data is encrypted before transmission, ensuring security. The input is the captured facial expression data, and the output is the encrypted data sent to the server.

[1435] Step 4:

[1436] The server analyzes the facial expression data using an emotion analysis engine.

[1437] The server inputs the received facial expression data into an emotion analysis engine. The emotion analysis engine analyzes the facial expression data and identifies the customer's emotion (e.g., confusion, joy, interest, etc.). The input is encrypted facial expression data, and the output is analyzed emotional information.

[1438] Step 5:

[1439] The server sends the analysis results to the generative AI model.

[1440] The server sends the analysis results to the generative AI model. The analysis results include the customer's emotional information. The input is the analyzed emotional information, and the output is the data sent to the generative AI model.

[1441] Step 6:

[1442] A generative AI model generates the slideshow.

[1443] The generative AI model generates an appropriate slideshow based on the emotional information it receives. For example, if a customer is confused, it generates a slideshow with detailed explanations. The input is the emotional information and prompt text, and the output is the generated slideshow.

[1444] Step 7:

[1445] The server transmits the generated slideshow to the terminal.

[1446] The server sends the generated slideshow to the device. The slideshow is sent in the appropriate format (e.g. PDF, PPT). The input is the generated slideshow and the output is the slideshow sent to the device.

[1447] Step 8:

[1448] Your device will display a slideshow.

[1449] The terminal automatically displays the received slideshow. The user can continue serving customers while checking the slideshow. The input is the slideshow sent to the terminal, and the output is the displayed slideshow.

[1450] Step 9:

[1451] The user serves customers using a slide show.

[1452] The user explains products and services to customers using a slideshow. The slideshow can also accommodate multiple attendees, allowing multiple crew members to use it simultaneously. The input is the displayed slideshow, and the output is the explanation given to the customer.

[1453] (Application example 3)

[1454] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1455] With conventional customer service systems, it was difficult to respond flexibly to customer emotions, and it was difficult to have multiple staff members serving customers at the same time or to shorten the service time. This led to problems such as a decrease in customer satisfaction and a deterioration in customer service efficiency. Furthermore, it was not possible to standardize customer service, and each staff member had their own different service style, which led to the issue of inconsistent service quality.

[1456] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[1457] In this invention, the server includes a means for inputting information about the customer's environment during service, a generative AI means for generating a picture-story show based on the input information, a means for using the generated slideshow to present a proposal method and product appeal tailored to the customer, an emotion engine means for analyzing the customer's emotions, a generative AI means for adjusting the slideshow based on information from the emotion engine means, and a means designed for simultaneous use by multiple users. This allows for flexible responses based on customer emotions, enabling simultaneous service by multiple people and shortening service time. It also allows for standardization of service and maintains consistent service quality.

[1458] "Means for inputting information about the customer's environment during customer service" refers to devices or methods for collecting information to understand the customer's needs and situation and inputting it into the system.

[1459] "Generative AI means" refers to artificial intelligence technology that automatically generates content or materials for specific purposes based on input information.

[1460] A "picture-story show slideshow" is a format that uses a sequence of images or slides to tell a story or explain something visually.

[1461] "Methods of proposing and promoting products that suit customers" refers to methods and devices for proposing optimal products and services according to the customer's needs and situation, and effectively communicating their appeal.

[1462] "Emotion engine means" is a technology that analyzes emotions from the customer's facial expressions, tone of voice, etc., and provides that information to the system.

[1463] The "generative AI means for adjusting the slideshow based on information from the emotion engine means" is an artificial intelligence technology that dynamically changes the content and order of the slideshow based on the emotion information obtained from the emotion engine means.

[1464] "Means designed to enable simultaneous use by multiple users" refers to designs or technologies that allow multiple users to use the system simultaneously.

[1465] The following system configuration will be described as an embodiment of the present invention.

[1466] System Configuration

[1467] This system includes a means for inputting environmental information about customers during customer service, a generative AI means, an emotion engine means, and a means designed to be usable by multiple users simultaneously.

[1468] Hardware and Software

[1469] Hardware: Smart glasses (with camera and display)

[1470] software:

[1471] OpenCV: Acquiring and displaying camera images

[1472] EmotionEngine: A library for analyzing customer emotions

[1473] SlideGenerator: A library for generating emotion-sensitive slideshows

[1474] Processing flow

[1475] 1. Environmental information input: Using the smart glasses' camera, environmental information such as the customer's facial expression and tone of voice is collected in real time, allowing us to understand the customer's needs and situation.

[1476] 2. Emotion analysis: The acquired environmental information is analyzed by the Emotion Engine to identify the customer's emotions. For example, if the customer shows a confused expression, the emotion engine will determine that the emotion is "confused."

[1477] 3. Slideshow generation: Based on the emotional information obtained from the emotion engine, the generative AI generates an appropriate slideshow. For example, if a confused expression is detected, a slideshow with more detailed explanations will be generated.

[1478] 4. Displaying the slideshow: The generated slideshow is displayed on the smart glasses display, allowing the salesperson to make appropriate suggestions and appeal to the customer.

[1479] 5. Multi-user support: The system is designed to be used by multiple store associates simultaneously, allowing each associate to individually assist customers.

[1480] Specific examples

[1481] For example, imagine a customer in a store is considering purchasing a new smartphone. If the customer shows a confused expression, the camera in the smart glasses captures that expression and the Emotion Engine determines that the expression is "confused." Based on this information, the generative AI generates a slideshow that details the smartphone's functions and features, and displays it on the smart glasses' display. The store clerk can then provide an appropriate explanation to the customer by viewing the slideshow.

[1482] Prompt Sentence Examples

[1483] "The customer has a confused look on their face. Please generate a slideshow with detailed product descriptions."

[1484] In this way, this invention enables flexible responses according to customer emotions, allows multiple staff to serve customers simultaneously, and shortens the time it takes to serve them.It also standardizes customer service and maintains a consistent quality of service.

[1485] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[1486] Step 1:

[1487] The server uses the smart glasses' camera to capture environmental information such as the customer's facial expressions and tone of voice in real time. The input is the camera image, and the output is environmental information data. This data is used for subsequent emotion analysis. Specifically, the camera captures the image and sends the image data to the server.

[1488] Step 2:

[1489] The server analyzes the acquired environmental information data using the Emotion Engine to identify the customer's emotions. The input is environmental information data, and the output is emotional information. Specifically, the Emotion Engine analyzes the video data using a facial expression recognition algorithm and classifies emotions into categories such as "confusion," "happiness," and "interest."

[1490] Step 3:

[1491] The server sends a prompt to the generative AI based on the emotional information obtained from the emotion engine, and generates an appropriate slideshow. The input is emotional information, and the output is slideshow data. Specifically, the generative AI receives the prompt, "The customer has a confused expression. Please generate a slideshow that includes a detailed description of the product," and generates a slideshow based on that.

[1492] Step 4:

[1493] The server displays the generated slideshow on the display of the smart glasses. The input is the slideshow data, and the output is the slideshow displayed on the display. In specific operations, the slideshow data is sent to the smart glasses and displayed on the display.

[1494] Step 5:

[1495] The user (store clerk) provides an appropriate explanation to the customer while watching the slideshow displayed on the smart glasses display. The input is the slideshow displayed on the display, and the output is the explanation to the customer. Specifically, the store clerk checks the contents of the slideshow and explains the product details to the customer based on that.

[1496] Step 6:

[1497] The server manages the system so that multiple users can use it simultaneously. The input is the customer service status data for each user, and the output is the system's resource allocation data. Specifically, the server monitors the customer service status of each user and dynamically allocates resources as needed.

[1498] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1499] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1500] Another example of generative AI is Gemini (internet search engine). <url: https: gemini.google.com ?hl="ja">) are listed.

[1501] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1502] [Third embodiment]

[1503] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1504] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1505] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1506] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1507] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1508] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1509] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1510] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1511] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1512] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1513] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1514] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[1515] "Example 1"

[1516] In this embodiment, the crew interviews the customer about their environment while serving them and inputs that information into the system. The input information is analyzed by generative AI, and a picture-story-style slideshow suited to the customer's situation is generated. The generated slideshow is used as a guide for the crew when presenting proposal methods and product appeals to the customer.

[1517] "Example 2"

[1518] As a specific example, if a crew member inputs information such as "I live in a newly built detached house" during a customer environment interview, the generative AI will generate a slideshow based on that information, including introductions to products and services suitable for newly built detached houses. The crew member will then use the generated slideshow to suggest products and services suitable for newly built detached houses.

[1519] "Example 3"

[1520] Generative AI also generates slideshows to shorten customer service time and enable multiple crew members to serve customers. For example, to shorten customer service time, the slideshow is designed to efficiently convey necessary information. To enable multiple crew members to serve customers, the slideshow is designed so that multiple crew members can use it simultaneously.

[1521] The processing flow of each embodiment will be described below.

[1522] "Example 1"

[1523] Step 1: When serving customers, the crew will ask about their environment and enter that information into the system.

[1524] Step 2: The input information is analyzed by generative AI.

[1525] Step 3: The generative AI generates a picture-story slideshow suited to the customer's situation.

[1526] Step 4: The generated slideshow will be used as a guide for the crew to present their proposals and products to the customer.

[1527] "Example 2"

[1528] Step 1: The crew will enter the information that "I live in a newly built detached house" during the customer's environmental interview.

[1529] Step 2: Based on this information, the generative AI generates a slideshow introducing products and services suitable for newly built single-family homes.

[1530] Step 3: The crew uses the generated slideshow to suggest products and services suitable for the newly built single-family home.

[1531] "Example 3"

[1532] Step 1: The generative AI generates a slideshow to shorten customer service time and enable multiple people to serve customers.

[1533] Step 2: To shorten customer service time, the slideshow is designed to efficiently convey the necessary information.

[1534] Step 3: To accommodate multiple guests, the slideshow is designed to be used by multiple crew members simultaneously.

[1535] Example 1

[1536] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1537] With the conventional customer service system, it was difficult to quickly and accurately provide suggestions that met the customer's needs, which tended to result in long service times. In addition, it was difficult to have multiple staff members provide service or standardize the service, and the system was heavily dependent on the skills of the crew. This resulted in the problem of inconsistent quality of customer service.

[1538] The identification process by the identification processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means. In this invention, the server includes a means for inputting customer environment interview information, a means for transmitting the input information to the server, a means for the server to transmit a prompt sentence to the generation AI model, a means for the generation AI model to perform analysis and generate a slideshow, a means for saving the generated slideshow and transmitting it to the terminal, a means for the terminal to display the slideshow, and a means for using the generated slideshow to present a proposal method and product appeal tailored to the customer. This reduces customer service time, enables multiple people to serve customers, and standardizes customer service.

[1539] "Customer environmental interview information" refers to information about customer needs and the environment collected by crew members while serving customers.

[1540] The "means of input" refers to an interface that allows crew members to input customer environmental hearing information using devices such as tablets or PCs.

[1541] The "means for transmitting to the server" is a communication means for encrypting information input from the terminal and transmitting the encrypted information to the server.

[1542] A "generative AI model" is an artificial intelligence model that analyzes input information and generates a slideshow that is appropriate for the customer's situation.

[1543] A "prompt statement" is an instruction statement created by the server to cause the generative AI model to perform analysis.

[1544] The "means for generating a slideshow" is a function that allows the generative AI model to analyze the prompt text and generate a picture-story show-style slideshow.

[1545] The "means for saving" is a function for saving the generated slide show in a presentation software format.

[1546] The "means for transmitting to the terminal" is a communication means for the server to transmit the generated slide show to the crew's terminal.

[1547] The "means for displaying a slide show" is a function for displaying a slide show received by a terminal.

[1548] The "method of proposal and means of appealing to merchandise" refers to a means by which the crew uses the generated slideshow to make specific proposals and appeal to the customer about merchandise.

[1549] This invention is a system that uses a generative AI model to generate a picture-story show-style slideshow based on environmental information about customers collected by the crew during customer service, and promotes proposal methods and products.

[1550] First, the user (crew member) uses a device such as a tablet or PC to input information about the customer's needs and environment. This information is input through a dedicated application. For example, specific information such as "The customer is considering a new office layout" is input.

[1551] The device then sends the entered information to the server in real time, using the HTTPS protocol to ensure security as the data is encrypted before transmission.

[1552] The server creates a prompt based on the received information and sends it to the generative AI model (e.g., GPT-4). The prompt might be something like, "The client is considering a new office layout. Please generate a slideshow about efficient desk arrangements and space utilization methods."

[1553] The generative AI model analyzes the prompt text and generates a picture-story-style slideshow that is appropriate for the customer's situation, including specific proposals and the selling points of the product.

[1554] The server saves the generated slideshow in a presentation software format (e.g., Microsoft PowerPoint or Google Slides), and then transmits the slideshow to the crew's device.

[1555] The device then displays the received slideshow, which the crew can use to provide specific suggestions to the customer. For example, a slideshow on optimizing office layout includes ideas for desk placement and efficient space utilization.

[1556] This system reduces customer service time, allows multiple people to serve customers, and standardizes customer service. An example of a specific prompt sentence is, "The customer is considering a new office layout. Please generate a slideshow about efficient desk arrangements and space utilization methods."

[1557] In this way, the system provides the crew with the tools to make optimal recommendations to customers based on the information they collect.

[1558] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1559] Step 1:

[1560] The user (crew) inputs the environmental hearing information.

[1561] Specifically, the crew uses a dedicated application on a tablet or PC to input information about the customer's needs and environment. For example, they input specific information such as, "The customer is considering the layout of a new office." The input information is temporarily saved in the application.

[1562] Input: Information about your needs and circumstances

[1563] Output: Temporarily stored hearing information

[1564] Step 2:

[1565] The terminal sends the input information to the server.

[1566] Specifically, the device encrypts the input information and sends it to the server using the HTTPS protocol. The server receives the data and stores it in a database.

[1567] Input: Temporarily stored hearing information

[1568] Output: Hearing information sent to the server

[1569] Step 3:

[1570] The server sends a prompt to the generative AI model.

[1571] Specifically, the server creates a prompt based on the received information. For example, it generates a prompt such as, "The client is considering a new office layout. Please generate a slideshow about efficient desk arrangements and space utilization methods." This prompt is then sent to the generative AI model.

[1572] Input: Hearing information stored on the server

[1573] Output: The prompt sent to the generative AI model

[1574] Step 4:

[1575] A generative AI model performs the analysis and generates a slideshow.

[1576] Specifically, a generative AI model (e.g., GPT-4) analyzes the prompt and generates a picture-story-style slideshow appropriate to the customer's situation, including specific proposals and selling points of the product.

[1577] Input: The prompt sent to the generative AI model

[1578] Output: Generated slideshow

[1579] Step 5:

[1580] The server saves the generated slideshow and transmits it to the device.

[1581] Specifically, the server saves the generated slideshow in a presentation software format (e.g., Microsoft PowerPoint or Google Slides), and then transmits the slideshow to the crew's device.

[1582] Input: Generated slideshow

[1583] Output: Slideshow sent to device

[1584] Step 6:

[1585] Your device will display a slideshow.

[1586] Specifically, the device displays the received slideshow. The crew member uses this slideshow to provide specific suggestions to the customer. For example, a slideshow on optimizing office layout includes ideas for desk arrangement and efficient space utilization.

[1587] Input: Slideshow sent to device

[1588] Output: Rendered slideshow

[1589] In this way, the system provides the crew with the tools to make optimal recommendations to customers based on the information they collect.

[1590] (Application example 1)

[1591] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1592] With the conventional customer service system, it was difficult to respond to customer needs quickly and accurately, which tended to result in long service times. Also, since the quality of customer service depended on the experience and skills of the crew, it was difficult to provide a consistent service. Furthermore, it was difficult to serve multiple customers at the same time, so efficient customer service was required.

[1593] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1594] In this invention, the server includes a means for inputting information about the customer's environment during the customer service session, a generation system AI means for generating a picture-story slideshow based on the input information, a means for using the generated slideshow to present a proposal method and product appeal tailored to the customer, a means for accepting voice input, a voice recognition means for converting voice into text, and a display means for displaying the generated slideshow. This allows for quick and accurate response to customer needs, shortening customer service time and enabling simultaneous response to multiple customers. It also standardizes the quality of customer service and realizes efficient service provision.

[1595] "Means for inputting environmental information from customers during customer service" refers to a means for collecting customer needs and environmental information through dialogue with customers and inputting that information into the system.

[1596] The "generative AI means for generating a picture-story show-style slideshow based on input hearing" refers to an artificial intelligence means for analyzing collected customer information and generating a picture-story show-style slideshow to make optimal proposals to customers.

[1597] "Means for making proposals and promoting products that suit customers using generated slideshows" refers to means for making appropriate proposals and promoting products to customers using generated slideshows.

[1598] The "means for accepting voice input" is a means for the crew to input the customer's needs by voice.

[1599] "Speech recognition means for converting speech into text" refers to means for analyzing input speech and converting it into text data.

[1600] The "display means for displaying the generated slide show" refers to a means for visually displaying the generated slide show.

[1601] A system for implementing this invention has the following configuration. First, it includes a voice input means for the crew to input customer needs and environmental information by voice while serving customers. This voice input means uses a microphone installed in a device such as smart glasses or a smartphone.

[1602] Next, the information input by voice is converted into text data by a voice recognition means. This voice recognition means uses a SpeechRecognition library or the like. The voice recognition means analyzes the voice data and converts it into text data.

[1603] The converted text data is sent to a generative AI means, which uses a generative AI model such as OpenAI GPT-3. This generative AI means analyzes the input text data and generates a picture-story show-style slideshow based on the customer's needs.

[1604] The generated slideshow is visually displayed by a display means, which may be a display of smart glasses or a smartphone screen. The display means provides the generated slideshow to the crew and serves as a guide for making effective proposals and promoting products to customers.

[1605] As a specific example, if a customer says, "I'm looking for a new smartphone, especially one with a good camera," the speech recognition means converts this speech into text, and the generative AI means generates the following prompt sentence:

[1606] Example prompt sentence:

[1607] "Based on the customer's needs, generate a Kamishibai-style slideshow with the following information: I'm looking for a new smartphone, especially one with a good camera."

[1608] Based on this prompt, the generative AI system generates a slideshow comparing the latest smartphone camera features, including the features and pricing information for each model. The generated slideshow is displayed on the smart glasses display and serves as a guide for the crew to make effective recommendations to the customer.

[1609] In this way, the present invention responds to customer needs quickly and accurately, shortens customer service time, and enables simultaneous service for multiple customers. It also standardizes the quality of customer service and provides efficient service.

[1610] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1611] Step 1:

[1612] The user uses the microphone on the smart glasses or smartphone to input customer needs and environmental information by voice.

[1613] Input: Customer voice information

[1614] Output: Audio data

[1615] Specific operation: The user interacts with the customer and inputs the customer's requests and questions by voice.

[1616] Step 2:

[1617] The device converts the voice input information into text data using a voice recognition method (e.g., SpeechRecognition library).

[1618] Input: Audio data

[1619] Output: Text data

[1620] Specific operation: The device analyzes the voice data and applies a voice recognition algorithm to convert it into text data.

[1621] Step 3:

[1622] The device sends the converted text data to a generative AI method (e.g., OpenAI GPT-3).

[1623] Input: Text data

[1624] Output: Request to generative AI

[1625] Specific operation: The device creates and sends a request to send text data to a generative AI model in the cloud.

[1626] Step 4:

[1627] The server uses generative AI means to generate a picture-story show-style slideshow based on the input text data.

[1628] Input: Text data

[1629] Output: Kamishibai style slideshow

[1630] Specific operation: The server inputs prompt text into the generative AI model and generates a slideshow based on the customer's needs.

[1631] Step 5:

[1632] The server transmits the generated slideshow to the terminal.

[1633] Input: Kamishibai style slideshow

[1634] Output: Slideshow data to the device

[1635] Specific operation: The server performs communication processing to transmit the generated slideshow data to the terminal.

[1636] Step 6:

[1637] The terminal displays the received slideshow on a display means (e.g., a display of smart glasses or a smartphone screen).

[1638] Input: Slideshow data

[1639] Output: Rendered slideshow

[1640] Specific operation: The terminal transmits the received slideshow data to the display device and displays it visually.

[1641] Step 7:

[1642] Based on the displayed slideshow, the user can make proposals and appeal to customers about products.

[1643] Input: Displayed slideshow

[1644] Output: Proposals and appeals to customers

[1645] Specific operation: The user watches the slideshow and makes appropriate suggestions and explains products to the customer.

[1646] Example 2

[1647] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1648] With conventional customer service systems, it took a lot of time and effort to propose appropriate products and services based on customer environment information. Furthermore, the quality of customer service varied depending on the staff member, making it difficult to provide consistent service. Furthermore, it was difficult to provide efficient service in a short amount of time with multiple staff members.

[1649] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1650] In this invention, the server includes means for inputting customer environment information, means for generating prompt sentences based on the input hearing information, means for calling a generative AI model using the generated prompt sentences to generate a slideshow, and means for proposing products and services suitable for the customer using the generated slideshow. This reduces customer service time, enables multiple people to serve customers, and further standardizes customer service.

[1651] "Customer environmental interview information" refers to information about the customer's current living environment and housing situation.

[1652] An "input means" is an interface or device that allows a user to input information into a system.

[1653] A "prompt" is an instruction that instructs a generative AI model to generate a specific output.

[1654] A "generative AI model" is an artificial intelligence model that generates a specific output based on an input prompt.

[1655] A "slide show" is a presentation material in which multiple slides are displayed in sequence.

[1656] The "means of proposal" refers to a method or tool for explaining and proposing products or services to customers using the generated slideshow.

[1657] A "server" is a computer system for processing and storing data.

[1658] This invention is a system that generates and proposes slideshows using a generative AI model based on customer environment interview information. Specific embodiments of this system are described below.

[1659] Server Roles

[1660] The server receives the customer's environmental interview information entered by the crew member on their device. This information includes the data that "you live in a newly built detached house." Based on this information, the server generates a prompt to be input into the generative AI model. Specifically, the server creates a prompt that reads, "Please generate a slideshow introducing products and services suitable for customers who live in newly built detached houses."

[1661] Next, the server uses the generated prompt to call a generative AI model (for example, OpenAI's GPT-4). The prompt is input into the AI ​​model to generate the content for the slideshow. Based on the content returned by the AI ​​model, the server generates the slideshow. The slideshow is created in PowerPoint or PDF format, for example. Finally, the server sends the generated slideshow to the crew's device.

[1662] Device Role

[1663] The terminal receives the customer's environmental interview information entered by the crew. For example, the information entered is "I live in a newly built detached house." The terminal then sends the entered information to the server. The terminal then receives the slideshow sent from the server. To display the received slideshow, the terminal uses software such as Microsoft PowerPoint or Google Slides. The crew then uses this software to show the slideshow to the customer.

[1664] User (Crew) Roles

[1665] The user (crew member) interviews the customer about their living environment and inputs the information that they "live in a newly built detached house" into the terminal. Next, the user checks the slideshow sent from the server on the terminal and proposes products and services suitable for the newly built detached house. The crew member uses the slideshow to explain specific products and services to the customer.

[1666] Specific examples

[1667] Example of information entered by crew

[1668] "You live in a newly built detached house."

[1669] Example prompts to input to the generative AI model

[1670] "Generate a slideshow showcasing products and services suitable for customers living in newly constructed single-family homes."

[1671] Example of generated slideshow content

[1672] 1. Slide 1: Title: "Best Products and Services for Newly Built Single-Family Homes"

[1673] 2. Slide 2: Product Introduction "Latest Security System"

[1674] 3. Slide 3: Service Introduction "Garden Design and Maintenance Service"

[1675] 4. Slide 4: Product Introduction "Energy-efficient Home Appliances"

[1676] 5. Slide 5: Service Introduction "Home Automation Installation Service"

[1677] The crew uses this slideshow to suggest products and services suitable for newly built single-family homes to customers.

[1678] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1679] Step 1:

[1680] The user (crew) interviews the customer about their living environment and collects information such as "I live in a newly built detached house." The user then enters this information into the device. The entered information is temporarily stored in the device's database.

[1681] Input: Customer's environmental information (e.g., "I live in a newly built detached house")

[1682] Output: Hearing information stored on the device

[1683] Step 2:

[1684] The terminal transmits the input hearing information to the server. The terminal uses a communication module to transmit data to the server via a network.

[1685] Input: Hearing information stored on the device

[1686] Output: Hearing information sent to the server

[1687] Step 3:

[1688] The server generates a prompt based on the received information. Specifically, it creates the prompt, "Please generate a slideshow introducing products and services suitable for customers living in newly built detached houses."

[1689] Input: Hearing information sent to the server

[1690] Output: Generated prompt statement

[1691] Step 4:

[1692] The server calls the generative AI model using the generated prompt sentence. The server inputs the prompt sentence into the generative AI model (e.g., GPT-4) to generate the content for the slideshow.

[1693] Input: Generated prompt text

[1694] Output: The slideshow content returned by the generative AI model

[1695] Step 5:

[1696] The server generates a slideshow based on the content returned by the generative AI model, for example in PowerPoint or PDF format.

[1697] Input: Slideshow content returned by a generative AI model

[1698] Output: Generated slideshow (PowerPoint and PDF formats)

[1699] Step 6:

[1700] The server transmits the generated slideshow to the crew's terminals. The server uses a communication module to transmit data to the terminals via a network.

[1701] Input: Generated slideshow

[1702] Output: Slideshow sent to device

[1703] Step 7:

[1704] The device receives the slideshow sent from the server and uses software such as Microsoft PowerPoint or Google Slides to display the received slideshow.

[1705] Input: Slideshow sent from server

[1706] Output: Slideshow displayed on device

[1707] Step 8:

[1708] The user (crew member) checks the slideshow received on the terminal and proposes products and services suitable for the newly built detached house. The crew member uses the slideshow to explain specific products and services to the customer.

[1709] Input: Slideshow displayed on device

[1710] Output: Products and services proposed to the customer

[1711] (Application example 2)

[1712] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1713] With conventional customer service systems, it was difficult to propose optimal products and services based on customer needs, which often resulted in long customer service times. Furthermore, because the quality of customer service depended on the skills and experience of the salesperson, there was also the problem of inconsistent quality. Furthermore, it was difficult to serve multiple customers at the same time.

[1714] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for inputting environmental interviews with customers during customer service, generative AI means for generating a picture-story show-style slideshow based on the input interviews, means for using the generated slideshow to present a proposal method and product appeal suited to the customer, means installed on the smart device for allowing the store clerk to propose optimal products and services based on the customer's needs, and means for displaying the generated slideshow. This makes it possible to efficiently propose optimal products and services based on the customer's needs, as well as shorten the time required for customer service and handle multiple customers simultaneously.

[1715] The "means for inputting information about the customer's environment during customer service" is an interface that allows store staff to collect information about the customer's living environment, needs, etc., and input it into the system.

[1716] "Generative AI means that generates a picture-story show-style slideshow based on input interview information" is an artificial intelligence technology that automatically generates a slideshow introducing optimal products and services based on collected customer information.

[1717] "Means of using generated slideshows to present proposals and products that suit customers" refers to a method of using generated slideshows to propose appropriate products and services to customers and effectively communicate their appeal.

[1718] "A means installed on a smart device that enables store clerks to suggest the most suitable products and services based on the customer's needs" refers to an application that is installed on a device such as a smartphone or tablet, and enables store clerks to input customer information, display the generated slideshow, and make suggestions.

[1719] The "means for displaying the generated slideshow" is a function for displaying the slideshow created by the generative AI means on the screen of the smart device.

[1720] A system for implementing this invention includes a means for inputting environmental information about a customer when serving them, a generative AI means for generating a picture-story slideshow in the form of a picture book based on the input information, a means for using the generated slideshow to present a proposal method and product appeal that is suited to the customer, a means that is installed on a smart device and enables a store clerk to propose optimal products and services based on the customer's needs, and a means for displaying the generated slideshow.

[1721] System program processing explanation

[1722] Hardware and Software

[1723] Hardware: smartphones, tablets, servers

[1724] Software: Python, OpenAI API, device application

[1725] Data processing and calculation

[1726] 1. Entering customer information: A salesperson uses a smart device to enter information about the customer's living environment and needs. For example, they enter information such as "I live in a newly built detached house."

[1727] 2. Utilizing generative AI models: Based on the customer information entered, a generative AI model (e.g., GPT-3) generates a slideshow introducing the most suitable products and services.

[1728] 3. Displaying the slideshow: The generated slideshow is displayed on the smart device screen, and the store clerk makes suggestions to the customer.

[1729] Specific examples

[1730] For example, if a customer enters "I live in a newly built detached house," the AI ​​will generate a slideshow like the one below.

[1731] Slide 1: Solar power generation system that supports eco-friendly living

[1732] Slide 2: The best smart home devices for new homes

[1733] Slide 3: Energy-efficient appliances

[1734] Prompt Sentence Examples

[1735] "Generate a slideshow showcasing products and services suitable for customers living in new homes. Customer Information: New home, eco-friendly lifestyle."

[1736] In this way, it is possible to efficiently propose optimal products and services based on customer needs, shorten customer service time, and handle multiple customers simultaneously.

[1737] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1738] Step 1:

[1739] The user uses a smart device to input customer information, specifically the customer's living environment and needs (e.g., "I live in a newly built detached house"). The input information is saved in the application on the device.

[1740] Input: Customer's living environment and needs

[1741] Output: Stored customer information

[1742] Step 2:

[1743] The device sends the stored customer information to the server, which analyzes the received customer information and generates a prompt to be passed to the generative AI model.

[1744] Input: Saved customer information

[1745] Output: Generated prompt statement

[1746] Step 3:

[1747] The server sends a request to a generative AI model (e.g., GPT-3) using the generated prompt, which then generates slideshow content to showcase the most suitable products and services.

[1748] Input: Generated prompt text

[1749] Output: The generated slideshow content

[1750] Step 4:

[1751] The server receives the generated slideshow content and transmits it to the terminal, which converts the received slideshow content into data for display.

[1752] Input: Generated slideshow content

[1753] Output: Slideshow data for display

[1754] Step 5:

[1755] The terminal uses the slideshow data for display to display a slideshow on the smart device screen, and the store clerk uses this slideshow to suggest the most suitable products and services to the customer.

[1756] Input: Slideshow data to display

[1757] Output: Slideshow displayed on smart device

[1758] In this way, it is possible to efficiently propose optimal products and services based on customer needs, shorten customer service time, and handle multiple customers simultaneously.

[1759] Example 3

[1760] Next, a third embodiment of the third embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1761] With the conventional customer service system, customer service times tended to be long and it was difficult for multiple staff to serve customers. In addition, the content of customer service often varied depending on the staff member, making it difficult to provide a consistent service. This led to problems such as a decline in customer satisfaction and a deterioration in efficiency.

[1762] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[1763] In this invention, the server includes means for a user to input a prompt text, means for a terminal to send the prompt text to the server, means for the server to receive the prompt text and input it to a generative AI model, means for the generative AI model to generate slideshow content based on the prompt text, means for the server to input the generated content to slideshow generation software, means for the slideshow generation software to generate a visually formatted slideshow, means for the server to send the generated slideshow to the terminal, and means for the terminal to display the slideshow to the user. This reduces customer service time, enables multiple people to serve customers, and further standardizes customer service.

[1764] A "user" is an entity that uses the system to input prompt sentences and generate a slide show.

[1765] A "prompt sentence" is an instruction sentence entered by the user, and is the information that the generative AI model uses to generate the content of the slideshow.

[1766] A "terminal" is a device through which a user enters prompts and communicates with a server.

[1767] The "server" is a computer system that receives prompt text, generates a slideshow using a generative AI model, and sends it to the device.

[1768] A "generative AI model" is an artificial intelligence algorithm that generates slideshow content based on prompt text.

[1769] "Slideshow generation software" is software that visually formats the content generated by a generative AI model and outputs it as a slideshow.

[1770] A "slideshow" is a visually stylized series of slides generated by generative AI models and slideshow generation software.

[1771] This invention is a system that generates a slideshow using a generative AI model, shortening the time it takes to serve customers and enabling multiple people to serve customers. Specific embodiments of this system are described below.

[1772] First, the user inputs a prompt using the device. The prompt is the information that the generative AI model uses to generate the content of the slideshow. For example, the user might input a prompt such as, "I would like you to create a slideshow that explains the features of our new product."

[1773] The device then sends the prompt to a server, which receives it and inputs it into a generative AI model, such as an advanced natural language processing model like GPT-4.

[1774] The generative AI model generates slideshow content based on prompts, which are presented in text format and include information such as the features, benefits, and usage of new products.

[1775] The server then inputs the generated text into slideshow generation software, such as the Microsoft PowerPoint API or Google Slides API, which visually formats the generated text and outputs it as a slideshow.

[1776] The generated slideshow is sent from the server to the terminal, which then displays it to the user. The user can then review the generated slideshow and make any necessary corrections or additions.

[1777] For example, if a user inputs the prompt "Please create a slideshow that explains the features of a new product," the generative AI model generates text describing the new product's features, benefits, and usage. The model then uses the Microsoft PowerPoint API to generate a visually formatted slideshow and sends it to the device. The user can then review the generated slideshow and make any necessary corrections or additions.

[1778] In this way, the slideshow generation system using the generative AI model functions as a tool for shortening customer service time and efficiently serving customers with multiple people. It is also possible to standardize customer service. The flow of the identification process in Example 3 will be described with reference to FIG. 15.

[1779] Step 1:

[1780] The user enters a prompt statement.

[1781] The user inputs a prompt into the input field of the device, for example, "I want you to create a slideshow that explains the features of our new product." The input prompt is temporarily stored in the device's memory.

[1782] Step 2:

[1783] The terminal sends the prompt to the server.

[1784] The terminal sends the prompt text entered by the user to the server. HTTP or HTTPS is used as the communication protocol. The input is the prompt text, and the output is a notification to the server that transmission has been completed.

[1785] Step 3:

[1786] The server receives the prompt sentence and inputs it into the generative AI model.

[1787] The server receives the prompt sentence sent from the terminal and inputs the received prompt sentence into the generative AI model. The input is the prompt sentence, and the output is a notification to the generative AI model that the input has been completed.

[1788] Step 4:

[1789] A generative AI model generates slideshow content based on prompts.

[1790] A generative AI model generates content for each slide in a slideshow based on a prompt, such as text about the features, benefits, and usage of a new product. The input is the prompt, and the output is the generated text.

[1791] Step 5:

[1792] The server inputs the generated content into the slideshow generation software.

[1793] The server inputs the text obtained from the generative AI model into the slideshow generation software. The input is the generated text, and the output is a notification to the slideshow generation software that the input has been completed.

[1794] Step 6:

[1795] Slideshow generation software generates a visually formatted slideshow.

[1796] Slideshow generation software generates a visually formatted slideshow from input text, for example by applying an appropriate layout or design to each slide. The input is the generated text and the output is a visually formatted slideshow.

[1797] Step 7:

[1798] The server transmits the generated slideshow to the terminal.

[1799] The server sends the generated slideshow to the device using HTTP or HTTPS as the communication protocol. The input is the visually formatted slideshow, and the output is a notification of completion of transmission to the device.

[1800] Step 8:

[1801] The terminal displays the slideshow to the user.

[1802] The terminal displays the slideshow received from the server to the user. The user can review the generated slideshow and make corrections or additions as needed. The input is a visually formatted slideshow, and the output is the display to the user.

[1803] (Application example 3)

[1804] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1805] With conventional customer service systems, each customer service representative must deal with each customer individually, which results in long service times and makes it difficult to serve multiple customers at the same time. Another problem is that the quality of customer service varies depending on the representative. Furthermore, while there is a demand for efficient information provision using smart devices, the current system is unable to adequately address this.

[1806] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[1807] In this invention, the server includes means for inputting information about the customer's environment during customer service, a generation system AI means for generating a picture-story show-style slideshow based on the input information, means for using the generated slideshow to present a proposal method and product appeal tailored to the customer, means for displaying the slideshow on a smart device, and means for generating and displaying individual slideshows so that multiple customer service representatives can simultaneously serve different customers. This reduces customer service time and enables simultaneous service to multiple customers, resulting in standardized customer service quality and efficient information provision using smart devices.

[1808] "Means for inputting environmental information about customers during customer service" refers to devices or methods for collecting information about customers' needs, interests, and environment and inputting it into the system.

[1809] A "picture-story show slideshow" is a presentation format that conveys information visually by displaying a series of images and text in sequence.

[1810] "Generative AI means" refers to technologies or devices that use artificial intelligence to automatically generate content based on input data.

[1811] "Methods of proposing and promoting products that suit customers" refers to methods and devices for proposing optimal products and services based on customers' needs and interests, and effectively communicating their appeal.

[1812] "Means for displaying a slideshow on a smart device" refers to techniques or devices for displaying the generated slideshow on a device such as smart glasses or a smartphone.

[1813] "Means for generating and displaying individual slideshows so that multiple customer service representatives can simultaneously serve different customers" refers to technology or devices that allow multiple customer service representatives to simultaneously generate and display individually customized slideshows for different customers.

[1814] The system for implementing this invention includes a means for inputting information about the customer's environment when serving customers, a generative AI means, a means for displaying a slideshow on a smart device, and a means for generating and displaying individual slideshows so that multiple staff members can serve different customers simultaneously.

[1815] System configuration

[1816] 1. A way to input information about the customer's environment during customer service

[1817] This is a device that uses input devices such as tablets and smartphones to collect information about customers' needs, interests, and environment and input it into the system.

[1818] 2. Generative AI means

[1819] The generative AI method uses generative AI models such as the OpenAI API to automatically generate a picture-story show based on the input interview data. The generated slideshow is used to present optimal proposal methods and product appeal to customers.

[1820] 3. How to display slideshows on smart devices

[1821] A technology or device that displays generated slideshows on devices such as smart glasses (e.g., Google Glass, Vuzix Blade) or smartphones, allowing customer service staff to provide information visually to customers.

[1822] 4. A way to generate and display individual slideshows for multiple salespeople to simultaneously serve different customers

[1823] The generative AI method allows multiple customer service representatives to simultaneously generate and display individually customized slideshows for different customers, making it possible to serve multiple customers at the same time.

[1824] Program processing explanation

[1825] The server first receives data collected through a means for inputting information about the customer's environment during service. Next, it uses a generative AI means to generate a picture-story show-style slideshow based on the input data. The generated slideshow is displayed on a smart device, which may include smart glasses or a smartphone. Furthermore, the generative AI means can generate and display individual slideshows so that multiple service representatives can simultaneously serve different customers.

[1826] Specific examples

[1827] For example, if a customer asks, "What are the features of my new smartphone?", the salesperson uses a tablet to input this question into the system. The generative AI means generates a slideshow based on this input and displays it on the smart glasses. If another salesperson is asked by another customer, "What are the latest fashion items?", a separate slideshow is similarly generated and displayed.

[1828] Prompt Sentence Examples

[1829] "Please tell me the features of your new smartphone."

[1830] In this way, the system can efficiently and effectively handle customer inquiries.

[1831] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[1832] Step 1:

[1833] Users use input devices such as tablets and smartphones to input information about their needs, interests, and surroundings. The input data is then sent to the server.

[1834] Input: Information about your needs, interests, and circumstances

[1835] Output: Hearing data sent to the server

[1836] Step 2:

[1837] The server passes the received hearing data to the generative AI means, which uses a generative AI model such as the OpenAI API to generate a picture-story show-style slideshow based on the input data.

[1838] Input: Hearing data

[1839] Output: Generated slideshow

[1840] Step 3:

[1841] The server transmits the generated slideshow to a smart device, which may include smart glasses or a smartphone.

[1842] Input: Generated slideshow

[1843] Output: Slideshow sent to smart device

[1844] Step 4:

[1845] The smart device displays the received slideshow, and the customer service representative provides the information visually to the customer through the smart device.

[1846] Input: Slideshow sent to smart device

[1847] Output: Slideshow displayed on smart device

[1848] Step 5:

[1849] When multiple customer service representatives are simultaneously serving different customers, the server passes each representative's interview data to the generative AI means to generate individual slideshows, which are then sent to and displayed on each smart device.

[1850] Input: Multiple hearing data

[1851] Output: Individually generated slideshow

[1852] In this way, the system can efficiently and effectively handle customer inquiries.

[1853] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1854] "Example 1"

[1855] One embodiment of the present invention is a system that incorporates an emotion engine. In this system, the environment of the customer is interviewed during the customer service session, and that information is input into the system. The input information is analyzed by a generative AI, which generates a picture-story slideshow suited to the customer's situation. The generated slideshow is used as a guide for the crew when presenting proposal methods and product appeals to the customer. Furthermore, the emotion engine recognizes the customer's emotions and adjusts the content of the slideshow and the customer service style based on those emotions.

[1856] "Example 2"

[1857] As a specific example, consider the case where a crew member inputs information about a customer's living environment, such as "I live in a newly built detached house," and the customer shows a pleased expression. Based on this information and the emotion recognition results of the emotion engine, the generative AI generates a slideshow introducing products and services suitable for a newly built detached house. The crew member then uses the generated slideshow to suggest products and services that the customer would enjoy.

[1858] "Example 3"

[1859] The emotion engine also adjusts the customer service style according to the customer's emotions. For example, if a customer shows a confused expression, the emotion engine conveys that information to the generative AI, which then generates a slideshow with more detailed explanations. This makes it possible to provide customer service that is easy for the customer to understand.

[1860] The processing flow of each embodiment will be described below.

[1861] "Example 1"

[1862] Step 1: Interview the customer about their environment during the service and enter that information into the system. Step 2: The input information is analyzed by generative AI, and a picture-story-style slideshow appropriate to the customer's situation is generated.

[1863] Step 3: The generated slideshow will be used as a guide for the crew to present their proposals and products to the customer.

[1864] Step 4: The emotion engine recognizes the customer's emotions and adjusts the slideshow content and customer service style based on those emotions.

[1865] "Example 2"

[1866] Step 1: The crew will enter the information that "I live in a newly built detached house" during the customer's environmental interview.

[1867] Step 2: The emotion engine recognizes facial expressions that the customer is likely to be happy with.

[1868] Step 3: Based on this information and the emotion recognition results from the emotion engine, the generative AI generates a slideshow introducing products and services suitable for newly built single-family homes.

[1869] Step 4: The crew uses the generated slideshow to suggest products and services that the customer might enjoy.

[1870] "Example 3"

[1871] Step 1: The emotion engine adjusts the customer service style according to the customer's emotions.

[1872] Step 2: For example, if the customer shows a confused expression, the emotion engine conveys that information to the generative AI.

[1873] Step 3: The generative AI generates a slideshow with more detailed explanations.

[1874] Step 4: This allows for customer service that is easy for customers to understand.

[1875] Example 1

[1876] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1877] With conventional customer service systems, it was difficult to quickly and effectively make proposals based on the customer's needs and situation. It was also difficult to standardize the quality of customer service and efficiently handle customer service with multiple staff. Furthermore, it was not possible to grasp the customer's emotions in real time and respond accordingly, making it difficult to improve customer satisfaction.

[1878] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1879] In this invention, the server includes: a means for inputting information about the customer's environment during customer service; a means for transmitting the input information to the server; a means for passing the information received by the server to a generative AI model for analysis; a means for transmitting the generated picture-story slideshow from the server to a terminal; a means for using the generated slideshow to present a proposal method and product appeal tailored to the customer; and a means for analyzing the customer's emotions using an emotion engine and adjusting the content of the slideshow and the customer service style based on the emotions. This enables prompt and effective proposals tailored to customer needs, standardizes the quality of customer service, and enables efficient customer service across multiple staff. Furthermore, by understanding customer emotions in real time and responding accordingly, customer satisfaction can be improved.

[1880] The "means for inputting information about the customer's environment when serving customers" is a function that provides an interface for users to input information about the customer's needs and situation using a terminal.

[1881] The "means for transmitting input hearing information to a server" is a communication function that enables the terminal to transmit information input by a user to a server.

[1882] "Means for passing the information received by the server to the generative AI model for analysis" refers to the processing function for passing the customer information received by the server to the generative AI model for analysis.

[1883] "Means for transmitting the generated picture-story show-style slideshow from the server to the terminal" refers to a communication function for transmitting the slideshow generated by the generative AI model from the server to the terminal.

[1884] "Means of using the generated slideshow to present a proposal method and product appeal that is suited to the customer" is a function that allows the user to present a generated slideshow to the customer using the terminal and appeal to the customer with a proposal method or product appeal.

[1885] "A means of analyzing customer emotions using an emotion engine and adjusting the content of the slideshow and customer service style based on those emotions" is a function that analyzes the customer's facial expressions using the device's camera and adjusts the content of the slideshow and customer service style based on that emotional data.

[1886] This invention is a system that listens to the customer's environment while serving them, generates a picture-story show-style slideshow using a generative AI model based on that information, and then uses an emotion engine to adjust the content of the slideshow and the serving style based on the customer's emotions.

[1887] The user uses a device (e.g., a tablet or PC) to conduct an environment interview with the customer...

Claims

1. A means for a user who is a salesperson to input the results of hearing about a customer's living environment by voice while serving the customer; A means for generating a slide show using a prompt sentence for instructing to generate a slide show showing a product and a method for proposing the product suited to the customer based on the result of the hearing and a generative AI model; means for displaying the generated slide show on a display unit of smart glasses worn by the user; A system including:

2. A means for recognizing an emotion of the customer from a facial expression of the customer using an emotion engine; and a means for modifying the generated slide show using a prompt sentence instructing modification of the generated slide show based on the recognition result of the customer's emotion and the generative AI model; The system of claim 1 , wherein the displaying means displays the modified slide show on a display of smart glasses worn by the user.

3. The input means inputs the results of the hearing by voice for each of the multiple customers when the user is serving multiple customers at the same time, said generating means generates said slide show for each of said plurality of customers; The system according to claim 1 , wherein the display means switches the display of the generated slide show for a different customer each time the user switches between customers he is serving.

Images