system

Abstract

We provide the system. [Solution] A means of receiving reservation information and survey responses and saving them in a database, A generation means that analyzes stored information and generates the optimal call script, A means for executing an auto-call based on the talk generated by the generation means, A means for collecting user response results via automated calls, analyzing them, and generating a summary, A means of suggesting customer service content to the terminal based on the generated summary, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] When customers visit the store, there are many cases where pre-call has not been made. Also, even when a call is made, the quality of the conversation content varies depending on the ability of the staff. As a result, there is a problem that promoting store visits by families and further exploring needs for other merchandise are insufficient, and the acquisition of store visitors has not been maximized. It is necessary to solve these problems and improve the service for customers who have made store reservation more efficiently and effectively.

Means for Solving the Problems

[0005] The present invention provides a means for receiving and storing reservation information and questionnaire responses. Next, a generation means is used in which a generating AI analyzes the stored information and generates an optimal pre-call script. Based on this generated script, the system provides a means for executing an auto-call to automatically make a call to the user. Furthermore, it provides a means for collecting the user's response results after the auto-call and creating a summary using the generating AI. Finally, the system provides a means for suggesting customer service content to the terminal based on this generated summary. This improves the call execution rate and the quality of the script, enabling personalized customer service for the user.

[0006] "Reservation information" refers to a collection of data, such as the date and time and the desired service, that users enter when making a reservation through the system.

[0007] "Survey responses" refer to information provided by users when making a reservation to visit the store, such as their family structure and their usage of other products.

[0008] "Generation method" refers to a function that uses generation AI to generate the optimal phone call script based on the entered reservation information and survey responses.

[0009] "AutoCall" is a process in which the system automatically makes a phone call to the user based on a generated message and delivers a voice message.

[0010] "Response results" refer to a collection of data that includes the user's reactions and responses to automated calls.

[0011] A "summary" is information provided for customer service suggestions, which is a compilation of key points analyzed based on the response results obtained from the user.

[0012] A "terminal" is a device used by customer service staff, and it is a tool for displaying generated summaries and suggested customer service content. [Brief explanation of the drawing]

[0013] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] This is a sequence diagram showing the processing flow of the data processing system in Example 2, which incorporates an emotion engine. [Figure 14] This is a sequence diagram showing the processing flow of the data processing system in Application Example 2, which combines an emotion engine. [Modes for carrying out the invention]

[0014] An example of an embodiment of the system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0015] First, the terms used in the following description will be explained.

[0016] In the following embodiments, a tagged processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0017] In the following embodiments, a tagged RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0018] In the following embodiments, a tagged storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, and the like.

[0019] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0020] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0021] [First Embodiment]

[0022] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0023] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0024] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0025] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0026] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0027] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0028] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0029] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0030] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0031] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0032] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0033] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0034] This invention aims to improve the efficiency of managing customer information and making pre-appointment calls when users make reservations to visit a store. The system consists of three main elements: a server, a user, and a terminal. Specific embodiments are shown below.

[0035] The server first receives the customer's reservation information and survey responses and stores them appropriately in a database. This information includes the reservation date and time, the type of service desired, family composition, and other services of interest. The server analyzes this stored information and uses AI to generate an optimal pre-call message for each user. The generated message is personalized based on the user's attributes and responses, and is tailored to the user's interests and needs.

[0036] Next, the server automatically calls the user via the auto-call system and delivers the generated conversation as an audio message. The user receives the sent auto-call message and can listen to its contents at home or elsewhere. The server collects the results of this auto-call, namely the user's responses and reactions, and stores them as data. The generating AI analyzes this data and generates a summary that highlights the key points.

[0037] Finally, the server sends the generated summary to a terminal used by the service staff. The terminal displays the summary, and the service staff uses it to provide personalized service to the user. For example, they can offer service suggestions for the whole family or specific suggestions for improving the user's current plan.

[0038] As a concrete example, suppose a user considering communication service plans for their family makes an appointment to visit a store and responds to a questionnaire stating, "Our family consists of four people, each using a different plan. We would like to review our plans." In this case, the server analyzes this information and generates a conversation that includes "a proposal for a unified plan for the whole family" and "questions about the benefits of each plan." Based on the user's response to the generated conversation, the server generates a summary to make a specific proposal for "a bundled family plan contract" and presents it to the device.

[0039] This system eliminates inconsistencies in communication styles due to individual differences, enabling more efficient and effective user support.

[0040] The following describes the processing flow.

[0041] Step 1:

[0042] The server receives reservation information and survey responses from users and stores this data in a database. This information includes the planned date and time of visit, services of interest, family structure, and information on other products currently being used.

[0043] Step 2:

[0044] The server uses stored reservation information and survey data to activate the generation AI. The generation AI analyzes the input information and automatically generates the optimal call script for each user. The generated script includes content customized based on the user's attributes and interests.

[0045] Step 3:

[0046] The server passes the generated call script to the auto-call system, which then automatically places a call to the user. The user receives the auto-call and can listen to the conversation over the phone. This process allows for pre-contact to encourage family visits or pique interest in other products.

[0047] Step 4:

[0048] The server collects the user's response after the auto-call is executed. Using this data, including the user's answers and reactions, the generating AI is reactivated to analyze the response. It then generates a summary that extracts the key points.

[0049] Step 5:

[0050] The server sends the generated summary to a terminal used by the store crew. The terminal displays this summary, which the crew can refer to during customer service, enabling a personalized approach to the user. For example, it can provide detailed suggestions for family-oriented communication plans.

[0051] (Example 1)

[0052] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0053] Current customer service systems often rely on manual processes for information management and customer communication, resulting in inefficiencies. Furthermore, the level of consistency and personalization provided to individual customers is insufficient. Therefore, there is a need to improve customer satisfaction while efficiently delivering personalized service.

[0054] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0055] In this invention, the server includes means for receiving reservation data and survey results and storing them in a storage means, means for processing the stored information and generating automatically generated communication content, and means for performing automated voice communication based on the generated communication content. This enables efficient management and processing of customer information, and improves the consistency and satisfaction of customer service through personalized automated communication.

[0056] "Reservation data" refers to information provided by users regarding the date and time of their visit and the services they wish to receive.

[0057] "Survey results" refer to data obtained through questionnaires and other information gathering activities that users have answered.

[0058] "Memory devices" refer to devices and technologies for long-term storage of digital information.

[0059] "Processing" refers to a series of operations that analyze received data and organize it into information.

[0060] "Automatically generated communication content" refers to the content of messages and conversations that are automatically created for the user.

[0061] "Automated voice communication" refers to a communication method that uses machine-generated voice messages to contact users.

[0062] A "summary" refers to a short, concise version of data or information, extracted from the original source to highlight the most important points.

[0063] "Display devices" refer to devices used by humans to visually confirm information.

[0064] "Response results" refer to data obtained as responses and feedback provided by users.

[0065] "Personalized services" refer to the provision of services tailored to the individual needs and characteristics of each user.

[0066] This invention provides a system for efficiently managing customer reservation data and survey results, and for providing personalized services. This system consists of three main elements: a server, a terminal, and a user.

[0067] The server receives reservation data and survey results and saves them to a relational database (e.g., MySQL®, PostgreSQL). The server analyzes the saved information using Python and generates automatically generated communication content using a generative AI model (such as TENSORFLOW® or PyTorch). This communication content is generated via prompt statements and is tailored to the individual needs of the user. For example, a prompt statement in the format of "Create the optimal proposal talk for user A, a family of four who is reviewing their communication plan" might be used.

[0068] Next, the server uses the automatically generated communication content to perform an automated voice call to the user via an auto-call system (e.g., a common voice communication API). The user receives the message through this voice call and can check its contents at home or elsewhere. The auto-call system collects the user's responses and transfers this data to the server.

[0069] The server analyzes the collected response data again using a regenerative AI model to generate a summary. This summary provides condensed information crucial for future customer service interactions. The generated summary is sent to a terminal used by the service staff, where detailed information is displayed. The service staff uses this summary to propose personalized services to users and improve the quality of service delivery.

[0070] As a concrete example, consider a scenario where a user requests either a unified plan proposal for the entire family or an explanation of the benefits of individual plans. In such cases, the system can effectively provide proposals that meet the user's expectations.

[0071] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0072] Step 1:

[0073] Users input reservation data and survey results using a dedicated application or web form. This includes information such as preferred date and time of visit, desired service type, family composition, and categories of interest. The input results are sent to the server. The arrival of the input data on the server provides the foundational data necessary for subsequent analysis.

[0074] Step 2:

[0075] The server stores reservation data and survey results received from users in a relational database. Specifically, it connects to a database management system (e.g., MySQL or PostgreSQL) and inserts the received data into the appropriate tables. This step ensures that the information is persistently stored and available for later analysis.

[0076] Step 3:

[0077] The server prepares a generative AI model to analyze the stored information. First, it runs a Python script to load input data into the model using the stored data. A prompt statement (e.g., "Create the best suggestion talk for a user who is reviewing their communication plan as a family of four") is input into the generative AI model, and the analysis begins. The model generates individual talk content based on this input information.

[0078] Step 4:

[0079] The server prepares an auto-call system based on the generated talk content. It then configures the system to automatically communicate with the user via voice. Specifically, it converts the generated talk into a voice message using a voice API and dials the user's phone number. This allows the user to receive personalized service information at home or elsewhere.

[0080] Step 5:

[0081] The user receives an automated call and confirms the message. The system captures the user's listening response and reactions (e.g., button presses or voice responses) and feeds them back to the server. This input data becomes core data for subsequent analysis.

[0082] Step 6:

[0083] The server analyzes the user's response and generates a summary using a generative AI model. Based on this summary, the server extracts important information useful for future customer service and prepares to provide it to the customer service staff.

[0084] Step 7:

[0085] The server sends the generated summary to a terminal used by the service staff. The terminal displays the received summary, providing the service staff with material to make personalized suggestions. The staff refer to the displayed information and make the best suggestions for the user. This enables the delivery of personalized service.

[0086] (Application Example 1)

[0087] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0088] For effective customer service in stores, it is crucial to thoroughly understand customer needs before they visit and provide appropriate service. However, traditional methods have been problematic because reservation information management and the content of pre-visit phone calls are highly dependent on individual staff members, leading to inconsistent results. Furthermore, for service staff to respond to customer needs, they are required to understand a large amount of information quickly and make immediate suggestions. Under these conditions, the burden on store staff becomes significant, making it difficult to maintain customer satisfaction.

[0089] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0090] In this invention, the server includes means for receiving reservation information and questionnaire responses and storing them in a storage medium; means for analyzing the stored information and generating an optimal call script; means for executing a voice call based on the script generated by the generation means; means for collecting the user's response results from the voice call, analyzing them, and generating a summary; and means for proposing and presenting the generated summary and personalized customer service suggestions to the terminal. This enables a detailed understanding of the user's needs when making a reservation and allows for personalized customer service.

[0091] "Reservation information" refers to information about the date and time of the customer's visit and the services they wish to receive.

[0092] "Survey responses" refer to information about the user's personal interests and needs provided when making a reservation.

[0093] A "storage medium" is a digital storage system used to store and preserve information.

[0094] A "generation method" is a system that has the function of creating talks and summaries based on collected and analyzed information.

[0095] "Voice communication" refers to the process of communicating using voice via electronic means.

[0096] A "user" is an individual or group that makes a reservation for a service and is the recipient of the customer service or services provided.

[0097] A "terminal" is an electronic device used by staff to verify the generated information.

[0098] A "personalized customer service plan" is a proposed customer service response tailored to the individual needs and interests of the user.

[0099] This invention is a system that enables effective communication and customer service with users. It consists of three main elements: a server, a terminal, and the user.

[0100] The server receives reservation information and survey responses entered by the user. This information is stored in a database such as MySQL or PostgreSQL. Subsequently, the stored information is analyzed using Python and Django, and optimal call scripts and summaries are generated by generative AI models such as OpenAI's GPT series.

[0101] The generated conversation is automatically transmitted to the user using a voice call system. Text-to-speech (TTS) technology is used for the text-to-speech conversion. After the user listens to the conversation, their response is collected on a server, re-analyzed, and a summary is generated.

[0102] The device, specifically a smartphone app developed using React Native, receives summaries and personalized service suggestions sent from the server. The app on the device displays this information in real time, helping store staff make the best service suggestions for customers.

[0103] For example, if a user responds to a survey by saying they "want to review their family's communication plan," the server uses AI to create a conversation that includes optimal suggestions and displays specific customer service proposals on the device, such as "Please suggest a new communication plan that can be used by the whole family."

[0104] An example of a prompt is, "To suggest the optimal communication plan based on family structure, please create a personalized conversation that takes into account the user's current usage and desired changes." This prompt instructs the generative AI model to generate specific conversation content.

[0105] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0106] Step 1:

[0107] The server receives reservation information and survey responses from users. The input includes information such as the date and time, desired services, and needs, which are then stored in the database. The output here is the stored customer information. Specifically, the received data is stored in a MySQL database via SQL queries.

[0108] Step 2:

[0109] The server analyzes the stored customer information. This analysis process uses a Python script to read information from the database and extract user attributes and needs. The input is user information retrieved from the database, and the output is the analyzed data. Specifically, the information is organized in JSON format.

[0110] Step 3:

[0111] The server generates the optimal call script using a generative AI model based on the analyzed data. A prompt is used to request the generative AI to create a script that matches the user's needs. The input at this stage is the analysis result and the prompt. The output is the generated call script. For example, data can be sent to the OpenAI API and the generated text can be received.

[0112] Step 4:

[0113] The server converts the generated talk into voice data using TTS technology and automatically makes a call using the voice call system. The input is the generated text talk, and the output is voice data. Specifically, the Google® Text-to-Speech API is used to convert the talk into voice.

[0114] Step 5:

[0115] The user receives and responds to an incoming voice call. The input is the voice call data, and the output is the user's response information. The user listens to the voice message using a regular telephone handset.

[0116] Step 6:

[0117] The server automatically collects user response information and analyzes it again. This analysis identifies key points. The input is the user's response, and the output is summarized data. Specifically, it converts speech to text and performs analysis using NLP (Neuro-Linguistic Programming) technology.

[0118] Step 7:

[0119] The terminal receives the analyzed summary data from the server, which is then accessed by staff. The input is the summary data sent from the server, and the output is the content displayed on the terminal. Specifically, the data is displayed in the user interface using a React Native application.

[0120] Step 8:

[0121] The staff app on the device presents personalized service suggestions to the user based on summarized information. The user receives service suggestions from the staff. The input is summarized information, and the output is the staff's suggestions. The staff responds based on the information displayed on the device.

[0122] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0123] This invention is a system that enables efficient information management when users make reservations and allows for pre-visit phone calls and customer service that take emotions into consideration. The invention provides a more personalized service by receiving, analyzing, and utilizing user reservation information and questionnaire responses, as well as emotion recognition.

[0124] The server first receives reservation information and questionnaire responses from users and stores this data in a database. This includes information such as the reservation date and time, expected services, family structure, and products of interest. Based on this received data, the server activates a generation AI to create an optimal pre-call script for each user.

[0125] The AI-generated conversation is tailored based on the user's attributes and interests, and an emotion engine is integrated into this process. The emotion engine analyzes all data, including the user's responses, to recognize the user's emotional state. This allows it to analyze what emotions the user expressed previously, what their satisfaction and dissatisfaction levels were, and incorporate this information into the call conversation.

[0126] The server passes this generated talk to the auto-calling system, which then automatically dials the user. The auto-calling system delivers the talk in an audio format that takes emotions into account, enabling a more empathetic and appropriate approach to the user. Users receive this call at home or on the go and listen to the content to deepen their understanding of the service.

[0127] After the automated call, the server collects the user's response again, and the emotion engine analyzes the user's reaction and emotions. Based on the resulting emotion information and response content, the generative AI creates a summary. This summary includes the user's emotion information and suggests appropriate customer service.

[0128] The terminal is a device used by the store crew to display this summary, which the crew uses in customer service. The crew can use this information to provide personalized suggestions and support to the user. For example, if a user expresses anxiety, they can provide detailed service explanations or reassuring explanations, resulting in more attentive and emotionally sensitive customer service.

[0129] As described above, the present invention enables automated pre-calling that takes into account the user's emotional state and personalized customer service, thereby realizing the provision of more efficient and effective customer service.

[0130] The following describes the processing flow.

[0131] Step 1:

[0132] The server receives customer reservation information and survey responses and stores them in a database. The reservation information includes the date and time and the services expected, while the survey includes family composition and information on products currently being used.

[0133] Step 2:

[0134] The server activates a generation AI using saved reservation information and survey responses. The generation AI analyzes this information and generates a customized pre-call message for each user. The message content is adjusted according to the user's attributes and past survey results.

[0135] Step 3:

[0136] The server utilizes an emotion engine to recognize the user's emotional state by analyzing past user responses and reservation information. This allows for further personalized conversations.

[0137] Step 4:

[0138] The server sends generated, emotion-driven messages to the auto-call system and automatically places a call to the user. The user can then listen to the customized message content through the received auto-call.

[0139] Step 5:

[0140] After an automated call, the server collects the user's response and uses an emotion engine to analyze the user's emotions and reactions in detail. Based on this analysis, a generative AI creates a summary.

[0141] Step 6:

[0142] The server sends the generated summary to the terminal. The terminal displays this summary, helping the store crew provide customer service based on it. The crew can then provide more personalized service based on the user's identified emotions and interests.

[0143] (Example 2)

[0144] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0145] In today's service industry, providing efficient and personalized customer service is a challenge, especially given the need for appropriate responses to users. In particular, the lack of prior contact and interaction that takes user emotions into account necessitates efficient information management while optimizing user satisfaction.

[0146] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0147] In this invention, the server includes means for receiving reservation information and questionnaire answers and storing them in a storage device, means for analyzing the stored information and generating optimal automated call content, and means for executing a machine-controlled call based on the call content generated by the generation means. This enables the generation of automated call content that takes into account the user's emotions and efficient information management and analysis.

[0148] "Reservation information" refers to detailed information such as the date, time, location, and content of the service that the user plans to provide.

[0149] "Questionnaire responses" refer to information collected from users through surveys and feedback, and are used to understand the characteristics and needs of users.

[0150] A "memory device" is a data storage system that stores received data and makes it available for use as needed.

[0151] "Generation means" refers to a process or algorithm for producing appropriate output based on stored information, and in this invention, it is used to generate automated call content.

[0152] "Automated call content" refers to the content of a call using pre-prepared voice messages or dialogue scripts, intended for prior notification or confirmation to the user.

[0153] "Machine-controlled communication" refers to automated voice calls conducted by a communication system, and is a means of telephone communication that is mechanically controlled.

[0154] The "generated summary" is a summary of the analyzed data, concisely describing the user's emotional state and necessary countermeasures.

[0155] "Emotional information" refers to information that indicates the emotional state and characteristics analyzed based on data provided by the user.

[0156] As the foundation for implementing this invention, the server receives reservation information and questionnaire responses from users. This data is stored in a database such as MySQL and serves as an important source of information for forming individual user profiles. Based on this information, the server uses a generative AI model to generate optimal automated call content. A general-purpose AI model with excellent natural language processing capabilities is used as the generative AI model.

[0157] The generated automated call content is adjusted via an emotion analysis engine to take into account the user's past responses and emotional information. This engine uses stored historical data to infer the user's emotional state and reflects it in the call content.

[0158] Next, the server converts the coordinated call content into speech using speech synthesis software and makes a phone call to the user through a machine-controlled call system. Speech synthesis technologies such as Amazon Polly are useful in this process. Machine-controlled calls can be received by users anywhere and provide them with a means to better understand the service.

[0159] Through the voice call, the user receives information about the service and how to make their next appointment, and responds as needed. The server collects these responses again and analyzes them in detail using an emotion engine. Based on the analysis, the generating AI creates a summary that again reflects the user's emotions and displays it on the device.

[0160] The terminal will display this overview as a device used by store crew members, who will then use it in customer service. Crew members can use this information to, for example, provide detailed explanations to customers who are feeling anxious about a new service, thereby reassuring them.

[0161] A concrete example of a prompt is, "The user is interested in the outdoors. Please create a call script to confirm a reservation for their next visit." Through this prompt, the generating AI model provides a personalized response and delivers information tailored to the user's needs.

[0162] In this way, the system of the present invention enables highly personalized services that take into account the user's emotions, and achieves efficient and highly accurate customer service.

[0163] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0164] Step 1:

[0165] The server receives reservation information and questionnaire responses from users. Input data includes reservation date and time, desired service, and hobbies / interests. This data is stored in a MySQL database and managed on a per-user basis. The stored data will be used for future analysis and as input for generating AI.

[0166] Step 2:

[0167] The server generates a prompt based on the saved reservation information and questionnaire responses. This prompt is used as input to a generative AI model. This generative model generates automated call content written in natural language based on the given prompt. For example, it might output something like, "The user is interested in outdoor activities. Please create a call script to confirm their next visit."

[0168] Step 3:

[0169] The generated call content is further analyzed by the emotion engine. Past response history and data on the user's emotional state are used as input. The emotion engine identifies the user's emotions and adjusts the call content to provide optimal information and tone, thereby creating an emotionally sensitive call output.

[0170] Step 4:

[0171] The server uses the adjusted call content as input data, which is then converted into speech using speech synthesis software. Specifically, it outputs the call content in text format as an audio file and sends it to the machine-controlled call system. This audio is then automatically dialed to the user, and the content is transmitted.

[0172] Step 5:

[0173] The user receives an automated voice call and confirms its content. The user responds as needed, following the voice guidance. The response data is returned to the server, saved again to the database, and analyzed.

[0174] Step 6:

[0175] The server analyzes user response data using an emotion engine. The user's most recent response information is used as input to identify user satisfaction and emotional state. A summary is generated based on this information, and this summary serves as the basis for preparing for the next customer interaction.

[0176] Step 7:

[0177] The terminal displays an analyzed summary on the device used by the store crew. The crew refers to this summary and provides customer service tailored to each customer's needs. Specifically, the crew can alleviate customer anxiety by providing detailed service explanations and emotional encouragement.

[0178] (Application Example 2)

[0179] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0180] Currently, in brick-and-mortar stores, there is a demand for personalized service based on the individual interests and emotional states of customers. However, implementing this requires considerable effort and time. In particular, there is a lack of efficient systems for providing customer service that meets the diverse needs of different customers, which increases the burden on store staff and hinders improvements in customer satisfaction.

[0181] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0182] In this invention, the server includes means for receiving reservation information and questionnaire responses and storing them in a storage means, means for analyzing the stored information and generating optimal communication content, and means for executing automated voice communication based on the communication generated by the generation means. This makes it possible to grasp the status information of each customer and efficiently provide personalized customer service.

[0183] "Reservation information" refers to information that includes the date and time the user wishes to receive the service, as well as the details of the service requested.

[0184] A "survey response" is the answer to a question asked to obtain details about a user's interests and expectations.

[0185] A "memory device" is a data storage system for holding received information.

[0186] "Analysis" is the process of analyzing the content and characteristics of acquired information.

[0187] "Communication content" refers to messages and audio information sent to the user.

[0188] "Generation means" refers to the process of creating new communication content based on the analyzed information.

[0189] "Automated voice communication" is a system that automatically transmits pre-generated voice information to the user.

[0190] "State information" refers to data about a user's internal situation, such as their emotions and interests.

[0191] An "information terminal" is an electronic device that displays prepared information and allows users to view it.

[0192] "Customer service information" refers to customer service methods and recommendations provided according to the user's characteristics and emotions.

[0193] The system for implementing this invention is primarily built around a server, a terminal, and user interaction. The server receives reservation information and survey responses from the user and stores them in a storage device. This allows for recording the service content the user expects and their past usage history, which can then be used to improve future services. For the storage device, it is generally recommended to use Firebase, a cloud-based data storage service.

[0194] Next, the server performs analysis using the stored information. This analysis includes sentiment analysis using machine learning frameworks such as TensorFlow. The user's emotional tendency data obtained through sentiment analysis is information that can lead to improved user satisfaction, and the generation method creates the optimal communication content based on this.

[0195] The generated communication content is delivered to the user via automated voice communication. This process utilizes the Google Cloud Text-to-Speech API to generate text-to-speech and automatically deliver it to the user. Through this voice communication, the user can check the details of the service they plan to use in advance.

[0196] The terminal will be used in stores as an information terminal. It will display customer service information that users should refer to when they visit the store. As a result, staff will be able to easily provide personalized service, improving the quality of customer service.

[0197] For example, in the case of a casual apparel store, if a user requests to "book a fitting appointment for a weekend afternoon," the AI ​​model can prepare in advance fitting outfits and related product information that are perfectly suited to that user.

[0198] An example of a prompt for the generating AI model could be, "Based on this week's booking data, please create casual fashion conversation content tailored to the user's preferences." This enables the provision of highly flexible and personalized services.

[0199] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0200] Step 1:

[0201] The server receives reservation information and survey responses from users. Inputs include the date and time, desired service details, and areas of interest, all entered by the user via a smartphone app. This input data is saved to cloud storage. The output is the accurately saved user reservation data.

[0202] Step 2:

[0203] The server analyzes reservation information and survey responses stored in memory to understand the user's emotional tendencies. This process uses TensorFlow, a machine learning framework. The input is the user data stored in step 1, and the output is the identification result of the user's emotional tendencies.

[0204] Step 3:

[0205] The server uses a generative AI model to generate optimal communication content based on the analyzed sentiment tendencies and other user characteristics. The input is the output data from step 2 and user attribute data, and the output is the text of the generated communication content.

[0206] Step 4:

[0207] The server converts the generated communication content into speech data using the Google Cloud Text-to-Speech API. The input is the text data generated in step 3, and the output is the speech data sent to the user.

[0208] Step 5:

[0209] The user receives an audio communication and uses it to preview the content of the service being offered. The input is the audio data from step 4, and the output is to improve the user's understanding and pique their interest.

[0210] Step 6:

[0211] The terminal displays customer service information necessary when a user visits the store. Input is user-specific customer service information sent from the server, and output is a specific customer service guide that can be used by store staff.

[0212] Step 7:

[0213] The staff operating the terminal will provide personalized service to the user based on the displayed service information. The input is the service information from step 6, and the output is a high-quality, personalized customer service experience.

[0214] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0215] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0216] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0217] [Second Embodiment]

[0218] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0219] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0220] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0221] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0222] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0223] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0224] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0225] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0226] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0227] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0228] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0229] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0230] This invention aims to improve the efficiency of managing customer information and making pre-appointment calls when users make reservations to visit a store. The system consists of three main elements: a server, a user, and a terminal. Specific embodiments are shown below.

[0231] The server first receives the customer's reservation information and survey responses and stores them appropriately in a database. This information includes the reservation date and time, the type of service desired, family composition, and other services of interest. The server analyzes this stored information and uses AI to generate an optimal pre-call message for each user. The generated message is personalized based on the user's attributes and responses, and is tailored to the user's interests and needs.

[0232] Next, the server automatically calls the user via the auto-call system and delivers the generated conversation as an audio message. The user receives the sent auto-call message and can listen to its contents at home or elsewhere. The server collects the results of this auto-call, namely the user's responses and reactions, and stores them as data. The generating AI analyzes this data and generates a summary that highlights the key points.

[0233] Finally, the server sends the generated summary to a terminal used by the service staff. The terminal displays the summary, and the service staff uses it to provide personalized service to the user. For example, they can offer service suggestions for the whole family or specific suggestions for improving the user's current plan.

[0234] As a concrete example, suppose a user considering communication service plans for their family makes an appointment to visit a store and responds to a questionnaire stating, "Our family consists of four people, each using a different plan. We would like to review our plans." In this case, the server analyzes this information and generates a conversation that includes "a proposal for a unified plan for the whole family" and "questions about the benefits of each plan." Based on the user's response to the generated conversation, the server generates a summary to make a specific proposal for "a bundled family plan contract" and presents it to the device.

[0235] This system eliminates inconsistencies in communication styles due to individual differences, enabling more efficient and effective user support.

[0236] The following describes the processing flow.

[0237] Step 1:

[0238] The server receives reservation information and survey responses from users and stores this data in a database. This information includes the planned date and time of visit, services of interest, family structure, and information on other products currently being used.

[0239] Step 2:

[0240] The server uses stored reservation information and survey data to activate the generation AI. The generation AI analyzes the input information and automatically generates the optimal call script for each user. The generated script includes content customized based on the user's attributes and interests.

[0241] Step 3:

[0242] The server passes the generated call script to the auto-call system, which then automatically places a call to the user. The user receives the auto-call and can listen to the conversation over the phone. This process allows for pre-contact to encourage family visits or pique interest in other products.

[0243] Step 4:

[0244] The server collects the user's response after the auto-call is executed. Using this data, including the user's answers and reactions, the generating AI is reactivated to analyze the response. It then generates a summary that extracts the key points.

[0245] Step 5:

[0246] The server sends the generated summary to a terminal used by the store crew. The terminal displays this summary, which the crew can refer to during customer service, enabling a personalized approach to the user. For example, it can provide detailed suggestions for family-oriented communication plans.

[0247] (Example 1)

[0248] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0249] Current customer service systems often rely on manual processes for information management and customer communication, resulting in inefficiencies. Furthermore, the level of consistency and personalization provided to individual customers is insufficient. Therefore, there is a need to improve customer satisfaction while efficiently delivering personalized service.

[0250] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0251] In this invention, the server includes means for receiving reservation data and survey results and storing them in a storage means, means for processing the stored information and generating automatically generated communication content, and means for performing automated voice communication based on the generated communication content. This enables efficient management and processing of customer information, and improves the consistency and satisfaction of customer service through personalized automated communication.

[0252] "Reservation data" refers to information provided by users regarding the date and time of their visit and the services they wish to receive.

[0253] "Survey results" refer to data obtained through questionnaires and other information gathering activities that users have answered.

[0254] "Memory devices" refer to devices and technologies for long-term storage of digital information.

[0255] "Processing" refers to a series of operations that analyze received data and organize it into information.

[0256] "Automatically generated communication content" refers to the content of messages and conversations that are automatically created for the user.

[0257] "Automated voice communication" refers to a communication method that uses machine-generated voice messages to contact users.

[0258] A "summary" refers to a short, concise version of data or information, extracted from the original source to highlight the most important points.

[0259] "Display devices" refer to devices used by humans to visually confirm information.

[0260] "Response results" refer to data obtained as responses and feedback provided by users.

[0261] "Personalized services" refer to the provision of services tailored to the individual needs and characteristics of each user.

[0262] This invention provides a system for efficiently managing customer reservation data and survey results, and for providing personalized services. This system consists of three main elements: a server, a terminal, and a user.

[0263] The server receives reservation data and survey results and saves them to a relational database (e.g., MySQL, PostgreSQL). The server analyzes the saved information using Python and generates automatically generated communication content using a generative AI model (using TensorFlow, PyTorch, etc.). This communication content is generated via prompt statements and is tailored to the individual needs of the user. For example, a prompt statement in the format of "Create the optimal suggestion talk for user A, a family of four who is reviewing their communication plan" might be used.

[0264] Next, the server uses the automatically generated communication content to perform an automated voice call to the user via an auto-call system (e.g., a common voice communication API). The user receives the message through this voice call and can check its contents at home or elsewhere. The auto-call system collects the user's responses and transfers this data to the server.

[0265] The server analyzes the collected response data again using a regenerative AI model to generate a summary. This summary provides condensed information crucial for future customer service interactions. The generated summary is sent to a terminal used by the service staff, where detailed information is displayed. The service staff uses this summary to propose personalized services to users and improve the quality of service delivery.

[0266] As a concrete example, consider a scenario where a user requests either a unified plan proposal for the entire family or an explanation of the benefits of individual plans. In such cases, the system can effectively provide proposals that meet the user's expectations.

[0267] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0268] Step 1:

[0269] Users input reservation data and survey results using a dedicated application or web form. This includes information such as preferred date and time of visit, desired service type, family composition, and categories of interest. The input results are sent to the server. The arrival of the input data on the server provides the foundational data necessary for subsequent analysis.

[0270] Step 2:

[0271] The server stores reservation data and survey results received from users in a relational database. Specifically, it connects to a database management system (e.g., MySQL or PostgreSQL) and inserts the received data into the appropriate tables. This step ensures that the information is persistently stored and available for later analysis.

[0272] Step 3:

[0273] The server prepares a generative AI model to analyze the stored information. First, it runs a Python script to load input data into the model using the stored data. A prompt statement (e.g., "Create the best suggestion talk for a user who is reviewing their communication plan as a family of four") is input into the generative AI model, and the analysis begins. The model generates individual talk content based on this input information.

[0274] Step 4:

[0275] The server prepares an auto-call system based on the generated talk content. It then configures the system to automatically communicate with the user via voice. Specifically, it converts the generated talk into a voice message using a voice API and dials the user's phone number. This allows the user to receive personalized service information at home or elsewhere.

[0276] Step 5:

[0277] The user receives an automated call and confirms the message. The system captures the user's listening response and reactions (e.g., button presses or voice responses) and feeds them back to the server. This input data becomes core data for subsequent analysis.

[0278] Step 6:

[0279] The server analyzes the user's response results and generates a summary using the generative AI model. Based on this summary, the server extracts important information useful for the next customer service, and prepares to provide it to the customer service staff.

[0280] Step 7:

[0281] The server transmits the generated summary to the terminal used by the customer service staff. The terminal displays the received summary and provides materials for the customer service staff to make personalized proposals. The staff refers to the displayed information and makes optimal proposals to the user. Thereby, individualized service provision is realized. [[ID=##**BLOCKQUOTE**##10]]

[0282] [[ID=##**BLOCKQUOTE**##11]] (Application Example 1)

[0283] Next, Application Example 1 will be described. In the following description, the data processing device 12 is referred to as the "server", and the smart glasses 214 are referred to as the "terminal".

[0284] For effective customer service in the store, it is important to understand the needs of the users in detail before they visit the store and provide appropriate services. However, in the conventional method, there are problems such that the management of reservation information and the content of the pre-call are personal, and variations in effects are likely to occur. Also, in order for the customer service staff to respond according to the needs of the users, it is required to understand a lot of information in a short time and make proposals immediately. Under such conditions, the burden on the store staff becomes large, and it is difficult to maintain customer satisfaction.

[0285] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following respective means.

[0286] In this invention, the server includes means for receiving reservation information and questionnaire responses and storing them in a storage medium, means for analyzing the stored information and generating an optimal opening talk, means for executing a voice call based on the talk generated by the generating means, means for collecting the response results of the user by the voice call, analyzing them, and generating a summary, and means for proposing and presenting the generated summary and personalized customer service plans to the terminal. Thereby, the needs of the user at the time of reservation can be grasped in detail, and personalized customer service can be provided.

[0287] The "reservation information" is information regarding the date and time when the user will visit the store and the service content desired.

[0288] The "questionnaire response" is information regarding the personal interests and needs provided by the user at the time of reservation.

[0289] The "storage medium" is a digital storage system for accumulating and storing information.

[0290] The "generating means" is a system having a function for creating talks and summaries based on the collected and analyzed information.

[0291] The "voice call" is a process of communicating by voice using electronic means.

[0292] The "user" is an individual or group that reserves a service and is the target of the provided customer service and service.

[0293] The "terminal" is an electronic device for the staff to confirm the generated information.

[0294] The "personalized customer service plan" is a plan for customer service created according to the individual needs and interests of the user.

[0295] This invention is a system that enables effective communication and customer service with users. It consists of three main elements: a server, a terminal, and the user.

[0296] The server receives reservation information and survey responses entered by the user. This information is stored in a database such as MySQL or PostgreSQL. Subsequently, the stored information is analyzed using Python and Django, and a generative AI model such as OpenAI's GPT series generates the optimal call script and summary.

[0297] The generated conversation is automatically transmitted to the user using a voice call system. Text-to-speech (TTS) technology is used for the text-to-speech conversion. After the user listens to the conversation, their response is collected on a server, re-analyzed, and a summary is generated.

[0298] The device, specifically a smartphone app developed using React Native, receives summaries and personalized service suggestions sent from the server. The app on the device displays this information in real time, helping store staff make the best service suggestions for customers.

[0299] For example, if a user responds to a survey by saying they "want to review their family's communication plan," the server uses AI to create a conversation that includes optimal suggestions and displays specific customer service proposals on the device, such as "Please suggest a new communication plan that can be used by the whole family."

[0300] An example of a prompt is, "To suggest the optimal communication plan based on family structure, please create a personalized conversation that takes into account the user's current usage and desired changes." This prompt instructs the generative AI model to generate specific conversation content.

[0301] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0302] Step 1:

[0303] The server receives reservation information and questionnaire responses from the user. As input, information regarding date and time, desired services, and needs is provided, and these are saved in the database. The output here is the saved customer information. Specifically, the received data is stored in a MySQL database by means of an SQL query.

[0304] Step 2:

[0305] The server analyzes the saved customer information. In this analysis process, the information saved in the database is read by a Python script to extract the attributes and needs of each user. There is user information obtained from the database as input, and the output is the analyzed data. Specifically, the information is organized in JSON format data.

[0306] Step 3:

[0307] The server generates an optimal call talk using an AI model generated based on the analyzed data. Using a prompt sentence, the generation AI is requested to generate a talk that matches the user's needs. The inputs at this time are the analysis result and the prompt sentence. The output is the generated call talk. For example, data is sent to the OpenAI API and the generated text is received.

[0308] Step 4:

[0309] The server converts the generated talk into voice data using TTS technology and performs an automatic call through the voice call system. The input is the generated text talk, and the output is voice data. Specifically, the Google Text-to-Speech API is utilized to vocalize the talk.

[0310] Step 5:

[0311] The user receives and responds to an incoming voice call. The input is the voice call data, and the output is the user's response information. The user listens to the voice message using a regular telephone handset.

[0312] Step 6:

[0313] The server automatically collects user response information and analyzes it again. This analysis identifies key points. The input is the user's response, and the output is summarized data. Specifically, it converts speech to text and performs analysis using NLP (Neuro-Linguistic Programming) technology.

[0314] Step 7:

[0315] The terminal receives the analyzed summary data from the server, which is then accessed by staff. The input is the summary data sent from the server, and the output is the content displayed on the terminal. Specifically, the data is displayed in the user interface using a React Native application.

[0316] Step 8:

[0317] The staff app on the device presents personalized service suggestions to the user based on summarized information. The user receives service suggestions from the staff. The input is summarized information, and the output is the staff's suggestions. The staff responds based on the information displayed on the device.

[0318] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0319] This invention is a system that enables efficient information management when users make reservations and allows for pre-visit phone calls and customer service that take emotions into consideration. The invention provides a more personalized service by receiving, analyzing, and utilizing user reservation information and questionnaire responses, as well as emotion recognition.

[0320] The server first receives reservation information and questionnaire responses from users and stores this data in a database. This includes information such as the reservation date and time, expected services, family structure, and products of interest. Based on this received data, the server activates a generation AI to create an optimal pre-call script for each user.

[0321] The AI-generated conversation is tailored based on the user's attributes and interests, and an emotion engine is integrated into this process. The emotion engine analyzes all data, including the user's responses, to recognize the user's emotional state. This allows it to analyze what emotions the user expressed previously, what their satisfaction and dissatisfaction levels were, and incorporate this information into the call conversation.

[0322] The server passes this generated talk to the auto-calling system, which then automatically dials the user. The auto-calling system delivers the talk in an audio format that takes emotions into account, enabling a more empathetic and appropriate approach to the user. Users receive this call at home or on the go and listen to the content to deepen their understanding of the service.

[0323] After the automated call, the server collects the user's response again, and the emotion engine analyzes the user's reaction and emotions. Based on the resulting emotion information and response content, the generative AI creates a summary. This summary includes the user's emotion information and suggests appropriate customer service.

[0324] The terminal is a device used by the store crew to display this summary, which the crew uses in customer service. The crew can use this information to provide personalized suggestions and support to the user. For example, if a user expresses anxiety, they can provide detailed service explanations or reassuring explanations, resulting in more attentive and emotionally sensitive customer service.

[0325] As described above, the present invention enables automated pre-calling that takes into account the user's emotional state and personalized customer service, thereby realizing the provision of more efficient and effective customer service.

[0326] The following describes the processing flow.

[0327] Step 1:

[0328] The server receives customer reservation information and survey responses and stores them in a database. The reservation information includes the date and time and the services expected, while the survey includes family composition and information on products currently being used.

[0329] Step 2:

[0330] The server activates a generation AI using saved reservation information and survey responses. The generation AI analyzes this information and generates a customized pre-call message for each user. The message content is adjusted according to the user's attributes and past survey results.

[0331] Step 3:

[0332] The server utilizes an emotion engine to recognize the user's emotional state by analyzing past user responses and reservation information. This allows for further personalized conversations.

[0333] Step 4:

[0334] The server sends generated, emotion-driven messages to the auto-call system and automatically places a call to the user. The user can then listen to the customized message content through the received auto-call.

[0335] Step 5:

[0336] After an automated call, the server collects the user's response and uses an emotion engine to analyze the user's emotions and reactions in detail. Based on this analysis, a generative AI creates a summary.

[0337] Step 6:

[0338] The server sends the generated summary to the terminal. The terminal displays this summary, helping the store crew provide customer service based on it. The crew can then provide more personalized service based on the user's identified emotions and interests.

[0339] (Example 2)

[0340] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0341] In today's service industry, providing efficient and personalized customer service is a challenge, especially given the need for appropriate responses to users. In particular, the lack of prior contact and interaction that takes user emotions into account necessitates efficient information management while optimizing user satisfaction.

[0342] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0343] In this invention, the server includes means for receiving reservation information and questionnaire answers and storing them in a storage device, means for analyzing the stored information and generating optimal automated call content, and means for executing a machine-controlled call based on the call content generated by the generation means. This enables the generation of automated call content that takes into account the user's emotions and efficient information management and analysis.

[0344] "Reservation information" refers to detailed information such as the date, time, location, and content of the service that the user plans to provide.

[0345] "Questionnaire responses" refer to information collected from users through surveys and feedback, and are used to understand the characteristics and needs of users.

[0346] A "memory device" is a data storage system that stores received data and makes it available for use as needed.

[0347] "Generation means" refers to a process or algorithm for producing appropriate output based on stored information, and in this invention, it is used to generate automated call content.

[0348] "Automated call content" refers to the content of a call using pre-prepared voice messages or dialogue scripts, intended for prior notification or confirmation to the user.

[0349] "Machine-controlled communication" refers to automated voice calls conducted by a communication system, and is a means of telephone communication that is mechanically controlled.

[0350] The "generated summary" is a summary of the analyzed data, concisely describing the user's emotional state and necessary countermeasures.

[0351] "Emotional information" refers to information that indicates the emotional state and characteristics analyzed based on data provided by the user.

[0352] As the foundation for implementing this invention, the server receives reservation information and questionnaire responses from users. This data is stored in a database such as MySQL and serves as an important source of information for forming individual user profiles. Based on this information, the server uses a generative AI model to generate optimal automated call content. A general-purpose AI model with excellent natural language processing capabilities is used as the generative AI model.

[0353] The generated automated call content is adjusted via an emotion analysis engine to take into account the user's past responses and emotional information. This engine uses stored historical data to infer the user's emotional state and reflects it in the call content.

[0354] Next, the server converts the coordinated call content into speech using speech synthesis software and makes a phone call to the user through a machine-controlled call system. Speech synthesis technologies such as Amazon Polly are useful in this process. Machine-controlled calls can be received by users anywhere and provide them with a means to better understand the service.

[0355] Through the voice call, the user receives information about the service and how to make their next appointment, and responds as needed. The server collects these responses again and analyzes them in detail using an emotion engine. Based on the analysis, the generating AI creates a summary that again reflects the user's emotions and displays it on the device.

[0356] The terminal will display this overview as a device used by store crew members, who will then use it in customer service. Crew members can use this information to, for example, provide detailed explanations to customers who are feeling anxious about a new service, thereby reassuring them.

[0357] A concrete example of a prompt is, "The user is interested in the outdoors. Please create a call script to confirm a reservation for their next visit." Through this prompt, the generating AI model provides a personalized response and delivers information tailored to the user's needs.

[0358] In this way, the system of the present invention enables highly personalized services that take into account the user's emotions, and achieves efficient and highly accurate customer service.

[0359] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0360] Step 1:

[0361] The server receives reservation information and questionnaire responses from users. Input data includes reservation date and time, desired service, and hobbies / interests. This data is stored in a MySQL database and managed on a per-user basis. The stored data will be used for future analysis and as input for generating AI.

[0362] Step 2:

[0363] The server generates a prompt based on the saved reservation information and questionnaire responses. This prompt is used as input to a generative AI model. This generative model generates automated call content written in natural language based on the given prompt. For example, it might output something like, "The user is interested in outdoor activities. Please create a call script to confirm their next visit."

[0364] Step 3:

[0365] The generated call content is further analyzed by the emotion engine. Past response history and data on the user's emotional state are used as input. The emotion engine identifies the user's emotions and adjusts the call content to provide optimal information and tone, thereby creating an emotionally sensitive call output.

[0366] Step 4:

[0367] The server uses the adjusted call content as input data, which is then converted into speech using speech synthesis software. Specifically, it outputs the call content in text format as an audio file and sends it to the machine-controlled call system. This audio is then automatically dialed to the user, and the content is transmitted.

[0368] Step 5:

[0369] The user receives an automated voice call and confirms its content. The user responds as needed, following the voice guidance. The response data is returned to the server, saved again to the database, and analyzed.

[0370] Step 6:

[0371] The server analyzes user response data using an emotion engine. The user's most recent response information is used as input to identify user satisfaction and emotional state. A summary is generated based on this information, and this summary serves as the basis for preparing for the next customer interaction.

[0372] Step 7:

[0373] The terminal displays an analyzed summary on the device used by the store crew. The crew refers to this summary and provides customer service tailored to each customer's needs. Specifically, the crew can alleviate customer anxiety by providing detailed service explanations and emotional encouragement.

[0374] (Application Example 2)

[0375] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0376] Currently, in brick-and-mortar stores, there is a demand for personalized service based on the individual interests and emotional states of customers. However, implementing this requires considerable effort and time. In particular, there is a lack of efficient systems for providing customer service that meets the diverse needs of different customers, which increases the burden on store staff and hinders improvements in customer satisfaction.

[0377] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0378] In this invention, the server includes means for receiving reservation information and questionnaire responses and storing them in a storage means, means for analyzing the stored information and generating optimal communication content, and means for executing automated voice communication based on the communication generated by the generation means. This makes it possible to grasp the status information of each customer and efficiently provide personalized customer service.

[0379] "Reservation information" refers to information that includes the date and time the user wishes to receive the service, as well as the details of the service requested.

[0380] A "survey response" is the answer to a question asked to obtain details about a user's interests and expectations.

[0381] A "memory device" is a data storage system for holding received information.

[0382] "Analysis" is the process of analyzing the content and characteristics of acquired information.

[0383] "Communication content" refers to messages and audio information sent to the user.

[0384] "Generation means" refers to the process of creating new communication content based on the analyzed information.

[0385] "Automated voice communication" is a system that automatically transmits pre-generated voice information to the user.

[0386] "State information" refers to data about a user's internal situation, such as their emotions and interests.

[0387] An "information terminal" is an electronic device that displays prepared information and allows users to view it.

[0388] "Customer service information" refers to customer service methods and recommendations provided according to the user's characteristics and emotions.

[0389] The system for implementing this invention is primarily built around a server, a terminal, and user interaction. The server receives reservation information and survey responses from the user and stores them in a storage device. This allows for recording the service content the user expects and their past usage history, which can then be used to improve future services. For the storage device, it is generally recommended to use Firebase, a cloud-based data storage service.

[0390] Next, the server performs analysis using the stored information. This analysis includes sentiment analysis using machine learning frameworks such as TensorFlow. The user's emotional tendency data obtained through sentiment analysis is information that can lead to improved user satisfaction, and the generation method creates the optimal communication content based on this.

[0391] The generated communication content is delivered to the user via automated voice communication. This process utilizes the Google Cloud Text-to-Speech API to generate text-to-speech and automatically deliver it to the user. Through this voice communication, the user can check the details of the service they plan to use in advance.

[0392] The terminal will be used in stores as an information terminal. It will display customer service information that users should refer to when they visit the store. As a result, staff will be able to easily provide personalized service, improving the quality of customer service.

[0393] For example, in the case of a casual apparel store, if a user requests to "book a fitting appointment for a weekend afternoon," the AI ​​model can prepare in advance fitting outfits and related product information that are perfectly suited to that user.

[0394] An example of a prompt for the generating AI model could be, "Based on this week's booking data, please create casual fashion conversation content tailored to the user's preferences." This enables the provision of highly flexible and personalized services.

[0395] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0396] Step 1:

[0397] The server receives reservation information and survey responses from users. Inputs include the date and time, desired service details, and areas of interest, all entered by the user via a smartphone app. This input data is saved to cloud storage. The output is the accurately saved user reservation data.

[0398] Step 2:

[0399] The server analyzes reservation information and survey responses stored in memory to understand the user's emotional tendencies. This process uses TensorFlow, a machine learning framework. The input is the user data stored in step 1, and the output is the identification result of the user's emotional tendencies.

[0400] Step 3:

[0401] The server uses a generative AI model to generate optimal communication content based on the analyzed sentiment tendencies and other user characteristics. The input is the output data from step 2 and user attribute data, and the output is the text of the generated communication content.

[0402] Step 4:

[0403] The server converts the generated communication content into speech data using the Google Cloud Text-to-Speech API. The input is the text data generated in step 3, and the output is the speech data sent to the user.

[0404] Step 5:

[0405] The user receives an audio communication and uses it to preview the content of the service being offered. The input is the audio data from step 4, and the output is to improve the user's understanding and pique their interest.

[0406] Step 6:

[0407] The terminal displays customer service information necessary when a user visits the store. Input is user-specific customer service information sent from the server, and output is a specific customer service guide that can be used by store staff.

[0408] Step 7:

[0409] The staff operating the terminal will provide personalized service to the user based on the displayed service information. The input is the service information from step 6, and the output is a high-quality, personalized customer service experience.

[0410] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0411] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0412] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0413] [Third Embodiment]

[0414] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0415] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0416] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0417] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0418] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0419] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0420] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0421] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0422] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0423] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0424] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0425] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0426] This invention aims to improve the efficiency of managing customer information and making pre-appointment calls when users make reservations to visit a store. The system consists of three main elements: a server, a user, and a terminal. Specific embodiments are shown below.

[0427] The server first receives the customer's reservation information and survey responses and stores them appropriately in a database. This information includes the reservation date and time, the type of service desired, family composition, and other services of interest. The server analyzes this stored information and uses AI to generate an optimal pre-call message for each user. The generated message is personalized based on the user's attributes and responses, and is tailored to the user's interests and needs.

[0428] Next, the server automatically calls the user via the auto-call system and delivers the generated conversation as an audio message. The user receives the sent auto-call message and can listen to its contents at home or elsewhere. The server collects the results of this auto-call, namely the user's responses and reactions, and stores them as data. The generating AI analyzes this data and generates a summary that highlights the key points.

[0429] Finally, the server sends the generated summary to a terminal used by the service staff. The terminal displays the summary, and the service staff uses it to provide personalized service to the user. For example, they can offer service suggestions for the whole family or specific suggestions for improving the user's current plan.

[0430] As a concrete example, suppose a user considering communication service plans for their family makes an appointment to visit a store and responds to a questionnaire stating, "Our family consists of four people, each using a different plan. We would like to review our plans." In this case, the server analyzes this information and generates a conversation that includes "a proposal for a unified plan for the whole family" and "questions about the benefits of each plan." Based on the user's response to the generated conversation, the server generates a summary to make a specific proposal for "a bundled family plan contract" and presents it to the device.

[0431] This system eliminates inconsistencies in communication styles due to individual differences, enabling more efficient and effective user support.

[0432] The following describes the processing flow.

[0433] Step 1:

[0434] The server receives reservation information and survey responses from users and stores this data in a database. This information includes the planned date and time of visit, services of interest, family structure, and information on other products currently being used.

[0435] Step 2:

[0436] The server uses stored reservation information and survey data to activate the generation AI. The generation AI analyzes the input information and automatically generates the optimal call script for each user. The generated script includes content customized based on the user's attributes and interests.

[0437] Step 3:

[0438] The server passes the generated call script to the auto-call system, which then automatically places a call to the user. The user receives the auto-call and can listen to the conversation over the phone. This process allows for pre-contact to encourage family visits or pique interest in other products.

[0439] Step 4:

[0440] The server collects the user's response after the auto-call is executed. Using this data, including the user's answers and reactions, the generating AI is reactivated to analyze the response. It then generates a summary that extracts the key points.

[0441] Step 5:

[0442] The server sends the generated summary to a terminal used by the store crew. The terminal displays this summary, which the crew can refer to during customer service, enabling a personalized approach to the user. For example, it can provide detailed suggestions for family-oriented communication plans.

[0443] (Example 1)

[0444] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0445] Current customer service systems often rely on manual processes for information management and customer communication, resulting in inefficiencies. Furthermore, the level of consistency and personalization provided to individual customers is insufficient. Therefore, there is a need to improve customer satisfaction while efficiently delivering personalized service.

[0446] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0447] In this invention, the server includes means for receiving reservation data and survey results and storing them in a storage means, means for processing the stored information and generating automatically generated communication content, and means for performing automated voice communication based on the generated communication content. This enables efficient management and processing of customer information, and improves the consistency and satisfaction of customer service through personalized automated communication.

[0448] "Reservation data" refers to information provided by users regarding the date and time of their visit and the services they wish to receive.

[0449] "Survey results" refer to data obtained through questionnaires and other information gathering activities that users have answered.

[0450] "Memory devices" refer to devices and technologies for long-term storage of digital information.

[0451] "Processing" refers to a series of operations that analyze received data and organize it into information.

[0452] "Automatically generated communication content" refers to the content of messages and conversations that are automatically created for the user.

[0453] "Automated voice communication" refers to a communication method that uses machine-generated voice messages to contact users.

[0454] A "summary" refers to a short, concise version of data or information, extracted from the original source to highlight the most important points.

[0455] "Display devices" refer to devices used by humans to visually confirm information.

[0456] "Response results" refer to data obtained as responses and feedback provided by users.

[0457] "Personalized services" refer to the provision of services tailored to the individual needs and characteristics of each user.

[0458] This invention provides a system for efficiently managing customer reservation data and survey results, and for providing personalized services. This system consists of three main elements: a server, a terminal, and a user.

[0459] The server receives reservation data and survey results and saves them to a relational database (e.g., MySQL, PostgreSQL). The server analyzes the saved information using Python and generates automatically generated communication content using a generative AI model (using TensorFlow, PyTorch, etc.). This communication content is generated via prompt statements and is tailored to the individual needs of the user. For example, a prompt statement in the format of "Create the optimal suggestion talk for user A, a family of four who is reviewing their communication plan" might be used.

[0460] Next, the server uses the automatically generated communication content to perform an automated voice call to the user via an auto-call system (e.g., a common voice communication API). The user receives the message through this voice call and can check its contents at home or elsewhere. The auto-call system collects the user's responses and transfers this data to the server.

[0461] The server analyzes the collected response data again using a regenerative AI model to generate a summary. This summary provides condensed information crucial for future customer service interactions. The generated summary is sent to a terminal used by the service staff, where detailed information is displayed. The service staff uses this summary to propose personalized services to users and improve the quality of service delivery.

[0462] As a concrete example, consider a scenario where a user requests either a unified plan proposal for the entire family or an explanation of the benefits of individual plans. In such cases, the system can effectively provide proposals that meet the user's expectations.

[0463] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0464] Step 1:

[0465] Users input reservation data and survey results using a dedicated application or web form. This includes information such as preferred date and time of visit, desired service type, family composition, and categories of interest. The input results are sent to the server. The arrival of the input data on the server provides the foundational data necessary for subsequent analysis.

[0466] Step 2:

[0467] The server stores reservation data and survey results received from users in a relational database. Specifically, it connects to a database management system (e.g., MySQL or PostgreSQL) and inserts the received data into the appropriate tables. This step ensures that the information is persistently stored and available for later analysis.

[0468] Step 3:

[0469] The server prepares a generative AI model to analyze the stored information. First, it runs a Python script to load input data into the model using the stored data. A prompt statement (e.g., "Create the best suggestion talk for a user who is reviewing their communication plan as a family of four") is input into the generative AI model, and the analysis begins. The model generates individual talk content based on this input information.

[0470] Step 4:

[0471] The server prepares an auto-call system based on the generated talk content. It then configures the system to automatically communicate with the user via voice. Specifically, it converts the generated talk into a voice message using a voice API and dials the user's phone number. This allows the user to receive personalized service information at home or elsewhere.

[0472] Step 5:

[0473] The user receives an automated call and confirms the message. The system captures the user's listening response and reactions (e.g., button presses or voice responses) and feeds them back to the server. This input data becomes core data for subsequent analysis.

[0474] Step 6:

[0475] The server analyzes the user's response and generates a summary using a generative AI model. Based on this summary, the server extracts important information useful for future customer service and prepares to provide it to the customer service staff.

[0476] Step 7:

[0477] The server sends the generated summary to a terminal used by the service staff. The terminal displays the received summary, providing the service staff with material to make personalized suggestions. The staff refer to the displayed information and make the best suggestions for the user. This enables the delivery of personalized service.

[0478] (Application Example 1)

[0479] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0480] For effective customer service in stores, it is crucial to thoroughly understand customer needs before they visit and provide appropriate service. However, traditional methods have been problematic because reservation information management and the content of pre-visit phone calls are highly dependent on individual staff members, leading to inconsistent results. Furthermore, for service staff to respond to customer needs, they are required to understand a large amount of information quickly and make immediate suggestions. Under these conditions, the burden on store staff becomes significant, making it difficult to maintain customer satisfaction.

[0481] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0482] In this invention, the server includes means for receiving reservation information and questionnaire responses and storing them in a storage medium; means for analyzing the stored information and generating an optimal call script; means for executing a voice call based on the script generated by the generation means; means for collecting the user's response results from the voice call, analyzing them, and generating a summary; and means for proposing and presenting the generated summary and personalized customer service suggestions to the terminal. This enables a detailed understanding of the user's needs when making a reservation and allows for personalized customer service.

[0483] "Reservation information" refers to information about the date and time of the customer's visit and the services they wish to receive.

[0484] "Survey responses" refer to information about the user's personal interests and needs provided when making a reservation.

[0485] A "storage medium" is a digital storage system used to store and preserve information.

[0486] A "generation method" is a system that has the function of creating talks and summaries based on collected and analyzed information.

[0487] "Voice communication" refers to the process of communicating using voice via electronic means.

[0488] A "user" is an individual or group that makes a reservation for a service and is the recipient of the customer service or services provided.

[0489] A "terminal" is an electronic device used by staff to verify the generated information.

[0490] A "personalized customer service plan" is a proposed customer service response tailored to the individual needs and interests of the user.

[0491] This invention is a system that enables effective communication and customer service with users. It consists of three main elements: a server, a terminal, and the user.

[0492] The server receives reservation information and survey responses entered by the user. This information is stored in a database such as MySQL or PostgreSQL. Subsequently, the stored information is analyzed using Python and Django, and a generative AI model such as OpenAI's GPT series generates the optimal call script and summary.

[0493] The generated conversation is automatically transmitted to the user using a voice call system. Text-to-speech (TTS) technology is used for the text-to-speech conversion. After the user listens to the conversation, their response is collected on a server, re-analyzed, and a summary is generated.

[0494] The device, specifically a smartphone app developed using React Native, receives summaries and personalized service suggestions sent from the server. The app on the device displays this information in real time, helping store staff make the best service suggestions for customers.

[0495] For example, if a user responds to a survey by saying they "want to review their family's communication plan," the server uses AI to create a conversation that includes optimal suggestions and displays specific customer service proposals on the device, such as "Please suggest a new communication plan that can be used by the whole family."

[0496] An example of a prompt is, "To suggest the optimal communication plan based on family structure, please create a personalized conversation that takes into account the user's current usage and desired changes." This prompt instructs the generative AI model to generate specific conversation content.

[0497] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0498] Step 1:

[0499] The server receives reservation information and survey responses from users. The input includes information such as the date and time, desired services, and needs, which are then stored in the database. The output here is the stored customer information. Specifically, the received data is stored in a MySQL database via SQL queries.

[0500] Step 2:

[0501] The server analyzes the stored customer information. This analysis process uses a Python script to read information from the database and extract user attributes and needs. The input is user information retrieved from the database, and the output is the analyzed data. Specifically, the information is organized in JSON format.

[0502] Step 3:

[0503] The server generates the optimal call script using a generative AI model based on the analyzed data. A prompt is used to request the generative AI to create a script that matches the user's needs. The input at this stage is the analysis result and the prompt. The output is the generated call script. For example, data can be sent to the OpenAI API and the generated text can be received.

[0504] Step 4:

[0505] The server converts the generated talk into audio data using TTS technology and automatically makes a call using the voice call system. The input is the generated text talk, and the output is audio data. Specifically, the Google Text-to-Speech API is used to convert the talk into speech.

[0506] Step 5:

[0507] The user receives and responds to an incoming voice call. The input is the voice call data, and the output is the user's response information. The user listens to the voice message using a regular telephone handset.

[0508] Step 6:

[0509] The server automatically collects user response information and analyzes it again. This analysis identifies key points. The input is the user's response, and the output is summarized data. Specifically, it converts speech to text and performs analysis using NLP (Neuro-Linguistic Programming) technology.

[0510] Step 7:

[0511] The terminal receives the analyzed summary data from the server, which is then accessed by staff. The input is the summary data sent from the server, and the output is the content displayed on the terminal. Specifically, the data is displayed in the user interface using a React Native application.

[0512] Step 8:

[0513] The staff app on the device presents personalized service suggestions to the user based on summarized information. The user receives service suggestions from the staff. The input is summarized information, and the output is the staff's suggestions. The staff responds based on the information displayed on the device.

[0514] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0515] This invention is a system that enables efficient information management when users make reservations and allows for pre-visit phone calls and customer service that take emotions into consideration. The invention provides a more personalized service by receiving, analyzing, and utilizing user reservation information and questionnaire responses, as well as emotion recognition.

[0516] The server first receives reservation information and questionnaire responses from users and stores this data in a database. This includes information such as the reservation date and time, expected services, family structure, and products of interest. Based on this received data, the server activates a generation AI to create an optimal pre-call script for each user.

[0517] The AI-generated conversation is tailored based on the user's attributes and interests, and an emotion engine is integrated into this process. The emotion engine analyzes all data, including the user's responses, to recognize the user's emotional state. This allows it to analyze what emotions the user expressed previously, what their satisfaction and dissatisfaction levels were, and incorporate this information into the call conversation.

[0518] The server passes this generated talk to the auto-calling system, which then automatically dials the user. The auto-calling system delivers the talk in an audio format that takes emotions into account, enabling a more empathetic and appropriate approach to the user. Users receive this call at home or on the go and listen to the content to deepen their understanding of the service.

[0519] After the automated call, the server collects the user's response again, and the emotion engine analyzes the user's reaction and emotions. Based on the resulting emotion information and response content, the generative AI creates a summary. This summary includes the user's emotion information and suggests appropriate customer service.

[0520] The terminal is a device used by the store crew to display this summary, which the crew uses in customer service. The crew can use this information to provide personalized suggestions and support to the user. For example, if a user expresses anxiety, they can provide detailed service explanations or reassuring explanations, resulting in more attentive and emotionally sensitive customer service.

[0521] As described above, the present invention enables automated pre-calling that takes into account the user's emotional state and personalized customer service, thereby realizing the provision of more efficient and effective customer service.

[0522] The following describes the processing flow.

[0523] Step 1:

[0524] The server receives customer reservation information and survey responses and stores them in a database. The reservation information includes the date and time and the services expected, while the survey includes family composition and information on products currently being used.

[0525] Step 2:

[0526] The server activates a generation AI using saved reservation information and survey responses. The generation AI analyzes this information and generates a customized pre-call message for each user. The message content is adjusted according to the user's attributes and past survey results.

[0527] Step 3:

[0528] The server utilizes an emotion engine to recognize the user's emotional state by analyzing past user responses and reservation information. This allows for further personalized conversations.

[0529] Step 4:

[0530] The server sends generated, emotion-driven messages to the auto-call system and automatically places a call to the user. The user can then listen to the customized message content through the received auto-call.

[0531] Step 5:

[0532] After an automated call, the server collects the user's response and uses an emotion engine to analyze the user's emotions and reactions in detail. Based on this analysis, a generative AI creates a summary.

[0533] Step 6:

[0534] The server sends the generated summary to the terminal. The terminal displays this summary, helping the store crew provide customer service based on it. The crew can then provide more personalized service based on the user's identified emotions and interests.

[0535] (Example 2)

[0536] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0537] In today's service industry, providing efficient and personalized customer service is a challenge, especially given the need for appropriate responses to users. In particular, the lack of prior contact and interaction that takes user emotions into account necessitates efficient information management while optimizing user satisfaction.

[0538] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0539] In this invention, the server includes means for receiving reservation information and questionnaire answers and storing them in a storage device, means for analyzing the stored information and generating optimal automated call content, and means for executing a machine-controlled call based on the call content generated by the generation means. This enables the generation of automated call content that takes into account the user's emotions and efficient information management and analysis.

[0540] "Reservation information" refers to detailed information such as the date, time, location, and content of the service that the user plans to provide.

[0541] "Questionnaire responses" refer to information collected from users through surveys and feedback, and are used to understand the characteristics and needs of users.

[0542] A "memory device" is a data storage system that stores received data and makes it available for use as needed.

[0543] "Generation means" refers to a process or algorithm for producing appropriate output based on stored information, and in this invention, it is used to generate automated call content.

[0544] "Automated call content" refers to the content of a call using pre-prepared voice messages or dialogue scripts, intended for prior notification or confirmation to the user.

[0545] "Machine-controlled communication" refers to automated voice calls conducted by a communication system, and is a means of telephone communication that is mechanically controlled.

[0546] The "generated summary" is a summary of the analyzed data, concisely describing the user's emotional state and necessary countermeasures.

[0547] "Emotional information" refers to information that indicates the emotional state and characteristics analyzed based on data provided by the user.

[0548] As the foundation for implementing this invention, the server receives reservation information and questionnaire responses from users. This data is stored in a database such as MySQL and serves as an important source of information for forming individual user profiles. Based on this information, the server uses a generative AI model to generate optimal automated call content. A general-purpose AI model with excellent natural language processing capabilities is used as the generative AI model.

[0549] The generated automated call content is adjusted via an emotion analysis engine to take into account the user's past responses and emotional information. This engine uses stored historical data to infer the user's emotional state and reflects it in the call content.

[0550] Next, the server converts the coordinated call content into speech using speech synthesis software and makes a phone call to the user through a machine-controlled call system. Speech synthesis technologies such as Amazon Polly are useful in this process. Machine-controlled calls can be received by users anywhere and provide them with a means to better understand the service.

[0551] Through the voice call, the user receives information about the service and how to make their next appointment, and responds as needed. The server collects these responses again and analyzes them in detail using an emotion engine. Based on the analysis, the generating AI creates a summary that again reflects the user's emotions and displays it on the device.

[0552] The terminal will display this overview as a device used by store crew members, who will then use it in customer service. Crew members can use this information to, for example, provide detailed explanations to customers who are feeling anxious about a new service, thereby reassuring them.

[0553] A concrete example of a prompt is, "The user is interested in the outdoors. Please create a call script to confirm a reservation for their next visit." Through this prompt, the generating AI model provides a personalized response and delivers information tailored to the user's needs.

[0554] In this way, the system of the present invention enables highly personalized services that take into account the user's emotions, and achieves efficient and highly accurate customer service.

[0555] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0556] Step 1:

[0557] The server receives reservation information and questionnaire responses from users. Input data includes reservation date and time, desired service, and hobbies / interests. This data is stored in a MySQL database and managed on a per-user basis. The stored data will be used for future analysis and as input for generating AI.

[0558] Step 2:

[0559] The server generates a prompt based on the saved reservation information and questionnaire responses. This prompt is used as input to a generative AI model. This generative model generates automated call content written in natural language based on the given prompt. For example, it might output something like, "The user is interested in outdoor activities. Please create a call script to confirm their next visit."

[0560] Step 3:

[0561] The generated call content is further analyzed by the emotion engine. Past response history and data on the user's emotional state are used as input. The emotion engine identifies the user's emotions and adjusts the call content to provide optimal information and tone, thereby creating an emotionally sensitive call output.

[0562] Step 4:

[0563] The server uses the adjusted call content as input data, which is then converted into speech using speech synthesis software. Specifically, it outputs the call content in text format as an audio file and sends it to the machine-controlled call system. This audio is then automatically dialed to the user, and the content is transmitted.

[0564] Step 5:

[0565] The user receives an automated voice call and confirms its content. The user responds as needed, following the voice guidance. The response data is returned to the server, saved again to the database, and analyzed.

[0566] Step 6:

[0567] The server analyzes user response data using an emotion engine. The user's most recent response information is used as input to identify user satisfaction and emotional state. A summary is generated based on this information, and this summary serves as the basis for preparing for the next customer interaction.

[0568] Step 7:

[0569] The terminal displays an analyzed summary on the device used by the store crew. The crew refers to this summary and provides customer service tailored to each customer's needs. Specifically, the crew can alleviate customer anxiety by providing detailed service explanations and emotional encouragement.

[0570] (Application Example 2)

[0571] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0572] Currently, in brick-and-mortar stores, there is a demand for personalized service based on the individual interests and emotional states of customers. However, implementing this requires considerable effort and time. In particular, there is a lack of efficient systems for providing customer service that meets the diverse needs of different customers, which increases the burden on store staff and hinders improvements in customer satisfaction.

[0573] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0574] In this invention, the server includes means for receiving reservation information and questionnaire responses and storing them in a storage means, means for analyzing the stored information and generating optimal communication content, and means for executing automated voice communication based on the communication generated by the generation means. This makes it possible to grasp the status information of each customer and efficiently provide personalized customer service.

[0575] "Reservation information" refers to information that includes the date and time the user wishes to receive the service, as well as the details of the service requested.

[0576] A "survey response" is the answer to a question asked to obtain details about a user's interests and expectations.

[0577] A "memory device" is a data storage system for holding received information.

[0578] "Analysis" is the process of analyzing the content and characteristics of acquired information.

[0579] "Communication content" refers to messages and audio information sent to the user.

[0580] "Generation means" refers to the process of creating new communication content based on the analyzed information.

[0581] "Automated voice communication" is a system that automatically transmits pre-generated voice information to the user.

[0582] "State information" refers to data about a user's internal situation, such as their emotions and interests.

[0583] An "information terminal" is an electronic device that displays prepared information and allows users to view it.

[0584] "Customer service information" refers to customer service methods and recommendations provided according to the user's characteristics and emotions.

[0585] The system for implementing this invention is primarily built around a server, a terminal, and user interaction. The server receives reservation information and survey responses from the user and stores them in a storage device. This allows for recording the service content the user expects and their past usage history, which can then be used to improve future services. For the storage device, it is generally recommended to use Firebase, a cloud-based data storage service.

[0586] Next, the server performs analysis using the stored information. This analysis includes sentiment analysis using machine learning frameworks such as TensorFlow. The user's emotional tendency data obtained through sentiment analysis is information that can lead to improved user satisfaction, and the generation method creates the optimal communication content based on this.

[0587] The generated communication content is delivered to the user via automated voice communication. This process utilizes the Google Cloud Text-to-Speech API to generate text-to-speech and automatically deliver it to the user. Through this voice communication, the user can check the details of the service they plan to use in advance.

[0588] The terminal will be used in stores as an information terminal. It will display customer service information that users should refer to when they visit the store. As a result, staff will be able to easily provide personalized service, improving the quality of customer service.

[0589] For example, in the case of a casual apparel store, if a user requests to "book a fitting appointment for a weekend afternoon," the AI ​​model can prepare in advance fitting outfits and related product information that are perfectly suited to that user.

[0590] An example of a prompt for the generating AI model could be, "Based on this week's booking data, please create casual fashion conversation content tailored to the user's preferences." This enables the provision of highly flexible and personalized services.

[0591] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0592] Step 1:

[0593] The server receives reservation information and survey responses from users. Inputs include the date and time, desired service details, and areas of interest, all entered by the user via a smartphone app. This input data is saved to cloud storage. The output is the accurately saved user reservation data.

[0594] Step 2:

[0595] The server analyzes reservation information and survey responses stored in memory to understand the user's emotional tendencies. This process uses TensorFlow, a machine learning framework. The input is the user data stored in step 1, and the output is the identification result of the user's emotional tendencies.

[0596] Step 3:

[0597] The server uses a generative AI model to generate optimal communication content based on the analyzed sentiment tendencies and other user characteristics. The input is the output data from step 2 and user attribute data, and the output is the text of the generated communication content.

[0598] Step 4:

[0599] The server converts the generated communication content into speech data using the Google Cloud Text-to-Speech API. The input is the text data generated in step 3, and the output is the speech data sent to the user.

[0600] Step 5:

[0601] The user receives an audio communication and uses it to preview the content of the service being offered. The input is the audio data from step 4, and the output is to improve the user's understanding and pique their interest.

[0602] Step 6:

[0603] The terminal displays customer service information necessary when a user visits the store. Input is user-specific customer service information sent from the server, and output is a specific customer service guide that can be used by store staff.

[0604] Step 7:

[0605] The staff operating the terminal will provide personalized service to the user based on the displayed service information. The input is the service information from step 6, and the output is a high-quality, personalized customer service experience.

[0606] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0607] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0608] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0609] [Fourth Embodiment]

[0610] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0611] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0612] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0613] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0614] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0615] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0616] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0617] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0618] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0619] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0620] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0621] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0622] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0623] This invention aims to improve the efficiency of managing customer information and making pre-appointment calls when users make reservations to visit a store. The system consists of three main elements: a server, a user, and a terminal. Specific embodiments are shown below.

[0624] The server first receives the customer's reservation information and survey responses and stores them appropriately in a database. This information includes the reservation date and time, the type of service desired, family composition, and other services of interest. The server analyzes this stored information and uses AI to generate an optimal pre-call message for each user. The generated message is personalized based on the user's attributes and responses, and is tailored to the user's interests and needs.

[0625] Next, the server automatically calls the user via the auto-call system and delivers the generated conversation as an audio message. The user receives the sent auto-call message and can listen to its contents at home or elsewhere. The server collects the results of this auto-call, namely the user's responses and reactions, and stores them as data. The generating AI analyzes this data and generates a summary that highlights the key points.

[0626] Finally, the server sends the generated summary to a terminal used by the service staff. The terminal displays the summary, and the service staff uses it to provide personalized service to the user. For example, they can offer service suggestions for the whole family or specific suggestions for improving the user's current plan.

[0627] As a concrete example, suppose a user considering communication service plans for their family makes an appointment to visit a store and responds to a questionnaire stating, "Our family consists of four people, each using a different plan. We would like to review our plans." In this case, the server analyzes this information and generates a conversation that includes "a proposal for a unified plan for the whole family" and "questions about the benefits of each plan." Based on the user's response to the generated conversation, the server generates a summary to make a specific proposal for "a bundled family plan contract" and presents it to the device.

[0628] This system eliminates inconsistencies in communication styles due to individual differences, enabling more efficient and effective user support.

[0629] The following describes the processing flow.

[0630] Step 1:

[0631] The server receives reservation information and survey responses from users and stores this data in a database. This information includes the planned date and time of visit, services of interest, family structure, and information on other products currently being used.

[0632] Step 2:

[0633] The server uses stored reservation information and survey data to activate the generation AI. The generation AI analyzes the input information and automatically generates the optimal call script for each user. The generated script includes content customized based on the user's attributes and interests.

[0634] Step 3:

[0635] The server passes the generated call script to the auto-call system, which then automatically places a call to the user. The user receives the auto-call and can listen to the conversation over the phone. This process allows for pre-contact to encourage family visits or pique interest in other products.

[0636] Step 4:

[0637] The server collects the user's response after the auto-call is executed. Using this data, including the user's answers and reactions, the generating AI is reactivated to analyze the response. It then generates a summary that extracts the key points.

[0638] Step 5:

[0639] The server sends the generated summary to a terminal used by the store crew. The terminal displays this summary, which the crew can refer to during customer service, enabling a personalized approach to the user. For example, it can provide detailed suggestions for family-oriented communication plans.

[0640] (Example 1)

[0641] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0642] Current customer service systems often rely on manual processes for information management and customer communication, resulting in inefficiencies. Furthermore, the level of consistency and personalization provided to individual customers is insufficient. Therefore, there is a need to improve customer satisfaction while efficiently delivering personalized service.

[0643] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0644] In this invention, the server includes means for receiving reservation data and survey results and storing them in a storage means, means for processing the stored information and generating automatically generated communication content, and means for performing automated voice communication based on the generated communication content. This enables efficient management and processing of customer information, and improves the consistency and satisfaction of customer service through personalized automated communication.

[0645] "Reservation data" refers to information provided by users regarding the date and time of their visit and the services they wish to receive.

[0646] "Survey results" refer to data obtained through questionnaires and other information gathering activities that users have answered.

[0647] "Memory devices" refer to devices and technologies for long-term storage of digital information.

[0648] "Processing" refers to a series of operations that analyze received data and organize it into information.

[0649] "Automatically generated communication content" refers to the content of messages and conversations that are automatically created for the user.

[0650] "Automated voice communication" refers to a communication method that uses machine-generated voice messages to contact users.

[0651] A "summary" refers to a short, concise version of data or information, extracted from the original source to highlight the most important points.

[0652] "Display devices" refer to devices used by humans to visually confirm information.

[0653] "Response results" refer to data obtained as responses and feedback provided by users.

[0654] "Personalized services" refer to the provision of services tailored to the individual needs and characteristics of each user.

[0655] This invention provides a system for efficiently managing customer reservation data and survey results, and for providing personalized services. This system consists of three main elements: a server, a terminal, and a user.

[0656] The server receives reservation data and survey results and saves them to a relational database (e.g., MySQL, PostgreSQL). The server analyzes the saved information using Python and generates automatically generated communication content using a generative AI model (using TensorFlow, PyTorch, etc.). This communication content is generated via prompt statements and is tailored to the individual needs of the user. For example, a prompt statement in the format of "Create the optimal suggestion talk for user A, a family of four who is reviewing their communication plan" might be used.

[0657] Next, the server uses the automatically generated communication content to perform an automated voice call to the user via an auto-call system (e.g., a common voice communication API). The user receives the message through this voice call and can check its contents at home or elsewhere. The auto-call system collects the user's responses and transfers this data to the server.

[0658] The server analyzes the collected response data again using a regenerative AI model to generate a summary. This summary provides condensed information crucial for future customer service interactions. The generated summary is sent to a terminal used by the service staff, where detailed information is displayed. The service staff uses this summary to propose personalized services to users and improve the quality of service delivery.

[0659] As a concrete example, consider a scenario where a user requests either a unified plan proposal for the entire family or an explanation of the benefits of individual plans. In such cases, the system can effectively provide proposals that meet the user's expectations.

[0660] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0661] Step 1:

[0662] Users input reservation data and survey results using a dedicated application or web form. This includes information such as preferred date and time of visit, desired service type, family composition, and categories of interest. The input results are sent to the server. The arrival of the input data on the server provides the foundational data necessary for subsequent analysis.

[0663] Step 2:

[0664] The server stores reservation data and survey results received from users in a relational database. Specifically, it connects to a database management system (e.g., MySQL or PostgreSQL) and inserts the received data into the appropriate tables. This step ensures that the information is persistently stored and available for later analysis.

[0665] Step 3:

[0666] The server prepares a generative AI model to analyze the stored information. First, it runs a Python script to load input data into the model using the stored data. A prompt statement (e.g., "Create the best suggestion talk for a user who is reviewing their communication plan as a family of four") is input into the generative AI model, and the analysis begins. The model generates individual talk content based on this input information.

[0667] Step 4:

[0668] The server prepares an auto-call system based on the generated talk content. It then configures the system to automatically communicate with the user via voice. Specifically, it converts the generated talk into a voice message using a voice API and dials the user's phone number. This allows the user to receive personalized service information at home or elsewhere.

[0669] Step 5:

[0670] The user receives an automated call and confirms the message. The system captures the user's listening response and reactions (e.g., button presses or voice responses) and feeds them back to the server. This input data becomes core data for subsequent analysis.

[0671] Step 6:

[0672] The server analyzes the user's response and generates a summary using a generative AI model. Based on this summary, the server extracts important information useful for future customer service and prepares to provide it to the customer service staff.

[0673] Step 7:

[0674] The server sends the generated summary to a terminal used by the service staff. The terminal displays the received summary, providing the service staff with material to make personalized suggestions. The staff refer to the displayed information and make the best suggestions for the user. This enables the delivery of personalized service.

[0675] (Application Example 1)

[0676] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0677] For effective customer service in stores, it is crucial to thoroughly understand customer needs before they visit and provide appropriate service. However, traditional methods have been problematic because reservation information management and the content of pre-visit phone calls are highly dependent on individual staff members, leading to inconsistent results. Furthermore, for service staff to respond to customer needs, they are required to understand a large amount of information quickly and make immediate suggestions. Under these conditions, the burden on store staff becomes significant, making it difficult to maintain customer satisfaction.

[0678] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0679] In this invention, the server includes means for receiving reservation information and questionnaire responses and storing them in a storage medium; means for analyzing the stored information and generating an optimal call script; means for executing a voice call based on the script generated by the generation means; means for collecting the user's response results from the voice call, analyzing them, and generating a summary; and means for proposing and presenting the generated summary and personalized customer service suggestions to the terminal. This enables a detailed understanding of the user's needs when making a reservation and allows for personalized customer service.

[0680] "Reservation information" refers to information about the date and time of the customer's visit and the services they wish to receive.

[0681] "Survey responses" refer to information about the user's personal interests and needs provided when making a reservation.

[0682] A "storage medium" is a digital storage system used to store and preserve information.

[0683] A "generation method" is a system that has the function of creating talks and summaries based on collected and analyzed information.

[0684] "Voice communication" refers to the process of communicating using voice via electronic means.

[0685] A "user" is an individual or group that makes a reservation for a service and is the recipient of the customer service or services provided.

[0686] A "terminal" is an electronic device used by staff to verify the generated information.

[0687] A "personalized customer service plan" is a proposed customer service response tailored to the individual needs and interests of the user.

[0688] This invention is a system that enables effective communication and customer service with users. It consists of three main elements: a server, a terminal, and the user.

[0689] The server receives reservation information and survey responses entered by the user. This information is stored in a database such as MySQL or PostgreSQL. Subsequently, the stored information is analyzed using Python and Django, and a generative AI model such as OpenAI's GPT series generates the optimal call script and summary.

[0690] The generated conversation is automatically transmitted to the user using a voice call system. Text-to-speech (TTS) technology is used for the text-to-speech conversion. After the user listens to the conversation, their response is collected on a server, re-analyzed, and a summary is generated.

[0691] The device, specifically a smartphone app developed using React Native, receives summaries and personalized service suggestions sent from the server. The app on the device displays this information in real time, helping store staff make the best service suggestions for customers.

[0692] For example, if a user responds to a survey by saying they "want to review their family's communication plan," the server uses AI to create a conversation that includes optimal suggestions and displays specific customer service proposals on the device, such as "Please suggest a new communication plan that can be used by the whole family."

[0693] An example of a prompt is, "To suggest the optimal communication plan based on family structure, please create a personalized conversation that takes into account the user's current usage and desired changes." This prompt instructs the generative AI model to generate specific conversation content.

[0694] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0695] Step 1:

[0696] The server receives reservation information and survey responses from users. The input includes information such as the date and time, desired services, and needs, which are then stored in the database. The output here is the stored customer information. Specifically, the received data is stored in a MySQL database via SQL queries.

[0697] Step 2:

[0698] The server analyzes the stored customer information. This analysis process uses a Python script to read information from the database and extract user attributes and needs. The input is user information retrieved from the database, and the output is the analyzed data. Specifically, the information is organized in JSON format.

[0699] Step 3:

[0700] The server generates the optimal call script using a generative AI model based on the analyzed data. A prompt is used to request the generative AI to create a script that matches the user's needs. The input at this stage is the analysis result and the prompt. The output is the generated call script. For example, data can be sent to the OpenAI API and the generated text can be received.

[0701] Step 4:

[0702] The server converts the generated talk into audio data using TTS technology and automatically makes a call using the voice call system. The input is the generated text talk, and the output is audio data. Specifically, the Google Text-to-Speech API is used to convert the talk into speech.

[0703] Step 5:

[0704] The user receives and responds to an incoming voice call. The input is the voice call data, and the output is the user's response information. The user listens to the voice message using a regular telephone handset.

[0705] Step 6:

[0706] The server automatically collects user response information and analyzes it again. This analysis identifies key points. The input is the user's response, and the output is summarized data. Specifically, it converts speech to text and performs analysis using NLP (Neuro-Linguistic Programming) technology.

[0707] Step 7:

[0708] The terminal receives the analyzed summary data from the server, which is then accessed by staff. The input is the summary data sent from the server, and the output is the content displayed on the terminal. Specifically, the data is displayed in the user interface using a React Native application.

[0709] Step 8:

[0710] The staff app on the device presents personalized service suggestions to the user based on summarized information. The user receives service suggestions from the staff. The input is summarized information, and the output is the staff's suggestions. The staff responds based on the information displayed on the device.

[0711] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0712] This invention is a system that enables efficient information management when users make reservations and allows for pre-visit phone calls and customer service that take emotions into consideration. The invention provides a more personalized service by receiving, analyzing, and utilizing user reservation information and questionnaire responses, as well as emotion recognition.

[0713] The server first receives reservation information and questionnaire responses from users and stores this data in a database. This includes information such as the reservation date and time, expected services, family structure, and products of interest. Based on this received data, the server activates a generation AI to create an optimal pre-call script for each user.

[0714] The AI-generated conversation is tailored based on the user's attributes and interests, and an emotion engine is integrated into this process. The emotion engine analyzes all data, including the user's responses, to recognize the user's emotional state. This allows it to analyze what emotions the user expressed previously, what their satisfaction and dissatisfaction levels were, and incorporate this information into the call conversation.

[0715] The server passes this generated talk to the auto-calling system, which then automatically dials the user. The auto-calling system delivers the talk in an audio format that takes emotions into account, enabling a more empathetic and appropriate approach to the user. Users receive this call at home or on the go and listen to the content to deepen their understanding of the service.

[0716] After the automated call, the server collects the user's response again, and the emotion engine analyzes the user's reaction and emotions. Based on the resulting emotion information and response content, the generative AI creates a summary. This summary includes the user's emotion information and suggests appropriate customer service.

[0717] The terminal is a device used by the store crew to display this summary, which the crew uses in customer service. The crew can use this information to provide personalized suggestions and support to the user. For example, if a user expresses anxiety, they can provide detailed service explanations or reassuring explanations, resulting in more attentive and emotionally sensitive customer service.

[0718] As described above, the present invention enables automated pre-calling that takes into account the user's emotional state and personalized customer service, thereby realizing the provision of more efficient and effective customer service.

[0719] The following describes the processing flow.

[0720] Step 1:

[0721] The server receives customer reservation information and survey responses and stores them in a database. The reservation information includes the date and time and the services expected, while the survey includes family composition and information on products currently being used.

[0722] Step 2:

[0723] The server activates a generation AI using saved reservation information and survey responses. The generation AI analyzes this information and generates a customized pre-call message for each user. The message content is adjusted according to the user's attributes and past survey results.

[0724] Step 3:

[0725] The server utilizes an emotion engine to recognize the user's emotional state by analyzing past user responses and reservation information. This allows for further personalized conversations.

[0726] Step 4:

[0727] The server sends generated, emotion-driven messages to the auto-call system and automatically places a call to the user. The user can then listen to the customized message content through the received auto-call.

[0728] Step 5:

[0729] After an automated call, the server collects the user's response and uses an emotion engine to analyze the user's emotions and reactions in detail. Based on this analysis, a generative AI creates a summary.

[0730] Step 6:

[0731] The server sends the generated summary to the terminal. The terminal displays this summary, helping the store crew provide customer service based on it. The crew can then provide more personalized service based on the user's identified emotions and interests.

[0732] (Example 2)

[0733] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0734] In today's service industry, providing efficient and personalized customer service is a challenge, especially given the need for appropriate responses to users. In particular, the lack of prior contact and interaction that takes user emotions into account necessitates efficient information management while optimizing user satisfaction.

[0735] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0736] In this invention, the server includes means for receiving reservation information and questionnaire answers and storing them in a storage device, means for analyzing the stored information and generating optimal automated call content, and means for executing a machine-controlled call based on the call content generated by the generation means. This enables the generation of automated call content that takes into account the user's emotions and efficient information management and analysis.

[0737] "Reservation information" refers to detailed information such as the date, time, location, and content of the service that the user plans to provide.

[0738] "Questionnaire responses" refer to information collected from users through surveys and feedback, and are used to understand the characteristics and needs of users.

[0739] A "memory device" is a data storage system that stores received data and makes it available for use as needed.

[0740] "Generation means" refers to a process or algorithm for producing appropriate output based on stored information, and in this invention, it is used to generate automated call content.

[0741] "Automated call content" refers to the content of a call using pre-prepared voice messages or dialogue scripts, intended for prior notification or confirmation to the user.

[0742] "Machine-controlled communication" refers to automated voice calls conducted by a communication system, and is a means of telephone communication that is mechanically controlled.

[0743] The "generated summary" is a summary of the analyzed data, concisely describing the user's emotional state and necessary countermeasures.

[0744] "Emotional information" refers to information that indicates the emotional state and characteristics analyzed based on data provided by the user.

[0745] As the foundation for implementing this invention, the server receives reservation information and questionnaire responses from users. This data is stored in a database such as MySQL and serves as an important source of information for forming individual user profiles. Based on this information, the server uses a generative AI model to generate optimal automated call content. A general-purpose AI model with excellent natural language processing capabilities is used as the generative AI model.

[0746] The generated automated call content is adjusted via an emotion analysis engine to take into account the user's past responses and emotional information. This engine uses stored historical data to infer the user's emotional state and reflects it in the call content.

[0747] Next, the server converts the coordinated call content into speech using speech synthesis software and makes a phone call to the user through a machine-controlled call system. Speech synthesis technologies such as Amazon Polly are useful in this process. Machine-controlled calls can be received by users anywhere and provide them with a means to better understand the service.

[0748] Through the voice call, the user receives information about the service and how to make their next appointment, and responds as needed. The server collects these responses again and analyzes them in detail using an emotion engine. Based on the analysis, the generating AI creates a summary that again reflects the user's emotions and displays it on the device.

[0749] The terminal will display this overview as a device used by store crew members, who will then use it in customer service. Crew members can use this information to, for example, provide detailed explanations to customers who are feeling anxious about a new service, thereby reassuring them.

[0750] A concrete example of a prompt is, "The user is interested in the outdoors. Please create a call script to confirm a reservation for their next visit." Through this prompt, the generating AI model provides a personalized response and delivers information tailored to the user's needs.

[0751] In this way, the system of the present invention enables highly personalized services that take into account the user's emotions, and achieves efficient and highly accurate customer service.

[0752] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0753] Step 1:

[0754] The server receives reservation information and questionnaire responses from users. Input data includes reservation date and time, desired service, and hobbies / interests. This data is stored in a MySQL database and managed on a per-user basis. The stored data will be used for future analysis and as input for generating AI.

[0755] Step 2:

[0756] The server generates a prompt based on the saved reservation information and questionnaire responses. This prompt is used as input to a generative AI model. This generative model generates automated call content written in natural language based on the given prompt. For example, it might output something like, "The user is interested in outdoor activities. Please create a call script to confirm their next visit."

[0757] Step 3:

[0758] The generated call content is further analyzed by the emotion engine. Past response history and data on the user's emotional state are used as input. The emotion engine identifies the user's emotions and adjusts the call content to provide optimal information and tone, thereby creating an emotionally sensitive call output.

[0759] Step 4:

[0760] The server uses the adjusted call content as input data, which is then converted into speech using speech synthesis software. Specifically, it outputs the call content in text format as an audio file and sends it to the machine-controlled call system. This audio is then automatically dialed to the user, and the content is transmitted.

[0761] Step 5:

[0762] The user receives an automated voice call and confirms its content. The user responds as needed, following the voice guidance. The response data is returned to the server, saved again to the database, and analyzed.

[0763] Step 6:

[0764] The server analyzes user response data using an emotion engine. The user's most recent response information is used as input to identify user satisfaction and emotional state. A summary is generated based on this information, and this summary serves as the basis for preparing for the next customer interaction.

[0765] Step 7:

[0766] The terminal displays an analyzed summary on the device used by the store crew. The crew refers to this summary and provides customer service tailored to each customer's needs. Specifically, the crew can alleviate customer anxiety by providing detailed service explanations and emotional encouragement.

[0767] (Application Example 2)

[0768] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0769] Currently, in brick-and-mortar stores, there is a demand for personalized service based on the individual interests and emotional states of customers. However, implementing this requires considerable effort and time. In particular, there is a lack of efficient systems for providing customer service that meets the diverse needs of different customers, which increases the burden on store staff and hinders improvements in customer satisfaction.

[0770] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0771] In this invention, the server includes means for receiving reservation information and questionnaire responses and storing them in a storage means, means for analyzing the stored information and generating optimal communication content, and means for executing automated voice communication based on the communication generated by the generation means. This makes it possible to grasp the status information of each customer and efficiently provide personalized customer service.

[0772] "Reservation information" refers to information that includes the date and time the user wishes to receive the service, as well as the details of the service requested.

[0773] A "survey response" is the answer to a question asked to obtain details about a user's interests and expectations.

[0774] A "memory device" is a data storage system for holding received information.

[0775] "Analysis" is the process of analyzing the content and characteristics of acquired information.

[0776] "Communication content" refers to messages and audio information sent to the user.

[0777] "Generation means" refers to the process of creating new communication content based on the analyzed information.

[0778] "Automated voice communication" is a system that automatically transmits pre-generated voice information to the user.

[0779] "State information" refers to data about a user's internal situation, such as their emotions and interests.

[0780] An "information terminal" is an electronic device that displays prepared information and allows users to view it.

[0781] "Customer service information" refers to customer service methods and recommendations provided according to the user's characteristics and emotions.

[0782] The system for implementing this invention is primarily built around a server, a terminal, and user interaction. The server receives reservation information and survey responses from the user and stores them in a storage device. This allows for recording the service content the user expects and their past usage history, which can then be used to improve future services. For the storage device, it is generally recommended to use Firebase, a cloud-based data storage service.

[0783] Next, the server performs analysis using the stored information. This analysis includes sentiment analysis using machine learning frameworks such as TensorFlow. The user's emotional tendency data obtained through sentiment analysis is information that can lead to improved user satisfaction, and the generation method creates the optimal communication content based on this.

[0784] The generated communication content is delivered to the user via automated voice communication. This process utilizes the Google Cloud Text-to-Speech API to generate text-to-speech and automatically deliver it to the user. Through this voice communication, the user can check the details of the service they plan to use in advance.

[0785] The terminal will be used in stores as an information terminal. It will display customer service information that users should refer to when they visit the store. As a result, staff will be able to easily provide personalized service, improving the quality of customer service.

[0786] For example, in the case of a casual apparel store, if a user requests to "book a fitting appointment for a weekend afternoon," the AI ​​model can prepare in advance fitting outfits and related product information that are perfectly suited to that user.

[0787] An example of a prompt for the generating AI model could be, "Based on this week's booking data, please create casual fashion conversation content tailored to the user's preferences." This enables the provision of highly flexible and personalized services.

[0788] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0789] Step 1:

[0790] The server receives reservation information and survey responses from users. Inputs include the date and time, desired service details, and areas of interest, all entered by the user via a smartphone app. This input data is saved to cloud storage. The output is the accurately saved user reservation data.

[0791] Step 2:

[0792] The server analyzes reservation information and survey responses stored in memory to understand the user's emotional tendencies. This process uses TensorFlow, a machine learning framework. The input is the user data stored in step 1, and the output is the identification result of the user's emotional tendencies.

[0793] Step 3:

[0794] The server uses a generative AI model to generate optimal communication content based on the analyzed sentiment tendencies and other user characteristics. The input is the output data from step 2 and user attribute data, and the output is the text of the generated communication content.

[0795] Step 4:

[0796] The server converts the generated communication content into speech data using the Google Cloud Text-to-Speech API. The input is the text data generated in step 3, and the output is the speech data sent to the user.

[0797] Step 5:

[0798] The user receives an audio communication and uses it to preview the content of the service being offered. The input is the audio data from step 4, and the output is to improve the user's understanding and pique their interest.

[0799] Step 6:

[0800] The terminal displays customer service information necessary when a user visits the store. Input is user-specific customer service information sent from the server, and output is a specific customer service guide that can be used by store staff.

[0801] Step 7:

[0802] The staff operating the terminal will provide personalized service to the user based on the displayed service information. The input is the service information from step 6, and the output is a high-quality, personalized customer service experience.

[0803] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0804] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0805] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0806] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0807] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0808] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0809] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0810] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0811] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0812] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0813] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0814] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0815] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0816] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0817] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0818] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0819] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0820] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0821] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0822] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0823] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0824] The following is further disclosed regarding the embodiments described above.

[0825] (Claim 1)

[0826] A means of receiving reservation information and survey responses and saving them in a database,

[0827] A generation means that analyzes stored information and generates the optimal call script,

[0828] A means for executing an auto-call based on the talk generated by the generation means,

[0829] A means for collecting user response results via automated calls, analyzing them, and generating a summary,

[0830] A means of suggesting customer service content to the terminal based on the generated summary,

[0831] A system that includes this.

[0832] (Claim 2)

[0833] The system according to claim 1, which generates personalized phone conversations based on the user's reservation information and survey responses.

[0834] (Claim 3)

[0835] The system according to claim 1, which sends a generated summary to a terminal and provides personalized services to the user based on the customer service content displayed on the terminal.

[0836] "Example 1"

[0837] (Claim 1)

[0838] A means for receiving reservation data and survey results and storing them in a storage device,

[0839] A generation means that processes stored information and generates automatically generated communication content,

[0840] A means of performing automated voice communication based on the generated communication content,

[0841] A means for acquiring user response results in automated voice communication, analyzing them, and generating a summary,

[0842] A means of proposing service content to a display device based on the generated summary,

[0843] A system that includes this.

[0844] (Claim 2)

[0845] The system according to claim 1, which generates personalized communication content based on user reservation data and survey results.

[0846] (Claim 3)

[0847] The system according to claim 1, which transmits a generated summary to a display device and provides personalized services to the user based on the service content presented on the display device.

[0848] "Application Example 1"

[0849] (Claim 1)

[0850] A means of receiving reservation information and survey responses and saving them to a storage medium,

[0851] A generation means that analyzes stored information and generates the optimal call script,

[0852] A means for performing a voice call based on the talk generated by the generation means,

[0853] A means for collecting user response results via voice calls, analyzing them, and generating summaries,

[0854] A means for proposing and presenting the generated summary and personalized customer service plan to the terminal,

[0855] A system that includes this.

[0856] (Claim 2)

[0857] The system according to claim 1, which generates personalized phone call scripts and customer service proposals based on user reservation information and questionnaire responses.

[0858] (Claim 3)

[0859] The system according to claim 1, which transmits a generated summary to a terminal and provides personalized services to the user based on the customer service suggestion displayed on the terminal.

[0860] "Example 2 of combining an emotion engine"

[0861] (Claim 1)

[0862] A means for receiving reservation information and questionnaire responses and saving them to a storage device,

[0863] A generation means that analyzes stored information and generates optimal automated call content,

[0864] A means for executing a machine-controlled call based on the call content generated by the generation means,

[0865] A means for collecting user response results via machine-controlled calls, analyzing them, and generating an overview,

[0866] A means of suggesting customer service content to a display device based on the generated summary,

[0867] A means of analyzing user emotions and incorporating the results into customer service,

[0868] A means of providing more empathetic responses to users based on the generated emotional information,

[0869] A system that includes this.

[0870] (Claim 2)

[0871] The system according to claim 1, which generates personalized automated call content based on the user's reservation information and questionnaire responses.

[0872] (Claim 3)

[0873] The system according to claim 1, which transmits the generated summary and emotional information to a display device, and provides personalized services to the user based on the customer service content displayed on the display device.

[0874] "Application example 2 when combining with an emotional engine"

[0875] (Claim 1)

[0876] A means for receiving reservation information and survey responses and storing them in a memory device,

[0877] A generation means that analyzes stored information and generates optimal communication content,

[0878] A means for performing automatic voice communication based on the communication generated by the generation means,

[0879] A means for collecting, analyzing, and generating summaries of user responses via automated voice communication,

[0880] A means of suggesting customer service content to an information terminal based on the generated summary,

[0881] A means of analyzing user status information and providing customer service information based on emotional tendencies,

[0882] A system that includes this.

[0883] (Claim 2)

[0884] The system according to claim 1, which generates personalized communication content based on the user's reservation information and survey responses.

[0885] (Claim 3)

[0886] The system according to claim 1, which transmits a generated summary to an information terminal and provides personalized services to the user based on the customer service content displayed on the information terminal. [Explanation of Symbols]

[0887] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means of receiving reservation information and survey responses and saving them in a database, A generation means that analyzes stored information and generates the optimal call script, A means for executing an auto-call based on the talk generated by the generation means, A means for collecting user response results via automated calls, analyzing them, and generating a summary, A means of suggesting customer service content to the terminal based on the generated summary, A system that includes this.

2. The system according to claim 1, which generates personalized phone call messages based on the user's reservation information and survey responses.

3. The system according to claim 1, which sends a generated summary to a terminal and provides personalized services to the user based on the customer service content displayed on the terminal.

Images