system

Abstract

Provide a system. An automatic response means for inputting reservation details by voice; A means for analyzing the input voice and generating a reservation schedule for the client; a means for notifying the client of the generated reservation schedule; a means of monitoring client bookings and detecting availability; means for automatically generating promotional text based on the detected free time; means for transmitting the generated promotional text to the client; A system including:

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] The purpose of this invention is to provide a system that reduces the burden of telephone response that occurs during busy periods in the retail and customer service industries, and also enables effective sales promotion activities during quiet times. In particular, the objective is to improve business efficiency and increase customer satisfaction by automating telephone reservation responses and sales activities. [Means for solving the problem]

[0005] The present invention solves the above problems by providing an automatic response means that allows reservation details to be input by voice. It then provides a means for analyzing the voice and generating a reservation schedule for the client. It also provides a means for notifying the client of the generated reservation schedule.

[0006] The system also includes a means for monitoring client reservation status in real time, detecting available time, and automatically generating promotional text based on the detected available time. It also includes a means for sending the generated promotional text to clients. This provides a system that enables effective sales activities while reducing the burden of answering the phone.

[0007] An "automatic response means" is a device or system that responds to telephone calls or voice inputs from users without human intervention and collects and processes the necessary information.

[0008] "Speech analysis" is a technology that converts voice data entered by a user into text, understands its content, and performs appropriate processing.

[0009] The "reservation schedule" is a table or data that systematically organizes dates and times determined based on the user's reservation details and other related information.

[0010] "Notification means" refers to a communication means or system for informing users of reservation schedules and other important information.

[0011] "Monitoring means" refers to devices or systems that track client reservations and store operations in real time to detect significant changes or vacancies.

[0012] "Free time" means time periods when there are no reservations or other commitments.

[0013] "Promotional text" is text used to advertise a store's services or products to customers.

[0014] A "messaging application" is software, such as LINE, for sending text messages to a user's device. [Brief explanation of the drawings]

[0015] [Figure 1]1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0016] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0017] First, the terms used in the following description will be explained.

[0018] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0019] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0020] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0021] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0022] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0023] [First embodiment]

[0024] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0025] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0026] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0027] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0028] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0029] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0030] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0031] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0032] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0033] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0034] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0035] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0036] The present invention relates to a system for automating telephone reservation responses and sales promotion activities. The system mainly comprises a server, a user terminal, an automatic response means, and a voice analysis means.

[0037] Configuring an Automated Response System

[0038] Reservations accepted

[0039] 1. Automated response methods:

[0040] When a user calls the store using their own terminal (for example, a mobile phone or landline), the server acts as an automatic answering means and answers the call. The server responds by saying, "Welcome. Please tell us the details of your reservation."

[0041] 2. Audio analysis methods:

[0042] The server uses speech recognition technology to analyze the user's voice input. For example, if the user says, "I'd like to make a reservation for two people tomorrow at 6 p.m.", the server converts that information into text data and accurately understands it.

[0043] 3. Reservation schedule generation method:

[0044] The server generates a reservation schedule based on the analyzed voice data. The reservation schedule includes the reservation date and time, number of people, and other necessary information. The generated reservation schedule is saved in a database.

[0045] 4. Means of notification:

[0046] The server uses the LINE API or similar to send a reservation confirmation message to the user's device to notify them of the reservation details. For example, a confirmation message such as "Tomorrow at 6 PM for two people. Your reservation has been confirmed."

[0047] Sales support

[0048] 1. Monitoring measures:

[0049] The server constantly monitors the client's reservation status and the store's operational status. The server periodically checks the reservation system database to find available time slots.

[0050] 2. Free time detection:

[0051] By monitoring, the server detects that there are no reservations between 14:00 and 16:00, for example. This free time information is used in the next step.

[0052] 3. Automatic sales promotion text generation method:

[0053] The server uses AI to generate promotional messages based on available times, such as, "We offer a special discount to customers who visit between 2:00 PM and 4:00 PM today. Please come and visit us."

[0054] 4. Message sending method:

[0055] The server sends the generated promotional message to all customers registered in the customer list via the LINE API, increasing the chances that customers will receive the message and become interested.

[0056] Specific examples

[0057] For example, suppose User A calls a restaurant and requests, "I'd like to make a reservation for two people tomorrow at 6 PM." The server automatically receives the call, converts the content into text using a voice recognition system, and generates a reservation schedule. The server then sends a reservation confirmation message to User A via LINE, confirming the reservation.

[0058] The server also detects when the store has fewer reservations and generates promotional messages based on that information, which are then sent to customers via LINE. For example, by sending a message offering a "special discount" when the store is open between 2:00 PM and 4:00 PM, it is possible to attract customers' interest and encourage them to visit the store.

[0059] In this way, the present invention provides a system that reduces the burden of answering telephone calls and allows effective sales promotion activities to be carried out even during idle time.

[0060] The processing flow will be explained below.

[0061] Specific processing steps for reservation handling

[0062] Step 1: User makes a call

[0063] The user calls the store's phone number using their own device (mobile phone or landline).

[0064] Step 2: The server automatically answers the call

[0065] The server acts as an automated answering machine and receives the call, saying, "Welcome. Please tell us your reservation details."

[0066] Step 3: The server analyzes the audio

[0067] The server analyzes the user's speech using an AI speech recognition system and converts the content into text data. For example, "I would like to make a reservation for two people tomorrow at 6 p.m."

[0068] Step 4: The server generates the reservation schedule

[0069] The server generates a reservation schedule based on the textual reservation details, and the generated reservation schedule is saved in a database.

[0070] Step 5: The server confirms the reservation

[0071] The server sends a reservation confirmation message to the user's device via the LINE API. For example, a message like "Tomorrow at 6 PM for two people. Your reservation has been confirmed."

[0072] Specific steps for sales responses

[0073] Step 1: The server monitors the situation

[0074] The server periodically checks the database to monitor client reservation status and store operation status in real time.

[0075] Step 2: The server finds free time

[0076] The server analyzes the reservation data and detects, for example, that there are no reservations between 2:00 p.m. and 4:00 p.m.

[0077] Step 3: The server automatically generates the promotional text

[0078] The server uses the generative AI model to automatically generate promotional messages based on the time when no reservations are available, such as "We offer a special discount to customers who visit between 2:00 PM and 4:00 PM today. Please come and visit us."

[0079] Step 4: The server sends the promotional message

[0080] The server uses the LINE API to send the generated promotional message to all customers registered in the customer list.

[0081] Step 5: Customer receives message

[0082] The customer receives the LINE message on their device and checks the contents.

[0083] Specific examples

[0084] Examples of reservation handling

[0085] Step 1: User A calls the store.

[0086] Step 2: The server will automatically answer the phone and say, "Welcome. Please tell us your reservation details."

[0087] Step 3: User A says, "I'd like to make a reservation for two people tomorrow at 6 PM." The server analyzes the speech and converts it into text.

[0088] Step 4: The server generates a reservation schedule based on the textual reservation details and saves it in the database.

[0089] Step 5: The server notifies User A via the LINE API, "Your reservation for two people at 6 PM tomorrow has been confirmed."

[0090] Specific examples of sales responses

[0091] Step 1: The server monitors the current reservation status.

[0092] Step 2: The server detects that there are no reservations between 2 PM and 4 PM.

[0093] Step 3: The server automatically generates a promotional message saying, "We will offer a special discount to customers who visit our store between 2:00 PM and 4:00 PM today. Please come and visit us."

[0094] Step 4: The server uses the LINE API to send the promotional message to all customers in the customer list.

[0095] Step 5: Customer B receives the LINE message, checks the contents, and then decides to visit the store.

[0096] These are the specific steps involved in handling reservations and sales. This system reduces the burden of answering phone calls during busy periods and allows for effective sales promotion activities during off-peak times.

[0097] Example 1

[0098] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0099] Conventional reservation systems have had issues such as the heavy workload associated with answering the phone and inadequate management of busy and quiet times. Furthermore, promotional activities to effectively fill empty time slots are time-consuming and difficult to carry out efficiently. Furthermore, there is a lack of automated systems to solve these issues, so there is a need for more efficient store operations.

[0100] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0101] In this invention, the server includes an automatic answering means, a means for analyzing input voice and generating a reservation schedule, a means for notifying the client of the generated reservation schedule, a means for monitoring the client's reservation status and detecting free time, a means for using a generative AI model to automatically generate promotional messages based on the detected free time, and a means for using a message sending application to send the generated promotional messages to the client. This makes it possible to automate telephone answering and promotional activities that effectively utilize free time.

[0102] The "automatic response means" is a means for allowing the user to input reservation details by voice, and has the function of accepting voice input and transmitting a guidance message.

[0103] The "voice analysis means" is a means for converting input voice into text data and analyzing the reservation details, and uses voice recognition technology.

[0104] The "reservation schedule generating means" is a means for generating a reservation schedule from the analyzed voice data and storing the necessary information in a database.

[0105] The "notification means" is a means for notifying the client of the created reservation schedule, and uses a message sending application.

[0106] The "monitoring means" is a means for monitoring the reservation status of clients and detecting available time.

[0107] The "promotional text generation means" is a means for automatically generating promotional text using an AI model based on the detected free time.

[0108] The "message sending means" is a means for sending the generated promotional text to the client, and typically uses a message sending application.

[0109] A "generative AI model" is an artificial intelligence model used to automatically generate promotional text, generating appropriate text based on input prompts.

[0110] A "prompt" is a sentence that is used as input to a generative AI model, providing the AI ​​with specific context and information to produce a specific output.

[0111] The present invention relates to a system for automating telephone reservation responses and sales promotion activities. The system mainly comprises a server, a user terminal, an automatic response means, and a voice analysis means.

[0112] Configuring an Automated Response System

[0113] Reservations accepted

[0114] 1. Automated response methods:

[0115] When a user calls the store using their own device (e.g., a mobile phone or landline), the server acts as an automated answering machine and receives the call. The server uses speech synthesis software (e.g., a speech synthesis engine) to respond with, "Welcome. Please tell us the details of your reservation."

[0116] 2. Audio analysis methods:

[0117] The server uses speech recognition technology (e.g., speech recognition API) to analyze the user's voice input. For example, if the user says, "I'd like to make a reservation for two people tomorrow at 6 p.m.", the server converts that content into text data and accurately understands it.

[0118] 3. Reservation schedule generation method:

[0119] The server generates a reservation schedule based on the analyzed voice data. The reservation schedule includes the reservation date and time, the number of people, and other necessary information. The generated reservation schedule is saved in a database (e.g., a relational database).

[0120] 4. Means of notification:

[0121] The server uses a messaging application (e.g., a messaging API) to notify the user of the created reservation schedule. For example, a confirmation message such as "Tomorrow at 6 PM for two people. Your reservation has been confirmed."

[0122] Sales support

[0123] 1. Monitoring measures:

[0124] The server constantly monitors the client's reservation status and the store's operational status. The server periodically checks the database to find available time slots.

[0125] 2. Free time detection method:

[0126] By monitoring, the server detects that there are no reservations between 2 pm and 4 pm, for example. This free time information is used to generate the next promotional message.

[0127] 3. Automatic sales promotion text generation method:

[0128] The server uses a generative AI model (e.g., a language model) to generate promotional messages based on available times, such as, "We offer a special discount to customers who visit our store between 2:00 PM and 4:00 PM today. Please come and visit us."

[0129] 4. Message sending method:

[0130] The server sends the generated promotional message to all customers registered in the customer list via a messaging application (e.g., messaging API). The message received by the customer increases the likelihood that they will be interested.

[0131] Specific examples

[0132] For example, suppose User A calls a restaurant and requests, "I'd like to make a reservation for two people tomorrow at 6 PM." The server automatically receives the call, converts the content into text using a speech recognition API, and generates a reservation schedule using a system using the Django framework. The server then uses a message API to send User A a confirmation message saying, "Tomorrow at 6 PM for two people. Your reservation has been confirmed," and the reservation is confirmed.

[0133] The server also uses a database to detect times when the store has few reservations, and based on that information, it uses a generative AI model to generate promotional messages such as, "We will offer a special discount to customers who visit between 2:00 p.m. and 4:00 p.m. today. Please come and visit us," which are then sent to customers using a messaging API.

[0134] Prompt Sentence Examples

[0135] An example prompt for a generative AI model is:

[0136] "We have openings between 2:00 PM and 4:00 PM. Please come up with some promotional text to attract customers to the store during this time."

[0137] In this way, the present invention provides a system that reduces the burden of answering telephone calls and enables effective sales promotion activities to be carried out even during idle time.

[0138] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0139] Specific processing steps for reservation handling

[0140] Step 1: User makes a call

[0141] How it works: The user dials the store's phone number from their device and presses the call button.

[0142] Input: The action of the user making a phone call.

[0143] Output: The call is connected to the server.

[0144] Step 2: The server receives the call

[0145] What it does: The server launches an automated attendant to receive the call, and uses text-to-speech software to say, "Welcome. Please tell us your reservation details."

[0146] Input: The user's phone connection.

[0147] Output: A voice prompt is played to the user.

[0148] Step 3: User speaks

[0149] Action: The user says, "I'd like to make a reservation for two people tomorrow at 6pm."

[0150] Input: User's voice input.

[0151] Output: The audio data is sent to the server.

[0152] Step 4: The server analyzes the audio

[0153] How it works: The server uses a speech recognition API to convert the user's speech into text data.

[0154] Input: Audio data.

[0155] Output: Reservation information converted to text data.

[0156] Step 5: The server generates the reservation schedule

[0157] Operation: The server generates a reservation schedule based on the text data and saves it in a database.

[0158] Input: Text data.

[0159] Output: Appointment schedule saved in the database.

[0160] Step 6: The server sends a reservation confirmation

[0161] How it works: The server uses the messaging API to send a reservation confirmation message to the user's device.

[0162] Input: The generated appointment schedule.

[0163] Output: A booking confirmation message sent to the user's device.

[0164] Specific steps for sales responses

[0165] Step 1: The server monitors the reservation status

[0166] How it works: The server periodically checks the reservation system database to see which time slots are available.

[0167] Input: Database reservation status.

[0168] Output: Free / busy information.

[0169] Step 2: The server finds free time

[0170] What it does: The server finds available times and uses that information for further processing, for example, checking that there are no reservations between 2 PM and 4 PM.

[0171] Input: Monitoring result data.

[0172] Output: Information about free time detection.

[0173] Step 3: The server automatically generates the promotional text

[0174] How it works: The server uses a generative AI model to generate promotional text based on available times. By providing a prompt, the server can generate a promotional text such as "We are available between 2:00 PM and 4:00 PM. Please come up with a promotional text to encourage customers to visit during this time."

[0175] Input: Free / busy information and prompt statement.

[0176] Output: The generated promotional text.

[0177] Step 4: The server sends the promotional message

[0178] How it works: The server uses the messaging API to send the generated promotional message to all customers on the customer list.

[0179] Input: Generated promotional text and customer list.

[0180] Output: Promotional message sent to customer's device.

[0181] By going through the above steps, the system that automates telephone reservations and promotional activities can be operated efficiently and effectively.

[0182] (Application example 1)

[0183] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0184] Conventional reservation systems not only required a lot of effort to handle phone reservations, but also required manual promotional activities to efficiently fill available time, reducing store operational efficiency. There was also a need to improve the user experience when using smartphones.

[0185] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0186] In this invention, the server includes an automatic response means for inputting reservation details by voice, a means for analyzing the input voice and generating a reservation schedule for the user, a means for notifying the user of the generated reservation schedule, a means for monitoring the reservation status of the user and detecting available time, a means for automatically generating promotional messages based on the detected available time, a means for sending the generated promotional messages to the user, and a means for sending the voice input to the server via a smartphone application. This enables the automation of reservations and efficient promotional activities during available time.

[0187] 1. "Reservation details" refers to information including the date and time the user wishes to make a reservation, the number of people, and other necessary information.

[0188] 2. "Automatic response means that allows voice input" refers to an automatic response system that allows users to input reservation details using voice.

[0189] 3. "Means for analyzing voice" refers to technology for converting input voice data into text data and understanding its content.

[0190] 4. "Means for generating reservation schedules" refers to technology for automatically creating reservation schedules based on reservation details obtained through voice analysis.

[0191] 5. "Means for notifying users of reservation schedules" refers to a system for notifying users of the generated reservation schedules, which is primarily done through a message sending application.

[0192] 6. "Means for monitoring reservation status" refers to a system that constantly checks the current reservation status and detects any available time or changes.

[0193] 7. "Means for detecting available times" refers to technology that monitors reservation status and detects when there are no reservations for a specific time period.

[0194] 8. "Means for automated promotional copy generation" means technology for automatically creating specific promotional messages based on detected available time.

[0195] 9. "Means for sending promotional messages to users" refers to a system for sending generated promotional messages to users, primarily using a message sending application.

[0196] 10. "Means for transmitting voice input to a server via a smartphone application" refers to technology that uses a smartphone application to transmit a user's voice input to a server.

[0197] 11. "Server" refers to the computer system that is the core of the entire system, which handles reservation reception, voice analysis, reservation schedule generation, automatic generation and transmission of promotional materials, etc.

[0198] This invention relates to a system that includes a server, a smartphone application, and artificial intelligence technology for streamlining reservation systems and sales promotion activities in brick-and-mortar stores.

[0199] 1. Voice input of reservation details

[0200] A user attempts to make a reservation using a smartphone application. When the user presses the "Record reservation" button in the smartphone application, the smartphone's microphone is activated and the user voice-records the reservation details.

[0201] 2. Voice analysis and reservation schedule generation

[0202] Once the voice input is complete, the smartphone application sends the voice data to the server. The server uses voice recognition technology to convert the voice data into text data and analyzes the reservation details. This process uses the "speech_recognition" library to convert the voice data into text. Based on the textual reservation details, the server generates a reservation schedule.

[0203] 3. Notification of reservation schedule

[0204] The generated reservation schedule is notified to the user using a messaging application such as the LINE API. For example, a confirmation message such as "Tomorrow at 6 PM for two people. Your reservation has been confirmed" is sent to the user's LINE account.

[0205] 4. Reservation monitoring and availability detection

[0206] The server periodically monitors the reservation system database and manages the store's reservation status. When an available time slot is detected, the information is used in the next step. In this process, the reservation status is periodically checked to detect, for example, that there are no reservations between 2:00 PM and 4:00 PM.

[0207] 5. Automatic generation and sending of promotional text

[0208] The server uses a generative AI model to automatically generate promotional text based on the detected free time. For example, "We offer a special discount to customers who visit the store between 2:00 PM and 4:00 PM today." The generated promotional text is sent to the user via the LINE API.

[0209] Hardware and software used

[0210] 1. Hardware

[0211] Smartphone: Voice input and application execution.

[0212] Server: Speech analysis, reservation schedule generation, notifications, promotional text generation.

[0213] 2. Software

[0214] Smartphone application (iOS, Swift): Records audio and sends it to the server.

[0215] Server (Python, Flask): Speech recognition (speech_recognition library), reservation schedule generation, message sending using LINE API.

[0216] Message sending application (LINE API): Sends reservation confirmation messages and promotional messages to users.

[0217] Specific examples

[0218] For example, a user might say through a smartphone application, "I'd like to make a reservation for two people tomorrow at 6 PM." The smartphone sends this voice data to a server, which analyzes it using voice recognition technology, converts it into text data, and generates a reservation schedule. The server then sends a message to the user via the LINE API, such as "Reservation confirmation: tomorrow at 6 PM for two people." The server also detects available times and creates promotional text using a generative AI model. A promotional text such as "We offer a special discount to customers who visit between 2 PM and 4 PM today" is sent to the user via the LINE API.

[0219] Prompt Sentence Examples

[0220] "User says: I'd like to make a reservation for two people tomorrow at 6pm."

[0221] "Promotional message: We are offering a special discount to customers who visit our store between 2pm and 4pm today."

[0222] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0223] Step 1:

[0224] The user launches the smartphone application and presses the "Record reservation" button. This activates the smartphone's microphone and starts voice input. The user then enters the reservation details by voice. The user's voice data is obtained as input.

[0225] Step 2:

[0226] Once the voice input is complete, the smartphone application sends the voice data to the server. At this time, the voice data file is passed to the server in the format included in the request body. The voice data is sent to the server as input.

[0227] Step 3:

[0228] The server converts the received voice data into text data using the "speech_recognition" library. It receives voice data as input and generates text data as output. At this stage, the voice recognition model analyzes the voice and obtains the reservation details as string information.

[0229] Step 4:

[0230] The server analyzes the textual reservation details and generates a reservation schedule. The server analyzes the text data as input, extracts information such as the reservation date, time, and number of people, and generates a reservation schedule as output. The reservation schedule is saved in a database.

[0231] Step 5:

[0232] The server notifies the user of the generated reservation schedule using the LINE API. Specifically, it sends a message within the LINE app saying, "Tomorrow at 6 PM for two people. Your reservation has been confirmed." It receives the reservation schedule information as input and sends a confirmation message to the user as output.

[0233] Step 6:

[0234] The server periodically monitors the reservation system database. It scans the reservation database and checks for time slots with few reservations. It takes the reservation status data as input and generates a list of available time slots as output.

[0235] Step 7:

[0236] The server uses a generative AI model to automatically generate promotional text based on available time slots. For example, it might say, "We offer a special discount to customers who visit between 2:00 PM and 4:00 PM today." It takes available time slot information as input and generates promotional text as output.

[0237] Step 8:

[0238] The server sends the generated promotional text to all users via the LINE API. It receives the promotional text as input and sends a message to users as output. Specifically, the promotional text is displayed within the LINE app, attracting users' attention and encouraging them to visit the store.

[0239] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0240] This invention is a system incorporating an emotion engine that recognizes user emotions in order to automate telephone reservation responses and sales promotion activities. The main components of this system include a server, a user terminal, an automatic response means, a voice analysis means, and an emotion engine.

[0241] Configuring an Automated Response System

[0242] Reservations accepted

[0243] 1. Automated response methods:

[0244] When a user calls the store using their own device (mobile phone or landline), the server acts as an automatic answering machine and answers the call. The server responds by saying, "Welcome. Please tell us the details of your reservation."

[0245] 2. Audio analysis methods:

[0246] The server uses speech recognition technology to analyze the user's voice input. For example, if the user says, "I'd like to make a reservation for two people tomorrow at 6 p.m.", the server converts that content into text data and understands it.

[0247] 3. Emotion Engine:

[0248] The server is equipped with an emotion engine that recognizes emotions from the user's voice and facial expressions. The emotion engine analyzes the user's tone and speed of speech, and in the case of video calls, facial expressions, to determine the user's emotional state.

[0249] 4. Appointment schedule generation method:

[0250] The server generates a reservation schedule based on the analyzed voice data. The reservation schedule includes the reservation date and time, number of people, and other necessary information. The generated reservation schedule is saved in a database.

[0251] 5. Means of notification:

[0252] The server notifies the user of the created reservation schedule by sending a reservation confirmation message to the user's device using the LINE API, etc. For example, a confirmation message such as "Tomorrow at 6 PM for two people. Your reservation has been confirmed."

[0253] Sales support

[0254] 1. Monitoring measures:

[0255] The server constantly monitors the client's reservation status and the store's operational status. The server periodically checks the reservation system database to detect available time slots.

[0256] 2. Free time detection:

[0257] By monitoring, the server detects, for example, that there are no reservations between 2 p.m. and 4 p.m. This free time information is used in the next step.

[0258] 3. Automatic sales promotion text generation method:

[0259] The server works in conjunction with the emotion engine to automatically generate appropriate promotional messages based on the user's emotional state. For example, if the user is feeling positive, a cheerful message such as "We will offer a special discount to customers who visit our store between 2:00 PM and 4:00 PM today. Please come and visit us." is generated.

[0260] 4. Message sending method:

[0261] The server sends the generated promotional text to all customers registered on the customer list via the LINE API.

[0262] Specific examples

[0263] Examples of reservation handling

[0264] Let's say User A calls a store. The server automatically receives the call and says, "Welcome. Please tell us your reservation details." When User A says, "I'd like to make a reservation for two people tomorrow at 6 PM," the server analyzes the voice, understands the content, and generates a reservation schedule. At the same time, the emotion engine analyzes User A's tone of voice and recognizes that he is satisfied with the confirmed reservation. As a result, the server notifies User A via LINE message, "Tomorrow at 6 PM for two people. Your reservation has been confirmed. Thank you."

[0265] Specific examples of sales responses

[0266] The server monitors the current reservation status of the store and detects that there are no reservations between 2:00 PM and 4:00 PM. Using an emotion engine, the server analyzes the emotional data of users who have previously visited the store and generates promotional messages for customers with positive emotions. For example, it generates a message such as, "We will offer a special discount to customers who visit our store between 2:00 PM and 4:00 PM today. Please come and visit us," and sends it to all customers via the LINE API.

[0267] In this way, the present invention provides a system that reduces the burden of answering the phone and allows for effective sales promotion activities that take into account the emotional state of the customer.

[0268] The processing flow will be explained below.

[0269] Specific processing steps for reservation handling

[0270] Step 1: User makes a call

[0271] The user calls the store's phone number using their own device (mobile phone or landline).

[0272] Step 2: The server automatically answers the call

[0273] The server acts as an automated answering machine and receives the call, saying, "Welcome. Please tell us your reservation details."

[0274] Step 3: The server analyzes the audio

[0275] The server analyzes the user's speech using an AI speech recognition system and converts it into text data. For example, "I would like to make a reservation for two people tomorrow at 6 p.m."

[0276] Step 4: The server performs sentiment analysis

[0277] The server uses an emotion engine to analyze the tone and rate of the user's speech to determine the user's emotional state, for example, whether the user is happy or nervous.

[0278] Step 5: The server generates the reservation schedule

[0279] The server generates a reservation schedule based on the textual reservation details and the results of emotion analysis, and the generated reservation schedule is saved in a database.

[0280] Step 6: The server confirms the reservation

[0281] The server sends a reservation confirmation message to the user's device via the LINE API. For example, a message like "Tomorrow at 6 PM for two people. Your reservation has been confirmed."

[0282] Specific steps for sales responses

[0283] Step 1: The server monitors the situation

[0284] The server periodically checks the database to monitor client reservation status and store operation status in real time.

[0285] Step 2: The server finds free time

[0286] The server monitors and detects, for example, that there are no reservations between 2:00 p.m. and 4:00 p.m.

[0287] Step 3: The server analyzes the emotion data

[0288] The server uses an emotion engine to analyze past customer emotion data and identify customers with positive emotions.

[0289] Step 4: The server automatically generates the promotional text

[0290] The server generates appropriate promotional messages based on the user's emotional state. For example, for customers with positive emotions, it generates a message such as, "We will offer a special discount to customers who visit our store between 2:00 PM and 4:00 PM today. Please come and visit us."

[0291] Step 5: The server sends the promotional message

[0292] The server uses the LINE API to send the generated promotional message to all customers registered in the customer list.

[0293] Step 6: Customer receives message

[0294] The customer receives the LINE message on their device and checks the contents.

[0295] Specific examples

[0296] Examples of reservation handling

[0297] Step 1: User A calls the store.

[0298] Step 2: The server will automatically answer the phone and say, "Welcome. Please tell us your reservation details."

[0299] Step 3: User A says, "I'd like to make a reservation for two people tomorrow at 6 PM." The server analyzes the speech and converts it into text.

[0300] Step 4: The server analyzes the speech tone of User A using the emotion engine and recognizes that the user is satisfied.

[0301] Step 5: The server generates a reservation schedule based on the textual reservation details and the results of sentiment analysis, and saves it in the database.

[0302] Step 6: The server notifies User A via the LINE API, "Your reservation for two people at 6 PM tomorrow has been confirmed. Thank you."

[0303] Specific examples of sales responses

[0304] Step 1: The server monitors the current reservation status of the store.

[0305] Step 2: The server detects that there are no reservations between 2 PM and 4 PM.

[0306] Step 3: The server analyzes past customer emotion data using an emotion engine to identify customers with positive emotions.

[0307] Step 4: The server automatically generates a promotional message saying, "We will offer a special discount to customers who visit our store between 2:00 PM and 4:00 PM today. Please come and visit us."

[0308] Step 5: The server uses the LINE API to send the promotional message to all customers in the customer list.

[0309] Step 6: Customer B receives the LINE message, checks the contents, and then decides to visit the store.

[0310] These are the specific processing steps for handling reservations and sales. This system reduces the burden of answering phone calls during busy periods and enables effective sales promotion activities that take customer emotions into consideration during slower times.

[0311] Example 2

[0312] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0313] Conventional reservation systems require users to make reservations manually by voice, resulting in inefficiency and a significant drain on human resources. Furthermore, promotional activities are conducted without considering the emotional state of the customer, making effective marketing difficult. A system that solves these problems and improves reservation efficiency and customer satisfaction is needed.

[0314] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0315] In this invention, the server includes an automatic response means for allowing reservation details to be input by voice, a means for analyzing the input voice, a means for generating a reservation schedule for a client based on the analyzed voice, a means for saving the generated reservation schedule in a database, a means for notifying the client of the saved reservation schedule, a means for monitoring the client's reservation status and detecting available time, a means for automatically generating promotional messages based on the detected available time, a means for sending the generated promotional messages to the client, a means for analyzing the user's speech tone and facial expressions to recognize the user's emotional state, and a means for adjusting the promotional messages based on the recognized emotional state. This enables efficient reservation handling and effective promotional activities that take the customer's emotional state into consideration.

[0316] "Reservation details" refers to information such as the date and time the user wishes to make a reservation, the number of people, and any special requests.

[0317] "Voice-activated automated response system" refers to an automated response system that allows users to enter reservation details by voice, which allows users to access the system via telephone or other voice input device.

[0318] "Means for analyzing voice" refers to technology that converts voice information entered by the user into text data and understands its content. Specifically, voice recognition technology is used.

[0319] "Means for generating a reservation schedule" refers to a system that extracts necessary reservation information from the analyzed voice data and creates a reservation schedule based on that information.

[0320] "Means for saving in a database" refers to the technology for saving in an appropriate database in order to manage the generated reservation schedule, which will make the accumulation and retrieval of information more efficient.

[0321] The "means for notifying the client" refers to a communication means for notifying the user of the created reservation schedule. For example, it includes a notification method using a messaging application.

[0322] "Means for monitoring reservation status and detecting available times" refers to a system that continuously monitors existing reservation data and identifies time slots that are not booked.

[0323] "Automatic promotional copy generation" refers to technology that automatically creates effective promotional messages based on detected free time, which may involve the use of generative AI models.

[0324] "Means for sending to client" refers to a means for delivering the generated promotional text to the user, which is achieved using a messaging application.

[0325] "Means for recognizing emotional state by analyzing speech tone and facial expressions" refers to technology that extracts emotional information from a user's voice and visual data to identify the user's current emotional state.

[0326] "Means for tailoring promotional text based on emotional state" refers to techniques for optimizing the content and tone of promotional messages depending on the perceived emotional state of the user.

[0327] This invention is a system incorporating an emotion engine that recognizes user emotions in order to automate telephone reservation responses and sales promotion activities. The main components of this system are a server, a user terminal, an automatic response means, a voice analysis means, an emotion engine, a reservation schedule generation means, a notification means, a monitoring means, an automatic promotional message generation means, and a message transmission means.

[0328] System configuration

[0329] 1. Automated Response Methods

[0330] When the server receives a call from a user, it acts as an automated answering system and responds with a voice message saying, "Welcome. Please tell us your reservation details." Specifically, it uses a calling API such as Twilio.

[0331] 2. Audio analysis methods

[0332] The server records the user's voice in real time and converts it into text using the Google® Cloud Speech-to-Text API. When a user says, "I'd like to make a reservation for two people tomorrow at 6 p.m.", the server converts the content into text data and extracts the necessary information.

[0333] 3. Emotion Engine

[0334] The server analyzes the tone and speed of the voice, and in the case of video calls, facial expressions, to recognize the user's emotional state using the Emotion API from Microsoft® Azure®. For example, if the user speaks in a relaxed tone, the emotion engine determines that the user is happy.

[0335] 4. Reservation Schedule Generation Method

[0336] The server generates a reservation schedule based on the analyzed voice data and the recognized emotional state. This schedule includes reservation dates, times, number of people, and other details. The generated reservation data is stored in a database such as MySQL®.

[0337] 5. Means of notification

[0338] The server uses the LINE API to send a reservation confirmation message to the user's device. For example, a notification might be sent to the user saying, "Your reservation for two people at 6:00 PM tomorrow has been confirmed."

[0339] 6. Monitoring measures

[0340] The server periodically checks the reservation system database to find available time slots by monitoring current reservations, a process automated using Python scripts.

[0341] 7. Free Time Detection

[0342] The server parses the reservation table and finds that there are no reservations between, say, 14:00 and 16:00. This information is stored in an internal data structure (a list or dictionary).

[0343] 8. Automatic generation of promotional text

[0344] The server uses a generative AI model (e.g., GPT-3 (registered trademark)) to automatically generate appropriate promotional text based on available time slots. The following prompt is used: "Based on previous customer information, please generate a promotional text offering a special discount to customers who visit between 2:00 PM and 4:00 PM today."

[0345] 9. Message Transmission Methods

[0346] The server then sends the generated promotional message to all customers via the LINE API, such as "We will offer a special discount to customers who visit our store between 2:00 PM and 4:00 PM today. Please come and visit us."

[0347] Specific examples

[0348] Examples of reservation handling

[0349] When User A calls the store using their mobile phone, the server automatically answers the call and says, "Welcome. Please tell us your reservation details." When User A says, "I'd like to make a reservation for two people tomorrow at 6 PM," the server analyzes the voice and understands the content. At the same time, the emotion engine analyzes the user's tone of voice and recognizes satisfaction. As a result, the server generates a reservation schedule, saves it in the database, and then notifies User A via LINE message, "Tomorrow at 6 PM for two people. Your reservation has been confirmed."

[0350] Specific examples of sales responses

[0351] The server monitors the current reservation status and detects that there are no reservations between 2:00 PM and 4:00 PM. Based on the emotional data of users who have previously visited the store, the server generates promotional messages for customers with positive emotions, such as "We offer a special discount to customers who visit us between 2:00 PM and 4:00 PM today. Please come and visit us," and sends them via the LINE API. As a result, it is expected that more reservations will be made for available time slots.

[0352] In this way, the present invention provides a system that aims to improve the efficiency of reservation handling and customer satisfaction, and realizes effective sales promotion activities.

[0353] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0354] Step 1:

[0355] Receiving calls

[0356] The server receives a call from the user's device. Using a call API such as Twilio, the server receives the call as an automated answering machine.

[0357] Input: A user makes a call from their device (mobile or landline).

[0358] Output: The server receives the user's call.

[0359] What happens: Your server uses the Twilio API to identify the incoming call and begin processing it.

[0360] Step 2:

[0361] Voice response

[0362] When the server receives a call, it acts as an automated answering system and plays a voice prompt saying, "Welcome. Please tell us your reservation details."

[0363] Input: The user's phone input.

[0364] Output: A voice prompt responds to the user.

[0365] Specific operation: Plays the audio guide message prepared by the server.

[0366] Step 3:

[0367] Audio analysis

[0368] The server records the user's speech in real time and converts it into text using the Google Cloud Speech-to-Text API.

[0369] Input: User's voice input.

[0370] Output: Text data as the result of speech analysis.

[0371] How it works: The server collects voice data in real time and converts it into text data using the Google Cloud Speech-to-Text API.

[0372] Step 4:

[0373] Analysis of audio data

[0374] The server analyzes the text data and extracts reservation information (date and time, number of people, special requests, etc.).

[0375] Input: Text data of speech analysis results.

[0376] Output: Reservation information (date, time, number of people, special requests, etc.).

[0377] What it does: The server parses the text data and extracts the required information using regular expressions and natural language processing techniques.

[0378] Step 5:

[0379] emotion recognition

[0380] The server uses Microsoft Azure's Emotion API to analyze the user's emotional state based on voice data and facial expression data from video calls.

[0381] Input: User's speech tone and facial expression data.

[0382] Output: Emotion recognition result (e.g., happy, anxious, etc.).

[0383] What it does: The server collects audio tones and visual data and uses the Emotion API to identify the emotional state.

[0384] Step 6:

[0385] Reservation schedule generation

[0386] The server generates a reservation schedule based on the analyzed voice data and the recognized emotional state and stores it in a database.

[0387] Input: Booking information and emotional state data.

[0388] Output: Appointment schedule.

[0389] Specific operation: The server organizes reservation information and stores it in a database such as MySQL.

[0390] Step 7:

[0391] Confirmation Notice

[0392] The server uses the LINE API to send a reservation confirmation message to the user's device.

[0393] Input: Appointment Schedule.

[0394] Output: Acknowledgement message.

[0395] Specific operation: The server generates a confirmation message based on the reservation schedule and sends it to the user via the LINE API.

[0396] Step 8:

[0397] Reservation status monitoring

[0398] The server periodically checks the reservation system database to monitor the reservation status and detects available time slots.

[0399] Input: Database reservation status.

[0400] Output: Free / busy information.

[0401] What it does: The server uses a Python script to periodically query the reservation data to identify available times.

[0402] Step 9:

[0403] Free Time Detection

[0404] The server analyzes the reservation table and finds that there are no reservations between, say, 2 PM and 4 PM, and stores this information in an internal data structure.

[0405] Input: Database reservation status.

[0406] Output: The free time found.

[0407] Specific operation: The server organizes and stores free time data in a data structure such as a list or dictionary.

[0408] Step 10:

[0409] Automatic sales promotion generation

[0410] The server uses a generative AI model (e.g., GPT-3) to automatically generate appropriate promotional text based on available time.

[0411] Input: Free / busy information and prompt text.

[0412] Output: Auto-generated promotional text.

[0413] Specific behavior: The server generates a promotional message using the following prompt: "Based on previous customer information, please generate a promotional message offering a special discount to customers who visit the store between 2:00 PM and 4:00 PM today."

[0414] Step 11:

[0415] Sending a message

[0416] The server sends the generated promotional message to all customers via the LINE API.

[0417] Input: Auto-generated promotional text.

[0418] Output: Promotional message sent.

[0419] Specific operation: The server uses the LINE API to send the generated promotional message to all customers at once.

[0420] (Application example 2)

[0421] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0422] In conventional reservation systems, when users make reservations over the phone, the burden on the staff who handle the reservations is heavy, resulting in inefficiency. Furthermore, the system often provides a uniform response without considering the user's feelings, resulting in low customer satisfaction. Furthermore, promotional activities to effectively utilize available time are complicated and time-consuming. There is a need to solve these issues and realize efficient reservation management and promotional activities while increasing user satisfaction.

[0423] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0424] In this invention, the server includes an automatic response means for inputting reservation details by voice, a means for analyzing the input voice and generating a reservation schedule for the client, a means for notifying the client of the generated reservation schedule, a means for monitoring the client's reservation status and detecting available time, a means for automatically generating promotional messages based on the detected available time, a means for sending the generated promotional messages to the client, an emotion recognition means as an application installed on the smartphone for generating personalized promotional messages based on prompt messages, and an emotion analysis means for analyzing the user's emotions based on the voice input. This enables effective reservation management and promotional activities that take user emotions into consideration.

[0425] An "automated response means for voice input of reservation details" is part of a system that allows users to input reservation information by voice via a smartphone or other device.

[0426] "Means for analyzing input voice and generating a client's reservation schedule" refers to a system function that uses voice recognition technology to analyze the user's voice data and automatically creates a reservation schedule based on that data.

[0427] The "means for notifying the client of the created reservation schedule" is a notification system for notifying the user of the created reservation schedule, and is a means using a message sending application or the like.

[0428] The "means for monitoring client reservation status and detecting available times" is part of a system that constantly monitors the reservation database and identifies time periods when no reservations have been made.

[0429] The "means for automatically generating promotional messages based on detected free time" is a system function that automatically creates appropriate promotional messages using emotion recognition means when free time is detected.

[0430] The "means for transmitting the generated promotional message to the client" refers to a system for transmitting the generated promotional message to the user, for example, a means for using a message sending application.

[0431] The "application installed on the smartphone" is software that runs on the smartphone and is a platform for executing the various means of the present invention.

[0432] The "emotion recognition means for generating personalized promotional text based on prompt text" is part of a system that includes emotion recognition technology for generating personalized promotional messages based on user input and historical data.

[0433] The "emotion analysis means for analyzing the user's emotions based on the voice input" is a part of the system that analyzes the emotions from the user's voice input and reflects them in the content of the interaction.

[0434] This invention is a system that uses an application installed on a smartphone to improve the efficiency of users' reservation management and promotional activities, thereby increasing user satisfaction.

[0435] Hardware and software used

[0436] The server uses the following hardware and software:

[0437] Smartphone: Used for user voice input and camera photography.

[0438] Speech recognition software: Use the speech_recognition Python package to convert audio data into text.

[0439] Sentiment analysis tools: The TextBlob Python package is used to analyze the sentiment of text, and the face_recognition library is used to recognize the facial expressions of users.

[0440] Database: SQLite is used to manage reservation data.

[0441] Message sending application: Uses the LINE API to send notification messages to users.

[0442] Data processing and calculation

[0443] 1. Voice analysis: When a user voices their reservation details on their smartphone, the voice recognition software converts the voice input into text data. At the same time, an emotion analysis tool also analyzes the emotion of the user's voice tone.

[0444] 2. Reservation schedule generation: After the voice data is converted to text, a reservation schedule is automatically generated. The necessary data (reservation date and time, number of people, etc.) is recorded in a database.

[0445] 3. Notification of reservation schedule: The generated reservation schedule is notified to the user using the LINE API, etc. For example, when a reservation is confirmed, a message is sent saying, "Tomorrow at 6 PM for two people. Your reservation has been confirmed."

[0446] 4. Monitoring and promotional message generation: The server constantly monitors the database to detect available time slots. When available time slots are detected, personalized promotional messages are automatically generated based on the results of sentiment analysis. For example, a message such as "We offer a special discount to customers who visit our store between 2:00 PM and 4:00 PM today. Please come and visit us." is generated. This is also sent to the user using the LINE API.

[0447] Specific examples

[0448] When a user opens the app and says, "Please make a reservation for two people tomorrow at 6 PM," the app uses its emotion engine to analyze the user's emotions and confirms the reservation.The app then uses the LINE API to send the user a message saying, "Tomorrow at 6 PM for two people. Your reservation is confirmed."

[0449] Prompt Sentence Examples

[0450] "I'd like to make a reservation for two people for tomorrow at 6pm."

[0451] "Thank you for your positive attitude! Your reservation is confirmed for two people tomorrow at 6pm."

[0452] "We are offering a special discount to customers who visit between 2pm and 4pm today. Please come and visit us!"

[0453] In this way, it becomes possible to effectively manage reservations and promote sales activities while taking into consideration the feelings of users.

[0454] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0455] Step 1:

[0456] The user enters the reservation details by voice via their smartphone.

[0457] Input: User's voice data

[0458] Specific operation: The user launches the app and says, "Please make a reservation for two people tomorrow at 6 p.m."

[0459] Step 2:

[0460] Voice recognition software on the smartphone converts the speech into text data.

[0461] Input: User's voice data

[0462] Data processing: Convert audio to text using the speech_recognition package.

[0463] Output: Text data (e.g., "Please make a reservation for two people tomorrow at 6 PM.")

[0464] What it does: Speech recognition software analyzes the audio data and converts it into text.

[0465] Step 3:

[0466] Sentiment analysis tools analyze user emotions from text data.

[0467] Input: Text data

[0468] Data Operations: Analyze text sentiment using the TextBlob package.

[0469] Output: Sentiment data (e.g., positive, negative, neutral)

[0470] Specific operation: Analyze the sentiment of text data and obtain analysis results.

[0471] Step 4:

[0472] The server generates a reservation schedule.

[0473] Input: Emotion data and text data

[0474] Data processing: Analyze the reservation details and register them in the schedule database.

[0475] Output: Reservation schedule (e.g. 2023-12-25 18:00, 2 people)

[0476] Specific operation: Organize reservation information based on the analyzed text data and emotion data and store it in a database.

[0477] Step 5:

[0478] The server notifies the user of the generated reservation schedule.

[0479] Input: Appointment Schedule

[0480] Output: Reservation confirmation message (e.g. "Tomorrow at 6pm for two people. Your reservation has been confirmed.")

[0481] Specific operation: A reservation confirmation message is sent to the user using the LINE API.

[0482] Step 6:

[0483] The server monitors the reservation status of the client and detects available times.

[0484] Input: Reservation database

[0485] Data calculation: Scans reservation database to identify available times.

[0486] Output: Free time information (e.g. 2023-12-25 14:00~16:00)

[0487] What it does: Continuously monitors the reservation database to detect times when there are no reservations.

[0488] Step 7:

[0489] The server automatically generates promotional text based on the detected free time.

[0490] Input: Free / busy information and emotion data

[0491] Data processing: Generate personalized promotional text based on the prompt text.

[0492] Output: Promotional message (e.g. "We offer a special discount to customers who visit us between 2pm and 4pm today. Please come and visit us.")

[0493] Specific behavior: Generate appropriate promotional messages by referencing free / busy information and sentiment data.

[0494] Step 8:

[0495] The server sends the generated promotional message to the user.

[0496] Input: Promotional Message

[0497] Output: Promotional message sent

[0498] Specific operation: Send promotional messages to users using LINE API etc.

[0499] In this way, by clearly indicating the specific operations and data processing / calculation performed at each step, it becomes easier to understand how the system works.

[0500] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0501] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0502] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0503] [Second embodiment]

[0504] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0505] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0506] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0507] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0508] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0509] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0510] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0511] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0512] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0513] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0514] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0515] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0516] The present invention relates to a system for automating telephone reservation responses and sales promotion activities. The system mainly comprises a server, a user terminal, an automatic response means, and a voice analysis means.

[0517] Configuring an Automated Response System

[0518] Reservations accepted

[0519] 1. Automated response methods:

[0520] When a user calls the store using their own terminal (for example, a mobile phone or landline), the server acts as an automatic answering means and answers the call. The server responds by saying, "Welcome. Please tell us the details of your reservation."

[0521] 2. Audio analysis methods:

[0522] The server uses speech recognition technology to analyze the user's voice input. For example, if the user says, "I'd like to make a reservation for two people tomorrow at 6 p.m.", the server converts that information into text data and accurately understands it.

[0523] 3. Reservation schedule generation method:

[0524] The server generates a reservation schedule based on the analyzed voice data. The reservation schedule includes the reservation date and time, number of people, and other necessary information. The generated reservation schedule is saved in a database.

[0525] 4. Means of notification:

[0526] The server uses the LINE API or similar to send a reservation confirmation message to the user's device to notify them of the reservation details. For example, a confirmation message such as "Tomorrow at 6 PM for two people. Your reservation has been confirmed."

[0527] Sales support

[0528] 1. Monitoring measures:

[0529] The server constantly monitors the client's reservation status and the store's operational status. The server periodically checks the reservation system database to find available time slots.

[0530] 2. Free time detection:

[0531] By monitoring, the server detects that there are no reservations between 14:00 and 16:00, for example. This free time information is used in the next step.

[0532] 3. Automatic sales promotion text generation method:

[0533] The server uses AI to generate promotional messages based on available times, such as, "We offer a special discount to customers who visit between 2:00 PM and 4:00 PM today. Please come and visit us."

[0534] 4. Message sending method:

[0535] The server sends the generated promotional message to all customers registered in the customer list via the LINE API, increasing the chances that customers will receive the message and become interested.

[0536] Specific examples

[0537] For example, suppose User A calls a restaurant and requests, "I'd like to make a reservation for two people tomorrow at 6 PM." The server automatically receives the call, converts the content into text using a voice recognition system, and generates a reservation schedule. The server then sends a reservation confirmation message to User A via LINE, confirming the reservation.

[0538] The server also detects when the store has fewer reservations and generates promotional messages based on that information, which are then sent to customers via LINE. For example, by sending a message offering a "special discount" when the store is open between 2:00 PM and 4:00 PM, it is possible to attract customers' interest and encourage them to visit the store.

[0539] In this way, the present invention provides a system that reduces the burden of answering telephone calls and allows effective sales promotion activities to be carried out even during idle time.

[0540] The processing flow will be explained below.

[0541] Specific processing steps for reservation handling

[0542] Step 1: User makes a call

[0543] The user calls the store's phone number using their own device (mobile phone or landline).

[0544] Step 2: The server automatically answers the call

[0545] The server acts as an automated answering machine and receives the call, saying, "Welcome. Please tell us your reservation details."

[0546] Step 3: The server analyzes the audio

[0547] The server analyzes the user's speech using an AI speech recognition system and converts the content into text data. For example, "I would like to make a reservation for two people tomorrow at 6 p.m."

[0548] Step 4: The server generates the reservation schedule

[0549] The server generates a reservation schedule based on the textual reservation details, and the generated reservation schedule is saved in a database.

[0550] Step 5: The server confirms the reservation

[0551] The server sends a reservation confirmation message to the user's device via the LINE API. For example, a message like "Tomorrow at 6 PM for two people. Your reservation has been confirmed."

[0552] Specific steps for sales responses

[0553] Step 1: The server monitors the situation

[0554] The server periodically checks the database to monitor client reservation status and store operation status in real time.

[0555] Step 2: The server finds free time

[0556] The server analyzes the reservation data and detects, for example, that there are no reservations between 2:00 p.m. and 4:00 p.m.

[0557] Step 3: The server automatically generates the promotional text

[0558] The server uses the generative AI model to automatically generate promotional messages based on the time when no reservations are available, such as "We offer a special discount to customers who visit between 2:00 PM and 4:00 PM today. Please come and visit us."

[0559] Step 4: The server sends the promotional message

[0560] The server uses the LINE API to send the generated promotional message to all customers registered in the customer list.

[0561] Step 5: Customer receives message

[0562] The customer receives the LINE message on their device and checks the contents.

[0563] Specific examples

[0564] Examples of reservation handling

[0565] Step 1: User A calls the store.

[0566] Step 2: The server will automatically answer the phone and say, "Welcome. Please tell us your reservation details."

[0567] Step 3: User A says, "I'd like to make a reservation for two people tomorrow at 6 PM." The server analyzes the speech and converts it into text.

[0568] Step 4: The server generates a reservation schedule based on the textual reservation details and saves it in the database.

[0569] Step 5: The server notifies User A via the LINE API, "Your reservation for two people at 6 PM tomorrow has been confirmed."

[0570] Specific examples of sales responses

[0571] Step 1: The server monitors the current reservation status.

[0572] Step 2: The server detects that there are no reservations between 2 PM and 4 PM.

[0573] Step 3: The server automatically generates a promotional message saying, "We will offer a special discount to customers who visit our store between 2:00 PM and 4:00 PM today. Please come and visit us."

[0574] Step 4: The server uses the LINE API to send the promotional message to all customers in the customer list.

[0575] Step 5: Customer B receives the LINE message, checks the contents, and then decides to visit the store.

[0576] These are the specific steps involved in handling reservations and sales. This system reduces the burden of answering phone calls during busy periods and allows for effective sales promotion activities during off-peak times.

[0577] Example 1

[0578] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0579] Conventional reservation systems have had issues such as the heavy workload associated with answering the phone and inadequate management of busy and quiet times. Furthermore, promotional activities to effectively fill empty time slots are time-consuming and difficult to carry out efficiently. Furthermore, there is a lack of automated systems to solve these issues, so there is a need for more efficient store operations.

[0580] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0581] In this invention, the server includes an automatic answering means, a means for analyzing input voice and generating a reservation schedule, a means for notifying the client of the generated reservation schedule, a means for monitoring the client's reservation status and detecting free time, a means for using a generative AI model to automatically generate promotional messages based on the detected free time, and a means for using a message sending application to send the generated promotional messages to the client. This makes it possible to automate telephone answering and promotional activities that effectively utilize free time.

[0582] The "automatic response means" is a means for allowing the user to input reservation details by voice, and has the function of accepting voice input and transmitting a guidance message.

[0583] The "voice analysis means" is a means for converting input voice into text data and analyzing the reservation details, and uses voice recognition technology.

[0584] The "reservation schedule generating means" is a means for generating a reservation schedule from the analyzed voice data and storing the necessary information in a database.

[0585] The "notification means" is a means for notifying the client of the created reservation schedule, and uses a message sending application.

[0586] The "monitoring means" is a means for monitoring the reservation status of clients and detecting available time.

[0587] The "promotional text generation means" is a means for automatically generating promotional text using an AI model based on the detected free time.

[0588] The "message sending means" is a means for sending the generated promotional text to the client, and typically uses a message sending application.

[0589] A "generative AI model" is an artificial intelligence model used to automatically generate promotional text, generating appropriate text based on input prompts.

[0590] A "prompt" is a sentence that is used as input to a generative AI model, providing the AI ​​with specific context and information to produce a specific output.

[0591] The present invention relates to a system for automating telephone reservation responses and sales promotion activities. The system mainly comprises a server, a user terminal, an automatic response means, and a voice analysis means.

[0592] Configuring an Automated Response System

[0593] Reservations accepted

[0594] 1. Automated response methods:

[0595] When a user calls the store using their own device (e.g., a mobile phone or landline), the server acts as an automated answering machine and receives the call. The server uses speech synthesis software (e.g., a speech synthesis engine) to respond with, "Welcome. Please tell us the details of your reservation."

[0596] 2. Audio analysis methods:

[0597] The server uses speech recognition technology (e.g., speech recognition API) to analyze the user's voice input. For example, if the user says, "I'd like to make a reservation for two people tomorrow at 6 p.m.", the server converts that content into text data and accurately understands it.

[0598] 3. Reservation schedule generation method:

[0599] The server generates a reservation schedule based on the analyzed voice data. The reservation schedule includes the reservation date and time, the number of people, and other necessary information. The generated reservation schedule is saved in a database (e.g., a relational database).

[0600] 4. Means of notification:

[0601] The server uses a messaging application (e.g., a messaging API) to notify the user of the created reservation schedule. For example, a confirmation message such as "Tomorrow at 6 PM for two people. Your reservation has been confirmed."

[0602] Sales support

[0603] 1. Monitoring measures:

[0604] The server constantly monitors the client's reservation status and the store's operational status. The server periodically checks the database to find available time slots.

[0605] 2. Free time detection method:

[0606] By monitoring, the server detects that there are no reservations between 2 pm and 4 pm, for example. This free time information is used to generate the next promotional message.

[0607] 3. Automatic sales promotion text generation method:

[0608] The server uses a generative AI model (e.g., a language model) to generate promotional messages based on available times, such as, "We offer a special discount to customers who visit our store between 2:00 PM and 4:00 PM today. Please come and visit us."

[0609] 4. Message sending method:

[0610] The server sends the generated promotional message to all customers registered in the customer list via a messaging application (e.g., messaging API). The message received by the customer increases the likelihood that they will be interested.

[0611] Specific examples

[0612] For example, suppose User A calls a restaurant and requests, "I'd like to make a reservation for two people tomorrow at 6 PM." The server automatically receives the call, converts the content into text using a speech recognition API, and generates a reservation schedule using a system using the Django framework. The server then uses a message API to send User A a confirmation message saying, "Tomorrow at 6 PM for two people. Your reservation has been confirmed," and the reservation is confirmed.

[0613] The server also uses a database to detect times when the store has few reservations, and based on that information, it uses a generative AI model to generate promotional messages such as, "We will offer a special discount to customers who visit between 2:00 p.m. and 4:00 p.m. today. Please come and visit us," which are then sent to customers using a messaging API.

[0614] Prompt Sentence Examples

[0615] An example prompt for a generative AI model is:

[0616] "We have openings between 2:00 PM and 4:00 PM. Please come up with some promotional text to attract customers to the store during this time."

[0617] In this way, the present invention provides a system that reduces the burden of answering telephone calls and enables effective sales promotion activities to be carried out even during idle time.

[0618] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0619] Specific processing steps for reservation handling

[0620] Step 1: User makes a call

[0621] How it works: The user dials the store's phone number from their device and presses the call button.

[0622] Input: The action of the user making a phone call.

[0623] Output: The call is connected to the server.

[0624] Step 2: The server receives the call

[0625] What it does: The server launches an automated attendant to receive the call, and uses text-to-speech software to say, "Welcome. Please tell us your reservation details."

[0626] Input: The user's phone connection.

[0627] Output: A voice prompt is played to the user.

[0628] Step 3: User speaks

[0629] Action: The user says, "I'd like to make a reservation for two people tomorrow at 6pm."

[0630] Input: User's voice input.

[0631] Output: The audio data is sent to the server.

[0632] Step 4: The server analyzes the audio

[0633] How it works: The server uses a speech recognition API to convert the user's speech into text data.

[0634] Input: Audio data.

[0635] Output: Reservation information converted to text data.

[0636] Step 5: The server generates the reservation schedule

[0637] Operation: The server generates a reservation schedule based on the text data and saves it in a database.

[0638] Input: Text data.

[0639] Output: Appointment schedule saved in the database.

[0640] Step 6: The server sends a reservation confirmation

[0641] How it works: The server uses the messaging API to send a reservation confirmation message to the user's device.

[0642] Input: The generated appointment schedule.

[0643] Output: A booking confirmation message sent to the user's device.

[0644] Specific steps for sales responses

[0645] Step 1: The server monitors the reservation status

[0646] How it works: The server periodically checks the reservation system database to see which time slots are available.

[0647] Input: Database reservation status.

[0648] Output: Free / busy information.

[0649] Step 2: The server finds free time

[0650] What it does: The server finds available times and uses that information for further processing, for example, checking that there are no reservations between 2 PM and 4 PM.

[0651] Input: Monitoring result data.

[0652] Output: Information about free time detection.

[0653] Step 3: The server automatically generates the promotional text

[0654] How it works: The server uses a generative AI model to generate promotional text based on available times. By providing a prompt, the server can generate a promotional text such as "We are available between 2:00 PM and 4:00 PM. Please come up with a promotional text to encourage customers to visit during this time."

[0655] Input: Free / busy information and prompt statement.

[0656] Output: The generated promotional text.

[0657] Step 4: The server sends the promotional message

[0658] How it works: The server uses the messaging API to send the generated promotional message to all customers on the customer list.

[0659] Input: Generated promotional text and customer list.

[0660] Output: Promotional message sent to customer's device.

[0661] By going through the above steps, the system that automates telephone reservations and promotional activities can be operated efficiently and effectively.

[0662] (Application example 1)

[0663] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0664] Conventional reservation systems not only required a lot of effort to handle phone reservations, but also required manual promotional activities to efficiently fill available time, reducing store operational efficiency. There was also a need to improve the user experience when using smartphones.

[0665] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0666] In this invention, the server includes an automatic response means for inputting reservation details by voice, a means for analyzing the input voice and generating a reservation schedule for the user, a means for notifying the user of the generated reservation schedule, a means for monitoring the reservation status of the user and detecting available time, a means for automatically generating promotional messages based on the detected available time, a means for sending the generated promotional messages to the user, and a means for sending the voice input to the server via a smartphone application. This enables the automation of reservations and efficient promotional activities during available time.

[0667] 1. "Reservation details" refers to information including the date and time the user wishes to make a reservation, the number of people, and other necessary information.

[0668] 2. "Automatic response means that allows voice input" refers to an automatic response system that allows users to input reservation details using voice.

[0669] 3. "Means for analyzing voice" refers to technology for converting input voice data into text data and understanding its content.

[0670] 4. "Means for generating reservation schedules" refers to technology for automatically creating reservation schedules based on reservation details obtained through voice analysis.

[0671] 5. "Means for notifying users of reservation schedules" refers to a system for notifying users of the generated reservation schedules, which is primarily done through a message sending application.

[0672] 6. "Means for monitoring reservation status" refers to a system that constantly checks the current reservation status and detects any available time or changes.

[0673] 7. "Means for detecting available times" refers to technology that monitors reservation status and detects when there are no reservations for a specific time period.

[0674] 8. "Means for automated promotional copy generation" means technology for automatically creating specific promotional messages based on detected available time.

[0675] 9. "Means for sending promotional messages to users" refers to a system for sending generated promotional messages to users, primarily using a message sending application.

[0676] 10. "Means for transmitting voice input to a server via a smartphone application" refers to technology that uses a smartphone application to transmit a user's voice input to a server.

[0677] 11. "Server" refers to the computer system that is the core of the entire system, which handles reservation reception, voice analysis, reservation schedule generation, automatic generation and transmission of promotional materials, etc.

[0678] This invention relates to a system that includes a server, a smartphone application, and artificial intelligence technology for streamlining reservation systems and sales promotion activities in brick-and-mortar stores.

[0679] 1. Voice input of reservation details

[0680] A user attempts to make a reservation using a smartphone application. When the user presses the "Record reservation" button in the smartphone application, the smartphone's microphone is activated and the user voice-records the reservation details.

[0681] 2. Voice analysis and reservation schedule generation

[0682] Once the voice input is complete, the smartphone application sends the voice data to the server. The server uses voice recognition technology to convert the voice data into text data and analyzes the reservation details. This process uses the "speech_recognition" library to convert the voice data into text. Based on the textual reservation details, the server generates a reservation schedule.

[0683] 3. Notification of reservation schedule

[0684] The generated reservation schedule is notified to the user using a messaging application such as the LINE API. For example, a confirmation message such as "Tomorrow at 6 PM for two people. Your reservation has been confirmed" is sent to the user's LINE account.

[0685] 4. Reservation monitoring and availability detection

[0686] The server periodically monitors the reservation system database and manages the store's reservation status. When an available time slot is detected, the information is used in the next step. In this process, the reservation status is periodically checked to detect, for example, that there are no reservations between 2:00 PM and 4:00 PM.

[0687] 5. Automatic generation and sending of promotional text

[0688] The server uses a generative AI model to automatically generate promotional text based on the detected free time. For example, "We offer a special discount to customers who visit the store between 2:00 PM and 4:00 PM today." The generated promotional text is sent to the user via the LINE API.

[0689] Hardware and software used

[0690] 1. Hardware

[0691] Smartphone: Voice input and application execution.

[0692] Server: Speech analysis, reservation schedule generation, notifications, promotional text generation.

[0693] 2. Software

[0694] Smartphone application (iOS, Swift): Records audio and sends it to the server.

[0695] Server (Python, Flask): Speech recognition (speech_recognition library), reservation schedule generation, message sending using LINE API.

[0696] Message sending application (LINE API): Sends reservation confirmation messages and promotional messages to users.

[0697] Specific examples

[0698] For example, a user might say through a smartphone application, "I'd like to make a reservation for two people tomorrow at 6 PM." The smartphone sends this voice data to a server, which analyzes it using voice recognition technology, converts it into text data, and generates a reservation schedule. The server then sends a message to the user via the LINE API, such as "Reservation confirmation: tomorrow at 6 PM for two people." The server also detects available times and creates promotional text using a generative AI model. A promotional text such as "We offer a special discount to customers who visit between 2 PM and 4 PM today" is sent to the user via the LINE API.

[0699] Prompt Sentence Examples

[0700] "User says: I'd like to make a reservation for two people tomorrow at 6pm."

[0701] "Promotional message: We are offering a special discount to customers who visit our store between 2pm and 4pm today."

[0702] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0703] Step 1:

[0704] The user launches the smartphone application and presses the "Record reservation" button. This activates the smartphone's microphone and starts voice input. The user then enters the reservation details by voice. The user's voice data is obtained as input.

[0705] Step 2:

[0706] Once the voice input is complete, the smartphone application sends the voice data to the server. At this time, the voice data file is passed to the server in the format included in the request body. The voice data is sent to the server as input.

[0707] Step 3:

[0708] The server converts the received voice data into text data using the "speech_recognition" library. It receives voice data as input and generates text data as output. At this stage, the voice recognition model analyzes the voice and obtains the reservation details as string information.

[0709] Step 4:

[0710] The server analyzes the textual reservation details and generates a reservation schedule. The server analyzes the text data as input, extracts information such as the reservation date, time, and number of people, and generates a reservation schedule as output. The reservation schedule is saved in a database.

[0711] Step 5:

[0712] The server notifies the user of the generated reservation schedule using the LINE API. Specifically, it sends a message within the LINE app saying, "Tomorrow at 6 PM for two people. Your reservation has been confirmed." It receives the reservation schedule information as input and sends a confirmation message to the user as output.

[0713] Step 6:

[0714] The server periodically monitors the reservation system database. It scans the reservation database and checks for time slots with few reservations. It takes the reservation status data as input and generates a list of available time slots as output.

[0715] Step 7:

[0716] The server uses a generative AI model to automatically generate promotional text based on available time slots. For example, it might say, "We offer a special discount to customers who visit between 2:00 PM and 4:00 PM today." It takes available time slot information as input and generates promotional text as output.

[0717] Step 8:

[0718] The server sends the generated promotional text to all users via the LINE API. It receives the promotional text as input and sends a message to users as output. Specifically, the promotional text is displayed within the LINE app, attracting users' attention and encouraging them to visit the store.

[0719] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0720] This invention is a system incorporating an emotion engine that recognizes user emotions in order to automate telephone reservation responses and sales promotion activities. The main components of this system include a server, a user terminal, an automatic response means, a voice analysis means, and an emotion engine.

[0721] Configuring an Automated Response System

[0722] Reservations accepted

[0723] 1. Automated response methods:

[0724] When a user calls the store using their own device (mobile phone or landline), the server acts as an automatic answering machine and answers the call. The server responds by saying, "Welcome. Please tell us the details of your reservation."

[0725] 2. Audio analysis methods:

[0726] The server uses speech recognition technology to analyze the user's voice input. For example, if the user says, "I'd like to make a reservation for two people tomorrow at 6 p.m.", the server converts that content into text data and understands it.

[0727] 3. Emotion Engine:

[0728] The server is equipped with an emotion engine that recognizes emotions from the user's voice and facial expressions. The emotion engine analyzes the user's tone and speed of speech, and in the case of video calls, facial expressions, to determine the user's emotional state.

[0729] 4. Appointment schedule generation method:

[0730] The server generates a reservation schedule based on the analyzed voice data. The reservation schedule includes the reservation date and time, number of people, and other necessary information. The generated reservation schedule is saved in a database.

[0731] 5. Means of notification:

[0732] The server notifies the user of the created reservation schedule by sending a reservation confirmation message to the user's device using the LINE API, etc. For example, a confirmation message such as "Tomorrow at 6 PM for two people. Your reservation has been confirmed."

[0733] Sales support

[0734] 1. Monitoring measures:

[0735] The server constantly monitors the client's reservation status and the store's operational status. The server periodically checks the reservation system database to detect available time slots.

[0736] 2. Free time detection:

[0737] By monitoring, the server detects, for example, that there are no reservations between 2 p.m. and 4 p.m. This free time information is used in the next step.

[0738] 3. Automatic sales promotion text generation method:

[0739] The server works in conjunction with the emotion engine to automatically generate appropriate promotional messages based on the user's emotional state. For example, if the user is feeling positive, a cheerful message such as "We will offer a special discount to customers who visit our store between 2:00 PM and 4:00 PM today. Please come and visit us." is generated.

[0740] 4. Message sending method:

[0741] The server sends the generated promotional text to all customers registered on the customer list via the LINE API.

[0742] Specific examples

[0743] Examples of reservation handling

[0744] Let's say User A calls a store. The server automatically receives the call and says, "Welcome. Please tell us your reservation details." When User A says, "I'd like to make a reservation for two people tomorrow at 6 PM," the server analyzes the voice, understands the content, and generates a reservation schedule. At the same time, the emotion engine analyzes User A's tone of voice and recognizes that he is satisfied with the confirmed reservation. As a result, the server notifies User A via LINE message, "Tomorrow at 6 PM for two people. Your reservation has been confirmed. Thank you."

[0745] Specific examples of sales responses

[0746] The server monitors the current reservation status of the store and detects that there are no reservations between 2:00 PM and 4:00 PM. Using an emotion engine, the server analyzes the emotional data of users who have previously visited the store and generates promotional messages for customers with positive emotions. For example, it generates a message such as, "We will offer a special discount to customers who visit our store between 2:00 PM and 4:00 PM today. Please come and visit us," and sends it to all customers via the LINE API.

[0747] In this way, the present invention provides a system that reduces the burden of answering the phone and allows for effective sales promotion activities that take into account the emotional state of the customer.

[0748] The processing flow will be explained below.

[0749] Specific processing steps for reservation handling

[0750] Step 1: User makes a call

[0751] The user calls the store's phone number using their own device (mobile phone or landline).

[0752] Step 2: The server automatically answers the call

[0753] The server acts as an automated answering machine and receives the call, saying, "Welcome. Please tell us your reservation details."

[0754] Step 3: The server analyzes the audio

[0755] The server analyzes the user's speech using an AI speech recognition system and converts it into text data. For example, "I would like to make a reservation for two people tomorrow at 6 p.m."

[0756] Step 4: The server performs sentiment analysis

[0757] The server uses an emotion engine to analyze the tone and rate of the user's speech to determine the user's emotional state, for example, whether the user is happy or nervous.

[0758] Step 5: The server generates the reservation schedule

[0759] The server generates a reservation schedule based on the textual reservation details and the results of emotion analysis, and the generated reservation schedule is saved in a database.

[0760] Step 6: The server confirms the reservation

[0761] The server sends a reservation confirmation message to the user's device via the LINE API. For example, a message like "Tomorrow at 6 PM for two people. Your reservation has been confirmed."

[0762] Specific steps for sales responses

[0763] Step 1: The server monitors the situation

[0764] The server periodically checks the database to monitor client reservation status and store operation status in real time.

[0765] Step 2: The server finds free time

[0766] The server monitors and detects, for example, that there are no reservations between 2:00 p.m. and 4:00 p.m.

[0767] Step 3: The server analyzes the emotion data

[0768] The server uses an emotion engine to analyze past customer emotion data and identify customers with positive emotions.

[0769] Step 4: The server automatically generates the promotional text

[0770] The server generates appropriate promotional messages based on the user's emotional state. For example, for customers with positive emotions, it generates a message such as, "We will offer a special discount to customers who visit our store between 2:00 PM and 4:00 PM today. Please come and visit us."

[0771] Step 5: The server sends the promotional message

[0772] The server uses the LINE API to send the generated promotional message to all customers registered in the customer list.

[0773] Step 6: Customer receives message

[0774] The customer receives the LINE message on their device and checks the contents.

[0775] Specific examples

[0776] Examples of reservation handling

[0777] Step 1: User A calls the store.

[0778] Step 2: The server will automatically answer the phone and say, "Welcome. Please tell us your reservation details."

[0779] Step 3: User A says, "I'd like to make a reservation for two people tomorrow at 6 PM." The server analyzes the speech and converts it into text.

[0780] Step 4: The server analyzes the speech tone of User A using the emotion engine and recognizes that the user is satisfied.

[0781] Step 5: The server generates a reservation schedule based on the textual reservation details and the results of sentiment analysis, and saves it in the database.

[0782] Step 6: The server notifies User A via the LINE API, "Your reservation for two people at 6 PM tomorrow has been confirmed. Thank you."

[0783] Specific examples of sales responses

[0784] Step 1: The server monitors the current reservation status of the store.

[0785] Step 2: The server detects that there are no reservations between 2 PM and 4 PM.

[0786] Step 3: The server analyzes past customer emotion data using an emotion engine to identify customers with positive emotions.

[0787] Step 4: The server automatically generates a promotional message saying, "We will offer a special discount to customers who visit our store between 2:00 PM and 4:00 PM today. Please come and visit us."

[0788] Step 5: The server uses the LINE API to send the promotional message to all customers in the customer list.

[0789] Step 6: Customer B receives the LINE message, checks the contents, and then decides to visit the store.

[0790] These are the specific processing steps for handling reservations and sales. This system reduces the burden of answering phone calls during busy periods and enables effective sales promotion activities that take customer emotions into consideration during slower times.

[0791] Example 2

[0792] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0793] Conventional reservation systems require users to make reservations manually by voice, resulting in inefficiency and a significant drain on human resources. Furthermore, promotional activities are conducted without considering the emotional state of the customer, making effective marketing difficult. A system that solves these problems and improves reservation efficiency and customer satisfaction is needed.

[0794] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0795] In this invention, the server includes an automatic response means for allowing reservation details to be input by voice, a means for analyzing the input voice, a means for generating a reservation schedule for a client based on the analyzed voice, a means for saving the generated reservation schedule in a database, a means for notifying the client of the saved reservation schedule, a means for monitoring the client's reservation status and detecting available time, a means for automatically generating promotional messages based on the detected available time, a means for sending the generated promotional messages to the client, a means for analyzing the user's speech tone and facial expressions to recognize the user's emotional state, and a means for adjusting the promotional messages based on the recognized emotional state. This enables efficient reservation handling and effective promotional activities that take the customer's emotional state into consideration.

[0796] "Reservation details" refers to information such as the date and time the user wishes to make a reservation, the number of people, and any special requests.

[0797] "Voice-activated automated response system" refers to an automated response system that allows users to enter reservation details by voice, which allows users to access the system via telephone or other voice input device.

[0798] "Means for analyzing voice" refers to technology that converts voice information entered by the user into text data and understands its content. Specifically, voice recognition technology is used.

[0799] "Means for generating a reservation schedule" refers to a system that extracts necessary reservation information from the analyzed voice data and creates a reservation schedule based on that information.

[0800] "Means for saving in a database" refers to the technology for saving in an appropriate database in order to manage the generated reservation schedule, which will make the accumulation and retrieval of information more efficient.

[0801] The "means for notifying the client" refers to a communication means for notifying the user of the created reservation schedule. For example, it includes a notification method using a messaging application.

[0802] "Means for monitoring reservation status and detecting available times" refers to a system that continuously monitors existing reservation data and identifies time slots that are not booked.

[0803] "Automatic promotional copy generation" refers to technology that automatically creates effective promotional messages based on detected free time, which may involve the use of generative AI models.

[0804] "Means for sending to client" refers to a means for delivering the generated promotional text to the user, which is achieved using a messaging application.

[0805] "Means for recognizing emotional state by analyzing speech tone and facial expressions" refers to technology that extracts emotional information from a user's voice and visual data to identify the user's current emotional state.

[0806] "Means for tailoring promotional text based on emotional state" refers to techniques for optimizing the content and tone of promotional messages depending on the perceived emotional state of the user.

[0807] This invention is a system incorporating an emotion engine that recognizes user emotions in order to automate telephone reservation responses and sales promotion activities. The main components of this system are a server, a user terminal, an automatic response means, a voice analysis means, an emotion engine, a reservation schedule generation means, a notification means, a monitoring means, an automatic promotional message generation means, and a message transmission means.

[0808] System configuration

[0809] 1. Automated Response Methods

[0810] When the server receives a call from a user, it acts as an automated answering system and responds with a voice message saying, "Welcome. Please tell us your reservation details." Specifically, it uses a calling API such as Twilio.

[0811] 2. Audio analysis methods

[0812] The server records the user's voice in real time and converts it to text using the Google Cloud Speech-to-Text API. When a user says, "I'd like to make a reservation for two people tomorrow at 6 p.m.", the server converts the voice to text data and extracts the necessary information.

[0813] 3. Emotion Engine

[0814] The server analyzes the tone and speed of the voice, and in the case of video calls, facial expressions, to recognize the user's emotional state using the Microsoft Azure Emotion API, etc. For example, if the user speaks in a relaxed tone, the emotion engine will determine that the user is happy.

[0815] 4. Reservation Schedule Generation Method

[0816] The server generates a reservation schedule based on the analyzed voice data and the recognized emotional state, including reservation dates and times, number of people, and other details, and stores the generated reservation data in a database such as MySQL.

[0817] 5. Means of notification

[0818] The server uses the LINE API to send a reservation confirmation message to the user's device. For example, a notification might be sent to the user saying, "Your reservation for two people at 6:00 PM tomorrow has been confirmed."

[0819] 6. Monitoring measures

[0820] The server periodically checks the reservation system database to find available time slots by monitoring current reservations, a process automated using Python scripts.

[0821] 7. Free Time Detection

[0822] The server parses the reservation table and finds that there are no reservations between, say, 14:00 and 16:00. This information is stored in an internal data structure (a list or dictionary).

[0823] 8. Automatic generation of promotional text

[0824] The server uses a generative AI model (e.g., GPT-3) to automatically generate appropriate promotional text based on available times, using a prompt like this: "Based on previous customer information, please generate a promotional text offering a special discount to customers who visit between 2:00 PM and 4:00 PM today."

[0825] 9. Message Transmission Methods

[0826] The server then sends the generated promotional message to all customers via the LINE API, such as "We will offer a special discount to customers who visit our store between 2:00 PM and 4:00 PM today. Please come and visit us."

[0827] Specific examples

[0828] Examples of reservation handling

[0829] When User A calls the store using their mobile phone, the server automatically answers the call and says, "Welcome. Please tell us your reservation details." When User A says, "I'd like to make a reservation for two people tomorrow at 6 PM," the server analyzes the voice and understands the content. At the same time, the emotion engine analyzes the user's tone of voice and recognizes satisfaction. As a result, the server generates a reservation schedule, saves it in the database, and then notifies User A via LINE message, "Tomorrow at 6 PM for two people. Your reservation has been confirmed."

[0830] Specific examples of sales responses

[0831] The server monitors the current reservation status and detects that there are no reservations between 2:00 PM and 4:00 PM. Based on the emotional data of users who have previously visited the store, the server generates promotional messages for customers with positive emotions, such as "We offer a special discount to customers who visit us between 2:00 PM and 4:00 PM today. Please come and visit us," and sends them via the LINE API. As a result, it is expected that more reservations will be made for available time slots.

[0832] In this way, the present invention provides a system that aims to improve the efficiency of reservation handling and customer satisfaction, and realizes effective sales promotion activities.

[0833] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0834] Step 1:

[0835] Receiving calls

[0836] The server receives a call from the user's device. Using a call API such as Twilio, the server receives the call as an automated answering machine.

[0837] Input: A user makes a call from their device (mobile or landline).

[0838] Output: The server receives the user's call.

[0839] What happens: Your server uses the Twilio API to identify the incoming call and begin processing it.

[0840] Step 2:

[0841] Voice response

[0842] When the server receives a call, it acts as an automated answering system and plays a voice prompt saying, "Welcome. Please tell us your reservation details."

[0843] Input: The user's phone input.

[0844] Output: A voice prompt responds to the user.

[0845] Specific operation: Plays the audio guide message prepared by the server.

[0846] Step 3:

[0847] Audio analysis

[0848] The server records the user's speech in real time and converts it into text using the Google Cloud Speech-to-Text API.

[0849] Input: User's voice input.

[0850] Output: Text data as the result of speech analysis.

[0851] How it works: The server collects voice data in real time and converts it into text data using the Google Cloud Speech-to-Text API.

[0852] Step 4:

[0853] Analysis of audio data

[0854] The server analyzes the text data and extracts reservation information (date and time, number of people, special requests, etc.).

[0855] Input: Text data of speech analysis results.

[0856] Output: Reservation information (date, time, number of people, special requests, etc.).

[0857] What it does: The server parses the text data and extracts the required information using regular expressions and natural language processing techniques.

[0858] Step 5:

[0859] emotion recognition

[0860] The server uses Microsoft Azure's Emotion API to analyze the user's emotional state based on voice data and facial expression data from video calls.

[0861] Input: User's speech tone and facial expression data.

[0862] Output: Emotion recognition result (e.g., happy, anxious, etc.).

[0863] What it does: The server collects audio tones and visual data and uses the Emotion API to identify the emotional state.

[0864] Step 6:

[0865] Reservation schedule generation

[0866] The server generates a reservation schedule based on the analyzed voice data and the recognized emotional state and stores it in a database.

[0867] Input: Booking information and emotional state data.

[0868] Output: Appointment schedule.

[0869] Specific operation: The server organizes reservation information and stores it in a database such as MySQL.

[0870] Step 7:

[0871] Confirmation Notice

[0872] The server uses the LINE API to send a reservation confirmation message to the user's device.

[0873] Input: Appointment Schedule.

[0874] Output: Acknowledgement message.

[0875] Specific operation: The server generates a confirmation message based on the reservation schedule and sends it to the user via the LINE API.

[0876] Step 8:

[0877] Reservation status monitoring

[0878] The server periodically checks the reservation system database to monitor the reservation status and detects available time slots.

[0879] Input: Database reservation status.

[0880] Output: Free / busy information.

[0881] What it does: The server uses a Python script to periodically query the reservation data to identify available times.

[0882] Step 9:

[0883] Free Time Detection

[0884] The server analyzes the reservation table and finds that there are no reservations between, say, 2 PM and 4 PM, and stores this information in an internal data structure.

[0885] Input: Database reservation status.

[0886] Output: The free time found.

[0887] Specific operation: The server organizes and stores free time data in a data structure such as a list or dictionary.

[0888] Step 10:

[0889] Automatic sales promotion generation

[0890] The server uses a generative AI model (e.g., GPT-3) to automatically generate appropriate promotional text based on available time.

[0891] Input: Free / busy information and prompt text.

[0892] Output: Auto-generated promotional text.

[0893] Specific behavior: The server generates a promotional message using the following prompt: "Based on previous customer information, please generate a promotional message offering a special discount to customers who visit the store between 2:00 PM and 4:00 PM today."

[0894] Step 11:

[0895] Sending a message

[0896] The server sends the generated promotional message to all customers via the LINE API.

[0897] Input: Auto-generated promotional text.

[0898] Output: Promotional message sent.

[0899] Specific operation: The server uses the LINE API to send the generated promotional message to all customers at once.

[0900] (Application example 2)

[0901] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0902] In conventional reservation systems, when users make reservations over the phone, the burden on the staff who handle the reservations is heavy, resulting in inefficiency. Furthermore, the system often provides a uniform response without considering the user's feelings, resulting in low customer satisfaction. Furthermore, promotional activities to effectively utilize available time are complicated and time-consuming. There is a need to solve these issues and realize efficient reservation management and promotional activities while increasing user satisfaction.

[0903] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0904] In this invention, the server includes an automatic response means for inputting reservation details by voice, a means for analyzing the input voice and generating a reservation schedule for the client, a means for notifying the client of the generated reservation schedule, a means for monitoring the client's reservation status and detecting available time, a means for automatically generating promotional messages based on the detected available time, a means for sending the generated promotional messages to the client, an emotion recognition means as an application installed on the smartphone for generating personalized promotional messages based on prompt messages, and an emotion analysis means for analyzing the user's emotions based on the voice input. This enables effective reservation management and promotional activities that take user emotions into consideration.

[0905] An "automated response means for voice input of reservation details" is part of a system that allows users to input reservation information by voice via a smartphone or other device.

[0906] "Means for analyzing input voice and generating a client's reservation schedule" refers to a system function that uses voice recognition technology to analyze the user's voice data and automatically creates a reservation schedule based on that data.

[0907] The "means for notifying the client of the created reservation schedule" is a notification system for notifying the user of the created reservation schedule, and is a means using a message sending application or the like.

[0908] The "means for monitoring client reservation status and detecting available times" is part of a system that constantly monitors the reservation database and identifies time periods when no reservations have been made.

[0909] The "means for automatically generating promotional messages based on detected free time" is a system function that automatically creates appropriate promotional messages using emotion recognition means when free time is detected.

[0910] The "means for transmitting the generated promotional message to the client" refers to a system for transmitting the generated promotional message to the user, for example, a means for using a message sending application.

[0911] The "application installed on the smartphone" is software that runs on the smartphone and is a platform for executing the various means of the present invention.

[0912] The "emotion recognition means for generating personalized promotional text based on prompt text" is part of a system that includes emotion recognition technology for generating personalized promotional messages based on user input and historical data.

[0913] The "emotion analysis means for analyzing the user's emotions based on the voice input" is a part of the system that analyzes the emotions from the user's voice input and reflects them in the content of the interaction.

[0914] This invention is a system that uses an application installed on a smartphone to improve the efficiency of users' reservation management and promotional activities, thereby increasing user satisfaction.

[0915] Hardware and software used

[0916] The server uses the following hardware and software:

[0917] Smartphone: Used for user voice input and camera photography.

[0918] Speech recognition software: Use the speech_recognition Python package to convert audio data into text.

[0919] Sentiment analysis tools: The TextBlob Python package is used to analyze the sentiment of text, and the face_recognition library is used to recognize the facial expressions of users.

[0920] Database: SQLite is used to manage reservation data.

[0921] Message sending application: Uses the LINE API to send notification messages to users.

[0922] Data processing and calculation

[0923] 1. Voice analysis: When a user voices their reservation details on their smartphone, the voice recognition software converts the voice input into text data. At the same time, an emotion analysis tool also analyzes the emotion of the user's voice tone.

[0924] 2. Reservation schedule generation: After the voice data is converted to text, a reservation schedule is automatically generated. The necessary data (reservation date and time, number of people, etc.) is recorded in a database.

[0925] 3. Notification of reservation schedule: The generated reservation schedule is notified to the user using the LINE API, etc. For example, when a reservation is confirmed, a message is sent saying, "Tomorrow at 6 PM for two people. Your reservation has been confirmed."

[0926] 4. Monitoring and promotional message generation: The server constantly monitors the database to detect available time slots. When available time slots are detected, personalized promotional messages are automatically generated based on the results of sentiment analysis. For example, a message such as "We offer a special discount to customers who visit our store between 2:00 PM and 4:00 PM today. Please come and visit us." is generated. This is also sent to the user using the LINE API.

[0927] Specific examples

[0928] When a user opens the app and says, "Please make a reservation for two people tomorrow at 6 PM," the app uses its emotion engine to analyze the user's emotions and confirms the reservation.The app then uses the LINE API to send the user a message saying, "Tomorrow at 6 PM for two people. Your reservation is confirmed."

[0929] Prompt Sentence Examples

[0930] "I'd like to make a reservation for two people for tomorrow at 6pm."

[0931] "Thank you for your positive attitude! Your reservation is confirmed for two people tomorrow at 6pm."

[0932] "We are offering a special discount to customers who visit between 2pm and 4pm today. Please come and visit us!"

[0933] In this way, it becomes possible to effectively manage reservations and promote sales activities while taking into consideration the feelings of users.

[0934] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0935] Step 1:

[0936] The user enters the reservation details by voice via their smartphone.

[0937] Input: User's voice data

[0938] Specific operation: The user launches the app and says, "Please make a reservation for two people tomorrow at 6 p.m."

[0939] Step 2:

[0940] Voice recognition software on the smartphone converts the speech into text data.

[0941] Input: User's voice data

[0942] Data processing: Convert audio to text using the speech_recognition package.

[0943] Output: Text data (e.g., "Please make a reservation for two people tomorrow at 6 PM.")

[0944] What it does: Speech recognition software analyzes the audio data and converts it into text.

[0945] Step 3:

[0946] Sentiment analysis tools analyze user emotions from text data.

[0947] Input: Text data

[0948] Data Operations: Analyze text sentiment using the TextBlob package.

[0949] Output: Sentiment data (e.g., positive, negative, neutral)

[0950] Specific operation: Analyze the sentiment of text data and obtain analysis results.

[0951] Step 4:

[0952] The server generates a reservation schedule.

[0953] Input: Emotion data and text data

[0954] Data processing: Analyze the reservation details and register them in the schedule database.

[0955] Output: Reservation schedule (e.g. 2023-12-25 18:00, 2 people)

[0956] Specific operation: Organize reservation information based on the analyzed text data and emotion data and store it in a database.

[0957] Step 5:

[0958] The server notifies the user of the generated reservation schedule.

[0959] Input: Appointment Schedule

[0960] Output: Reservation confirmation message (e.g. "Tomorrow at 6pm for two people. Your reservation has been confirmed.")

[0961] Specific operation: A reservation confirmation message is sent to the user using the LINE API.

[0962] Step 6:

[0963] The server monitors the reservation status of the client and detects available times.

[0964] Input: Reservation database

[0965] Data calculation: Scans reservation database to identify available times.

[0966] Output: Free time information (e.g. 2023-12-25 14:00~16:00)

[0967] What it does: Continuously monitors the reservation database to detect times when there are no reservations.

[0968] Step 7:

[0969] The server automatically generates promotional text based on the detected free time.

[0970] Input: Free / busy information and emotion data

[0971] Data processing: Generate personalized promotional text based on the prompt text.

[0972] Output: Promotional message (e.g. "We offer a special discount to customers who visit us between 2pm and 4pm today. Please come and visit us.")

[0973] Specific behavior: Generate appropriate promotional messages by referencing free / busy information and sentiment data.

[0974] Step 8:

[0975] The server sends the generated promotional message to the user.

[0976] Input: Promotional Message

[0977] Output: Promotional message sent

[0978] Specific operation: Send promotional messages to users using LINE API etc.

[0979] In this way, by clearly indicating the specific operations and data processing / calculation performed at each step, it becomes easier to understand how the system works.

[0980] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0981] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0982] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0983] [Third embodiment]

[0984] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0985] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0986] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0987] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0988] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0989] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0990] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0991] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0992] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0993] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0994] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0995] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0996] The present invention relates to a system for automating telephone reservation responses and sales promotion activities. The system mainly comprises a server, a user terminal, an automatic response means, and a voice analysis means.

[0997] Configuring an Automated Response System

[0998] Reservations accepted

[0999] 1. Automated response methods:

[1000] When a user calls the store using their own terminal (for example, a mobile phone or landline), the server acts as an automatic answering means and answers the call. The server responds by saying, "Welcome. Please tell us the details of your reservation."

[1001] 2. Audio analysis methods:

[1002] The server uses speech recognition technology to analyze the user's voice input. For example, if the user says, "I'd like to make a reservation for two people tomorrow at 6 p.m.", the server converts that information into text data and accurately understands it.

[1003] 3. Reservation schedule generation method:

[1004] The server generates a reservation schedule based on the analyzed voice data. The reservation schedule includes the reservation date and time, number of people, and other necessary information. The generated reservation schedule is saved in a database.

[1005] 4. Means of notification:

[1006] The server uses the LINE API or similar to send a reservation confirmation message to the user's device to notify them of the reservation details. For example, a confirmation message such as "Tomorrow at 6 PM for two people. Your reservation has been confirmed."

[1007] Sales support

[1008] 1. Monitoring measures:

[1009] The server constantly monitors the client's reservation status and the store's operational status. The server periodically checks the reservation system database to find available time slots.

[1010] 2. Free time detection:

[1011] By monitoring, the server detects that there are no reservations between 14:00 and 16:00, for example. This free time information is used in the next step.

[1012] 3. Automatic sales promotion text generation method:

[1013] The server uses AI to generate promotional messages based on available times, such as, "We offer a special discount to customers who visit between 2:00 PM and 4:00 PM today. Please come and visit us."

[1014] 4. Message sending method:

[1015] The server sends the generated promotional message to all customers registered in the customer list via the LINE API, increasing the chances that customers will receive the message and become interested.

[1016] Specific examples

[1017] For example, suppose User A calls a restaurant and requests, "I'd like to make a reservation for two people tomorrow at 6 PM." The server automatically receives the call, converts the content into text using a voice recognition system, and generates a reservation schedule. The server then sends a reservation confirmation message to User A via LINE, confirming the reservation.

[1018] The server also detects when the store has fewer reservations and generates promotional messages based on that information, which are then sent to customers via LINE. For example, by sending a message offering a "special discount" when the store is open between 2:00 PM and 4:00 PM, it is possible to attract customers' interest and encourage them to visit the store.

[1019] In this way, the present invention provides a system that reduces the burden of answering telephone calls and allows effective sales promotion activities to be carried out even during idle time.

[1020] The processing flow will be explained below.

[1021] Specific processing steps for reservation handling

[1022] Step 1: User makes a call

[1023] The user calls the store's phone number using their own device (mobile phone or landline).

[1024] Step 2: The server automatically answers the call

[1025] The server acts as an automated answering machine and receives the call, saying, "Welcome. Please tell us your reservation details."

[1026] Step 3: The server analyzes the audio

[1027] The server analyzes the user's speech using an AI speech recognition system and converts the content into text data. For example, "I would like to make a reservation for two people tomorrow at 6 p.m."

[1028] Step 4: The server generates the reservation schedule

[1029] The server generates a reservation schedule based on the textual reservation details, and the generated reservation schedule is saved in a database.

[1030] Step 5: The server confirms the reservation

[1031] The server sends a reservation confirmation message to the user's device via the LINE API. For example, a message like "Tomorrow at 6 PM for two people. Your reservation has been confirmed."

[1032] Specific steps for sales responses

[1033] Step 1: The server monitors the situation

[1034] The server periodically checks the database to monitor client reservation status and store operation status in real time.

[1035] Step 2: The server finds free time

[1036] The server analyzes the reservation data and detects, for example, that there are no reservations between 2:00 p.m. and 4:00 p.m.

[1037] Step 3: The server automatically generates the promotional text

[1038] The server uses the generative AI model to automatically generate promotional messages based on the time when no reservations are available, such as "We offer a special discount to customers who visit between 2:00 PM and 4:00 PM today. Please come and visit us."

[1039] Step 4: The server sends the promotional message

[1040] The server uses the LINE API to send the generated promotional message to all customers registered in the customer list.

[1041] Step 5: Customer receives message

[1042] The customer receives the LINE message on their device and checks the contents.

[1043] Specific examples

[1044] Examples of reservation handling

[1045] Step 1: User A calls the store.

[1046] Step 2: The server will automatically answer the phone and say, "Welcome. Please tell us your reservation details."

[1047] Step 3: User A says, "I'd like to make a reservation for two people tomorrow at 6 PM." The server analyzes the speech and converts it into text.

[1048] Step 4: The server generates a reservation schedule based on the textual reservation details and saves it in the database.

[1049] Step 5: The server notifies User A via the LINE API, "Your reservation for two people at 6 PM tomorrow has been confirmed."

[1050] Specific examples of sales responses

[1051] Step 1: The server monitors the current reservation status.

[1052] Step 2: The server detects that there are no reservations between 2 PM and 4 PM.

[1053] Step 3: The server automatically generates a promotional message saying, "We will offer a special discount to customers who visit our store between 2:00 PM and 4:00 PM today. Please come and visit us."

[1054] Step 4: The server uses the LINE API to send the promotional message to all customers in the customer list.

[1055] Step 5: Customer B receives the LINE message, checks the contents, and then decides to visit the store.

[1056] These are the specific steps involved in handling reservations and sales. This system reduces the burden of answering phone calls during busy periods and allows for effective sales promotion activities during off-peak times.

[1057] Example 1

[1058] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1059] Conventional reservation systems have had issues such as the heavy workload associated with answering the phone and inadequate management of busy and quiet times. Furthermore, promotional activities to effectively fill empty time slots are time-consuming and difficult to carry out efficiently. Furthermore, there is a lack of automated systems to solve these issues, so there is a need for more efficient store operations.

[1060] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1061] In this invention, the server includes an automatic answering means, a means for analyzing input voice and generating a reservation schedule, a means for notifying the client of the generated reservation schedule, a means for monitoring the client's reservation status and detecting free time, a means for using a generative AI model to automatically generate promotional messages based on the detected free time, and a means for using a message sending application to send the generated promotional messages to the client. This makes it possible to automate telephone answering and promotional activities that effectively utilize free time.

[1062] The "automatic response means" is a means for allowing the user to input reservation details by voice, and has the function of accepting voice input and transmitting a guidance message.

[1063] The "voice analysis means" is a means for converting input voice into text data and analyzing the reservation details, and uses voice recognition technology.

[1064] The "reservation schedule generating means" is a means for generating a reservation schedule from the analyzed voice data and storing the necessary information in a database.

[1065] The "notification means" is a means for notifying the client of the created reservation schedule, and uses a message sending application.

[1066] The "monitoring means" is a means for monitoring the reservation status of clients and detecting available time.

[1067] The "promotional text generation means" is a means for automatically generating promotional text using an AI model based on the detected free time.

[1068] The "message sending means" is a means for sending the generated promotional text to the client, and typically uses a message sending application.

[1069] A "generative AI model" is an artificial intelligence model used to automatically generate promotional text, generating appropriate text based on input prompts.

[1070] A "prompt" is a sentence that is used as input to a generative AI model, providing the AI ​​with specific context and information to produce a specific output.

[1071] The present invention relates to a system for automating telephone reservation responses and sales promotion activities. The system mainly comprises a server, a user terminal, an automatic response means, and a voice analysis means.

[1072] Configuring an Automated Response System

[1073] Reservations accepted

[1074] 1. Automated response methods:

[1075] When a user calls the store using their own device (e.g., a mobile phone or landline), the server acts as an automated answering machine and receives the call. The server uses speech synthesis software (e.g., a speech synthesis engine) to respond with, "Welcome. Please tell us the details of your reservation."

[1076] 2. Audio analysis methods:

[1077] The server uses speech recognition technology (e.g., speech recognition API) to analyze the user's voice input. For example, if the user says, "I'd like to make a reservation for two people tomorrow at 6 p.m.", the server converts that content into text data and accurately understands it.

[1078] 3. Reservation schedule generation method:

[1079] The server generates a reservation schedule based on the analyzed voice data. The reservation schedule includes the reservation date and time, the number of people, and other necessary information. The generated reservation schedule is saved in a database (e.g., a relational database).

[1080] 4. Means of notification:

[1081] The server uses a messaging application (e.g., a messaging API) to notify the user of the created reservation schedule. For example, a confirmation message such as "Tomorrow at 6 PM for two people. Your reservation has been confirmed."

[1082] Sales support

[1083] 1. Monitoring measures:

[1084] The server constantly monitors the client's reservation status and the store's operational status. The server periodically checks the database to find available time slots.

[1085] 2. Free time detection method:

[1086] By monitoring, the server detects that there are no reservations between 2 pm and 4 pm, for example. This free time information is used to generate the next promotional message.

[1087] 3. Automatic sales promotion text generation method:

[1088] The server uses a generative AI model (e.g., a language model) to generate promotional messages based on available times, such as, "We offer a special discount to customers who visit our store between 2:00 PM and 4:00 PM today. Please come and visit us."

[1089] 4. Message sending method:

[1090] The server sends the generated promotional message to all customers registered in the customer list via a messaging application (e.g., messaging API). The message received by the customer increases the likelihood that they will be interested.

[1091] Specific examples

[1092] For example, suppose User A calls a restaurant and requests, "I'd like to make a reservation for two people tomorrow at 6 PM." The server automatically receives the call, converts the content into text using a speech recognition API, and generates a reservation schedule using a system using the Django framework. The server then uses a message API to send User A a confirmation message saying, "Tomorrow at 6 PM for two people. Your reservation has been confirmed," and the reservation is confirmed.

[1093] The server also uses a database to detect times when the store has few reservations, and based on that information, it uses a generative AI model to generate promotional messages such as, "We will offer a special discount to customers who visit between 2:00 p.m. and 4:00 p.m. today. Please come and visit us," which are then sent to customers using a messaging API.

[1094] Prompt Sentence Examples

[1095] An example prompt for a generative AI model is:

[1096] "We have openings between 2:00 PM and 4:00 PM. Please come up with some promotional text to attract customers to the store during this time."

[1097] In this way, the present invention provides a system that reduces the burden of answering telephone calls and enables effective sales promotion activities to be carried out even during idle time.

[1098] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1099] Specific processing steps for reservation handling

[1100] Step 1: User makes a call

[1101] How it works: The user dials the store's phone number from their device and presses the call button.

[1102] Input: The action of the user making a phone call.

[1103] Output: The call is connected to the server.

[1104] Step 2: The server receives the call

[1105] What it does: The server launches an automated attendant to receive the call, and uses text-to-speech software to say, "Welcome. Please tell us your reservation details."

[1106] Input: The user's phone connection.

[1107] Output: A voice prompt is played to the user.

[1108] Step 3: User speaks

[1109] Action: The user says, "I'd like to make a reservation for two people tomorrow at 6pm."

[1110] Input: User's voice input.

[1111] Output: The audio data is sent to the server.

[1112] Step 4: The server analyzes the audio

[1113] How it works: The server uses a speech recognition API to convert the user's speech into text data.

[1114] Input: Audio data.

[1115] Output: Reservation information converted to text data.

[1116] Step 5: The server generates the reservation schedule

[1117] Operation: The server generates a reservation schedule based on the text data and saves it in a database.

[1118] Input: Text data.

[1119] Output: Appointment schedule saved in the database.

[1120] Step 6: The server sends a reservation confirmation

[1121] How it works: The server uses the messaging API to send a reservation confirmation message to the user's device.

[1122] Input: The generated appointment schedule.

[1123] Output: A booking confirmation message sent to the user's device.

[1124] Specific steps for sales responses

[1125] Step 1: The server monitors the reservation status

[1126] How it works: The server periodically checks the reservation system database to see which time slots are available.

[1127] Input: Database reservation status.

[1128] Output: Free / busy information.

[1129] Step 2: The server finds free time

[1130] What it does: The server finds available times and uses that information for further processing, for example, checking that there are no reservations between 2 PM and 4 PM.

[1131] Input: Monitoring result data.

[1132] Output: Information about free time detection.

[1133] Step 3: The server automatically generates the promotional text

[1134] How it works: The server uses a generative AI model to generate promotional text based on available times. By providing a prompt, the server can generate a promotional text such as "We are available between 2:00 PM and 4:00 PM. Please come up with a promotional text to encourage customers to visit during this time."

[1135] Input: Free / busy information and prompt statement.

[1136] Output: The generated promotional text.

[1137] Step 4: The server sends the promotional message

[1138] How it works: The server uses the messaging API to send the generated promotional message to all customers on the customer list.

[1139] Input: Generated promotional text and customer list.

[1140] Output: Promotional message sent to customer's device.

[1141] By going through the above steps, the system that automates telephone reservations and promotional activities can be operated efficiently and effectively.

[1142] (Application example 1)

[1143] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1144] Conventional reservation systems not only required a lot of effort to handle phone reservations, but also required manual promotional activities to efficiently fill available time, reducing store operational efficiency. There was also a need to improve the user experience when using smartphones.

[1145] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1146] In this invention, the server includes an automatic response means for inputting reservation details by voice, a means for analyzing the input voice and generating a reservation schedule for the user, a means for notifying the user of the generated reservation schedule, a means for monitoring the reservation status of the user and detecting available time, a means for automatically generating promotional messages based on the detected available time, a means for sending the generated promotional messages to the user, and a means for sending the voice input to the server via a smartphone application. This enables the automation of reservations and efficient promotional activities during available time.

[1147] 1. "Reservation details" refers to information including the date and time the user wishes to make a reservation, the number of people, and other necessary information.

[1148] 2. "Automatic response means that allows voice input" refers to an automatic response system that allows users to input reservation details using voice.

[1149] 3. "Means for analyzing voice" refers to technology for converting input voice data into text data and understanding its content.

[1150] 4. "Means for generating reservation schedules" refers to technology for automatically creating reservation schedules based on reservation details obtained through voice analysis.

[1151] 5. "Means for notifying users of reservation schedules" refers to a system for notifying users of the generated reservation schedules, which is primarily done through a message sending application.

[1152] 6. "Means for monitoring reservation status" refers to a system that constantly checks the current reservation status and detects any available time or changes.

[1153] 7. "Means for detecting available times" refers to technology that monitors reservation status and detects when there are no reservations for a specific time period.

[1154] 8. "Means for automated promotional copy generation" means technology for automatically creating specific promotional messages based on detected available time.

[1155] 9. "Means for sending promotional messages to users" refers to a system for sending generated promotional messages to users, primarily using a message sending application.

[1156] 10. "Means for transmitting voice input to a server via a smartphone application" refers to technology that uses a smartphone application to transmit a user's voice input to a server.

[1157] 11. "Server" refers to the computer system that is the core of the entire system, which handles reservation reception, voice analysis, reservation schedule generation, automatic generation and transmission of promotional materials, etc.

[1158] This invention relates to a system that includes a server, a smartphone application, and artificial intelligence technology for streamlining reservation systems and sales promotion activities in brick-and-mortar stores.

[1159] 1. Voice input of reservation details

[1160] A user attempts to make a reservation using a smartphone application. When the user presses the "Record reservation" button in the smartphone application, the smartphone's microphone is activated and the user voice-records the reservation details.

[1161] 2. Voice analysis and reservation schedule generation

[1162] Once the voice input is complete, the smartphone application sends the voice data to the server. The server uses voice recognition technology to convert the voice data into text data and analyzes the reservation details. This process uses the "speech_recognition" library to convert the voice data into text. Based on the textual reservation details, the server generates a reservation schedule.

[1163] 3. Notification of reservation schedule

[1164] The generated reservation schedule is notified to the user using a messaging application such as the LINE API. For example, a confirmation message such as "Tomorrow at 6 PM for two people. Your reservation has been confirmed" is sent to the user's LINE account.

[1165] 4. Reservation monitoring and availability detection

[1166] The server periodically monitors the reservation system database and manages the store's reservation status. When an available time slot is detected, the information is used in the next step. In this process, the reservation status is periodically checked to detect, for example, that there are no reservations between 2:00 PM and 4:00 PM.

[1167] 5. Automatic generation and sending of promotional text

[1168] The server uses a generative AI model to automatically generate promotional text based on the detected free time. For example, "We offer a special discount to customers who visit the store between 2:00 PM and 4:00 PM today." The generated promotional text is sent to the user via the LINE API.

[1169] Hardware and software used

[1170] 1. Hardware

[1171] Smartphone: Voice input and application execution.

[1172] Server: Speech analysis, reservation schedule generation, notifications, promotional text generation.

[1173] 2. Software

[1174] Smartphone application (iOS, Swift): Records audio and sends it to the server.

[1175] Server (Python, Flask): Speech recognition (speech_recognition library), reservation schedule generation, message sending using LINE API.

[1176] Message sending application (LINE API): Sends reservation confirmation messages and promotional messages to users.

[1177] Specific examples

[1178] For example, a user might say through a smartphone application, "I'd like to make a reservation for two people tomorrow at 6 PM." The smartphone sends this voice data to a server, which analyzes it using voice recognition technology, converts it into text data, and generates a reservation schedule. The server then sends a message to the user via the LINE API, such as "Reservation confirmation: tomorrow at 6 PM for two people." The server also detects available times and creates promotional text using a generative AI model. A promotional text such as "We offer a special discount to customers who visit between 2 PM and 4 PM today" is sent to the user via the LINE API.

[1179] Prompt Sentence Examples

[1180] "User says: I'd like to make a reservation for two people tomorrow at 6pm."

[1181] "Promotional message: We are offering a special discount to customers who visit our store between 2pm and 4pm today."

[1182] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1183] Step 1:

[1184] The user launches the smartphone application and presses the "Record reservation" button. This activates the smartphone's microphone and starts voice input. The user then enters the reservation details by voice. The user's voice data is obtained as input.

[1185] Step 2:

[1186] Once the voice input is complete, the smartphone application sends the voice data to the server. At this time, the voice data file is passed to the server in the format included in the request body. The voice data is sent to the server as input.

[1187] Step 3:

[1188] The server converts the received voice data into text data using the "speech_recognition" library. It receives voice data as input and generates text data as output. At this stage, the voice recognition model analyzes the voice and obtains the reservation details as string information.

[1189] Step 4:

[1190] The server analyzes the textual reservation details and generates a reservation schedule. The server analyzes the text data as input, extracts information such as the reservation date, time, and number of people, and generates a reservation schedule as output. The reservation schedule is saved in a database.

[1191] Step 5:

[1192] The server notifies the user of the generated reservation schedule using the LINE API. Specifically, it sends a message within the LINE app saying, "Tomorrow at 6 PM for two people. Your reservation has been confirmed." It receives the reservation schedule information as input and sends a confirmation message to the user as output.

[1193] Step 6:

[1194] The server periodically monitors the reservation system database. It scans the reservation database and checks for time slots with few reservations. It takes the reservation status data as input and generates a list of available time slots as output.

[1195] Step 7:

[1196] The server uses a generative AI model to automatically generate promotional text based on available time slots. For example, it might say, "We offer a special discount to customers who visit between 2:00 PM and 4:00 PM today." It takes available time slot information as input and generates promotional text as output.

[1197] Step 8:

[1198] The server sends the generated promotional text to all users via the LINE API. It receives the promotional text as input and sends a message to users as output. Specifically, the promotional text is displayed within the LINE app, attracting users' attention and encouraging them to visit the store.

[1199] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1200] This invention is a system incorporating an emotion engine that recognizes user emotions in order to automate telephone reservation responses and sales promotion activities. The main components of this system include a server, a user terminal, an automatic response means, a voice analysis means, and an emotion engine.

[1201] Configuring an Automated Response System

[1202] Reservations accepted

[1203] 1. Automated response methods:

[1204] When a user calls the store using their own device (mobile phone or landline), the server acts as an automatic answering machine and answers the call. The server responds by saying, "Welcome. Please tell us the details of your reservation."

[1205] 2. Audio analysis methods:

[1206] The server uses speech recognition technology to analyze the user's voice input. For example, if the user says, "I'd like to make a reservation for two people tomorrow at 6 p.m.", the server converts that content into text data and understands it.

[1207] 3. Emotion Engine:

[1208] The server is equipped with an emotion engine that recognizes emotions from the user's voice and facial expressions. The emotion engine analyzes the user's tone and speed of speech, and in the case of video calls, facial expressions, to determine the user's emotional state.

[1209] 4. Appointment schedule generation method:

[1210] The server generates a reservation schedule based on the analyzed voice data. The reservation schedule includes the reservation date and time, number of people, and other necessary information. The generated reservation schedule is saved in a database.

[1211] 5. Means of notification:

[1212] The server notifies the user of the created reservation schedule by sending a reservation confirmation message to the user's device using the LINE API, etc. For example, a confirmation message such as "Tomorrow at 6 PM for two people. Your reservation has been confirmed."

[1213] Sales support

[1214] 1. Monitoring measures:

[1215] The server constantly monitors the client's reservation status and the store's operational status. The server periodically checks the reservation system database to detect available time slots.

[1216] 2. Free time detection:

[1217] By monitoring, the server detects, for example, that there are no reservations between 2 p.m. and 4 p.m. This free time information is used in the next step.

[1218] 3. Automatic sales promotion text generation method:

[1219] The server works in conjunction with the emotion engine to automatically generate appropriate promotional messages based on the user's emotional state. For example, if the user is feeling positive, a cheerful message such as "We will offer a special discount to customers who visit our store between 2:00 PM and 4:00 PM today. Please come and visit us." is generated.

[1220] 4. Message sending method:

[1221] The server sends the generated promotional text to all customers registered on the customer list via the LINE API.

[1222] Specific examples

[1223] Examples of reservation handling

[1224] Let's say User A calls a store. The server automatically receives the call and says, "Welcome. Please tell us your reservation details." When User A says, "I'd like to make a reservation for two people tomorrow at 6 PM," the server analyzes the voice, understands the content, and generates a reservation schedule. At the same time, the emotion engine analyzes User A's tone of voice and recognizes that he is satisfied with the confirmed reservation. As a result, the server notifies User A via LINE message, "Tomorrow at 6 PM for two people. Your reservation has been confirmed. Thank you."

[1225] Specific examples of sales responses

[1226] The server monitors the current reservation status of the store and detects that there are no reservations between 2:00 PM and 4:00 PM. Using an emotion engine, the server analyzes the emotional data of users who have previously visited the store and generates promotional messages for customers with positive emotions. For example, it generates a message such as, "We will offer a special discount to customers who visit our store between 2:00 PM and 4:00 PM today. Please come and visit us," and sends it to all customers via the LINE API.

[1227] In this way, the present invention provides a system that reduces the burden of answering the phone and allows for effective sales promotion activities that take into account the emotional state of the customer.

[1228] The processing flow will be explained below.

[1229] Specific processing steps for reservation handling

[1230] Step 1: User makes a call

[1231] The user calls the store's phone number using their own device (mobile phone or landline).

[1232] Step 2: The server automatically answers the call

[1233] The server acts as an automated answering machine and receives the call, saying, "Welcome. Please tell us your reservation details."

[1234] Step 3: The server analyzes the audio

[1235] The server analyzes the user's speech using an AI speech recognition system and converts it into text data. For example, "I would like to make a reservation for two people tomorrow at 6 p.m."

[1236] Step 4: The server performs sentiment analysis

[1237] The server uses an emotion engine to analyze the tone and rate of the user's speech to determine the user's emotional state, for example, whether the user is happy or nervous.

[1238] Step 5: The server generates the reservation schedule

[1239] The server generates a reservation schedule based on the textual reservation details and the results of emotion analysis, and the generated reservation schedule is saved in a database.

[1240] Step 6: The server confirms the reservation

[1241] The server sends a reservation confirmation message to the user's device via the LINE API. For example, a message like "Tomorrow at 6 PM for two people. Your reservation has been confirmed."

[1242] Specific steps for sales responses

[1243] Step 1: The server monitors the situation

[1244] The server periodically checks the database to monitor client reservation status and store operation status in real time.

[1245] Step 2: The server finds free time

[1246] The server monitors and detects, for example, that there are no reservations between 2:00 p.m. and 4:00 p.m.

[1247] Step 3: The server analyzes the emotion data

[1248] The server uses an emotion engine to analyze past customer emotion data and identify customers with positive emotions.

[1249] Step 4: The server automatically generates the promotional text

[1250] The server generates appropriate promotional messages based on the user's emotional state. For example, for customers with positive emotions, it generates a message such as, "We will offer a special discount to customers who visit our store between 2:00 PM and 4:00 PM today. Please come and visit us."

[1251] Step 5: The server sends the promotional message

[1252] The server uses the LINE API to send the generated promotional message to all customers registered in the customer list.

[1253] Step 6: Customer receives message

[1254] The customer receives the LINE message on their device and checks the contents.

[1255] Specific examples

[1256] Examples of reservation handling

[1257] Step 1: User A calls the store.

[1258] Step 2: The server will automatically answer the phone and say, "Welcome. Please tell us your reservation details."

[1259] Step 3: User A says, "I'd like to make a reservation for two people tomorrow at 6 PM." The server analyzes the speech and converts it into text.

[1260] Step 4: The server analyzes the speech tone of User A using the emotion engine and recognizes that the user is satisfied.

[1261] Step 5: The server generates a reservation schedule based on the textual reservation details and the results of sentiment analysis, and saves it in the database.

[1262] Step 6: The server notifies User A via the LINE API, "Your reservation for two people at 6 PM tomorrow has been confirmed. Thank you."

[1263] Specific examples of sales responses

[1264] Step 1: The server monitors the current reservation status of the store.

[1265] Step 2: The server detects that there are no reservations between 2 PM and 4 PM.

[1266] Step 3: The server analyzes past customer emotion data using an emotion engine to identify customers with positive emotions.

[1267] Step 4: The server automatically generates a promotional message saying, "We will offer a special discount to customers who visit our store between 2:00 PM and 4:00 PM today. Please come and visit us."

[1268] Step 5: The server uses the LINE API to send the promotional message to all customers in the customer list.

[1269] Step 6: Customer B receives the LINE message, checks the contents, and then decides to visit the store.

[1270] These are the specific processing steps for handling reservations and sales. This system reduces the burden of answering phone calls during busy periods and enables effective sales promotion activities that take customer emotions into consideration during slower times.

[1271] Example 2

[1272] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1273] Conventional reservation systems require users to make reservations manually by voice, resulting in inefficiency and a significant drain on human resources. Furthermore, promotional activities are conducted without considering the emotional state of the customer, making effective marketing difficult. A system that solves these problems and improves reservation efficiency and customer satisfaction is needed.

[1274] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1275] In this invention, the server includes an automatic response means for allowing reservation details to be input by voice, a means for analyzing the input voice, a means for generating a reservation schedule for a client based on the analyzed voice, a means for saving the generated reservation schedule in a database, a means for notifying the client of the saved reservation schedule, a means for monitoring the client's reservation status and detecting available time, a means for automatically generating promotional messages based on the detected available time, a means for sending the generated promotional messages to the client, a means for analyzing the user's speech tone and facial expressions to recognize the user's emotional state, and a means for adjusting the promotional messages based on the recognized emotional state. This enables efficient reservation handling and effective promotional activities that take the customer's emotional state into consideration.

[1276] "Reservation details" refers to information such as the date and time the user wishes to make a reservation, the number of people, and any special requests.

[1277] "Voice-activated automated response system" refers to an automated response system that allows users to enter reservation details by voice, which allows users to access the system via telephone or other voice input device.

[1278] "Means for analyzing voice" refers to technology that converts voice information entered by the user into text data and understands its content. Specifically, voice recognition technology is used.

[1279] "Means for generating a reservation schedule" refers to a system that extracts necessary reservation information from the analyzed voice data and creates a reservation schedule based on that information.

[1280] "Means for saving in a database" refers to the technology for saving in an appropriate database in order to manage the generated reservation schedule, which will make the accumulation and retrieval of information more efficient.

[1281] The "means for notifying the client" refers to a communication means for notifying the user of the created reservation schedule. For example, it includes a notification method using a messaging application.

[1282] "Means for monitoring reservation status and detecting available times" refers to a system that continuously monitors existing reservation data and identifies time slots that are not booked.

[1283] "Automatic promotional copy generation" refers to technology that automatically creates effective promotional messages based on detected free time, which may involve the use of generative AI models.

[1284] "Means for sending to client" refers to a means for delivering the generated promotional text to the user, which is achieved using a messaging application.

[1285] "Means for recognizing emotional state by analyzing speech tone and facial expressions" refers to technology that extracts emotional information from a user's voice and visual data to identify the user's current emotional state.

[1286] "Means for tailoring promotional text based on emotional state" refers to techniques for optimizing the content and tone of promotional messages depending on the perceived emotional state of the user.

[1287] This invention is a system incorporating an emotion engine that recognizes user emotions in order to automate telephone reservation responses and sales promotion activities. The main components of this system are a server, a user terminal, an automatic response means, a voice analysis means, an emotion engine, a reservation schedule generation means, a notification means, a monitoring means, an automatic promotional message generation means, and a message transmission means.

[1288] System configuration

[1289] 1. Automated Response Methods

[1290] When the server receives a call from a user, it acts as an automated answering system and responds with a voice message saying, "Welcome. Please tell us your reservation details." Specifically, it uses a calling API such as Twilio.

[1291] 2. Audio analysis methods

[1292] The server records the user's voice in real time and converts it to text using the Google Cloud Speech-to-Text API. When a user says, "I'd like to make a reservation for two people tomorrow at 6 p.m.", the server converts the voice to text data and extracts the necessary information.

[1293] 3. Emotion Engine

[1294] The server analyzes the tone and speed of the voice, and in the case of video calls, facial expressions, to recognize the user's emotional state using the Microsoft Azure Emotion API, etc. For example, if the user speaks in a relaxed tone, the emotion engine will determine that the user is happy.

[1295] 4. Reservation Schedule Generation Method

[1296] The server generates a reservation schedule based on the analyzed voice data and the recognized emotional state, including reservation dates and times, number of people, and other details, and stores the generated reservation data in a database such as MySQL.

[1297] 5. Means of notification

[1298] The server uses the LINE API to send a reservation confirmation message to the user's device. For example, a notification might be sent to the user saying, "Your reservation for two people at 6:00 PM tomorrow has been confirmed."

[1299] 6. Monitoring measures

[1300] The server periodically checks the reservation system database to find available time slots by monitoring current reservations, a process automated using Python scripts.

[1301] 7. Free Time Detection

[1302] The server parses the reservation table and finds that there are no reservations between, say, 14:00 and 16:00. This information is stored in an internal data structure (a list or dictionary).

[1303] 8. Automatic generation of promotional text

[1304] The server uses a generative AI model (e.g., GPT-3) to automatically generate appropriate promotional text based on available times, using a prompt like this: "Based on previous customer information, please generate a promotional text offering a special discount to customers who visit between 2:00 PM and 4:00 PM today."

[1305] 9. Message Transmission Methods

[1306] The server then sends the generated promotional message to all customers via the LINE API, such as "We will offer a special discount to customers who visit our store between 2:00 PM and 4:00 PM today. Please come and visit us."

[1307] Specific examples

[1308] Examples of reservation handling

[1309] When User A calls the store using their mobile phone, the server automatically answers the call and says, "Welcome. Please tell us your reservation details." When User A says, "I'd like to make a reservation for two people tomorrow at 6 PM," the server analyzes the voice and understands the content. At the same time, the emotion engine analyzes the user's tone of voice and recognizes satisfaction. As a result, the server generates a reservation schedule, saves it in the database, and then notifies User A via LINE message, "Tomorrow at 6 PM for two people. Your reservation has been confirmed."

[1310] Specific examples of sales responses

[1311] The server monitors the current reservation status and detects that there are no reservations between 2:00 PM and 4:00 PM. Based on the emotional data of users who have previously visited the store, the server generates promotional messages for customers with positive emotions, such as "We offer a special discount to customers who visit us between 2:00 PM and 4:00 PM today. Please come and visit us," and sends them via the LINE API. As a result, it is expected that more reservations will be made for available time slots.

[1312] In this way, the present invention provides a system that aims to improve the efficiency of reservation handling and customer satisfaction, and realizes effective sales promotion activities.

[1313] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1314] Step 1:

[1315] Receiving calls

[1316] The server receives a call from the user's device. Using a call API such as Twilio, the server receives the call as an automated answering machine.

[1317] Input: A user makes a call from their device (mobile or landline).

[1318] Output: The server receives the user's call.

[1319] What happens: Your server uses the Twilio API to identify the incoming call and begin processing it.

[1320] Step 2:

[1321] Voice response

[1322] When the server receives a call, it acts as an automated answering system and plays a voice prompt saying, "Welcome. Please tell us your reservation details."

[1323] Input: The user's phone input.

[1324] Output: A voice prompt responds to the user.

[1325] Specific operation: Plays the audio guide message prepared by the server.

[1326] Step 3:

[1327] Audio analysis

[1328] The server records the user's speech in real time and converts it into text using the Google Cloud Speech-to-Text API.

[1329] Input: User's voice input.

[1330] Output: Text data as the result of speech analysis.

[1331] How it works: The server collects voice data in real time and converts it into text data using the Google Cloud Speech-to-Text API.

[1332] Step 4:

[1333] Analysis of audio data

[1334] The server analyzes the text data and extracts reservation information (date and time, number of people, special requests, etc.).

[1335] Input: Text data of speech analysis results.

[1336] Output: Reservation information (date, time, number of people, special requests, etc.).

[1337] What it does: The server parses the text data and extracts the required information using regular expressions and natural language processing techniques.

[1338] Step 5:

[1339] emotion recognition

[1340] The server uses Microsoft Azure's Emotion API to analyze the user's emotional state based on voice data and facial expression data from video calls.

[1341] Input: User's speech tone and facial expression data.

[1342] Output: Emotion recognition result (e.g., happy, anxious, etc.).

[1343] What it does: The server collects audio tones and visual data and uses the Emotion API to identify the emotional state.

[1344] Step 6:

[1345] Reservation schedule generation

[1346] The server generates a reservation schedule based on the analyzed voice data and the recognized emotional state and stores it in a database.

[1347] Input: Booking information and emotional state data.

[1348] Output: Appointment schedule.

[1349] Specific operation: The server organizes reservation information and stores it in a database such as MySQL.

[1350] Step 7:

[1351] Confirmation Notice

[1352] The server uses the LINE API to send a reservation confirmation message to the user's device.

[1353] Input: Appointment Schedule.

[1354] Output: Acknowledgement message.

[1355] Specific operation: The server generates a confirmation message based on the reservation schedule and sends it to the user via the LINE API.

[1356] Step 8:

[1357] Reservation status monitoring

[1358] The server periodically checks the reservation system database to monitor the reservation status and detects available time slots.

[1359] Input: Database reservation status.

[1360] Output: Free / busy information.

[1361] What it does: The server uses a Python script to periodically query the reservation data to identify available times.

[1362] Step 9:

[1363] Free Time Detection

[1364] The server analyzes the reservation table and finds that there are no reservations between, say, 2 PM and 4 PM, and stores this information in an internal data structure.

[1365] Input: Database reservation status.

[1366] Output: The free time found.

[1367] Specific operation: The server organizes and stores free time data in a data structure such as a list or dictionary.

[1368] Step 10:

[1369] Automatic sales promotion generation

[1370] The server uses a generative AI model (e.g., GPT-3) to automatically generate appropriate promotional text based on available time.

[1371] Input: Free / busy information and prompt text.

[1372] Output: Auto-generated promotional text.

[1373] Specific behavior: The server generates a promotional message using the following prompt: "Based on previous customer information, please generate a promotional message offering a special discount to customers who visit the store between 2:00 PM and 4:00 PM today."

[1374] Step 11:

[1375] Sending a message

[1376] The server sends the generated promotional message to all customers via the LINE API.

[1377] Input: Auto-generated promotional text.

[1378] Output: Promotional message sent.

[1379] Specific operation: The server uses the LINE API to send the generated promotional message to all customers at once.

[1380] (Application example 2)

[1381] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1382] In conventional reservation systems, when users make reservations over the phone, the burden on the staff who handle the reservations is heavy, resulting in inefficiency. Furthermore, the system often provides a uniform response without considering the user's feelings, resulting in low customer satisfaction. Furthermore, promotional activities to effectively utilize available time are complicated and time-consuming. There is a need to solve these issues and realize efficient reservation management and promotional activities while increasing user satisfaction.

[1383] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1384] In this invention, the server includes an automatic response means for inputting reservation details by voice, a means for analyzing the input voice and generating a reservation schedule for the client, a means for notifying the client of the generated reservation schedule, a means for monitoring the client's reservation status and detecting available time, a means for automatically generating promotional messages based on the detected available time, a means for sending the generated promotional messages to the client, an emotion recognition means as an application installed on the smartphone for generating personalized promotional messages based on prompt messages, and an emotion analysis means for analyzing the user's emotions based on the voice input. This enables effective reservation management and promotional activities that take user emotions into consideration.

[1385] An "automated response means for voice input of reservation details" is part of a system that allows users to input reservation information by voice via a smartphone or other device.

[1386] "Means for analyzing input voice and generating a client's reservation schedule" refers to a system function that uses voice recognition technology to analyze the user's voice data and automatically creates a reservation schedule based on that data.

[1387] The "means for notifying the client of the created reservation schedule" is a notification system for notifying the user of the created reservation schedule, and is a means using a message sending application or the like.

[1388] The "means for monitoring client reservation status and detecting available times" is part of a system that constantly monitors the reservation database and identifies time periods when no reservations have been made.

[1389] The "means for automatically generating promotional messages based on detected free time" is a system function that automatically creates appropriate promotional messages using emotion recognition means when free time is detected.

[1390] The "means for transmitting the generated promotional message to the client" refers to a system for transmitting the generated promotional message to the user, for example, a means for using a message sending application.

[1391] The "application installed on the smartphone" is software that runs on the smartphone and is a platform for executing the various means of the present invention.

[1392] The "emotion recognition means for generating personalized promotional text based on prompt text" is part of a system that includes emotion recognition technology for generating personalized promotional messages based on user input and historical data.

[1393] The "emotion analysis means for analyzing the user's emotions based on the voice input" is a part of the system that analyzes the emotions from the user's voice input and reflects them in the content of the interaction.

[1394] This invention is a system that uses an application installed on a smartphone to improve the efficiency of users' reservation management and promotional activities, thereby increasing user satisfaction.

[1395] Hardware and software used

[1396] The server uses the following hardware and software:

[1397] Smartphone: Used for user voice input and camera photography.

[1398] Speech recognition software: Use the speech_recognition Python package to convert audio data into text.

[1399] Sentiment analysis tools: The TextBlob Python package is used to analyze the sentiment of text, and the face_recognition library is used to recognize the facial expressions of users.

[1400] Database: SQLite is used to manage reservation data.

[1401] Message sending application: Uses the LINE API to send notification messages to users.

[1402] Data processing and calculation

[1403] 1. Voice analysis: When a user voices their reservation details on their smartphone, the voice recognition software converts the voice input into text data. At the same time, an emotion analysis tool also analyzes the emotion of the user's voice tone.

[1404] 2. Reservation schedule generation: After the voice data is converted to text, a reservation schedule is automatically generated. The necessary data (reservation date and time, number of people, etc.) is recorded in a database.

[1405] 3. Notification of reservation schedule: The generated reservation schedule is notified to the user using the LINE API, etc. For example, when a reservation is confirmed, a message is sent saying, "Tomorrow at 6 PM for two people. Your reservation has been confirmed."

[1406] 4. Monitoring and promotional message generation: The server constantly monitors the database to detect available time slots. When available time slots are detected, personalized promotional messages are automatically generated based on the results of sentiment analysis. For example, a message such as "We offer a special discount to customers who visit our store between 2:00 PM and 4:00 PM today. Please come and visit us." is generated. This is also sent to the user using the LINE API.

[1407] Specific examples

[1408] When a user opens the app and says, "Please make a reservation for two people tomorrow at 6 PM," the app uses its emotion engine to analyze the user's emotions and confirms the reservation.The app then uses the LINE API to send the user a message saying, "Tomorrow at 6 PM for two people. Your reservation is confirmed."

[1409] Prompt Sentence Examples

[1410] "I'd like to make a reservation for two people for tomorrow at 6pm."

[1411] "Thank you for your positive attitude! Your reservation is confirmed for two people tomorrow at 6pm."

[1412] "We are offering a special discount to customers who visit between 2pm and 4pm today. Please come and visit us!"

[1413] In this way, it becomes possible to effectively manage reservations and promote sales activities while taking into consideration the feelings of users.

[1414] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1415] Step 1:

[1416] The user enters the reservation details by voice via their smartphone.

[1417] Input: User's voice data

[1418] Specific operation: The user launches the app and says, "Please make a reservation for two people tomorrow at 6 p.m."

[1419] Step 2:

[1420] Voice recognition software on the smartphone converts the speech into text data.

[1421] Input: User's voice data

[1422] Data processing: Convert audio to text using the speech_recognition package.

[1423] Output: Text data (e.g., "Please make a reservation for two people tomorrow at 6 PM.")

[1424] What it does: Speech recognition software analyzes the audio data and converts it into text.

[1425] Step 3:

[1426] Sentiment analysis tools analyze user emotions from text data.

[1427] Input: Text data

[1428] Data Operations: Analyze text sentiment using the TextBlob package.

[1429] Output: Sentiment data (e.g., positive, negative, neutral)

[1430] Specific operation: Analyze the sentiment of text data and obtain analysis results.

[1431] Step 4:

[1432] The server generates a reservation schedule.

[1433] Input: Emotion data and text data

[1434] Data processing: Analyze the reservation details and register them in the schedule database.

[1435] Output: Reservation schedule (e.g. 2023-12-25 18:00, 2 people)

[1436] Specific operation: Organize reservation information based on the analyzed text data and emotion data and store it in a database.

[1437] Step 5:

[1438] The server notifies the user of the generated reservation schedule.

[1439] Input: Appointment Schedule

[1440] Output: Reservation confirmation message (e.g. "Tomorrow at 6pm for two people. Your reservation has been confirmed.")

[1441] Specific operation: A reservation confirmation message is sent to the user using the LINE API.

[1442] Step 6:

[1443] The server monitors the reservation status of the client and detects available times.

[1444] Input: Reservation database

[1445] Data calculation: Scans reservation database to identify available times.

[1446] Output: Free time information (e.g. 2023-12-25 14:00~16:00)

[1447] What it does: Continuously monitors the reservation database to detect times when there are no reservations.

[1448] Step 7:

[1449] The server automatically generates promotional text based on the detected free time.

[1450] Input: Free / busy information and emotion data

[1451] Data processing: Generate personalized promotional text based on the prompt text.

[1452] Output: Promotional message (e.g. "We offer a special discount to customers who visit us between 2pm and 4pm today. Please come and visit us.")

[1453] Specific behavior: Generate appropriate promotional messages by referencing free / busy information and sentiment data.

[1454] Step 8:

[1455] The server sends the generated promotional message to the user.

[1456] Input: Promotional Message

[1457] Output: Promotional message sent

[1458] Specific operation: Send promotional messages to users using LINE API etc.

[1459] In this way, by clearly indicating the specific operations and data processing / calculation performed at each step, it becomes easier to understand how the system works.

[1460] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1461] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1462] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1463] [Fourth embodiment]

[1464] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1465] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1466] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1467] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1468] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1469] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1470] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1471] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1472] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1473] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1474] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1475] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1476] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1477] The present invention relates to a system for automating telephone reservation responses and sales promotion activities. The system mainly comprises a server, a user terminal, an automatic response means, and a voice analysis means.

[1478] Configuring an Automated Response System

[1479] Reservations accepted

[1480] 1. Automated response methods:

[1481] When a user calls the store using their own terminal (for example, a mobile phone or landline), the server acts as an automatic answering means and answers the call. The server responds by saying, "Welcome. Please tell us the details of your reservation."

[1482] 2. Audio analysis methods:

[1483] The server uses speech recognition technology to analyze the user's voice input. For example, if the user says, "I'd like to make a reservation for two people tomorrow at 6 p.m.", the server converts that information into text data and accurately understands it.

[1484] 3. Reservation schedule generation method:

[1485] The server generates a reservation schedule based on the analyzed voice data. The reservation schedule includes the reservation date and time, number of people, and other necessary information. The generated reservation schedule is saved in a database.

[1486] 4. Means of notification:

[1487] The server uses the LINE API or similar to send a reservation confirmation message to the user's device to notify them of the reservation details. For example, a confirmation message such as "Tomorrow at 6 PM for two people. Your reservation has been confirmed."

[1488] Sales support

[1489] 1. Monitoring measures:

[1490] The server constantly monitors the client's reservation status and the store's operational status. The server periodically checks the reservation system database to find available time slots.

[1491] 2. Free time detection:

[1492] By monitoring, the server detects that there are no reservations between 14:00 and 16:00, for example. This free time information is used in the next step.

[1493] 3. Automatic sales promotion text generation method:

[1494] The server uses AI to generate promotional messages based on available times, such as, "We offer a special discount to customers who visit between 2:00 PM and 4:00 PM today. Please come and visit us."

[1495] 4. Message sending method:

[1496] The server sends the generated promotional message to all customers registered in the customer list via the LINE API, increasing the chances that customers will receive the message and become interested.

[1497] Specific examples

[1498] For example, suppose User A calls a restaurant and requests, "I'd like to make a reservation for two people tomorrow at 6 PM." The server automatically receives the call, converts the content into text using a voice recognition system, and generates a reservation schedule. The server then sends a reservation confirmation message to User A via LINE, confirming the reservation.

[1499] The server also detects when the store has fewer reservations and generates promotional messages based on that information, which are then sent to customers via LINE. For example, by sending a message offering a "special discount" when the store is open between 2:00 PM and 4:00 PM, it is possible to attract customers' interest and encourage them to visit the store.

[1500] In this way, the present invention provides a system that reduces the burden of answering telephone calls and allows effective sales promotion activities to be carried out even during idle time.

[1501] The processing flow will be explained below.

[1502] Specific processing steps for reservation handling

[1503] Step 1: User makes a call

[1504] The user calls the store's phone number using their own device (mobile phone or landline).

[1505] Step 2: The server automatically answers the call

[1506] The server acts as an automated answering machine and receives the call, saying, "Welcome. Please tell us your reservation details."

[1507] Step 3: The server analyzes the audio

[1508] The server analyzes the user's speech using an AI speech recognition system and converts the content into text data. For example, "I would like to make a reservation for two people tomorrow at 6 p.m."

[1509] Step 4: The server generates the reservation schedule

[1510] The server generates a reservation schedule based on the textual reservation details, and the generated reservation schedule is saved in a database.

[1511] Step 5: The server confirms the reservation

[1512] The server sends a reservation confirmation message to the user's device via the LINE API. For example, a message like "Tomorrow at 6 PM for two people. Your reservation has been confirmed."

[1513] Specific steps for sales responses

[1514] Step 1: The server monitors the situation

[1515] The server periodically checks the database to monitor client reservation status and store operation status in real time.

[1516] Step 2: The server finds free time

[1517] The server analyzes the reservation data and detects, for example, that there are no reservations between 2:00 p.m. and 4:00 p.m.

[1518] Step 3: The server automatically generates the promotional text

[1519] The server uses the generative AI model to automatically generate promotional messages based on the time when no reservations are available, such as "We offer a special discount to customers who visit between 2:00 PM and 4:00 PM today. Please come and visit us."

[1520] Step 4: The server sends the promotional message

[1521] The server uses the LINE API to send the generated promotional message to all customers registered in the customer list.

[1522] Step 5: Customer receives message

[1523] The customer receives the LINE message on their device and checks the contents.

[1524] Specific examples

[1525] Examples of reservation handling

[1526] Step 1: User A calls the store.

[1527] Step 2: The server will automatically answer the phone and say, "Welcome. Please tell us your reservation details."

[1528] Step 3: User A says, "I'd like to make a reservation for two people tomorrow at 6 PM." The server analyzes the speech and converts it into text.

[1529] Step 4: The server generates a reservation schedule based on the textual reservation details and saves it in the database.

[1530] Step 5: The server notifies User A via the LINE API, "Your reservation for two people at 6 PM tomorrow has been confirmed."

[1531] Specific examples of sales responses

[1532] Step 1: The server monitors the current reservation status.

[1533] Step 2: The server detects that there are no reservations between 2 PM and 4 PM.

[1534] Step 3: The server automatically generates a promotional message saying, "We will offer a special discount to customers who visit our store between 2:00 PM and 4:00 PM today. Please come and visit us."

[1535] Step 4: The server uses the LINE API to send the promotional message to all customers in the customer list.

[1536] Step 5: Customer B receives the LINE message, checks the contents, and then decides to visit the store.

[1537] These are the specific steps involved in handling reservations and sales. This system reduces the burden of answering phone calls during busy periods and allows for effective sales promotion activities during off-peak times.

[1538] Example 1

[1539] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1540] Conventional reservation systems have had issues such as the heavy workload associated with answering the phone and inadequate management of busy and quiet times. Furthermore, promotional activities to effectively fill empty time slots are time-consuming and difficult to carry out efficiently. Furthermore, there is a lack of automated systems to solve these issues, so there is a need for more efficient store operations.

[1541] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1542] In this invention, the server includes an automatic answering means, a means for analyzing input voice and generating a reservation schedule, a means for notifying the client of the generated reservation schedule, a means for monitoring the client's reservation status and detecting free time, a means for using a generative AI model to automatically generate promotional messages based on the detected free time, and a means for using a message sending application to send the generated promotional messages to the client. This makes it possible to automate telephone answering and promotional activities that effectively utilize free time.

[1543] The "automatic response means" is a means for allowing the user to input reservation details by voice, and has the function of accepting voice input and transmitting a guidance message.

[1544] The "voice analysis means" is a means for converting input voice into text data and analyzing the reservation details, and uses voice recognition technology.

[1545] The "reservation schedule generating means" is a means for generating a reservation schedule from the analyzed voice data and storing the necessary information in a database.

[1546] The "notification means" is a means for notifying the client of the created reservation schedule, and uses a message sending application.

[1547] The "monitoring means" is a means for monitoring the reservation status of clients and detecting available time.

[1548] The "promotional text generation means" is a means for automatically generating promotional text using an AI model based on the detected free time.

[1549] The "message sending means" is a means for sending the generated promotional text to the client, and typically uses a message sending application.

[1550] A "generative AI model" is an artificial intelligence model used to automatically generate promotional text, generating appropriate text based on input prompts.

[1551] A "prompt" is a sentence that is used as input to a generative AI model, providing the AI ​​with specific context and information to produce a specific output.

[1552] The present invention relates to a system for automating telephone reservation responses and sales promotion activities. The system mainly comprises a server, a user terminal, an automatic response means, and a voice analysis means.

[1553] Configuring an Automated Response System

[1554] Reservations accepted

[1555] 1. Automated response methods:

[1556] When a user calls the store using their own device (e.g., a mobile phone or landline), the server acts as an automated answering machine and receives the call. The server uses speech synthesis software (e.g., a speech synthesis engine) to respond with, "Welcome. Please tell us the details of your reservation."

[1557] 2. Audio analysis methods:

[1558] The server uses speech recognition technology (e.g., speech recognition API) to analyze the user's voice input. For example, if the user says, "I'd like to make a reservation for two people tomorrow at 6 p.m.", the server converts that content into text data and accurately understands it.

[1559] 3. Reservation schedule generation method:

[1560] The server generates a reservation schedule based on the analyzed voice data. The reservation schedule includes the reservation date and time, the number of people, and other necessary information. The generated reservation schedule is saved in a database (e.g., a relational database).

[1561] 4. Means of notification:

[1562] The server uses a messaging application (e.g., a messaging API) to notify the user of the created reservation schedule. For example, a confirmation message such as "Tomorrow at 6 PM for two people. Your reservation has been confirmed."

[1563] Sales support

[1564] 1. Monitoring measures:

[1565] The server constantly monitors the client's reservation status and the store's operational status. The server periodically checks the database to find available time slots.

[1566] 2. Free time detection method:

[1567] By monitoring, the server detects that there are no reservations between 2 pm and 4 pm, for example. This free time information is used to generate the next promotional message.

[1568] 3. Automatic sales promotion text generation method:

[1569] The server uses a generative AI model (e.g., a language model) to generate promotional messages based on available times, such as, "We offer a special discount to customers who visit our store between 2:00 PM and 4:00 PM today. Please come and visit us."

[1570] 4. Message sending method:

[1571] The server sends the generated promotional message to all customers registered in the customer list via a messaging application (e.g., messaging API). The message received by the customer increases the likelihood that they will be interested.

[1572] Specific examples

[1573] For example, suppose User A calls a restaurant and requests, "I'd like to make a reservation for two people tomorrow at 6 PM." The server automatically receives the call, converts the content into text using a speech recognition API, and generates a reservation schedule using a system using the Django framework. The server then uses a message API to send User A a confirmation message saying, "Tomorrow at 6 PM for two people. Your reservation has been confirmed," and the reservation is confirmed.

[1574] The server also uses a database to detect times when the store has few reservations, and based on that information, it uses a generative AI model to generate promotional messages such as, "We will offer a special discount to customers who visit between 2:00 p.m. and 4:00 p.m. today. Please come and visit us," which are then sent to customers using a messaging API.

[1575] Prompt Sentence Examples

[1576] An example prompt for a generative AI model is:

[1577] "We have openings between 2:00 PM and 4:00 PM. Please come up with some promotional text to attract customers to the store during this time."

[1578] In this way, the present invention provides a system that reduces the burden of answering telephone calls and enables effective sales promotion activities to be carried out even during idle time.

[1579] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1580] Specific processing steps for reservation handling

[1581] Step 1: User makes a call

[1582] How it works: The user dials the store's phone number from their device and presses the call button.

[1583] Input: The action of the user making a phone call.

[1584] Output: The call is connected to the server.

[1585] Step 2: The server receives the call

[1586] What it does: The server launches an automated attendant to receive the call, and uses text-to-speech software to say, "Welcome. Please tell us your reservation details."

[1587] Input: The user's phone connection.

[1588] Output: A voice prompt is played to the user.

[1589] Step 3: User speaks

[1590] Action: The user says, "I'd like to make a reservation for two people tomorrow at 6pm."

[1591] Input: User's voice input.

[1592] Output: The audio data is sent to the server.

[1593] Step 4: The server analyzes the audio

[1594] How it works: The server uses a speech recognition API to convert the user's speech into text data.

[1595] Input: Audio data.

[1596] Output: Reservation information converted to text data.

[1597] Step 5: The server generates the reservation schedule

[1598] Operation: The server generates a reservation schedule based on the text data and saves it in a database.

[1599] Input: Text data.

[1600] Output: Appointment schedule saved in the database.

[1601] Step 6: The server sends a reservation confirmation

[1602] How it works: The server uses the messaging API to send a reservation confirmation message to the user's device.

[1603] Input: The generated appointment schedule.

[1604] Output: A booking confirmation message sent to the user's device.

[1605] Specific steps for sales responses

[1606] Step 1: The server monitors the reservation status

[1607] How it works: The server periodically checks the reservation system database to see which time slots are available.

[1608] Input: Database reservation status.

[1609] Output: Free / busy information.

[1610] Step 2: The server finds free time

[1611] What it does: The server finds available times and uses that information for further processing, for example, checking that there are no reservations between 2 PM and 4 PM.

[1612] Input: Monitoring result data.

[1613] Output: Information about free time detection.

[1614] Step 3: The server automatically generates the promotional text

[1615] How it works: The server uses a generative AI model to generate promotional text based on available times. By providing a prompt, the server can generate a promotional text such as "We are available between 2:00 PM and 4:00 PM. Please come up with a promotional text to encourage customers to visit during this time."

[1616] Input: Free / busy information and prompt statement.

[1617] Output: The generated promotional text.

[1618] Step 4: The server sends the promotional message

[1619] How it works: The server uses the messaging API to send the generated promotional message to all customers on the customer list.

[1620] Input: Generated promotional text and customer list.

[1621] Output: Promotional message sent to customer's device.

[1622] By going through the above steps, the system that automates telephone reservations and promotional activities can be operated efficiently and effectively.

[1623] (Application example 1)

[1624] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1625] Conventional reservation systems not only required a lot of effort to handle phone reservations, but also required manual promotional activities to efficiently fill available time, reducing store operational efficiency. There was also a need to improve the user experience when using smartphones.

[1626] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1627] In this invention, the server includes an automatic response means for inputting reservation details by voice, a means for analyzing the input voice and generating a reservation schedule for the user, a means for notifying the user of the generated reservation schedule, a means for monitoring the reservation status of the user and detecting available time, a means for automatically generating promotional messages based on the detected available time, a means for sending the generated promotional messages to the user, and a means for sending the voice input to the server via a smartphone application. This enables the automation of reservations and efficient promotional activities during available time.

[1628] 1. "Reservation details" refers to information including the date and time the user wishes to make a reservation, the number of people, and other necessary information.

[1629] 2. "Automatic response means that allows voice input" refers to an automatic response system that allows users to input reservation details using voice.

[1630] 3. "Means for analyzing voice" refers to technology for converting input voice data into text data and understanding its content.

[1631] 4. "Means for generating reservation schedules" refers to technology for automatically creating reservation schedules based on reservation details obtained through voice analysis.

[1632] 5. "Means for notifying users of reservation schedules" refers to a system for notifying users of the generated reservation schedules, which is primarily done through a message sending application.

[1633] 6. "Means for monitoring reservation status" refers to a system that constantly checks the current reservation status and detects any available time or changes.

[1634] 7. "Means for detecting available times" refers to technology that monitors reservation status and detects when there are no reservations for a specific time period.

[1635] 8. "Means for automated promotional copy generation" means technology for automatically creating specific promotional messages based on detected available time.

[1636] 9. "Means for sending promotional messages to users" refers to a system for sending generated promotional messages to users, primarily using a message sending application.

[1637] 10. "Means for transmitting voice input to a server via a smartphone application" refers to technology that uses a smartphone application to transmit a user's voice input to a server.

[1638] 11. "Server" refers to the computer system that is the core of the entire system, which handles reservation reception, voice analysis, reservation schedule generation, automatic generation and transmission of promotional materials, etc.

[1639] This invention relates to a system that includes a server, a smartphone application, and artificial intelligence technology for streamlining reservation systems and sales promotion activities in brick-and-mortar stores.

[1640] 1. Voice input of reservation details

[1641] A user attempts to make a reservation using a smartphone application. When the user presses the "Record reservation" button in the smartphone application, the smartphone's microphone is activated and the user voice-records the reservation details.

[1642] 2. Voice analysis and reservation schedule generation

[1643] Once the voice input is complete, the smartphone application sends the voice data to the server. The server uses voice recognition technology to convert the voice data into text data and analyzes the reservation details. This process uses the "speech_recognition" library to convert the voice data into text. Based on the textual reservation details, the server generates a reservation schedule.

[1644] 3. Notification of reservation schedule

[1645] The generated reservation schedule is notified to the user using a messaging application such as the LINE API. For example, a confirmation message such as "Tomorrow at 6 PM for two people. Your reservation has been confirmed" is sent to the user's LINE account.

[1646] 4. Reservation monitoring and availability detection

[1647] The server periodically monitors the reservation system database and manages the store's reservation status. When an available time slot is detected, the information is used in the next step. In this process, the reservation status is periodically checked to detect, for example, that there are no reservations between 2:00 PM and 4:00 PM.

[1648] 5. Automatic generation and sending of promotional text

[1649] The server uses a generative AI model to automatically generate promotional text based on the detected free time. For example, "We offer a special discount to customers who visit the store between 2:00 PM and 4:00 PM today." The generated promotional text is sent to the user via the LINE API.

[1650] Hardware and software used

[1651] 1. Hardware

[1652] Smartphone: Voice input and application execution.

[1653] Server: Speech analysis, reservation schedule generation, notifications, promotional text generation.

[1654] 2. Software

[1655] Smartphone application (iOS, Swift): Records audio and sends it to the server.

[1656] Server (Python, Flask): Speech recognition (speech_recognition library), reservation schedule generation, message sending using LINE API.

[1657] Message sending application (LINE API): Sends reservation confirmation messages and promotional messages to users.

[1658] Specific examples

[1659] For example, a user might say through a smartphone application, "I'd like to make a reservation for two people tomorrow at 6 PM." The smartphone sends this voice data to a server, which analyzes it using voice recognition technology, converts it into text data, and generates a reservation schedule. The server then sends a message to the user via the LINE API, such as "Reservation confirmation: tomorrow at 6 PM for two people." The server also detects available times and creates promotional text using a generative AI model. A promotional text such as "We offer a special discount to customers who visit between 2 PM and 4 PM today" is sent to the user via the LINE API.

[1660] Prompt Sentence Examples

[1661] "User says: I'd like to make a reservation for two people tomorrow at 6pm."

[1662] "Promotional message: We are offering a special discount to customers who visit our store between 2pm and 4pm today."

[1663] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1664] Step 1:

[1665] The user launches the smartphone application and presses the "Record reservation" button. This activates the smartphone's microphone and starts voice input. The user then enters the reservation details by voice. The user's voice data is obtained as input.

[1666] Step 2:

[1667] Once the voice input is complete, the smartphone application sends the voice data to the server. At this time, the voice data file is passed to the server in the format included in the request body. The voice data is sent to the server as input.

[1668] Step 3:

[1669] The server converts the received voice data into text data using the "speech_recognition" library. It receives voice data as input and generates text data as output. At this stage, the voice recognition model analyzes the voice and obtains the reservation details as string information.

[1670] Step 4:

[1671] The server analyzes the textual reservation details and generates a reservation schedule. The server analyzes the text data as input, extracts information such as the reservation date, time, and number of people, and generates a reservation schedule as output. The reservation schedule is saved in a database.

[1672] Step 5:

[1673] The server notifies the user of the generated reservation schedule using the LINE API. Specifically, it sends a message within the LINE app saying, "Tomorrow at 6 PM for two people. Your reservation has been confirmed." It receives the reservation schedule information as input and sends a confirmation message to the user as output.

[1674] Step 6:

[1675] The server periodically monitors the reservation system database. It scans the reservation database and checks for time slots with few reservations. It takes the reservation status data as input and generates a list of available time slots as output.

[1676] Step 7:

[1677] The server uses a generative AI model to automatically generate promotional text based on available time slots. For example, it might say, "We offer a special discount to customers who visit between 2:00 PM and 4:00 PM today." It takes available time slot information as input and generates promotional text as output.

[1678] Step 8:

[1679] The server sends the generated promotional text to all users via the LINE API. It receives the promotional text as input and sends a message to users as output. Specifically, the promotional text is displayed within the LINE app, attracting users' attention and encouraging them to visit the store.

[1680] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1681] This invention is a system incorporating an emotion engine that recognizes user emotions in order to automate telephone reservation responses and sales promotion activities. The main components of this system include a server, a user terminal, an automatic response means, a voice analysis means, and an emotion engine.

[1682] Configuring an Automated Response System

[1683] Reservations accepted

[1684] 1. Automated response methods:

[1685] When a user calls the store using their own device (mobile phone or landline), the server acts as an automatic answering machine and answers the call. The server responds by saying, "Welcome. Please tell us the details of your reservation."

[1686] 2. Audio analysis methods:

[1687] The server uses speech recognition technology to analyze the user's voice input. For example, if the user says, "I'd like to make a reservation for two people tomorrow at 6 p.m.", the server converts that content into text data and understands it.

[1688] 3. Emotion Engine:

[1689] The server is equipped with an emotion engine that recognizes emotions from the user's voice and facial expressions. The emotion engine analyzes the user's tone and speed of speech, and in the case of video calls, facial expressions, to determine the user's emotional state.

[1690] 4. Appointment schedule generation method:

[1691] The server generates a reservation schedule based on the analyzed voice data. The reservation schedule includes the reservation date and time, number of people, and other necessary information. The generated reservation schedule is saved in a database.

[1692] 5. Means of notification:

[1693] The server notifies the user of the created reservation schedule by sending a reservation confirmation message to the user's device using the LINE API, etc. For example, a confirmation message such as "Tomorrow at 6 PM for two people. Your reservation has been confirmed."

[1694] Sales support

[1695] 1. Monitoring measures:

[1696] The server constantly monitors the client's reservation status and the store's operational status. The server periodically checks the reservation system database to detect available time slots.

[1697] 2. Free time detection:

[1698] By monitoring, the server detects, for example, that there are no reservations between 2 p.m. and 4 p.m. This free time information is used in the next step.

[1699] 3. Automatic sales promotion text generation method:

[1700] The server works in conjunction with the emotion engine to automatically generate appropriate promotional messages based on the user's emotional state. For example, if the user is feeling positive, a cheerful message such as "We will offer a special discount to customers who visit our store between 2:00 PM and 4:00 PM today. Please come and visit us." is generated.

[1701] 4. Message sending method:

[1702] The server sends the generated promotional text to all customers registered on the customer list via the LINE API.

[1703] Specific examples

[1704] Examples of reservation handling

[1705] Let's say User A calls a store. The server automatically receives the call and says, "Welcome. Please tell us your reservation details." When User A says, "I'd like to make a reservation for two people tomorrow at 6 PM," the server analyzes the voice, understands the content, and generates a reservation schedule. At the same time, the emotion engine analyzes User A's tone of voice and recognizes that he is satisfied with the confirmed reservation. As a result, the server notifies User A via LINE message, "Tomorrow at 6 PM for two people. Your reservation has been confirmed. Thank you."

[1706] Specific examples of sales responses

[1707] The server monitors the current reservation status of the store and detects that there are no reservations between 2:00 PM and 4:00 PM. Using an emotion engine, the server analyzes the emotional data of users who have previously visited the store and generates promotional messages for customers with positive emotions. For example, it generates a message such as, "We will offer a special discount to customers who visit our store between 2:00 PM and 4:00 PM today. Please come and visit us," and sends it to all customers via the LINE API.

[1708] In this way, the present invention provides a system that reduces the burden of answering the phone and allows for effective sales promotion activities that take into account the emotional state of the customer.

[1709] The processing flow will be explained below.

[1710] Specific processing steps for reservation handling

[1711] Step 1: User makes a call

[1712] The user calls the store's phone number using their own device (mobile phone or landline).

[1713] Step 2: The server automatically answers the call

[1714] The server acts as an automated answering machine and receives the call, saying, "Welcome. Please tell us your reservation details."

[1715] Step 3: The server analyzes the audio

[1716] The server analyzes the user's speech using an AI speech recognition system and converts it into text data. For example, "I would like to make a reservation for two people tomorrow at 6 p.m."

[1717] Step 4: The server performs sentiment analysis

[1718] The server uses an emotion engine to analyze the tone and rate of the user's speech to determine the user's emotional state, for example, whether the user is happy or nervous.

[1719] Step 5: The server generates the reservation schedule

[1720] The server generates a reservation schedule based on the textual reservation details and the results of emotion analysis, and the generated reservation schedule is saved in a database.

[1721] Step 6: The server confirms the reservation

[1722] The server sends a reservation confirmation message to the user's device via the LINE API. For example, a message like "Tomorrow at 6 PM for two people. Your reservation has been confirmed."

[1723] Specific steps for sales responses

[1724] Step 1: The server monitors the situation

[1725] The server periodically checks the database to monitor client reservation status and store operation status in real time.

[1726] Step 2: The server finds free time

[1727] The server monitors and detects, for example, that there are no reservations between 2:00 p.m. and 4:00 p.m.

[1728] Step 3: The server analyzes the emotion data

[1729] The server uses an emotion engine to analyze past customer emotion data and identify customers with positive emotions.

[1730] Step 4: The server automatically generates the promotional text

[1731] The server generates appropriate promotional messages based on the user's emotional state. For example, for customers with positive emotions, it generates a message such as, "We will offer a special discount to customers who visit our store between 2:00 PM and 4:00 PM today. Please come and visit us."

[1732] Step 5: The server sends the promotional message

[1733] The server uses the LINE API to send the generated promotional message to all customers registered in the customer list.

[1734] Step 6: Customer receives message

[1735] The customer receives the LINE message on their device and checks the contents.

[1736] Specific examples

[1737] Examples of reservation handling

[1738] Step 1: User A calls the store.

[1739] Step 2: The server will automatically answer the phone and say, "Welcome. Please tell us your reservation details."

[1740] Step 3: User A says, "I'd like to make a reservation for two people tomorrow at 6 PM." The server analyzes the speech and converts it into text.

[1741] Step 4: The server analyzes the speech tone of User A using the emotion engine and recognizes that the user is satisfied.

[1742] Step 5: The server generates a reservation schedule based on the textual reservation details and the results of sentiment analysis, and saves it in the database.

[1743] Step 6: The server notifies User A via the LINE API, "Your reservation for two people at 6 PM tomorrow has been confirmed. Thank you."

[1744] Specific examples of sales responses

[1745] Step 1: The server monitors the current reservation status of the store.

[1746] Step 2: The server detects that there are no reservations between 2 PM and 4 PM.

[1747] Step 3: The server analyzes past customer emotion data using an emotion engine to identify customers with positive emotions.

[1748] Step 4: The server automatically generates a promotional message saying, "We will offer a special discount to customers who visit our store between 2:00 PM and 4:00 PM today. Please come and visit us."

[1749] Step 5: The server uses the LINE API to send the promotional message to all customers in the customer list.

[1750] Step 6: Customer B receives the LINE message, checks the contents, and then decides to visit the store.

[1751] These are the specific processing steps for handling reservations and sales. This system reduces the burden of answering phone calls during busy periods and enables effective sales promotion activities that take customer emotions into consideration during slower times.

[1752] Example 2

[1753] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1754] Conventional reservation systems require users to make reservations manually by voice, resulting in inefficiency and a significant drain on human resources. Furthermore, promotional activities are conducted without considering the emotional state of the customer, making effective marketing difficult. A system that solves these problems and improves reservation efficiency and customer satisfaction is needed.

[1755] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1756] In this invention, the server includes an automatic response means for allowing reservation details to be input by voice, a means for analyzing the input voice, a means for generating a reservation schedule for a client based on the analyzed voice, a means for saving the generated reservation schedule in a database, a means for notifying the client of the saved reservation schedule, a means for monitoring the client's reservation status and detecting available time, a means for automatically generating promotional messages based on the detected available time, a means for sending the generated promotional messages to the client, a means for analyzing the user's speech tone and facial expressions to recognize the user's emotional state, and a means for adjusting the promotional messages based on the recognized emotional state. This enables efficient reservation handling and effective promotional activities that take the customer's emotional state into consideration.

[1757] "Reservation details" refers to information such as the date and time the user wishes to make a reservation, the number of people, and any special requests.

[1758] "Voice-activated automated response system" refers to an automated response system that allows users to enter reservation details by voice, which allows users to access the system via telephone or other voice input device.

[1759] "Means for analyzing voice" refers to technology that converts voice information entered by the user into text data and understands its content. Specifically, voice recognition technology is used.

[1760] "Means for generating a reservation schedule" refers to a system that extracts necessary reservation information from the analyzed voice data and creates a reservation schedule based on that information.

[1761] "Means for saving in a database" refers to the technology for saving in an appropriate database in order to manage the generated reservation schedule, which will make the accumulation and retrieval of information more efficient.

[1762] The "means for notifying the client" refers to a communication means for notifying the user of the created reservation schedule. For example, it includes a notification method using a messaging application.

[1763] "Means for monitoring reservation status and detecting available times" refers to a system that continuously monitors existing reservation data and identifies time slots that are not booked.

[1764] "Automatic promotional copy generation" refers to technology that automatically creates effective promotional messages based on detected free time, which may involve the use of generative AI models.

[1765] "Means for sending to client" refers to a means for delivering the generated promotional text to the user, which is achieved using a messaging application.

[1766] "Means for recognizing emotional state by analyzing speech tone and facial expressions" refers to technology that extracts emotional information from a user's voice and visual data to identify the user's current emotional state.

[1767] "Means for tailoring promotional text based on emotional state" refers to techniques for optimizing the content and tone of promotional messages depending on the perceived emotional state of the user.

[1768] This invention is a system incorporating an emotion engine that recognizes user emotions in order to automate telephone reservation responses and sales promotion activities. The main components of this system are a server, a user terminal, an automatic response means, a voice analysis means, an emotion engine, a reservation schedule generation means, a notification means, a monitoring means, an automatic promotional message generation means, and a message transmission means.

[1769] System configuration

[1770] 1. Automated Response Methods

[1771] When the server receives a call from a user, it acts as an automated answering system and responds with a voice message saying, "Welcome. Please tell us your reservation details." Specifically, it uses a calling API such as Twilio.

[1772] 2. Audio analysis methods

[1773] The server records the user's voice in real time and converts it to text using the Google Cloud Speech-to-Text API. When a user says, "I'd like to make a reservation for two people tomorrow at 6 p.m.", the server converts the voice to text data and extracts the necessary information.

[1774] 3. Emotion Engine

[1775] The server analyzes the tone and speed of the voice, and in the case of video calls, facial expressions, to recognize the user's emotional state using the Microsoft Azure Emotion API, etc. For example, if the user speaks in a relaxed tone, the emotion engine will determine that the user is happy.

[1776] 4. Reservation Schedule Generation Method

[1777] The server generates a reservation schedule based on the analyzed voice data and the recognized emotional state, including reservation dates and times, number of people, and other details, and stores the generated reservation data in a database such as MySQL.

[1778] 5. Means of notification

[1779] The server uses the LINE API to send a reservation confirmation message to the user's device. For example, a notification might be sent to the user saying, "Your reservation for two people at 6:00 PM tomorrow has been confirmed."

[1780] 6. Monitoring measures

[1781] The server periodically checks the reservation system database to find available time slots by monitoring current reservations, a process automated using Python scripts.

[1782] 7. Free Time Detection

[1783] The server parses the reservation table and finds that there are no reservations between, say, 14:00 and 16:00. This information is stored in an internal data structure (a list or dictionary).

[1784] 8. Automatic generation of promotional text

[1785] The server uses a generative AI model (e.g., GPT-3) to automatically generate appropriate promotional text based on available times, using a prompt like this: "Based on previous customer information, please generate a promotional text offering a special discount to customers who visit between 2:00 PM and 4:00 PM today."

[1786] 9. Message Transmission Methods

[1787] The server then sends the generated promotional message to all customers via the LINE API, such as "We will offer a special discount to customers who visit our store between 2:00 PM and 4:00 PM today. Please come and visit us."

[1788] Specific examples

[1789] Examples of reservation handling

[1790] When User A calls the store using their mobile phone, the server automatically answers the call and says, "Welcome. Please tell us your reservation details." When User A says, "I'd like to make a reservation for two people tomorrow at 6 PM," the server analyzes the voice and understands the content. At the same time, the emotion engine analyzes the user's tone of voice and recognizes satisfaction. As a result, the server generates a reservation schedule, saves it in the database, and then notifies User A via LINE message, "Tomorrow at 6 PM for two people. Your reservation has been confirmed."

[1791] Specific examples of sales responses

[1792] The server monitors the current reservation status and detects that there are no reservations between 2:00 PM and 4:00 PM. Based on the emotional data of users who have previously visited the store, the server generates promotional messages for customers with positive emotions, such as "We offer a special discount to customers who visit us between 2:00 PM and 4:00 PM today. Please come and visit us," and sends them via the LINE API. As a result, it is expected that more reservations will be made for available time slots.

[1793] In this way, the present invention provides a system that aims to improve the efficiency of reservation handling and customer satisfaction, and realizes effective sales promotion activities.

[1794] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1795] Step 1:

[1796] Receiving calls

[1797] The server receives a call from the user's device. Using a call API such as Twilio, the server receives the call as an automated answering machine.

[1798] Input: A user makes a call from their device (mobile or landline).

[1799] Output: The server receives the user's call.

[1800] What happens: Your server uses the Twilio API to identify the incoming call and begin processing it.

[1801] Step 2:

[1802] Voice response

[1803] When the server receives a call, it acts as an automated answering system and plays a voice prompt saying, "Welcome. Please tell us your reservation details."

[1804] Input: The user's phone input.

[1805] Output: A voice prompt responds to the user.

[1806] Specific operation: Plays the audio guide message prepared by the server.

[1807] Step 3:

[1808] Audio analysis

[1809] The server records the user's speech in real time and converts it into text using the Google Cloud Speech-to-Text API.

[1810] Input: User's voice input.

[1811] Output: Text data as the result of speech analysis.

[1812] How it works: The server collects voice data in real time and converts it into text data using the Google Cloud Speech-to-Text API.

[1813] Step 4:

[1814] Analysis of audio data

[1815] The server analyzes the text data and extracts reservation information (date and time, number of people, special requests, etc.).

[1816] Input: Text data of speech analysis results.

[1817] Output: Reservation information (date, time, number of people, special requests, etc.).

[1818] What it does: The server parses the text data and extracts the required information using regular expressions and natural language processing techniques.

[1819] Step 5:

[1820] emotion recognition

[1821] The server uses Microsoft Azure's Emotion API to analyze the user's emotional state based on voice data and facial expression data from video calls.

[1822] Input: User's speech tone and facial expression data.

[1823] Output: Emotion recognition result (e.g., happy, anxious, etc.).

[1824] What it does: The server collects audio tones and visual data and uses the Emotion API to identify the emotional state.

[1825] Step 6:

[1826] Reservation schedule generation

[1827] The server generates a reservation schedule based on the analyzed voice data and the recognized emotional state and stores it in a database.

[1828] Input: Booking information and emotional state data.

[1829] Output: Appointment schedule.

[1830] Specific operation: The server organizes reservation information and stores it in a database such as MySQL.

[1831] Step 7:

[1832] Confirmation Notice

[1833] The server uses the LINE API to send a reservation confirmation message to the user's device.

[1834] Input: Appointment Schedule.

[1835] Output: Acknowledgement message.

[1836] Specific operation: The server generates a confirmation message based on the reservation schedule and sends it to the user via the LINE API.

[1837] Step 8:

[1838] Reservation status monitoring

[1839] The server periodically checks the reservation system database to monitor the reservation status and detects available time slots.

[1840] Input: Database reservation status.

[1841] Output: Free / busy information.

[1842] What it does: The server uses a Python script to periodically query the reservation data to identify available times.

[1843] Step 9:

[1844] Free Time Detection

[1845] The server analyzes the reservation table and finds that there are no reservations between, say, 2 PM and 4 PM, and stores this information in an internal data structure.

[1846] Input: Database reservation status.

[1847] Output: The free time found.

[1848] Specific operation: The server organizes and stores free time data in a data structure such as a list or dictionary.

[1849] Step 10:

[1850] Automatic sales promotion generation

[1851] The server uses a generative AI model (e.g., GPT-3) to automatically generate appropriate promotional text based on available time.

[1852] Input: Free / busy information and prompt text.

[1853] Output: Auto-generated promotional text.

[1854] Specific behavior: The server generates a promotional message using the following prompt: "Based on previous customer information, please generate a promotional message offering a special discount to customers who visit the store between 2:00 PM and 4:00 PM today."

[1855] Step 11:

[1856] Sending a message

[1857] The server sends the generated promotional message to all customers via the LINE API.

[1858] Input: Auto-generated promotional text.

[1859] Output: Promotional message sent.

[1860] Specific operation: The server uses the LINE API to send the generated promotional message to all customers at once.

[1861] (Application example 2)

[1862] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1863] In conventional reservation systems, when users make reservations over the phone, the burden on the staff who handle the reservations is heavy, resulting in inefficiency. Furthermore, the system often provides a uniform response without considering the user's feelings, resulting in low customer satisfaction. Furthermore, promotional activities to effectively utilize available time are complicated and time-consuming. There is a need to solve these issues and realize efficient reservation management and promotional activities while increasing user satisfaction.

[1864] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1865] In this invention, the server includes an automatic response means for inputting reservation details by voice, a means for analyzing the input voice and generating a reservation schedule for the client, a means for notifying the client of the generated reservation schedule, a means for monitoring the client's reservation status and detecting available time, a means for automatically generating promotional messages based on the detected available time, a means for sending the generated promotional messages to the client, an emotion recognition means as an application installed on the smartphone for generating personalized promotional messages based on prompt messages, and an emotion analysis means for analyzing the user's emotions based on the voice input. This enables effective reservation management and promotional activities that take user emotions into consideration.

[1866] An "automated response means for voice input of reservation details" is part of a system that allows users to input reservation information by voice via a smartphone or other device.

[1867] "Means for analyzing input voice and generating a client's reservation schedule" refers to a system function that uses voice recognition technology to analyze the user's voice data and automatically creates a reservation schedule based on that data.

[1868] The "means for notifying the client of the created reservation schedule" is a notification system for notifying the user of the created reservation schedule, and is a means using a message sending application or the like.

[1869] The "means for monitoring client reservation status and detecting available times" is part of a system that constantly monitors the reservation database and identifies time periods when no reservations have been made.

[1870] The "means for automatically generating promotional messages based on detected free time" is a system function that automatically creates appropriate promotional messages using emotion recognition means when free time is detected.

[1871] The "means for transmitting the generated promotional message to the client" refers to a system for transmitting the generated promotional message to the user, for example, a means for using a message sending application.

[1872] The "application installed on the smartphone" is software that runs on the smartphone and is a platform for executing the various means of the present invention.

[1873] The "emotion recognition means for generating personalized promotional text based on prompt text" is part of a system that includes emotion recognition technology for generating personalized promotional messages based on user input and historical data.

[1874] The "emotion analysis means for analyzing the user's emotions based on the voice input" is a part of the system that analyzes the emotions from the user's voice input and reflects them in the content of the interaction.

[1875] This invention is a system that uses an application installed on a smartphone to improve the efficiency of users' reservation management and promotional activities, thereby increasing user satisfaction.

[1876] Hardware and software used

[1877] The server uses the following hardware and software:

[1878] Smartphone: Used for user voice input and camera photography.

[1879] Speech recognition software: Use the speech_recognition Python package to convert audio data into text.

[1880] Sentiment analysis tools: The TextBlob Python package is used to analyze the sentiment of text, and the face_recognition library is used to recognize the facial expressions of users.

[1881] Database: SQLite is used to manage reservation data.

[1882] Message sending application: Uses the LINE API to send notification messages to users.

[1883] Data processing and calculation

[1884] 1. Voice analysis: When a user voices their reservation details on their smartphone, the voice recognition software converts the voice input into text data. At the same time, an emotion analysis tool also analyzes the emotion of the user's voice tone.

[1885] 2. Reservation schedule generation: After the voice data is converted to text, a reservation schedule is automatically generated. The necessary data (reservation date and time, number of people, etc.) is recorded in a database.

[1886] 3. Notification of reservation schedule: The generated reservation schedule is notified to the user using the LINE API, etc. For example, when a reservation is confirmed, a message is sent saying, "Tomorrow at 6 PM for two people. Your reservation has been confirmed."

[1887] 4. Monitoring and promotional message generation: The server constantly monitors the database to detect available time slots. When available time slots are detected, personalized promotional messages are automatically generated based on the results of sentiment analysis. For example, a message such as "We offer a special discount to customers who visit our store between 2:00 PM and 4:00 PM today. Please come and visit us." is generated. This is also sent to the user using the LINE API.

[1888] Specific examples

[1889] When a user opens the app and says, "Please make a reservation for two people tomorrow at 6 PM," the app uses its emotion engine to analyze the user's emotions and confirms the reservation.The app then uses the LINE API to send the user a message saying, "Tomorrow at 6 PM for two people. Your reservation is confirmed."

[1890] Prompt Sentence Examples

[1891] "I'd like to make a reservation for two people for tomorrow at 6pm."

[1892] "Thank you for your positive attitude! Your reservation is confirmed for two people tomorrow at 6pm."

[1893] "We are offering a special discount to customers who visit between 2pm and 4pm today. Please come and visit us!"

[1894] In this way, it becomes possible to effectively manage reservations and promote sales activities while taking into consideration the feelings of users.

[1895] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1896] Step 1:

[1897] The user enters the reservation details by voice via their smartphone.

[1898] Input: User's voice data

[1899] Specific operation: The user launches the app and says, "Please make a reservation for two people tomorrow at 6 p.m."

[1900] Step 2:

[1901] Voice recognition software on the smartphone converts the speech into text data.

[1902] Input: User's voice data

[1903] Data processing: Convert audio to text using the speech_recognition package.

[1904] Output: Text data (e.g., "Please make a reservation for two people tomorrow at 6 PM.")

[1905] What it does: Speech recognition software analyzes the audio data and converts it into text.

[1906] Step 3:

[1907] Sentiment analysis tools analyze user emotions from text data.

[1908] Input: Text data

[1909] Data Operations: Analyze text sentiment using the TextBlob package.

[1910] Output: Sentiment data (e.g., positive, negative, neutral)

[1911] Specific operation: Analyze the sentiment of text data and obtain analysis results.

[1912] Step 4:

[1913] The server generates a reservation schedule.

[1914] Input: Emotion data and text data

[1915] Data processing: Analyze the reservation details and register them in the schedule database.

[1916] Output: Reservation schedule (e.g. 2023-12-25 18:00, 2 people)

[1917] Specific operation: Organize reservation information based on the analyzed text data and emotion data and store it in a database.

[1918] Step 5:

[1919] The server notifies the user of the generated reservation schedule.

[1920] Input: Appointment Schedule

[1921] Output: Reservation confirmation message (e.g. "Tomorrow at 6pm for two people. Your reservation has been confirmed.")

[1922] Specific operation: A reservation confirmation message is sent to the user using the LINE API.

[1923] Step 6:

[1924] The server monitors the reservation status of the client and detects available times.

[1925] Input: Reservation database

[1926] Data calculation: Scans reservation database to identify available times.

[1927] Output: Free time information (e.g. 2023-12-25 14:00~16:00)

[1928] What it does: Continuously monitors the reservation database to detect times when there are no reservations.

[1929] Step 7:

[1930] The server automatically generates promotional text based on the detected free time.

[1931] Input: Free / busy information and emotion data

[1932] Data processing: Generate personalized promotional text based on the prompt text.

[1933] Output: Promotional message (e.g. "We offer a special discount to customers who visit us between 2pm and 4pm today. Please come and visit us.")

[1934] Specific behavior: Generate appropriate promotional messages by referencing free / busy information and sentiment data.

[1935] Step 8:

[1936] The server sends the generated promotional message to the user.

[1937] Input: Promotional Message

[1938] Output: Promotional message sent

[1939] Specific operation: Send promotional messages to users using LINE API etc.

[1940] In this way, by clearly indicating the specific operations and data processing / calculation performed at each step, it becomes easier to understand how the system works.

[1941] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1942] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1943] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1944] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1945] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1946] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1947] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1948] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1949] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1950] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1951] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1952] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1953] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1954] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1955] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1956] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1957] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1958] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1959] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1960] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1961] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1962] The following is further disclosed regarding the above embodiment.

[1963] (Claim 1)

[1964] An automated response system that allows you to input reservation details by voice;

[1965] A means for analyzing the input voice and generating a reservation schedule for the client;

[1966] a means for notifying the client of the generated reservation schedule;

[1967] a means of monitoring client bookings and detecting availability;

[1968] means for automatically generating promotional text based on the detected free time;

[1969] means for transmitting the generated promotional text to the client;

[1970] A system including:

[1971] (Claim 2)

[1972] 2. The system of claim 1, wherein the automated response means includes an artificial intelligence based voice recognition system.

[1973] (Claim 3)

[1974] 10. The system of claim 1, wherein the means for sending promotional text uses a messaging application.

[1975] "Example 1"

[1976] (Claim 1)

[1977] An automated response system that allows you to input reservation details by voice;

[1978] A means for analyzing the input voice and generating a reservation schedule for the client;

[1979] a means for notifying the client of the generated reservation schedule;

[1980] a means of monitoring client bookings and detecting availability;

[1981] a means for using a generative AI model to automatically generate promotional copy based on the detected free time;

[1982] means for using a messaging application to send the generated promotional text to the client;

[1983] A system including:

[1984] (Claim 2)

[1985] 2. The system of claim 1, wherein the automated response means includes an artificial intelligence based voice recognition system.

[1986] (Claim 3)

[1987] 10. The system of claim 1, further comprising means for inputting a prompt statement to generate the promotional statement.

[1988] "Application Example 1"

[1989] (Claim 1)

[1990] An automated response system that allows you to input reservation details by voice;

[1991] A means for analyzing input voice and generating a reservation schedule for a user;

[1992] a means for notifying a user of the generated reservation schedule;

[1993] A means for monitoring the reservation status of users and detecting available times;

[1994] means for automatically generating promotional text based on the detected free time;

[1995] means for transmitting the generated promotional text to the user;

[1996] means for transmitting the voice input to a server through a smartphone application;

[1997] A system including:

[1998] (Claim 2)

[1999] 2. The system of claim 1, wherein the automated response means includes an artificial intelligence based voice recognition system.

[2000] (Claim 3)

[2001] 10. The system of claim 1, wherein the means for sending promotional text uses a messaging application.

[2002] "Example 2: Combining Emotion Engines"

[2003] (Claim 1)

[2004] An automated response system that allows you to input reservation details by voice;

[2005] A means for analyzing input voice;

[2006] A means for generating a reservation schedule for a client based on the analyzed voice;

[2007] A means for storing the generated reservation schedule in a database;

[2008] a means of notifying clients of saved appointment schedules;

[2009] a means of monitoring client bookings and detecting availability;

[2010] means for automatically generating promotional text based on the detected free time;

[2011] means for transmitting the generated promotional text to the client;

[2012] means for recognizing a user's emotional state by analyzing the user's speech tone and facial expressions;

[2013] a means for adjusting promotional text based on the perceived emotional...

Claims

1. An automated response system that allows you to input reservation details by voice; A means for analyzing the input voice and generating a reservation schedule for the client; a means for notifying the client of the generated reservation schedule; a means of monitoring client bookings and detecting availability; means for automatically generating promotional text based on the detected free time; means for transmitting the generated promotional text to the client; A system including:

2. 2. The system of claim 1, wherein the automated response means includes an artificial intelligence voice recognition system.

3. 10. The system of claim 1, wherein the means for sending promotional text uses a messaging application.

Images