System

Abstract

A system is provided.SOLUTION: The system includes a means for receiving information inputted by a user, a means for storing the received information in a database, a means for analyzing a question submitted by the user, a means for automatically generating an answer based on the analysis result, a means for converting voice data into text data, a means for converting image data of a business card into text data, and a means for notifying a business department of the converted business card information.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In conventional call center operations, responding to customer inquiries and requests relies on human resources, resulting in slow response times and human error. Furthermore, it is difficult to effectively integrate and operate multiple methods, such as voice response and collecting business card information. This results in lower customer satisfaction and inefficient operations. [Means for solving the problem]

[0005] The present invention solves these problems by providing a system equipped with a means for receiving and saving information input from users, a means for analyzing questions submitted by users and generating appropriate answers, a means for converting voice data into text, and a means for converting business card image data into text and notifying the sales department. Specifically, the system automatically analyzes user questions and generates appropriate answers, enabling prompt and accurate responses. Furthermore, by using voice recognition and OCR technologies, the system can accommodate a variety of input formats, efficiently collect business card information, and utilize it in sales activities. This can improve business efficiency and customer satisfaction.

[0006] "User" means a person or entity that uses the System to provide information or submit questions or requests.

[0007] The "information input means" is an interface that allows users to input basic information such as their name, email address, and phone number into the system.

[0008] A "database" is a structured data storage within a computer system for storing received user information, analysis results, business card information, etc.

[0009] The "question analysis means" is a means for analyzing a question submitted by a user using natural language processing technology or the like, and understanding the intent of the question.

[0010] The "automatic answer generation means" is a means for automatically generating an appropriate answer based on the analysis results and providing it to the user.

[0011] The "voice recognition means" is a means that uses voice recognition technology to convert voice data provided by a user into text data.

[0012] "OCR means" refers to a means that uses optical character recognition (OCR) technology to extract character information from image data such as business cards and convert it into text data.

[0013] The "sales department notification means" is a means for notifying the sales department of the converted business card information and supporting sales activities.

[0014] The "marketing message generation means" is a means for generating customized marketing messages based on the analysis results of business card information and other collected data. [Brief explanation of the drawings]

[0015] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0016] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0017] First, the terms used in the following description will be explained.

[0018] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0019] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0020] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0021] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0022] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0023] [First embodiment]

[0024] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0025] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0026] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0027] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0028] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0029] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0030] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0031] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0032] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0033] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0034] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0035] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0036] This invention is a system for improving the efficiency of call center operations and customer satisfaction. This system automatically responds to inquiries and requests from users, performs voice recognition, and collects business card information. It also effectively obtains information useful for sales activities and generates marketing messages.

[0037] Explanation of program processing

[0038] User reception process

[0039] 1. A user accesses the system

[0040] A user accesses the system from a terminal through a web browser or application. For example, the user accesses the system's URL in a browser.

[0041] 2. The user enters basic information

[0042] The user enters basic information such as "name," "email address," and "phone number" into the input form. Specifically, the user enters "Taro," "taro@example.com," "090-1234-5678," etc.

[0043] 3. The device sends the input information to the server

[0044] The terminal converts the input information into an appropriate format and sends it to the server.

[0045] 4. The server stores the information in a database

[0046] The server stores the received user information in the database. For example, it executes an SQL query like "INSERT INTO UserTable (Name, Email, Phone) VALUES ('Taro', 'taro@example.com', '090-1234-5678')".

[0047] Auto-Response Processing

[0048] 1. The user enters a question

[0049] The user types a question into the chat box on the device, for example, "What are your business hours?"

[0050] 2. The device sends a question to the server

[0051] The terminal sends the entered question to the server.

[0052] 3. The server parses the question

[0053] The server uses a natural language processing (NLP) engine to parse the question, in this example extracting the keyword "business hours."

[0054] 4. The server searches the FAQ database

[0055] The server queries the FAQ database based on the analysis results and retrieves the appropriate answer, for example, "Business hours are Monday through Friday, 9:00 AM to 6:00 PM."

[0056] 5. The server sends the answer to the device

[0057] The server sends the obtained answer to the terminal.

[0058] 6. The device displays the answer to the user

[0059] The device will display the response it receives in the chat box.

[0060] Voice-enabled processing

[0061] 1. The user selects a voice input

[0062] The user clicks the "Speech Input" button in the chat box.

[0063] 2. The device records the audio

[0064] The device activates the microphone and records the user's voice. For example, the user might say, "I'd like to know more about this product."

[0065] 3. The device sends the audio data to the server

[0066] The device sends the recorded audio data to the server.

[0067] 4. The server converts the speech to text

[0068] The server uses a speech recognition engine to convert the voice data into text. In this example, the voice "I want to know more about the product" is converted into the text "I want to know more about the product."

[0069] 5. The server generates the answer

[0070] The server analyzes the text information and generates an appropriate response, such as "This product was developed using the latest technology. Please visit our website for more information."

[0071] 6. The server sends the answer to the device

[0072] The server generates a response text and sends it to the terminal.

[0073] 7. The device displays the answer to the user

[0074] The device will display the response it receives in the chat box.

[0075] Business card information collection and processing

[0076] 1. The user provides business card information

[0077] The user takes a photo of the business card with a camera and uploads the image data to the terminal.

[0078] 2. The device sends the business card image to the server

[0079] The terminal transmits the business card image data to the server.

[0080] 3. The server converts the image into text

[0081] The server uses an OCR (Optical Character Recognition) engine to extract text information from the image. In this example, it extracts information such as "Yamada Taro," "ABC Co., Ltd.", and "Manager."

[0082] 4. The server saves the business card information in a database

[0083] The server saves the extracted business card information in the database. For example, it saves it as follows: INSERT INTO BusinessCardTable (Name, Company, Title) VALUES ('Yamada Taro', 'ABC Co., Ltd.', 'Manager').

[0084] 5. The server notifies the sales department

[0085] The server sends the stored information to the sales department staff via email.

[0086] DM policy processing

[0087] 1. The server analyzes the business card information

[0088] The server retrieves the business card information from the database and analyzes it, for example to generate a list of customers in a particular industry.

[0089] 2. The server generates the marketing message

[0090] Based on the analysis results, the server generates a marketing message tailored to each customer, such as "A special offer for Taro Yamada of ABC Co., Ltd."

[0091] 3. The server sends the DM

[0092] Sending server-generated marketing messages to targeted users via email and post.

[0093] As described above, this system responds to user inquiries and requests from multiple angles, thereby improving the efficiency of call center operations and customer satisfaction.

[0094] The processing flow will be explained below.

[0095] User reception process

[0096] Step 1:

[0097] A user accesses the system

[0098] A user accesses the system from a terminal through a web browser or application. For example, the user accesses the system's URL in a browser.

[0099] Step 2:

[0100] The user enters basic information

[0101] The user enters basic information such as "name," "email address," and "phone number" into the input form. Specifically, the user enters "Taro," "taro@example.com," "090-1234-5678," etc.

[0102] Step 3:

[0103] The device sends the input information to the server

[0104] The terminal converts the input information into an appropriate format and sends it to the server.

[0105] Step 4:

[0106] The server stores the information in a database

[0107] The server stores the received user information in the database. For example, it executes an SQL query like "INSERT INTO UserTable (Name, Email, Phone) VALUES ('Taro', 'taro@example.com', '090-1234-5678')".

[0108] Auto-Response Processing

[0109] Step 1:

[0110] The user enters a question

[0111] The user types a question into the chat box on the device, for example, "What are your business hours?"

[0112] Step 2:

[0113] The device sends a question to the server

[0114] The terminal sends the entered question to the server.

[0115] Step 3:

[0116] The server parses the question

[0117] The server uses a natural language processing (NLP) engine to parse the question, in this example extracting the keyword "business hours."

[0118] Step 4:

[0119] The server searches the FAQ database

[0120] The server queries the FAQ database based on the analysis results and retrieves the appropriate answer, for example, "Business hours are Monday through Friday, 9:00 AM to 6:00 PM."

[0121] Step 5:

[0122] The server sends the answer to the device

[0123] The server sends the obtained answer to the terminal.

[0124] Step 6:

[0125] The device displays the answer to the user

[0126] The device will display the response it receives in the chat box.

[0127] Voice-enabled processing

[0128] Step 1:

[0129] The user selects a voice input

[0130] The user clicks the "Speech Input" button in the chat box.

[0131] Step 2:

[0132] The device records audio

[0133] The device activates the microphone and records the user's voice. For example, the user might say, "I'd like to know more about this product."

[0134] Step 3:

[0135] The device sends the audio data to the server.

[0136] The device sends the recorded audio data to the server.

[0137] Step 4:

[0138] The server converts the speech to text

[0139] The server uses a speech recognition engine to convert the voice data into text. In this example, the voice "I want to know more about the product" is converted into the text "I want to know more about the product."

[0140] Step 5:

[0141] The server generates the answer

[0142] The server analyzes the text information and generates an appropriate response, such as "This product was developed using the latest technology. Please visit our website for more information."

[0143] Step 6:

[0144] The server sends the answer to the device

[0145] The server generates a response text and sends it to the terminal.

[0146] Step 7:

[0147] The device displays the answer to the user

[0148] The device will display the response it receives in the chat box.

[0149] Business card information collection and processing

[0150] Step 1:

[0151] User provides business card information

[0152] The user takes a photo of the business card with a camera and uploads the image data to the terminal.

[0153] Step 2:

[0154] The device sends the business card image to the server.

[0155] The terminal transmits the business card image data to the server.

[0156] Step 3:

[0157] The server converts the image into text

[0158] The server uses an OCR (Optical Character Recognition) engine to extract text information from the image. In this example, it extracts information such as "Yamada Taro," "ABC Co., Ltd.", and "Manager."

[0159] Step 4:

[0160] The server stores the business card information in a database

[0161] The server saves the extracted business card information in the database. For example, it saves it as follows: INSERT INTO BusinessCardTable (Name, Company, Title) VALUES ('Yamada Taro', 'ABC Co., Ltd.', 'Manager').

[0162] Step 5:

[0163] The server notifies the sales department

[0164] The server sends the stored information to the sales department staff via email.

[0165] DM policy processing

[0166] Step 1:

[0167] The server analyzes the business card information

[0168] The server retrieves the business card information from the database and analyzes it, for example to generate a list of customers in a particular industry.

[0169] Step 2:

[0170] The server generates the marketing message

[0171] Based on the analysis results, the server generates a marketing message tailored to each customer, such as "A special offer for Taro Yamada of ABC Co., Ltd."

[0172] Step 3:

[0173] The server sends a DM

[0174] Sending server-generated marketing messages to targeted users via email and post.

[0175] Example 1

[0176] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0177] Call center operations must efficiently respond to a large number of inquiries, which requires the rapid and accurate processing of data in various formats. However, conventional systems have difficulty integrating voice recognition, business card information collection, and natural language processing, resulting in problems such as reduced work efficiency and low customer satisfaction. Furthermore, the creation and transmission of marketing messages is also done manually, which is inefficient. There is a need to solve these problems, improve the efficiency of call center operations, and increase customer satisfaction.

[0178] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0179] In this invention, the server includes means for receiving information input by a user, means for storing the received information in a database, means for analyzing questions submitted by users, means for automatically generating answers based on the analysis results, means for converting voice data into text data, means for converting business card image data into text data, means for notifying the sales department of the converted business card information, means for analyzing questions using natural language processing technology, means for searching an FAQ database to obtain an appropriate answer, means for making voice input selectable, means for transmitting recorded voice data to the server, means for displaying answers to the user, means for formatting the input information and transmitting it to the server, means for analyzing the business card information and generating a marketing message, and means for transmitting the generated marketing message to the user. This enables efficient response to inquiries, voice recognition, automatic collection of business card information, and automatic generation and transmission of marketing messages.

[0180] "Means for receiving information input by a user" refers to an interface for receiving data input by a user.

[0181] "Means for saving received information to a database" refers to the process of saving the data entered by the user in a database for storing the data.

[0182] "Means for analyzing a question submitted by a user" refers to a processing technology for analyzing a question entered by a user and understanding its intent.

[0183] "Means for automatically generating answers based on the analysis results" refers to a process for automatically generating appropriate answers to the analyzed questions.

[0184] "Means for converting voice data into text data" refers to technology for converting information input by voice into text format.

[0185] "Means for converting business card image data into text data" refers to technology for converting business card photos or scanned data into text information.

[0186] "Means for notifying the sales department of the converted business card information" refers to the process of notifying the sales department of the data extracted from the business card.

[0187] "Means for analyzing questions using natural language processing technology" refers to technology for analyzing user questions using natural language processing technology.

[0188] "Means for searching the FAQ database to obtain an appropriate answer" refers to the process of searching the FAQ database to obtain an appropriate answer to a user's question.

[0189] "Means for enabling selection of voice input" refers to a technology that provides an interface that allows a user to select a voice input.

[0190] "Means for transmitting recorded voice data to a server" refers to a communication technology for transmitting recorded voice data to a server.

[0191] The "means for displaying the answer to the user" refers to an interface for displaying the generated answer on the user's terminal.

[0192] "Means for formatting input information and sending it to the server" refers to the process of converting the information input by the user into an appropriate format and sending it to the server.

[0193] "Means for analyzing business card information and generating a marketing message" refers to technology for analyzing information obtained from a business card and generating a specific marketing message.

[0194] "Means for sending generated marketing messages to users" refers to the process of sending generated marketing messages to target users.

[0195] This invention is a system aimed at improving the efficiency of call center operations and customer satisfaction. This system automatically responds to inquiries and requests from users, performs voice recognition, and collects business card information. It also effectively obtains information useful for sales activities and generates marketing messages.

[0196] The main components of the system are a terminal for users to input information, a server to process the input information, and a database to store the data. The specific hardware and software configurations are described below.

[0197] Hardware and software used

[0198] 1. Device:

[0199] A device used by a user, such as a smartphone, tablet, or PC. The device accesses the system through a web browser or a dedicated application.

[0200] 2. Server:

[0201] Data processing is performed using high-performance servers (e.g., AWS (registered trademark), Google (registered trademark) Cloud) on which a natural language processing engine (e.g., Google Natural Language API), a voice recognition engine (e.g., Google Speech-to-Text API), and an OCR engine (e.g., Tesseract OCR) are installed.

[0202] 3. Database:

[0203] A relational database (e.g., MySQL (registered trademark), PostgreSQL) that stores user information, FAQ data, and business card information.

[0204] User reception process

[0205] User accesses and enters basic information: The user uses a terminal to access the system through a web browser or application. Then, the user enters basic information such as "name," "email address," and "phone number." For example, "Taro," "taro@example.com," and "090-1234-5678" are entered.

[0206] Sending and saving input information: The device sends the input information to the server, which stores it in a database.

[0207] Auto-Response Processing

[0208] User's question input and transmission: The user inputs a question into the chat box, and the terminal transmits the question to the server. For example, "What are your business hours?" is input.

[0209] Question analysis and answer generation: The server uses a natural language processing engine to analyze the question and search the FAQ database to get the appropriate answer, which is then sent to the device and displayed to the user.

[0210] Voice-enabled processing

[0211] Select and record voice input: The user clicks the voice input button in the chat box, and the device activates the microphone to record the voice. For example, "I would like to know more about this product."

[0212] Sending and converting voice data: The device sends the recorded voice data to the server, which then converts it into text using a voice recognition engine.

[0213] Generate and display an answer: The server analyzes the text information, generates an appropriate answer, sends it to the device, and displays it to the user. For example, "This product has been developed using the latest technology. Please visit our website for more information." is generated and displayed.

[0214] Business card information collection and processing

[0215] Providing and sending business card information: The user takes a photo of the business card with the camera and uploads it to the device, which then sends the image data to the server.

[0216] Image data conversion and storage: The server uses an OCR engine to extract text information from the image and store it in a database. It also sends the stored information to the sales department via email.

[0217] DM policy processing

[0218] Analyzing business card information and generating marketing messages: The server retrieves business card information from the database and generates a list of customers in a specific industry. The server then generates marketing messages tailored to each customer based on the analysis results.

[0219] Sending direct mail: Sending server-generated marketing messages to targeted users via email or post. For example, a message such as "We'd like to share a special offer with you" is generated.

[0220] With the above-described configuration and procedures, the system can improve the efficiency of call center operations in a variety of ways and increase customer satisfaction.

[0221] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0222] User reception process

[0223] Step 1:

[0224] A user accesses the system

[0225] A user starts a web browser or application on a device such as a PC or smartphone and accesses the system's URL. The input is the system's URL, and the output is the display of the system's web page.

[0226] Step 2:

[0227] The user enters basic information

[0228] A user enters basic information such as "name," "email address," and "phone number" into an input form displayed on the system. The input is the user's personal information (e.g., "Taro," "taro@example.com," "090-1234-5678"), and the output is that this information is displayed in the input fields.

[0229] Step 3:

[0230] The device sends the input information to the server

[0231] The device converts the information entered by the user into an appropriate format, such as JSON, and sends it to the server using an HTTP POST request. The input is the personal information entered by the user, and the output is the formatted data. As a concrete example, the following JSON data is sent from the device to the server:

[0232] json

[0233] {

[0234] "name": "Taro",

[0235] "email": "taro@example.com",

[0236] "phone": "090-1234-5678"

[0237] }

[0238] Step 4:

[0239] The server stores the information in a database

[0240] The server analyzes the received user information and executes SQL queries to store it in the database. The input is the personal information sent from the terminal, and the output is the data stored in the database. Specifically, the following SQL queries are executed:

[0241] sql

[0242] INSERT INTO UserTable (Name, Email, Phone) VALUES ('Taro', 'taro@example.com', '090-1234-5678')

[0243] Auto-Response Processing

[0244] Step 1:

[0245] The user enters a question

[0246] The user types a question into the chat box on the device and clicks the send button. The input is the user's question (e.g., "What are your business hours?"), and the output is the question displayed in the chat box.

[0247] Step 2:

[0248] The device sends a question to the server

[0249] The device sends the entered question to the server in an appropriate format (e.g. JSON format). The input is the question entered by the user, and the output is the data sent to the server. The content sent is as follows:

[0250] json

[0251] {

[0252] "question": "What are your opening hours?"

[0253] }

[0254] Step 3:

[0255] The server parses the question

[0256] The server uses a natural language processing engine to analyze the question and extract important keywords and intent. The input is the user's question data, and the output is the analyzed intent and keywords. For example, the keyword "business hours" is extracted.

[0257] Step 4:

[0258] The server searches the FAQ database

[0259] The server queries the FAQ database based on the analysis results to obtain the appropriate answer. The input is the analyzed keyword (e.g., "business hours"), and the output is the answer data from the FAQ database. Specifically, the following SQL query is executed:

[0260] sql

[0261] SELECT Answer FROM FAQTable WHERE Question LIKE '%Business Hours%'

[0262] And you find the answer, "Our business hours are Monday through Friday, 9:00 AM to 6:00 PM."

[0263] Step 5:

[0264] The server sends the answer to the device

[0265] The server converts the answer it gets into the appropriate format and sends it to the terminal as an HTTP response. The input is the answer from the FAQ database, and the output is the formatted response sent. An example response is as follows:

[0266] json

[0267] {

[0268] "answer": "Our business hours are Monday to Friday, 9am to 6pm."

[0269] }

[0270] Step 6:

[0271] The device displays the answer to the user

[0272] The terminal formats the answer received from the server and displays it in the chat box. The input is the answer data from the server, and the output is the display of the answer to the user.

[0273] Voice-enabled processing

[0274] Step 1:

[0275] The user selects a voice input

[0276] The user clicks the "Voice Input" button in the chat box. The input is an operation that enables voice input, and the output is an operation that starts voice recording.

[0277] Step 2:

[0278] The device records audio

[0279] The device activates the microphone and records the user's voice. The input is the user's voice (e.g., "I want to know more about this product"), and the output is the recorded voice data.

[0280] Step 3:

[0281] The device sends the audio data to the server.

[0282] The device sends the recorded audio data to the server in the appropriate format. The input is the recorded audio data, and the output is the data sent to the server.

[0283] Step 4:

[0284] The server converts the speech to text

[0285] The server uses a speech recognition engine to convert the voice data into text. The input is the voice data sent to the server, and the output is text data. For example, the text might be "I'd like to know more about this product."

[0286] Step 5:

[0287] The server generates the answer

[0288] The server analyzes the text information and generates an appropriate answer. The input is speech data converted into text, and the output is the generated answer text. For example, the answer generated might be, "This product was developed using the latest technology. Please visit our website for more information."

[0289] Step 6:

[0290] The server sends the answer to the device

[0291] The server generates a response and sends it to the terminal in text format. The input is the generated response text, and the output is the formatted response transmission.

[0292] Step 7:

[0293] The device displays the answer to the user

[0294] The terminal displays the answer received from the server in the chat box. The input is the answer data from the server, and the output is the display of the answer to the user.

[0295] Business card information collection and processing

[0296] Step 1:

[0297] User provides business card information

[0298] A user takes a photo of a business card with a camera and uploads it to a terminal as image data. The input is the image data of the business card, and the output is the image uploaded to the terminal.

[0299] Step 2:

[0300] The device sends the business card image to the server.

[0301] The terminal sends the business card image data to the server in the required format. The input is the business card image data, and the output is the data sent to the server.

[0302] Step 3:

[0303] The server converts the image into text

[0304] The server uses an OCR engine to extract text information from the business card image. The input is the business card image data, and the output is the extracted text data. For example, the extracted information is "Yamada Taro," "ABC Co., Ltd.", and "Manager."

[0305] Step 4:

[0306] The server stores the business card information in a database

[0307] The server saves the extracted business card information in a database. The input is the extracted business card information, and the output is the data stored in the database. Specifically, the following SQL query is executed:

[0308] sql

[0309] INSERT INTO BusinessCardTable (Name, Company, Title) VALUES ('Yamada Taro', 'ABC Co., Ltd.', 'Manager')

[0310] Step 5:

[0311] The server notifies the sales department

[0312] The server sends a notification email to the sales department based on the saved business card information. The input is the business card information saved in the database, and the output is a notification email to the sales department.

[0313] DM policy processing

[0314] Step 1:

[0315] The server analyzes the business card information

[0316] The server retrieves business card information from a database and analyzes it according to specific criteria. The input is the business card information retrieved from the database, and the output is the analysis result. For example, a customer list for a specific industry is generated.

[0317] Step 2:

[0318] The server generates the marketing message

[0319] The server generates a marketing message tailored to each customer based on the analysis results. The input is the analysis results, and the output is the generated marketing message. For example, a message such as "We'd like to share a special offer with you" is generated.

[0320] Step 3:

[0321] The server sends a DM

[0322] The server generates a marketing message and sends it to the target user via email or post. The input is the generated marketing message, and the output is the message sent to the target user.

[0323] By following these specific steps, the system will improve the efficiency of call center operations and customer satisfaction.

[0324] (Application example 1)

[0325] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0326] In food delivery services, efficient response to customer inquiries, ordering support, and the creation and transmission of marketing messages to specific customers requires the rapid and accurate processing of multiple pieces of information. However, with conventional systems, user input and responses to inquiries are often done manually, resulting in inefficiencies. Furthermore, managing voice data and business card information is cumbersome, and notifications to specific departments and the creation of marketing messages are not automated, limiting the ability to improve operational efficiency and customer satisfaction.

[0327] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0328] In this invention, the server includes means for receiving information input by a user, means for storing the received information in a database, means for analyzing questions submitted by the user, means for automatically generating answers based on the analysis results, means for converting voice data into text data, means for converting business card image data into text data, means for notifying a specific department of the converted business card information, means for receiving voice data, means for providing specific information based on the generated text data, means for analyzing input information received from the user and generating a message based on the specific information, and means for transmitting the generated message using a communication means. This allows inquiries, order support, and the generation and transmission of marketing messages in a food delivery service to be handled centrally and automatically, thereby significantly improving business efficiency and customer satisfaction.

[0329] The "means for receiving information input by the user" is a function for acquiring information input by the user into the application and incorporating it into the system.

[0330] The "means for saving received information in a database" is a function for storing acquired user information in a database so that it can be reused as needed.

[0331] The "means for analyzing questions submitted by users" is a function for interpreting the content of inquiries submitted by users and analyzing them to generate appropriate answers.

[0332] "Means for automatically generating an answer based on the analysis results" is a function that enables the system to automatically generate an appropriate response based on the analysis results of the question.

[0333] The "means for converting voice data into text data" is a function for converting voice data provided by the user into character data.

[0334] The "means for converting business card image data into text data" is a function for extracting character information from the image data of a business card provided by the user and saving it as text data.

[0335] The "means for notifying a specific department of the converted business card information" is a function for automatically notifying the relevant department of the extracted business card information.

[0336] The "means for receiving voice data" is a function for accepting voice input from the user.

[0337] The "means for providing specific information based on the generated text data" is a function for providing necessary information to the user based on information obtained from voice or character data.

[0338] The "means for analyzing input information received from a user and generating a message based on specific information" is a function for analyzing input information from a user and automatically creating a message based on specific conditions.

[0339] The "means for transmitting the generated message using a communication means" is a function for transmitting the automatically generated message to the user or the relevant department.

[0340] This invention is a system for improving the efficiency of customer service in food delivery services. The system is implemented using a smartphone and includes functions for receiving information entered by users, storing the received information in a database, converting voice input into text, converting business card images into text, and generating and sending marketing messages based on this information.

[0341] The server provides a dedicated application to the smartphone to receive the information entered by the user. This application has an interface where the user can enter basic information such as name, address, and phone number. The entered information is sent to the server in real time and stored in a database.

[0342] In the food delivery business, inquiries from customers occur frequently. When a customer enters a question via chat or voice through the application, the question is sent to the server. The server analyzes the question using natural language processing technology and automatically generates an appropriate answer from the FAQ database. At this time, the voice data is converted into text data using the Google Speech-to-Text API, and the text data is analyzed using the Google Cloud Natural Language API. Based on the analyzed data, an appropriate answer is extracted from the FAQ database and displayed to the customer.

[0343] Furthermore, if a customer provides image data of their business card, the application will activate the smartphone camera to take a picture of the business card. The captured image data will be sent to the server and converted into text data using the Google Cloud Vision API. The converted business card information will be stored in a database and notified to a specific department.

[0344] Regarding the generation of marketing messages, the server analyzes relevant data and automatically generates messages based on specific conditions. The generated messages are then sent to customers and related departments via communication means, which allows for targeting specific customers and improving the effectiveness of campaigns.

[0345] Here's an example of what happens when a customer uses voice input:

[0346] 1. User: "Launch the app."

[0347] 2. Application: "Hello, how can I help you?"

[0348] 3. User: "I'd like to order a pizza. Can you tell me what kind?"

[0349] 4. The application converts the speech to text and returns the appropriate information.

[0350] And here's an example of a prompt for the generative AI model:

[0351] A user asked, "What's today's special?" Find the right answer in our FAQ database.

[0352] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0353] Step 1:

[0354] The user launches the application and enters the required basic information (name, address, phone number, etc.).

[0355] Input: Basic information entered by the user.

[0356] Output: The device converts the basic information into an appropriate format and sends it to the server.

[0357] Step 2:

[0358] The server stores the received basic information in a database.

[0359] Input: Basic information sent from the device.

[0360] Output: Basic information is stored in the database. Specifically, the SQL query is executed and saved as follows: "INSERT INTO UserTable (Name, Address, Phone) VALUES ('Taro', '123 Main St', '090-1234-5678')".

[0361] Step 3:

[0362] Users use the chat box or voice input function within the application to make inquiries or orders.

[0363] Input: Text or voice input by the user.

[0364] Output: The device sends the entered text or voice data to the server.

[0365] Step 4:

[0366] The server uses a speech recognition engine to convert the voice data into text data.

[0367] Input: Audio data sent from the device.

[0368] Output: The audio data is converted to text using the Google Speech-to-Text API.

[0369] Step 5:

[0370] The server uses a natural language processing (NLP) engine to analyze the text data and generate appropriate answers.

[0371] Input: Text data.

[0372] Output: Answers based on the analysis results. Specifically, the text data is analyzed using the Google Cloud Natural Language API, and appropriate answers are extracted from the FAQ database.

[0373] Step 6:

[0374] The server generates a response and sends it to the terminal.

[0375] Input: The server-generated answer.

[0376] Output: The terminal displays the answer it received to the user, specifically, in the chat box.

[0377] Step 7:

[0378] The user provides a business card image through the application.

[0379] Input: Business card image data taken by the user with the smartphone camera.

[0380] Output: The terminal sends the business card image data to the server.

[0381] Step 8:

[0382] The server uses an OCR engine to convert the business card image data into text data.

[0383] Input: Business card image data sent from the device.

[0384] Output: The business card image data is converted to text data using the Google Cloud Vision API.

[0385] Step 9:

[0386] The server stores the converted business card information in a database and notifies a specific department.

[0387] Input: Business card information extracted by the OCR engine.

[0388] Output: The business card information is stored in a database and a notification is sent to a specific department. Specifically, an SQL query is executed, saved, and notified via email or the internal notification system.

[0389] Step 10:

[0390] A server generates a marketing message and transmits it using a communication means.

[0391] Input: Customer data and business card information stored in a database.

[0392] Output: The generated marketing messages are sent to the target users by analyzing the information in the database, generating messages based on the conditions, and sending them via email or in-app notifications.

[0393] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0394] This invention is a system for improving the efficiency of call center operations and customer satisfaction. This system automatically responds to inquiries and requests from users, performs voice recognition, collects business card information, and combines it with an emotion engine that recognizes the user's emotions to provide a more personalized response.

[0395] Explanation of program processing

[0396] User reception process

[0397] 1. A user accesses the system

[0398] A user accesses the system from a terminal through a web browser or application. For example, the user accesses the system's URL in a browser.

[0399] 2. The user enters basic information

[0400] The user enters basic information such as "name," "email address," and "phone number" into the input form. Specifically, the user enters "Taro," "taro@example.com," "090-1234-5678," etc.

[0401] 3. The device sends the input information to the server

[0402] The terminal converts the input information into an appropriate format and sends it to the server.

[0403] 4. The server stores the information in a database

[0404] The server stores the received user information in the database. For example, it executes an SQL query like "INSERT INTO UserTable (Name, Email, Phone) VALUES ('Taro', 'taro@example.com', '090-1234-5678')".

[0405] Auto-Response Processing

[0406] 1. The user enters a question

[0407] The user types a question into the chat box on the device, for example, "What are your business hours?"

[0408] 2. The device sends a question to the server

[0409] The terminal sends the entered question to the server.

[0410] 3. The server parses the question

[0411] The server uses a natural language processing (NLP) engine to parse the question, in this example extracting the keyword "business hours."

[0412] 4. The server searches the FAQ database

[0413] The server queries the FAQ database based on the analysis results and retrieves the appropriate answer, for example, "Business hours are Monday through Friday, 9:00 AM to 6:00 PM."

[0414] 5. The server sends the answer to the device

[0415] The server sends the obtained answer to the terminal.

[0416] 6. The device displays the answer to the user

[0417] The device will display the response it receives in the chat box.

[0418] Voice-enabled processing

[0419] 1. The user selects a voice input

[0420] The user clicks the "Speech Input" button in the chat box.

[0421] 2. The device records the audio

[0422] The device activates the microphone and records the user's voice. For example, the user might say, "I'd like to know more about this product."

[0423] 3. The device sends the audio data to the server

[0424] The device sends the recorded audio data to the server.

[0425] 4. The server converts the speech to text

[0426] The server uses a speech recognition engine to convert the voice data into text. In this example, the voice "I want to know more about the product" is converted into the text "I want to know more about the product."

[0427] 5. The server generates the answer

[0428] The server analyzes the text information and generates an appropriate response, such as "This product was developed using the latest technology. Please visit our website for more information."

[0429] 6. The server sends the answer to the device

[0430] The server generates a response text and sends it to the terminal.

[0431] 7. The device displays the answer to the user

[0432] The device will display the response it receives in the chat box.

[0433] Business card information collection and processing

[0434] 1. The user provides business card information

[0435] The user takes a photo of the business card with a camera and uploads the image data to the terminal.

[0436] 2. The device sends the business card image to the server

[0437] The terminal transmits the business card image data to the server.

[0438] 3. The server converts the image into text

[0439] The server uses an OCR (Optical Character Recognition) engine to extract text information from the image. In this example, it extracts information such as "Yamada Taro," "ABC Co., Ltd.", and "Manager."

[0440] 4. The server saves the business card information in a database

[0441] The server saves the extracted business card information in the database. For example, it saves it as follows: INSERT INTO BusinessCardTable (Name, Company, Title) VALUES ('Yamada Taro', 'ABC Co., Ltd.', 'Manager').

[0442] 5. The server notifies the sales department

[0443] The server sends the stored information to the sales department staff via email.

[0444] DM policy processing

[0445] 1. The server analyzes the business card information

[0446] The server retrieves the business card information from the database and analyzes it, for example to generate a list of customers in a particular industry.

[0447] 2. The server generates the marketing message

[0448] Based on the analysis results, the server generates a marketing message tailored to each customer, such as "A special offer for Taro Yamada of ABC Co., Ltd."

[0449] 3. The server sends the DM

[0450] Sending server-generated marketing messages to targeted users via email and post.

[0451] Emotion engine processing

[0452] 1. Recognize user emotions

[0453] The server uses an emotion engine to recognize emotions from user input data, for example, extracting emotions such as "anger" or "joy" from user speech or voice recordings.

[0454] 2. Tailor responses based on emotional data

[0455] The server automatically adjusts responses based on the recognized emotion data, for example, generating a calming tone if the user is angry.

[0456] 3. Save emotion data

[0457] The server stores the recognized emotion data in a database for later analysis, for example, to analyze the emotional tendencies of a particular user.

[0458] 4. Generating Marketing Messages Using Emotional Data

[0459] The server generates more personalized marketing messages based on the emotional data analyzed by the emotion engine. For example, it might suggest a "relaxing offer" to a user with a neutral emotion.

[0460] This allows the system to respond quickly and accurately to user inquiries and requests, and by combining voice recognition and emotion recognition, it will improve business efficiency and customer satisfaction.

[0461] The processing flow will be explained below.

[0462] User reception process

[0463] Step 1:

[0464] A user accesses the system

[0465] A user accesses the system from a terminal through a web browser or application. For example, the user accesses the system's URL in a browser.

[0466] Step 2:

[0467] The user enters basic information

[0468] The user enters basic information such as "name," "email address," and "phone number" into the input form. Specifically, the user enters "Taro," "taro@example.com," "090-1234-5678," etc.

[0469] Step 3:

[0470] The device sends the input information to the server

[0471] The terminal converts the input information into an appropriate format and sends it to the server.

[0472] Step 4:

[0473] The server stores the information in a database

[0474] The server stores the received user information in the database. For example, it executes an SQL query like "INSERT INTO UserTable (Name, Email, Phone) VALUES ('Taro', 'taro@example.com', '090-1234-5678')".

[0475] Auto-Response Processing

[0476] Step 1:

[0477] The user enters a question

[0478] The user types a question into the chat box on the device, for example, "What are your business hours?"

[0479] Step 2:

[0480] The device sends a question to the server

[0481] The terminal sends the entered question to the server.

[0482] Step 3:

[0483] The server parses the question

[0484] The server uses a natural language processing (NLP) engine to parse the question, in this example extracting the keyword "business hours."

[0485] Step 4:

[0486] The server searches the FAQ database

[0487] The server queries the FAQ database based on the analysis results and retrieves the appropriate answer, for example, "Business hours are Monday through Friday, 9:00 AM to 6:00 PM."

[0488] Step 5:

[0489] The server sends the answer to the device

[0490] The server sends the obtained answer to the terminal.

[0491] Step 6:

[0492] The device displays the answer to the user

[0493] The device will display the response it receives in the chat box.

[0494] Voice-enabled processing

[0495] Step 1:

[0496] The user selects a voice input

[0497] The user clicks the "Speech Input" button in the chat box.

[0498] Step 2:

[0499] The device records audio

[0500] The device activates the microphone and records the user's voice. For example, the user might say, "I'd like to know more about this product."

[0501] Step 3:

[0502] The device sends the audio data to the server.

[0503] The device sends the recorded audio data to the server.

[0504] Step 4:

[0505] The server converts the speech to text

[0506] The server uses a speech recognition engine to convert the voice data into text. In this example, the voice "I want to know more about the product" is converted into the text "I want to know more about the product."

[0507] Step 5:

[0508] The server generates the answer

[0509] The server analyzes the text information and generates an appropriate response, such as "This product was developed using the latest technology. Please visit our website for more information."

[0510] Step 6:

[0511] The server sends the answer to the device

[0512] The server generates a response text and sends it to the terminal.

[0513] Step 7:

[0514] The device displays the answer to the user

[0515] The device will display the response it receives in the chat box.

[0516] Business card information collection and processing

[0517] Step 1:

[0518] User provides business card information

[0519] The user takes a photo of the business card with a camera and uploads the image data to the terminal.

[0520] Step 2:

[0521] The device sends the business card image to the server.

[0522] The terminal transmits the business card image data to the server.

[0523] Step 3:

[0524] The server converts the image into text

[0525] The server uses an OCR (Optical Character Recognition) engine to extract text information from the image. In this example, it extracts information such as "Yamada Taro," "ABC Co., Ltd.", and "Manager."

[0526] Step 4:

[0527] The server stores the business card information in a database

[0528] The server saves the extracted business card information in the database. For example, it saves it as follows: INSERT INTO BusinessCardTable (Name, Company, Title) VALUES ('Yamada Taro', 'ABC Co., Ltd.', 'Manager').

[0529] Step 5:

[0530] The server notifies the sales department

[0531] The server sends the stored information to the sales department staff via email.

[0532] DM policy processing

[0533] Step 1:

[0534] The server analyzes the business card information

[0535] The server retrieves the business card information from the database and analyzes it, for example to generate a list of customers in a particular industry.

[0536] Step 2:

[0537] The server generates the marketing message

[0538] Based on the analysis results, the server generates a marketing message tailored to each customer, such as "A special offer for Taro Yamada of ABC Co., Ltd."

[0539] Step 3:

[0540] The server sends a DM

[0541] Sending server-generated marketing messages to targeted users via email and post.

[0542] Emotion engine processing

[0543] Step 1:

[0544] Recognize user emotions

[0545] The server uses an emotion engine to recognize emotions from user input data, for example, extracting emotions such as "anger" or "joy" from user speech or voice recordings.

[0546] Step 2:

[0547] Tailor responses based on emotional data

[0548] The server automatically adjusts responses based on the recognized emotion data, for example, generating a calming tone if the user is angry.

[0549] Step 3:

[0550] Storing emotion data

[0551] The server stores the recognized emotion data in a database for later analysis, for example, to analyze the emotional tendencies of a particular user.

[0552] Step 4:

[0553] Generating marketing messages using emotional data

[0554] The server generates more personalized marketing messages based on the emotional data analyzed by the emotion engine. For example, it might suggest a "relaxing offer" to a user with a neutral emotion.

[0555] The above is the specific processing flow of the system of the present invention, which enables quick and accurate responses to user inquiries and requests, and by combining speech recognition and emotion recognition, improves the efficiency of call center operations and customer satisfaction.

[0556] Example 2

[0557] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0558] Conventional call center systems have limited capabilities for responding to user inquiries, making it difficult to provide personalized responses tailored to individual needs. Furthermore, there are insufficient means for improving customer satisfaction by utilizing advanced technologies such as voice recognition and emotion recognition. Furthermore, collecting business card information and generating marketing messages based on that information is time-consuming, requiring increased efficiency. To address these issues, the present invention provides a system that integrates the automation of inquiry responses, personalized responses using emotion recognition, and the collection and analysis of business card information.

[0559] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for receiving information input by a user, means for saving the received information in a database, means for analyzing a question submitted by a user, means for automatically generating an answer based on the analysis result, means for converting voice data into text data, means for converting business card image data into text data, means for notifying the sales department of the converted business card information, means for recognizing the user's emotion, and means for adjusting a response based on the recognized emotion. This improves the accuracy and speed of inquiry responses, enables personalized responses according to customer emotions, and further realizes more efficient marketing measures through the automatic collection and analysis of business card information.

[0560] "Means for receiving user-entered information" refers to the functionality of devices or software that capture data entered by users through forms or chat boxes and incorporate it into the system.

[0561] "Means for storing received information in a database" refers to a technology for permanently storing data received from a user via a database system such as SQL.

[0562] "Means for analyzing questions submitted by users" refers to software functionality that uses natural language processing (NLP) technology to understand the content of users' questions and extract keywords and intent.

[0563] "Means for automatically generating answers based on the analysis results" refers to the functions of algorithms and databases that automatically create appropriate answers based on the analyzed question content and provide them to users.

[0564] "Means for converting voice data into text data" refers to technology that uses a voice recognition engine to convert voice files or real-time voice input into text format.

[0565] The "means for converting business card image data into text data" is a software function for extracting text information from business card images using optical character recognition (OCR) technology.

[0566] "Means for notifying the sales department of the converted business card information" is a function for notifying the sales staff of the extracted business card information via email or a notification system.

[0567] "Means for recognizing user emotions" refers to technology that uses an emotion analysis engine to extract emotions from user input data and voice.

[0568] The "means for tailoring responses based on recognized emotions" is a software function for automatically generating and delivering responses with appropriate tone and content depending on the user's emotional state.

[0569] The present invention is a system that automatically responds to inquiries and requests from users, providing more personalized responses by performing voice recognition, collecting business card information, and even emotion recognition. This system operates using the following hardware and software.

[0570] Hardware

[0571] 1. Terminal: A device (e.g., computer, smartphone, tablet) through which a user enters information.

[0572] 2. Server: A computer system running hardware and software for processing and storing data.

[0573] 3. Database: A data storage system for persistently storing information (e.g., MySQL, PostgreSQL).

[0574] software

[0575] 1. Web Browser or Application: The interface through which the user accesses the system.

[0576] 2. Natural Language Processing (NLP) engine: Software for analyzing user questions (e.g., Google Dialogflow).

[0577] 3. Speech recognition engine: Software for converting voice data into text (e.g., Google Cloud Speech-to-Text).

[0578] 4. Optical Character Recognition (OCR) engine: Software for extracting text information from business card images (e.g., Tesseract OCR).

[0579] 5. Sentiment analysis engine: Software for recognizing user emotions (e.g., IBM Watson® Tone Analyzer).

[0580] 6. Generative AI models: Software for tailoring responses based on user sentiment (e.g., GPT-3®).

[0581] Specific examples

[0582] 1. User acceptance process

[0583] A user accesses the system's web page using a smartphone and enters their name (e.g., "Taro"), email address (e.g., "taro@example.com"), and phone number (e.g., "090-1234-5678"). The device sends this information to the server, which then stores the received information in a database.

[0584] 2. Auto-Response Processing

[0585] The user enters a question into the chat box (e.g., "What are your business hours?"). The device sends the question to the server, which analyzes the question using an NLP engine. Based on the analysis results, the server retrieves an appropriate answer from the FAQ database (e.g., "Business hours are Monday through Friday, 9 a.m. to 6 p.m.") and displays it to the user through the device.

[0586] 3. Voice-enabled processing

[0587] The user clicks the voice input button and says, "I'd like to know more about this product." The device records the voice and sends it to the server. The server uses a speech recognition engine to convert the voice data into text and generate an answer (e.g., "This product was developed using the latest technology. Please visit our website for more information."). The answer is returned to the device and displayed to the user.

[0588] 4. Business card information collection and processing

[0589] A user takes a photo of a business card with their smartphone and uploads it to the system. The device sends the business card image to the server, which uses an OCR engine to extract text information from the image. The extracted information (e.g., "Yamada Taro," "XYZ Co., Ltd.", "Manager") is stored in a database and notified to the sales department.

[0590] 5. Emotional Engine Processing

[0591] The emotion analysis engine recognizes emotional data (e.g., "anger" or "happiness") from the user's comments and questions, and the server generates a response that corresponds to that emotion. For example, if the user is angry, the server generates a response in a calming tone (e.g., "We apologize for any inconvenience this may have caused you"). The generated response is provided to the user via the device.

[0592] Prompt Sentence Examples

[0593] The prompts that users enter into the system include:

[0594] 1. "What are your opening hours?"

[0595] 2. "I want to know more about the product"

[0596] 3. "I want to know about new product release information."

[0597] As a result, this system responds quickly and accurately to user inquiries and requests, and by combining voice recognition and emotion recognition, it achieves improved business efficiency and customer satisfaction.

[0598] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0599] Program processing flow

[0600] User reception process

[0601] Step 1:

[0602] A user accesses the system. The user opens a web browser or application on the terminal and accesses the system's URL. Input: Access request is made by the URL output of the web browser. Output: Login screen or initial screen is displayed.

[0603] Step 2:

[0604] The user enters basic information. The user enters their name, email address, and phone number into the system form. Specific examples of input include "Taro," "taro@example.com," and "090-1234-5678." Input: Basic information of the user. Output: Entered information data.

[0605] Step 3:

[0606] The terminal sends the input information to the server. The terminal converts the input data into JSON format and sends it to the server via a POST request. Input: JSON format of user input information. Output: JSON data is sent to the server.

[0607] Step 4:

[0608] The server saves the information in the database. The server saves the received data in the database using an SQL query. Specifically, it executes the query "INSERT INTO UserTable (Name, Email, Phone) VALUES ('Taro', 'taro@example.com', '090-1234-5678')". Input: User information in JSON format. Output: Information saved in the database.

[0609] Auto-Response Processing

[0610] Step 1:

[0611] User types a question. User types a question into the chat box (e.g., "What are your business hours?"). Input: User's question text. Output: Question typed into the chat box.

[0612] Step 2:

[0613] The terminal sends the question to the server. The terminal converts the entered question into JSON format and sends it to the server. Input: JSON format of the question text. Output: JSON data sent to the server.

[0614] Step 3:

[0615] The server analyzes the question. The server uses a natural language processing engine to analyze the question and extract keywords. Input: The question text "What are your business hours?" Output: The keyword "business hours".

[0616] Step 4:

[0617] The server searches the FAQ database. The server queries the FAQ table based on the extracted keywords to get the appropriate answer. For example, it executes the query "SELECT Answer FROM FAQTable WHERE Question LIKE '%Business Hours%'". Input: Keyword "Business Hours". Output: Answer text from the FAQ database.

[0618] Step 5:

[0619] The server sends the answer to the terminal. The server converts the answer text to JSON format and sends it to the terminal. Input: Answer from the FAQ database. Output: Answer data in JSON format.

[0620] Step 6:

[0621] The terminal displays the answer to the user. The terminal displays the received answer in the chat box. Input: JSON data sent from the server. Output: Answer displayed in the chat box.

[0622] Voice-enabled processing

[0623] Step 1:

[0624] User selects audio input. User clicks "Audio Input" button in chat box. Input: Select audio input. Output: Activate microphone.

[0625] Step 2:

[0626] The device records the voice. The user speaks (e.g., "I'd like to know more about this product") and the device records the voice. Input: User's speech. Output: Recorded voice data.

[0627] Step 3:

[0628] The device sends audio data to the server. The device sends the recorded audio data to the server via a POST request. Input: Recorded audio data. Output: Audio data sent to the server.

[0629] Step 4:

[0630] The server converts speech to text. The server uses a speech recognition engine to convert the speech data into text format. Input: speech data. Output: converted text data.

[0631] Step 5:

[0632] The server generates the answer. The server analyzes the text question content and generates the corresponding answer. Input: Converted text data. Output: Generated answer text.

[0633] Step 6:

[0634] The server sends the answer to the terminal. The server converts the answer to JSON format and sends it to the terminal. Input: Generated answer text. Output: JSON format answer data.

[0635] Step 7:

[0636] The terminal displays the answer to the user. The terminal displays the received answer in the chat box. Input: JSON data sent from the server. Output: Answer displayed in the chat box.

[0637] Business card information collection and processing

[0638] Step 1:

[0639] The user provides business card information. The user takes a photo of the business card with the smartphone camera and uploads it to the system. Input: Business card image. Output: Uploaded image data.

[0640] Step 2:

[0641] The terminal sends the business card image to the server. The terminal sends the business card image data to the server. Input: Business card image data. Output: Image data sent to the server.

[0642] Step 3:

[0643] The server converts the image to text. The server uses an OCR engine to extract text information from the image data. Input: Business card image data. Output: Converted text information.

[0644] Step 4:

[0645] The server saves the business card information in the database. The server saves the extracted business card information in the database using an SQL query. For example, execute the query "INSERT INTO BusinessCardTable (Name, Company, Title) VALUES ('Yamada Taro', 'XYZ Co., Ltd.', 'Manager')". Input: Converted text information. Output: Information saved in the database.

[0646] Step 5:

[0647] The server notifies the sales department. The server sends a notification email to the sales department based on the saved information. Input: Business card information. Output: Notification email to the sales department.

[0648] Emotion engine processing

[0649] Step 1:

[0650] The server recognizes the user's emotions. The server uses an emotion analysis engine to extract emotions from the user's input data. Input: User's input data. Output: Extracted emotion data.

[0651] Step 2:

[0652] The server tailors the response based on the emotion data. The server tailors the response using a generative AI model based on the extracted emotion data. Input: Emotion data. Output: Tailored response text.

[0653] Step 3:

[0654] The server stores the emotion data. The server stores the recognized emotion data in a database and uses it for later analysis. Input: Emotion data. Output: Emotion data saved in the database.

[0655] Step 4:

[0656] The server generates a marketing message using the emotion data. The server generates a personalized marketing message for each user based on the emotion data. For example, a user with a calm emotion might be offered a "relaxing offer." Input: Emotion data. Output: Generated marketing message.

[0657] (Application example 2)

[0658] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0659] Autonomous vehicles require rapid and appropriate responses to passenger inquiries and requests. However, conventional systems have struggled to recognize passenger emotions and provide personalized responses based on those emotions. They also lacked efficient means to digitize passenger business card information and generate marketing messages. The present invention aims to solve these problems and improve passenger service.

[0660] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0661] In this invention, the server includes means for receiving information input by a user, means for storing the received information in a database, means for analyzing questions submitted by users, means for automatically generating answers based on the analysis results, means for converting voice data to text data, means for converting business card image data to text data, means for notifying the sales department of the converted business card information, means for receiving and analyzing voice input from passengers in the vehicle and generating an automatic response, means for recognizing passenger emotions and providing a response according to the emotions, and means for digitizing and storing passenger business card information, thereby making it possible to respond quickly and appropriately to passenger inquiries and requests and provide personalized services based on emotions.

[0662] "Information entered by the user" refers to data such as the name, email address, phone number, and questions that the user provides to the system.

[0663] "Means for storing in a database" refers to a method using a server or storage device to organize and efficiently store the received information.

[0664] "Means of analysis" refers to the process of using natural language processing technology to understand the content of questions and input data submitted by users and determine how to respond.

[0665] "Means for automatically generating answers" refers to algorithms or programs that generate optimal answers from analysis results or pre-set information and present them to the user.

[0666] "Means for converting voice data into text data" refers to the process of converting a user's voice input into text information using voice recognition technology.

[0667] The "means of converting business card image data into text data" is the process of extracting text information from business card images using OCR (optical character recognition) technology.

[0668] The "means of notifying the sales department" is a method of transmitting the extracted business card information and other important information to the sales department staff via email or a notification system.

[0669] "Means for receiving and analyzing passenger voice inputs within a vehicle and generating an automatic response" refers to a system for recognizing passenger voice instructions within an automated vehicle, analyzing the content of the instructions, and generating an appropriate response.

[0670] "Means for recognizing passenger emotions and providing emotionally appropriate responses" refers to a process for determining the passenger's emotional state from the passenger's tone of voice and content of speech, and generating a response with an appropriate tone and content.

[0671] The "means for digitizing and storing passenger business card information" refers to a method of capturing business cards provided by passengers as digital images, converting them into text data, and storing them in the system.

[0672] The present invention provides a system for streamlining passenger service and providing personalized responses in autonomous vehicles, which implements a series of processes for efficiently acquiring, analyzing, and responding to passenger input information.

[0673] System Overview

[0674] This system is composed of the following means:

[0675] 1. A means of receiving information entered by the user

[0676] Passengers enter information such as their name, destination, and questions using a smartphone app or a touch panel inside the vehicle, and send it to the system.

[0677] 2. A means of storing the received information in a database

[0678] The received information is stored in a database, for example, using a relational database such as PostgreSQL.

[0679] 3. A means of analyzing user-submitted questions

[0680] Questions are parsed using natural language processing (NLP) techniques, using the Google Cloud Natural Language API to extract keywords and intent from the text.

[0681] 4. A means of automatically generating answers based on analysis results

[0682] Based on the analysis results, an appropriate response is generated, utilizing a pre-configured FAQ database and generative AI models.

[0683] 5. Means of converting audio data into text data

[0684] Passenger voice input is converted into text data using Google Cloud Speech-to-Text.

[0685] 6. How to convert business card image data into text data

[0686] When a passenger provides their business card information, OCR (optical character recognition) technology, such as Tesseract OCR, is used to extract text data from the image of the business card.

[0687] 7. A method for notifying the sales department of the converted business card information

[0688] Business card information is digitized and sent to the sales department via email or internal notification system.

[0689] 8. Means for receiving and analyzing passenger voice inputs in the vehicle to generate automated responses

[0690] Passenger voice commands are received through an in-car microphone, analyzed, and an appropriate response is generated, combining voice recognition and NLP technology.

[0691] 9. A means of recognizing passenger emotions and providing emotional responses

[0692] The system determines the passenger's emotions from their tone of voice and the content of their comments, and uses IBM Watson Emotion Analysis to generate responses based on those emotions.

[0693] 10. A means of digitizing and storing passenger business card information

[0694] The business card information is digitized, stored, and used to generate subsequent marketing messages.

[0695] Specific examples

[0696] Specific examples of voice input

[0697] A passenger says, "Please make the car's air conditioning a little cooler." This speech is collected through a microphone in the car and converted into text using Google Cloud Speech-to-Text: "Please make the car's air conditioning a little cooler." The system then analyzes this text using NLP technology and generates the response message, "The temperature has been adjusted."

[0698] Example prompts for generative AI models

[0699] "Please determine the emotion from the text below. Then suggest an appropriate response: \"How's the weather today?\""

[0700] This enables the system of the present invention to provide personalized services in autonomous vehicles and improve passenger satisfaction.

[0701] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0702] Step 1:

[0703] Users access the system through a touch panel in the car or a smartphone app. They input their destination, name, and other necessary information. For example, a user might input "Destination: Chuo Station" and "Name: Yamada Taro" into the touch panel.

[0704] Input: Destination, name, and other basic information

[0705] Output: The entered information is saved in JSON format on the device.

[0706] How it works: The device displays an input form, and once the user completes the form, it converts the information into JSON format and sends it to the server.

[0707] Step 2:

[0708] The terminal sends the entered information to a database. The server receives it and stores it in the database. A relational database such as PostgreSQL is used.

[0709] Input: User input information in JSON format

[0710] Output: User information stored in the database

[0711] How it works: The server receives the JSON data, generates the appropriate queries, and saves them to the database.

[0712] Step 3:

[0713] The user types a question into the touch panel or chat box of the app, for example, "Please lower the temperature inside the car."

[0714] Input: User's question

[0715] Output: The question sent to the server

[0716] Operation: The device sends the entered question to the server.

[0717] Step 4:

[0718] The server analyzes the user's question using natural language processing (NLP) techniques, including the Google Cloud Natural Language API, to extract keywords and intent from the question.

[0719] Input: User's question

[0720] Output: Parsed keywords and intent

[0721] How it works: The server uses an NLP engine to parse the incoming question and extract keywords (e.g., temperature) and intent (e.g., lower).

[0722] Step 5:

[0723] The server generates an automated response based on the analysis results, for example, "The temperature inside the car has been lowered."

[0724] Input: Parsed keywords and intent

[0725] Output: Auto-generated response message

[0726] How it works: The server references the FAQ database and generative AI models to generate the best response.

[0727] Step 6:

[0728] When a user speaks, the car's microphone records the speech and converts it into text using Google Cloud Speech-to-Text. For example, "Turn up the air conditioning" is converted into text as "Turn up the air conditioning."

[0729] Input: Audio data

[0730] Output: Text data

[0731] How it works: The device records audio and sends the audio data to the server, which uses a speech recognition engine to convert it into text.

[0732] Step 7:

[0733] When a user provides business card information, the system uses an in-car camera to take a picture of the business card and then uses OCR technology to extract the text data from the card. For example, the business card image is converted into "Yamada Taro, company name, job title."

[0734] Input: Business card image data

[0735] Output: Business card text data

[0736] How it works: The device scans a business card and sends the image data to the server, which uses Tesseract OCR to extract the text data.

[0737] Step 8:

[0738] The server stores the converted business card information in a database and notifies the sales department, for example, by sending an email to the sales department.

[0739] Input: Business card text data

[0740] Output: Business card information stored in the database, notification email

[0741] What it does: The server saves the business card information in a database and sends a notification to the specified email address.

[0742] Step 9:

[0743] The server recognizes the passenger's emotions and generates responses based on their emotions using IBM Watson Emotion Analysis. For example, if the user is angry, it generates a calmer response.

[0744] Input: What the user said

[0745] Output: Sentiment analysis results, tailored response message

[0746] How it works: The server analyzes the voice or text data using an emotion recognition engine and generates a response based on the user's emotional state.

[0747] Step 10:

[0748] The server uses the business card information to generate and send marketing messages, which allows for personalized marketing.

[0749] Input: Business card information, user data

[0750] Output: Marketing message

[0751] How it works: The server analyzes the business card information, generates marketing messages tailored to the specific user, and sends them via email or notification system.

[0752] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0753] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0754] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0755] [Second embodiment]

[0756] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0757] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0758] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0759] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0760] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0761] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0762] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0763] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0764] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0765] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0766] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0767] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0768] This invention is a system for improving the efficiency of call center operations and customer satisfaction. This system automatically responds to inquiries and requests from users, performs voice recognition, and collects business card information. It also effectively obtains information useful for sales activities and generates marketing messages.

[0769] Explanation of program processing

[0770] User reception process

[0771] 1. A user accesses the system

[0772] A user accesses the system from a terminal through a web browser or application. For example, the user accesses the system's URL in a browser.

[0773] 2. The user enters basic information

[0774] The user enters basic information such as "name," "email address," and "phone number" into the input form. Specifically, the user enters "Taro," "taro@example.com," "090-1234-5678," etc.

[0775] 3. The device sends the input information to the server

[0776] The terminal converts the input information into an appropriate format and sends it to the server.

[0777] 4. The server stores the information in a database

[0778] The server stores the received user information in the database. For example, it executes an SQL query like "INSERT INTO UserTable (Name, Email, Phone) VALUES ('Taro', 'taro@example.com', '090-1234-5678')".

[0779] Auto-Response Processing

[0780] 1. The user enters a question

[0781] The user types a question into the chat box on the device, for example, "What are your business hours?"

[0782] 2. The device sends a question to the server

[0783] The terminal sends the entered question to the server.

[0784] 3. The server parses the question

[0785] The server uses a natural language processing (NLP) engine to parse the question, in this example extracting the keyword "business hours."

[0786] 4. The server searches the FAQ database

[0787] The server queries the FAQ database based on the analysis results and retrieves the appropriate answer, for example, "Business hours are Monday through Friday, 9:00 AM to 6:00 PM."

[0788] 5. The server sends the answer to the device

[0789] The server sends the obtained answer to the terminal.

[0790] 6. The device displays the answer to the user

[0791] The device will display the response it receives in the chat box.

[0792] Voice-enabled processing

[0793] 1. The user selects a voice input

[0794] The user clicks the "Speech Input" button in the chat box.

[0795] 2. The device records the audio

[0796] The device activates the microphone and records the user's voice. For example, the user might say, "I'd like to know more about this product."

[0797] 3. The device sends the audio data to the server

[0798] The device sends the recorded audio data to the server.

[0799] 4. The server converts the speech to text

[0800] The server uses a speech recognition engine to convert the voice data into text. In this example, the voice "I want to know more about the product" is converted into the text "I want to know more about the product."

[0801] 5. The server generates the answer

[0802] The server analyzes the text information and generates an appropriate response, such as "This product was developed using the latest technology. Please visit our website for more information."

[0803] 6. The server sends the answer to the device

[0804] The server generates a response text and sends it to the terminal.

[0805] 7. The device displays the answer to the user

[0806] The device will display the response it receives in the chat box.

[0807] Business card information collection and processing

[0808] 1. The user provides business card information

[0809] The user takes a photo of the business card with a camera and uploads the image data to the terminal.

[0810] 2. The device sends the business card image to the server

[0811] The terminal transmits the business card image data to the server.

[0812] 3. The server converts the image into text

[0813] The server uses an OCR (Optical Character Recognition) engine to extract text information from the image. In this example, it extracts information such as "Yamada Taro," "ABC Co., Ltd.", and "Manager."

[0814] 4. The server saves the business card information in a database

[0815] The server saves the extracted business card information in the database. For example, it saves it as follows: INSERT INTO BusinessCardTable (Name, Company, Title) VALUES ('Yamada Taro', 'ABC Co., Ltd.', 'Manager').

[0816] 5. The server notifies the sales department

[0817] The server sends the stored information to the sales department staff via email.

[0818] DM policy processing

[0819] 1. The server analyzes the business card information

[0820] The server retrieves the business card information from the database and analyzes it, for example to generate a list of customers in a particular industry.

[0821] 2. The server generates the marketing message

[0822] Based on the analysis results, the server generates a marketing message tailored to each customer, such as "A special offer for Taro Yamada of ABC Co., Ltd."

[0823] 3. The server sends the DM

[0824] Sending server-generated marketing messages to targeted users via email and post.

[0825] As described above, this system responds to user inquiries and requests from multiple angles, thereby improving the efficiency of call center operations and customer satisfaction.

[0826] The processing flow will be explained below.

[0827] User reception process

[0828] Step 1:

[0829] A user accesses the system

[0830] A user accesses the system from a terminal through a web browser or application. For example, the user accesses the system's URL in a browser.

[0831] Step 2:

[0832] The user enters basic information

[0833] The user enters basic information such as "name," "email address," and "phone number" into the input form. Specifically, the user enters "Taro," "taro@example.com," "090-1234-5678," etc.

[0834] Step 3:

[0835] The device sends the input information to the server

[0836] The terminal converts the input information into an appropriate format and sends it to the server.

[0837] Step 4:

[0838] The server stores the information in a database

[0839] The server stores the received user information in the database. For example, it executes an SQL query like "INSERT INTO UserTable (Name, Email, Phone) VALUES ('Taro', 'taro@example.com', '090-1234-5678')".

[0840] Auto-Response Processing

[0841] Step 1:

[0842] The user enters a question

[0843] The user types a question into the chat box on the device, for example, "What are your business hours?"

[0844] Step 2:

[0845] The device sends a question to the server

[0846] The terminal sends the entered question to the server.

[0847] Step 3:

[0848] The server parses the question

[0849] The server uses a natural language processing (NLP) engine to parse the question, in this example extracting the keyword "business hours."

[0850] Step 4:

[0851] The server searches the FAQ database

[0852] The server queries the FAQ database based on the analysis results and retrieves the appropriate answer, for example, "Business hours are Monday through Friday, 9:00 AM to 6:00 PM."

[0853] Step 5:

[0854] The server sends the answer to the device

[0855] The server sends the obtained answer to the terminal.

[0856] Step 6:

[0857] The device displays the answer to the user

[0858] The device will display the response it receives in the chat box.

[0859] Voice-enabled processing

[0860] Step 1:

[0861] The user selects a voice input

[0862] The user clicks the "Speech Input" button in the chat box.

[0863] Step 2:

[0864] The device records audio

[0865] The device activates the microphone and records the user's voice. For example, the user might say, "I'd like to know more about this product."

[0866] Step 3:

[0867] The device sends the audio data to the server.

[0868] The device sends the recorded audio data to the server.

[0869] Step 4:

[0870] The server converts the speech to text

[0871] The server uses a speech recognition engine to convert the voice data into text. In this example, the voice "I want to know more about the product" is converted into the text "I want to know more about the product."

[0872] Step 5:

[0873] The server generates the answer

[0874] The server analyzes the text information and generates an appropriate response, such as "This product was developed using the latest technology. Please visit our website for more information."

[0875] Step 6:

[0876] The server sends the answer to the device

[0877] The server generates a response text and sends it to the terminal.

[0878] Step 7:

[0879] The device displays the answer to the user

[0880] The device will display the response it receives in the chat box.

[0881] Business card information collection and processing

[0882] Step 1:

[0883] User provides business card information

[0884] The user takes a photo of the business card with a camera and uploads the image data to the terminal.

[0885] Step 2:

[0886] The device sends the business card image to the server.

[0887] The terminal transmits the business card image data to the server.

[0888] Step 3:

[0889] The server converts the image into text

[0890] The server uses an OCR (Optical Character Recognition) engine to extract text information from the image. In this example, it extracts information such as "Yamada Taro," "ABC Co., Ltd.", and "Manager."

[0891] Step 4:

[0892] The server stores the business card information in a database

[0893] The server saves the extracted business card information in the database. For example, it saves it as follows: INSERT INTO BusinessCardTable (Name, Company, Title) VALUES ('Yamada Taro', 'ABC Co., Ltd.', 'Manager').

[0894] Step 5:

[0895] The server notifies the sales department

[0896] The server sends the stored information to the sales department staff via email.

[0897] DM policy processing

[0898] Step 1:

[0899] The server analyzes the business card information

[0900] The server retrieves the business card information from the database and analyzes it, for example to generate a list of customers in a particular industry.

[0901] Step 2:

[0902] The server generates the marketing message

[0903] Based on the analysis results, the server generates a marketing message tailored to each customer, such as "A special offer for Taro Yamada of ABC Co., Ltd."

[0904] Step 3:

[0905] The server sends a DM

[0906] Sending server-generated marketing messages to targeted users via email and post.

[0907] Example 1

[0908] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0909] Call center operations must efficiently respond to a large number of inquiries, which requires the rapid and accurate processing of data in various formats. However, conventional systems have difficulty integrating voice recognition, business card information collection, and natural language processing, resulting in problems such as reduced work efficiency and low customer satisfaction. Furthermore, the creation and transmission of marketing messages is also done manually, which is inefficient. There is a need to solve these problems, improve the efficiency of call center operations, and increase customer satisfaction.

[0910] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0911] In this invention, the server includes means for receiving information input by a user, means for storing the received information in a database, means for analyzing questions submitted by users, means for automatically generating answers based on the analysis results, means for converting voice data into text data, means for converting business card image data into text data, means for notifying the sales department of the converted business card information, means for analyzing questions using natural language processing technology, means for searching an FAQ database to obtain an appropriate answer, means for making voice input selectable, means for transmitting recorded voice data to the server, means for displaying answers to the user, means for formatting the input information and transmitting it to the server, means for analyzing the business card information and generating a marketing message, and means for transmitting the generated marketing message to the user. This enables efficient response to inquiries, voice recognition, automatic collection of business card information, and automatic generation and transmission of marketing messages.

[0912] "Means for receiving information input by a user" refers to an interface for receiving data input by a user.

[0913] "Means for saving received information to a database" refers to the process of saving the data entered by the user in a database for storing the data.

[0914] "Means for analyzing a question submitted by a user" refers to a processing technology for analyzing a question entered by a user and understanding its intent.

[0915] "Means for automatically generating answers based on the analysis results" refers to a process for automatically generating appropriate answers to the analyzed questions.

[0916] "Means for converting voice data into text data" refers to technology for converting information input by voice into text format.

[0917] "Means for converting business card image data into text data" refers to technology for converting business card photos or scanned data into text information.

[0918] "Means for notifying the sales department of the converted business card information" refers to the process of notifying the sales department of the data extracted from the business card.

[0919] "Means for analyzing questions using natural language processing technology" refers to technology for analyzing user questions using natural language processing technology.

[0920] "Means for searching the FAQ database to obtain an appropriate answer" refers to the process of searching the FAQ database to obtain an appropriate answer to a user's question.

[0921] "Means for enabling selection of voice input" refers to a technology that provides an interface that allows a user to select a voice input.

[0922] "Means for transmitting recorded voice data to a server" refers to a communication technology for transmitting recorded voice data to a server.

[0923] The "means for displaying the answer to the user" refers to an interface for displaying the generated answer on the user's terminal.

[0924] "Means for formatting input information and sending it to the server" refers to the process of converting the information input by the user into an appropriate format and sending it to the server.

[0925] "Means for analyzing business card information and generating a marketing message" refers to technology for analyzing information obtained from a business card and generating a specific marketing message.

[0926] "Means for sending generated marketing messages to users" refers to the process of sending generated marketing messages to target users.

[0927] This invention is a system aimed at improving the efficiency of call center operations and customer satisfaction. This system automatically responds to inquiries and requests from users, performs voice recognition, and collects business card information. It also effectively obtains information useful for sales activities and generates marketing messages.

[0928] The main components of the system are a terminal for users to input information, a server to process the input information, and a database to store the data. The specific hardware and software configurations are described below.

[0929] Hardware and software used

[0930] 1. Device:

[0931] A device used by a user, such as a smartphone, tablet, or PC. The device accesses the system through a web browser or a dedicated application.

[0932] 2. Server:

[0933] Data processing is performed using high-performance servers (e.g., AWS, Google Cloud) with a natural language processing engine (e.g., Google Natural Language API), a speech recognition engine (e.g., Google Speech-to-Text API), and an OCR engine (e.g., Tesseract OCR) installed.

[0934] 3. Database:

[0935] A relational database (e.g., MySQL, PostgreSQL) that stores user information, FAQ data, and business card information.

[0936] User reception process

[0937] User accesses and enters basic information: The user uses a terminal to access the system through a web browser or application. Then, the user enters basic information such as "name," "email address," and "phone number." For example, "Taro," "taro@example.com," and "090-1234-5678" are entered.

[0938] Sending and saving input information: The device sends the input information to the server, which stores it in a database.

[0939] Auto-Response Processing

[0940] User's question input and transmission: The user inputs a question into the chat box, and the terminal transmits the question to the server. For example, "What are your business hours?" is input.

[0941] Question analysis and answer generation: The server uses a natural language processing engine to analyze the question and search the FAQ database to get the appropriate answer, which is then sent to the device and displayed to the user.

[0942] Voice-enabled processing

[0943] Select and record voice input: The user clicks the voice input button in the chat box, and the device activates the microphone to record the voice. For example, "I would like to know more about this product."

[0944] Sending and converting voice data: The device sends the recorded voice data to the server, which then converts it into text using a voice recognition engine.

[0945] Generate and display an answer: The server analyzes the text information, generates an appropriate answer, sends it to the device, and displays it to the user. For example, "This product has been developed using the latest technology. Please visit our website for more information." is generated and displayed.

[0946] Business card information collection and processing

[0947] Providing and sending business card information: The user takes a photo of the business card with the camera and uploads it to the device, which then sends the image data to the server.

[0948] Image data conversion and storage: The server uses an OCR engine to extract text information from the image and store it in a database. It also sends the stored information to the sales department via email.

[0949] DM policy processing

[0950] Analyzing business card information and generating marketing messages: The server retrieves business card information from the database and generates a list of customers in a specific industry. The server then generates marketing messages tailored to each customer based on the analysis results.

[0951] Sending direct mail: Sending server-generated marketing messages to targeted users via email or post. For example, a message such as "We'd like to share a special offer with you" is generated.

[0952] With the above-described configuration and procedures, the system can improve the efficiency of call center operations in a variety of ways and increase customer satisfaction.

[0953] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0954] User reception process

[0955] Step 1:

[0956] A user accesses the system

[0957] A user starts a web browser or application on a device such as a PC or smartphone and accesses the system's URL. The input is the system's URL, and the output is the display of the system's web page.

[0958] Step 2:

[0959] The user enters basic information

[0960] A user enters basic information such as "name," "email address," and "phone number" into an input form displayed on the system. The input is the user's personal information (e.g., "Taro," "taro@example.com," "090-1234-5678"), and the output is that this information is displayed in the input fields.

[0961] Step 3:

[0962] The device sends the input information to the server

[0963] The device converts the information entered by the user into an appropriate format, such as JSON, and sends it to the server using an HTTP POST request. The input is the personal information entered by the user, and the output is the formatted data. As a concrete example, the following JSON data is sent from the device to the server:

[0964] json

[0965] {

[0966] "name": "Taro",

[0967] "email": "taro@example.com",

[0968] "phone": "090-1234-5678"

[0969] }

[0970] Step 4:

[0971] The server stores the information in a database

[0972] The server analyzes the received user information and executes SQL queries to store it in the database. The input is the personal information sent from the terminal, and the output is the data stored in the database. Specifically, the following SQL queries are executed:

[0973] sql

[0974] INSERT INTO UserTable (Name, Email, Phone) VALUES ('Taro', 'taro@example.com', '090-1234-5678')

[0975] Auto-Response Processing

[0976] Step 1:

[0977] The user enters a question

[0978] The user types a question into the chat box on the device and clicks the send button. The input is the user's question (e.g., "What are your business hours?"), and the output is the question displayed in the chat box.

[0979] Step 2:

[0980] The device sends a question to the server

[0981] The device sends the entered question to the server in an appropriate format (e.g. JSON format). The input is the question entered by the user, and the output is the data sent to the server. The content sent is as follows:

[0982] json

[0983] {

[0984] "question": "What are your opening hours?"

[0985] }

[0986] Step 3:

[0987] The server parses the question

[0988] The server uses a natural language processing engine to analyze the question and extract important keywords and intent. The input is the user's question data, and the output is the analyzed intent and keywords. For example, the keyword "business hours" is extracted.

[0989] Step 4:

[0990] The server searches the FAQ database

[0991] The server queries the FAQ database based on the analysis results to obtain the appropriate answer. The input is the analyzed keyword (e.g., "business hours"), and the output is the answer data from the FAQ database. Specifically, the following SQL query is executed:

[0992] sql

[0993] SELECT Answer FROM FAQTable WHERE Question LIKE '%Business Hours%'

[0994] And you find the answer, "Our business hours are Monday through Friday, 9:00 AM to 6:00 PM."

[0995] Step 5:

[0996] The server sends the answer to the device

[0997] The server converts the answer it gets into the appropriate format and sends it to the terminal as an HTTP response. The input is the answer from the FAQ database, and the output is the formatted response sent. An example response is as follows:

[0998] json

[0999] {

[1000] "answer": "Our business hours are Monday to Friday, 9am to 6pm."

[1001] }

[1002] Step 6:

[1003] The device displays the answer to the user

[1004] The terminal formats the answer received from the server and displays it in the chat box. The input is the answer data from the server, and the output is the display of the answer to the user.

[1005] Voice-enabled processing

[1006] Step 1:

[1007] The user selects a voice input

[1008] The user clicks the "Voice Input" button in the chat box. The input is an operation that enables voice input, and the output is an operation that starts voice recording.

[1009] Step 2:

[1010] The device records audio

[1011] The device activates the microphone and records the user's voice. The input is the user's voice (e.g., "I want to know more about this product"), and the output is the recorded voice data.

[1012] Step 3:

[1013] The device sends the audio data to the server.

[1014] The device sends the recorded audio data to the server in the appropriate format. The input is the recorded audio data, and the output is the data sent to the server.

[1015] Step 4:

[1016] The server converts the speech to text

[1017] The server uses a speech recognition engine to convert the voice data into text. The input is the voice data sent to the server, and the output is text data. For example, the text might be "I'd like to know more about this product."

[1018] Step 5:

[1019] The server generates the answer

[1020] The server analyzes the text information and generates an appropriate answer. The input is speech data converted into text, and the output is the generated answer text. For example, the answer generated might be, "This product was developed using the latest technology. Please visit our website for more information."

[1021] Step 6:

[1022] The server sends the answer to the device

[1023] The server generates a response and sends it to the terminal in text format. The input is the generated response text, and the output is the formatted response transmission.

[1024] Step 7:

[1025] The device displays the answer to the user

[1026] The terminal displays the answer received from the server in the chat box. The input is the answer data from the server, and the output is the display of the answer to the user.

[1027] Business card information collection and processing

[1028] Step 1:

[1029] User provides business card information

[1030] A user takes a photo of a business card with a camera and uploads it to a terminal as image data. The input is the image data of the business card, and the output is the image uploaded to the terminal.

[1031] Step 2:

[1032] The device sends the business card image to the server.

[1033] The terminal sends the business card image data to the server in the required format. The input is the business card image data, and the output is the data sent to the server.

[1034] Step 3:

[1035] The server converts the image into text

[1036] The server uses an OCR engine to extract text information from the business card image. The input is the business card image data, and the output is the extracted text data. For example, the extracted information is "Yamada Taro," "ABC Co., Ltd.", and "Manager."

[1037] Step 4:

[1038] The server stores the business card information in a database

[1039] The server saves the extracted business card information in a database. The input is the extracted business card information, and the output is the data stored in the database. Specifically, the following SQL query is executed:

[1040] sql

[1041] INSERT INTO BusinessCardTable (Name, Company, Title) VALUES ('Yamada Taro', 'ABC Co., Ltd.', 'Manager')

[1042] Step 5:

[1043] The server notifies the sales department

[1044] The server sends a notification email to the sales department based on the saved business card information. The input is the business card information saved in the database, and the output is a notification email to the sales department.

[1045] DM policy processing

[1046] Step 1:

[1047] The server analyzes the business card information

[1048] The server retrieves business card information from a database and analyzes it according to specific criteria. The input is the business card information retrieved from the database, and the output is the analysis result. For example, a customer list for a specific industry is generated.

[1049] Step 2:

[1050] The server generates the marketing message

[1051] The server generates a marketing message tailored to each customer based on the analysis results. The input is the analysis results, and the output is the generated marketing message. For example, a message such as "We'd like to share a special offer with you" is generated.

[1052] Step 3:

[1053] The server sends a DM

[1054] The server generates a marketing message and sends it to the target user via email or post. The input is the generated marketing message, and the output is the message sent to the target user.

[1055] By following these specific steps, the system will improve the efficiency of call center operations and customer satisfaction.

[1056] (Application example 1)

[1057] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1058] In food delivery services, efficient response to customer inquiries, ordering support, and the creation and transmission of marketing messages to specific customers requires the rapid and accurate processing of multiple pieces of information. However, with conventional systems, user input and responses to inquiries are often done manually, resulting in inefficiencies. Furthermore, managing voice data and business card information is cumbersome, and notifications to specific departments and the creation of marketing messages are not automated, limiting the ability to improve operational efficiency and customer satisfaction.

[1059] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1060] In this invention, the server includes means for receiving information input by a user, means for storing the received information in a database, means for analyzing questions submitted by the user, means for automatically generating answers based on the analysis results, means for converting voice data into text data, means for converting business card image data into text data, means for notifying a specific department of the converted business card information, means for receiving voice data, means for providing specific information based on the generated text data, means for analyzing input information received from the user and generating a message based on the specific information, and means for transmitting the generated message using a communication means. This allows inquiries, order support, and the generation and transmission of marketing messages in a food delivery service to be handled centrally and automatically, thereby significantly improving business efficiency and customer satisfaction.

[1061] The "means for receiving information input by the user" is a function for acquiring information input by the user into the application and incorporating it into the system.

[1062] The "means for saving received information in a database" is a function for storing acquired user information in a database so that it can be reused as needed.

[1063] The "means for analyzing questions submitted by users" is a function for interpreting the content of inquiries submitted by users and analyzing them to generate appropriate answers.

[1064] "Means for automatically generating an answer based on the analysis results" is a function that enables the system to automatically generate an appropriate response based on the analysis results of the question.

[1065] The "means for converting voice data into text data" is a function for converting voice data provided by the user into character data.

[1066] The "means for converting business card image data into text data" is a function for extracting character information from the image data of a business card provided by the user and saving it as text data.

[1067] The "means for notifying a specific department of the converted business card information" is a function for automatically notifying the relevant department of the extracted business card information.

[1068] The "means for receiving voice data" is a function for accepting voice input from the user.

[1069] The "means for providing specific information based on the generated text data" is a function for providing necessary information to the user based on information obtained from voice or character data.

[1070] The "means for analyzing input information received from a user and generating a message based on specific information" is a function for analyzing input information from a user and automatically creating a message based on specific conditions.

[1071] The "means for transmitting the generated message using a communication means" is a function for transmitting the automatically generated message to the user or the relevant department.

[1072] This invention is a system for improving the efficiency of customer service in food delivery services. The system is implemented using a smartphone and includes functions for receiving information entered by users, storing the received information in a database, converting voice input into text, converting business card images into text, and generating and sending marketing messages based on this information.

[1073] The server provides a dedicated application to the smartphone to receive the information entered by the user. This application has an interface where the user can enter basic information such as name, address, and phone number. The entered information is sent to the server in real time and stored in a database.

[1074] In the food delivery business, inquiries from customers occur frequently. When a customer enters a question via chat or voice through the application, the question is sent to the server. The server analyzes the question using natural language processing technology and automatically generates an appropriate answer from the FAQ database. At this time, the voice data is converted into text data using the Google Speech-to-Text API, and the text data is analyzed using the Google Cloud Natural Language API. Based on the analyzed data, an appropriate answer is extracted from the FAQ database and displayed to the customer.

[1075] Furthermore, if a customer provides image data of their business card, the application will activate the smartphone camera to take a picture of the business card. The captured image data will be sent to the server and converted into text data using the Google Cloud Vision API. The converted business card information will be stored in a database and notified to a specific department.

[1076] Regarding the generation of marketing messages, the server analyzes relevant data and automatically generates messages based on specific conditions. The generated messages are then sent to customers and related departments via communication means, which allows for targeting specific customers and improving the effectiveness of campaigns.

[1077] Here's an example of what happens when a customer uses voice input:

[1078] 1. User: "Launch the app."

[1079] 2. Application: "Hello, how can I help you?"

[1080] 3. User: "I'd like to order a pizza. Can you tell me what kind?"

[1081] 4. The application converts the speech to text and returns the appropriate information.

[1082] And here's an example of a prompt for the generative AI model:

[1083] A user asked, "What's today's special?" Find the right answer in our FAQ database.

[1084] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1085] Step 1:

[1086] The user launches the application and enters the required basic information (name, address, phone number, etc.).

[1087] Input: Basic information entered by the user.

[1088] Output: The device converts the basic information into an appropriate format and sends it to the server.

[1089] Step 2:

[1090] The server stores the received basic information in a database.

[1091] Input: Basic information sent from the device.

[1092] Output: Basic information is stored in the database. Specifically, the SQL query is executed and saved as follows: "INSERT INTO UserTable (Name, Address, Phone) VALUES ('Taro', '123 Main St', '090-1234-5678')".

[1093] Step 3:

[1094] Users use the chat box or voice input function within the application to make inquiries or orders.

[1095] Input: Text or voice input by the user.

[1096] Output: The device sends the entered text or voice data to the server.

[1097] Step 4:

[1098] The server uses a speech recognition engine to convert the voice data into text data.

[1099] Input: Audio data sent from the device.

[1100] Output: The audio data is converted to text using the Google Speech-to-Text API.

[1101] Step 5:

[1102] The server uses a natural language processing (NLP) engine to analyze the text data and generate appropriate answers.

[1103] Input: Text data.

[1104] Output: Answers based on the analysis results. Specifically, the text data is analyzed using the Google Cloud Natural Language API, and appropriate answers are extracted from the FAQ database.

[1105] Step 6:

[1106] The server generates a response and sends it to the terminal.

[1107] Input: The server-generated answer.

[1108] Output: The terminal displays the answer it received to the user, specifically, in the chat box.

[1109] Step 7:

[1110] The user provides a business card image through the application.

[1111] Input: Business card image data taken by the user with the smartphone camera.

[1112] Output: The terminal sends the business card image data to the server.

[1113] Step 8:

[1114] The server uses an OCR engine to convert the business card image data into text data.

[1115] Input: Business card image data sent from the device.

[1116] Output: The business card image data is converted to text data using the Google Cloud Vision API.

[1117] Step 9:

[1118] The server stores the converted business card information in a database and notifies a specific department.

[1119] Input: Business card information extracted by the OCR engine.

[1120] Output: The business card information is stored in a database and a notification is sent to a specific department. Specifically, an SQL query is executed, saved, and notified via email or the internal notification system.

[1121] Step 10:

[1122] A server generates a marketing message and transmits it using a communication means.

[1123] Input: Customer data and business card information stored in a database.

[1124] Output: The generated marketing messages are sent to the target users by analyzing the information in the database, generating messages based on the conditions, and sending them via email or in-app notifications.

[1125] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1126] This invention is a system for improving the efficiency of call center operations and customer satisfaction. This system automatically responds to inquiries and requests from users, performs voice recognition, collects business card information, and combines it with an emotion engine that recognizes the user's emotions to provide a more personalized response.

[1127] Explanation of program processing

[1128] User reception process

[1129] 1. A user accesses the system

[1130] A user accesses the system from a terminal through a web browser or application. For example, the user accesses the system's URL in a browser.

[1131] 2. The user enters basic information

[1132] The user enters basic information such as "name," "email address," and "phone number" into the input form. Specifically, the user enters "Taro," "taro@example.com," "090-1234-5678," etc.

[1133] 3. The device sends the input information to the server

[1134] The terminal converts the input information into an appropriate format and sends it to the server.

[1135] 4. The server stores the information in a database

[1136] The server stores the received user information in the database. For example, it executes an SQL query like "INSERT INTO UserTable (Name, Email, Phone) VALUES ('Taro', 'taro@example.com', '090-1234-5678')".

[1137] Auto-Response Processing

[1138] 1. The user enters a question

[1139] The user types a question into the chat box on the device, for example, "What are your business hours?"

[1140] 2. The device sends a question to the server

[1141] The terminal sends the entered question to the server.

[1142] 3. The server parses the question

[1143] The server uses a natural language processing (NLP) engine to parse the question, in this example extracting the keyword "business hours."

[1144] 4. The server searches the FAQ database

[1145] The server queries the FAQ database based on the analysis results and retrieves the appropriate answer, for example, "Business hours are Monday through Friday, 9:00 AM to 6:00 PM."

[1146] 5. The server sends the answer to the device

[1147] The server sends the obtained answer to the terminal.

[1148] 6. The device displays the answer to the user

[1149] The device will display the response it receives in the chat box.

[1150] Voice-enabled processing

[1151] 1. The user selects a voice input

[1152] The user clicks the "Speech Input" button in the chat box.

[1153] 2. The device records the audio

[1154] The device activates the microphone and records the user's voice. For example, the user might say, "I'd like to know more about this product."

[1155] 3. The device sends the audio data to the server

[1156] The device sends the recorded audio data to the server.

[1157] 4. The server converts the speech to text

[1158] The server uses a speech recognition engine to convert the voice data into text. In this example, the voice "I want to know more about the product" is converted into the text "I want to know more about the product."

[1159] 5. The server generates the answer

[1160] The server analyzes the text information and generates an appropriate response, such as "This product was developed using the latest technology. Please visit our website for more information."

[1161] 6. The server sends the answer to the device

[1162] The server generates a response text and sends it to the terminal.

[1163] 7. The device displays the answer to the user

[1164] The device will display the response it receives in the chat box.

[1165] Business card information collection and processing

[1166] 1. The user provides business card information

[1167] The user takes a photo of the business card with a camera and uploads the image data to the terminal.

[1168] 2. The device sends the business card image to the server

[1169] The terminal transmits the business card image data to the server.

[1170] 3. The server converts the image into text

[1171] The server uses an OCR (Optical Character Recognition) engine to extract text information from the image. In this example, it extracts information such as "Yamada Taro," "ABC Co., Ltd.", and "Manager."

[1172] 4. The server saves the business card information in a database

[1173] The server saves the extracted business card information in the database. For example, it saves it as follows: INSERT INTO BusinessCardTable (Name, Company, Title) VALUES ('Yamada Taro', 'ABC Co., Ltd.', 'Manager').

[1174] 5. The server notifies the sales department

[1175] The server sends the stored information to the sales department staff via email.

[1176] DM policy processing

[1177] 1. The server analyzes the business card information

[1178] The server retrieves the business card information from the database and analyzes it, for example to generate a list of customers in a particular industry.

[1179] 2. The server generates the marketing message

[1180] Based on the analysis results, the server generates a marketing message tailored to each customer, such as "A special offer for Taro Yamada of ABC Co., Ltd."

[1181] 3. The server sends the DM

[1182] Sending server-generated marketing messages to targeted users via email and post.

[1183] Emotion engine processing

[1184] 1. Recognize user emotions

[1185] The server uses an emotion engine to recognize emotions from user input data, for example, extracting emotions such as "anger" or "joy" from user speech or voice recordings.

[1186] 2. Tailor responses based on emotional data

[1187] The server automatically adjusts responses based on the recognized emotion data, for example, generating a calming tone if the user is angry.

[1188] 3. Save emotion data

[1189] The server stores the recognized emotion data in a database for later analysis, for example, to analyze the emotional tendencies of a particular user.

[1190] 4. Generating Marketing Messages Using Emotional Data

[1191] The server generates more personalized marketing messages based on the emotional data analyzed by the emotion engine. For example, it might suggest a "relaxing offer" to a user with a neutral emotion.

[1192] This allows the system to respond quickly and accurately to user inquiries and requests, and by combining voice recognition and emotion recognition, it will improve business efficiency and customer satisfaction.

[1193] The processing flow will be explained below.

[1194] User reception process

[1195] Step 1:

[1196] A user accesses the system

[1197] A user accesses the system from a terminal through a web browser or application. For example, the user accesses the system's URL in a browser.

[1198] Step 2:

[1199] The user enters basic information

[1200] The user enters basic information such as "name," "email address," and "phone number" into the input form. Specifically, the user enters "Taro," "taro@example.com," "090-1234-5678," etc.

[1201] Step 3:

[1202] The device sends the input information to the server

[1203] The terminal converts the input information into an appropriate format and sends it to the server.

[1204] Step 4:

[1205] The server stores the information in a database

[1206] The server stores the received user information in the database. For example, it executes an SQL query like "INSERT INTO UserTable (Name, Email, Phone) VALUES ('Taro', 'taro@example.com', '090-1234-5678')".

[1207] Auto-Response Processing

[1208] Step 1:

[1209] The user enters a question

[1210] The user types a question into the chat box on the device, for example, "What are your business hours?"

[1211] Step 2:

[1212] The device sends a question to the server

[1213] The terminal sends the entered question to the server.

[1214] Step 3:

[1215] The server parses the question

[1216] The server uses a natural language processing (NLP) engine to parse the question, in this example extracting the keyword "business hours."

[1217] Step 4:

[1218] The server searches the FAQ database

[1219] The server queries the FAQ database based on the analysis results and retrieves the appropriate answer, for example, "Business hours are Monday through Friday, 9:00 AM to 6:00 PM."

[1220] Step 5:

[1221] The server sends the answer to the device

[1222] The server sends the obtained answer to the terminal.

[1223] Step 6:

[1224] The device displays the answer to the user

[1225] The device will display the response it receives in the chat box.

[1226] Voice-enabled processing

[1227] Step 1:

[1228] The user selects a voice input

[1229] The user clicks the "Speech Input" button in the chat box.

[1230] Step 2:

[1231] The device records audio

[1232] The device activates the microphone and records the user's voice. For example, the user might say, "I'd like to know more about this product."

[1233] Step 3:

[1234] The device sends the audio data to the server.

[1235] The device sends the recorded audio data to the server.

[1236] Step 4:

[1237] The server converts the speech to text

[1238] The server uses a speech recognition engine to convert the voice data into text. In this example, the voice "I want to know more about the product" is converted into the text "I want to know more about the product."

[1239] Step 5:

[1240] The server generates the answer

[1241] The server analyzes the text information and generates an appropriate response, such as "This product was developed using the latest technology. Please visit our website for more information."

[1242] Step 6:

[1243] The server sends the answer to the device

[1244] The server generates a response text and sends it to the terminal.

[1245] Step 7:

[1246] The device displays the answer to the user

[1247] The device will display the response it receives in the chat box.

[1248] Business card information collection and processing

[1249] Step 1:

[1250] User provides business card information

[1251] The user takes a photo of the business card with a camera and uploads the image data to the terminal.

[1252] Step 2:

[1253] The device sends the business card image to the server.

[1254] The terminal transmits the business card image data to the server.

[1255] Step 3:

[1256] The server converts the image into text

[1257] The server uses an OCR (Optical Character Recognition) engine to extract text information from the image. In this example, it extracts information such as "Yamada Taro," "ABC Co., Ltd.", and "Manager."

[1258] Step 4:

[1259] The server stores the business card information in a database

[1260] The server saves the extracted business card information in the database. For example, it saves it as follows: INSERT INTO BusinessCardTable (Name, Company, Title) VALUES ('Yamada Taro', 'ABC Co., Ltd.', 'Manager').

[1261] Step 5:

[1262] The server notifies the sales department

[1263] The server sends the stored information to the sales department staff via email.

[1264] DM policy processing

[1265] Step 1:

[1266] The server analyzes the business card information

[1267] The server retrieves the business card information from the database and analyzes it, for example to generate a list of customers in a particular industry.

[1268] Step 2:

[1269] The server generates the marketing message

[1270] Based on the analysis results, the server generates a marketing message tailored to each customer, such as "A special offer for Taro Yamada of ABC Co., Ltd."

[1271] Step 3:

[1272] The server sends a DM

[1273] Sending server-generated marketing messages to targeted users via email and post.

[1274] Emotion engine processing

[1275] Step 1:

[1276] Recognize user emotions

[1277] The server uses an emotion engine to recognize emotions from user input data, for example, extracting emotions such as "anger" or "joy" from user speech or voice recordings.

[1278] Step 2:

[1279] Tailor responses based on emotional data

[1280] The server automatically adjusts responses based on the recognized emotion data, for example, generating a calming tone if the user is angry.

[1281] Step 3:

[1282] Storing emotion data

[1283] The server stores the recognized emotion data in a database for later analysis, for example, to analyze the emotional tendencies of a particular user.

[1284] Step 4:

[1285] Generating marketing messages using emotional data

[1286] The server generates more personalized marketing messages based on the emotional data analyzed by the emotion engine. For example, it might suggest a "relaxing offer" to a user with a neutral emotion.

[1287] The above is the specific processing flow of the system of the present invention, which enables quick and accurate responses to user inquiries and requests, and by combining speech recognition and emotion recognition, improves the efficiency of call center operations and customer satisfaction.

[1288] Example 2

[1289] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1290] Conventional call center systems have limited capabilities for responding to user inquiries, making it difficult to provide personalized responses tailored to individual needs. Furthermore, there are insufficient means for improving customer satisfaction by utilizing advanced technologies such as voice recognition and emotion recognition. Furthermore, collecting business card information and generating marketing messages based on that information is time-consuming, requiring increased efficiency. To address these issues, the present invention provides a system that integrates the automation of inquiry responses, personalized responses using emotion recognition, and the collection and analysis of business card information.

[1291] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for receiving information input by a user, means for saving the received information in a database, means for analyzing a question submitted by a user, means for automatically generating an answer based on the analysis result, means for converting voice data into text data, means for converting business card image data into text data, means for notifying the sales department of the converted business card information, means for recognizing the user's emotion, and means for adjusting a response based on the recognized emotion. This improves the accuracy and speed of inquiry responses, enables personalized responses according to customer emotions, and further realizes more efficient marketing measures through the automatic collection and analysis of business card information.

[1292] "Means for receiving user-entered information" refers to the functionality of devices or software that capture data entered by users through forms or chat boxes and incorporate it into the system.

[1293] "Means for storing received information in a database" refers to a technology for permanently storing data received from a user via a database system such as SQL.

[1294] "Means for analyzing questions submitted by users" refers to software functionality that uses natural language processing (NLP) technology to understand the content of users' questions and extract keywords and intent.

[1295] "Means for automatically generating answers based on the analysis results" refers to the functions of algorithms and databases that automatically create appropriate answers based on the analyzed question content and provide them to users.

[1296] "Means for converting voice data into text data" refers to technology that uses a voice recognition engine to convert voice files or real-time voice input into text format.

[1297] The "means for converting business card image data into text data" is a software function for extracting text information from business card images using optical character recognition (OCR) technology.

[1298] "Means for notifying the sales department of the converted business card information" is a function for notifying the sales staff of the extracted business card information via email or a notification system.

[1299] "Means for recognizing user emotions" refers to technology that uses an emotion analysis engine to extract emotions from user input data and voice.

[1300] The "means for tailoring responses based on recognized emotions" is a software function for automatically generating and delivering responses with appropriate tone and content depending on the user's emotional state.

[1301] The present invention is a system that automatically responds to inquiries and requests from users, providing more personalized responses by performing voice recognition, collecting business card information, and even emotion recognition. This system operates using the following hardware and software.

[1302] Hardware

[1303] 1. Terminal: A device (e.g., computer, smartphone, tablet) through which a user enters information.

[1304] 2. Server: A computer system running hardware and software for processing and storing data.

[1305] 3. Database: A data storage system for persistently storing information (e.g., MySQL, PostgreSQL).

[1306] software

[1307] 1. Web Browser or Application: The interface through which the user accesses the system.

[1308] 2. Natural Language Processing (NLP) engine: Software for analyzing user questions (e.g., Google Dialogflow).

[1309] 3. Speech recognition engine: Software for converting voice data into text (e.g., Google Cloud Speech-to-Text).

[1310] 4. Optical Character Recognition (OCR) engine: Software for extracting text information from business card images (e.g., Tesseract OCR).

[1311] 5. Sentiment analysis engine: Software for recognizing user emotions (e.g., IBM Watson Tone Analyzer).

[1312] 6. Generative AI models: Software for tailoring responses based on user sentiment (e.g., GPT-3).

[1313] Specific examples

[1314] 1. User acceptance process

[1315] A user accesses the system's web page using a smartphone and enters their name (e.g., "Taro"), email address (e.g., "taro@example.com"), and phone number (e.g., "090-1234-5678"). The device sends this information to the server, which then stores the received information in a database.

[1316] 2. Auto-Response Processing

[1317] The user enters a question into the chat box (e.g., "What are your business hours?"). The device sends the question to the server, which analyzes the question using an NLP engine. Based on the analysis results, the server retrieves an appropriate answer from the FAQ database (e.g., "Business hours are Monday through Friday, 9 a.m. to 6 p.m.") and displays it to the user through the device.

[1318] 3. Voice-enabled processing

[1319] The user clicks the voice input button and says, "I'd like to know more about this product." The device records the voice and sends it to the server. The server uses a speech recognition engine to convert the voice data into text and generate an answer (e.g., "This product was developed using the latest technology. Please visit our website for more information."). The answer is returned to the device and displayed to the user.

[1320] 4. Business card information collection and processing

[1321] A user takes a photo of a business card with their smartphone and uploads it to the system. The device sends the business card image to the server, which uses an OCR engine to extract text information from the image. The extracted information (e.g., "Yamada Taro," "XYZ Co., Ltd.", "Manager") is stored in a database and notified to the sales department.

[1322] 5. Emotional Engine Processing

[1323] The emotion analysis engine recognizes emotional data (e.g., "anger" or "happiness") from the user's comments and questions, and the server generates a response that corresponds to that emotion. For example, if the user is angry, the server generates a response in a calming tone (e.g., "We apologize for any inconvenience this may have caused you"). The generated response is provided to the user via the device.

[1324] Prompt Sentence Examples

[1325] The prompts that users enter into the system include:

[1326] 1. "What are your opening hours?"

[1327] 2. "I want to know more about the product"

[1328] 3. "I want to know about new product release information."

[1329] As a result, this system responds quickly and accurately to user inquiries and requests, and by combining voice recognition and emotion recognition, it achieves improved business efficiency and customer satisfaction.

[1330] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1331] Program processing flow

[1332] User reception process

[1333] Step 1:

[1334] A user accesses the system. The user opens a web browser or application on the terminal and accesses the system's URL. Input: Access request is made by the URL output of the web browser. Output: Login screen or initial screen is displayed.

[1335] Step 2:

[1336] The user enters basic information. The user enters their name, email address, and phone number into the system form. Specific examples of input include "Taro," "taro@example.com," and "090-1234-5678." Input: Basic information of the user. Output: Entered information data.

[1337] Step 3:

[1338] The terminal sends the input information to the server. The terminal converts the input data into JSON format and sends it to the server via a POST request. Input: JSON format of user input information. Output: JSON data is sent to the server.

[1339] Step 4:

[1340] The server saves the information in the database. The server saves the received data in the database using an SQL query. Specifically, it executes the query "INSERT INTO UserTable (Name, Email, Phone) VALUES ('Taro', 'taro@example.com', '090-1234-5678')". Input: User information in JSON format. Output: Information saved in the database.

[1341] Auto-Response Processing

[1342] Step 1:

[1343] User types a question. User types a question into the chat box (e.g., "What are your business hours?"). Input: User's question text. Output: Question typed into the chat box.

[1344] Step 2:

[1345] The terminal sends the question to the server. The terminal converts the entered question into JSON format and sends it to the server. Input: JSON format of the question text. Output: JSON data sent to the server.

[1346] Step 3:

[1347] The server analyzes the question. The server uses a natural language processing engine to analyze the question and extract keywords. Input: The question text "What are your business hours?" Output: The keyword "business hours".

[1348] Step 4:

[1349] The server searches the FAQ database. The server queries the FAQ table based on the extracted keywords to get the appropriate answer. For example, it executes the query "SELECT Answer FROM FAQTable WHERE Question LIKE '%Business Hours%'". Input: Keyword "Business Hours". Output: Answer text from the FAQ database.

[1350] Step 5:

[1351] The server sends the answer to the terminal. The server converts the answer text to JSON format and sends it to the terminal. Input: Answer from the FAQ database. Output: Answer data in JSON format.

[1352] Step 6:

[1353] The terminal displays the answer to the user. The terminal displays the received answer in the chat box. Input: JSON data sent from the server. Output: Answer displayed in the chat box.

[1354] Voice-enabled processing

[1355] Step 1:

[1356] User selects audio input. User clicks "Audio Input" button in chat box. Input: Select audio input. Output: Activate microphone.

[1357] Step 2:

[1358] The device records the voice. The user speaks (e.g., "I'd like to know more about this product") and the device records the voice. Input: User's speech. Output: Recorded voice data.

[1359] Step 3:

[1360] The device sends audio data to the server. The device sends the recorded audio data to the server via a POST request. Input: Recorded audio data. Output: Audio data sent to the server.

[1361] Step 4:

[1362] The server converts speech to text. The server uses a speech recognition engine to convert the speech data into text format. Input: speech data. Output: converted text data.

[1363] Step 5:

[1364] The server generates the answer. The server analyzes the text question content and generates the corresponding answer. Input: Converted text data. Output: Generated answer text.

[1365] Step 6:

[1366] The server sends the answer to the terminal. The server converts the answer to JSON format and sends it to the terminal. Input: Generated answer text. Output: JSON format answer data.

[1367] Step 7:

[1368] The terminal displays the answer to the user. The terminal displays the received answer in the chat box. Input: JSON data sent from the server. Output: Answer displayed in the chat box.

[1369] Business card information collection and processing

[1370] Step 1:

[1371] The user provides business card information. The user takes a photo of the business card with the smartphone camera and uploads it to the system. Input: Business card image. Output: Uploaded image data.

[1372] Step 2:

[1373] The terminal sends the business card image to the server. The terminal sends the business card image data to the server. Input: Business card image data. Output: Image data sent to the server.

[1374] Step 3:

[1375] The server converts the image to text. The server uses an OCR engine to extract text information from the image data. Input: Business card image data. Output: Converted text information.

[1376] Step 4:

[1377] The server saves the business card information in the database. The server saves the extracted business card information in the database using an SQL query. For example, execute the query "INSERT INTO BusinessCardTable (Name, Company, Title) VALUES ('Yamada Taro', 'XYZ Co., Ltd.', 'Manager')". Input: Converted text information. Output: Information saved in the database.

[1378] Step 5:

[1379] The server notifies the sales department. The server sends a notification email to the sales department based on the saved information. Input: Business card information. Output: Notification email to the sales department.

[1380] Emotion engine processing

[1381] Step 1:

[1382] The server recognizes the user's emotions. The server uses an emotion analysis engine to extract emotions from the user's input data. Input: User's input data. Output: Extracted emotion data.

[1383] Step 2:

[1384] The server tailors the response based on the emotion data. The server tailors the response using a generative AI model based on the extracted emotion data. Input: Emotion data. Output: Tailored response text.

[1385] Step 3:

[1386] The server stores the emotion data. The server stores the recognized emotion data in a database and uses it for later analysis. Input: Emotion data. Output: Emotion data saved in the database.

[1387] Step 4:

[1388] The server generates a marketing message using the emotion data. The server generates a personalized marketing message for each user based on the emotion data. For example, a user with a calm emotion might be offered a "relaxing offer." Input: Emotion data. Output: Generated marketing message.

[1389] (Application example 2)

[1390] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1391] Autonomous vehicles require rapid and appropriate responses to passenger inquiries and requests. However, conventional systems have struggled to recognize passenger emotions and provide personalized responses based on those emotions. They also lacked efficient means to digitize passenger business card information and generate marketing messages. The present invention aims to solve these problems and improve passenger service.

[1392] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1393] In this invention, the server includes means for receiving information input by a user, means for storing the received information in a database, means for analyzing questions submitted by users, means for automatically generating answers based on the analysis results, means for converting voice data to text data, means for converting business card image data to text data, means for notifying the sales department of the converted business card information, means for receiving and analyzing voice input from passengers in the vehicle and generating an automatic response, means for recognizing passenger emotions and providing a response according to the emotions, and means for digitizing and storing passenger business card information, thereby making it possible to respond quickly and appropriately to passenger inquiries and requests and provide personalized services based on emotions.

[1394] "Information entered by the user" refers to data such as the name, email address, phone number, and questions that the user provides to the system.

[1395] "Means for storing in a database" refers to a method using a server or storage device to organize and efficiently store the received information.

[1396] "Means of analysis" refers to the process of using natural language processing technology to understand the content of questions and input data submitted by users and determine how to respond.

[1397] "Means for automatically generating answers" refers to algorithms or programs that generate optimal answers from analysis results or pre-set information and present them to the user.

[1398] "Means for converting voice data into text data" refers to the process of converting a user's voice input into text information using voice recognition technology.

[1399] The "means of converting business card image data into text data" is the process of extracting text information from business card images using OCR (optical character recognition) technology.

[1400] The "means of notifying the sales department" is a method of transmitting the extracted business card information and other important information to the sales department staff via email or a notification system.

[1401] "Means for receiving and analyzing passenger voice inputs within a vehicle and generating an automatic response" refers to a system for recognizing passenger voice instructions within an automated vehicle, analyzing the content of the instructions, and generating an appropriate response.

[1402] "Means for recognizing passenger emotions and providing emotionally appropriate responses" refers to a process for determining the passenger's emotional state from the passenger's tone of voice and content of speech, and generating a response with an appropriate tone and content.

[1403] The "means for digitizing and storing passenger business card information" refers to a method of capturing business cards provided by passengers as digital images, converting them into text data, and storing them in the system.

[1404] The present invention provides a system for streamlining passenger service and providing personalized responses in autonomous vehicles, which implements a series of processes for efficiently acquiring, analyzing, and responding to passenger input information.

[1405] System Overview

[1406] This system is composed of the following means:

[1407] 1. A means of receiving information entered by the user

[1408] Passengers enter information such as their name, destination, and questions using a smartphone app or a touch panel inside the vehicle, and send it to the system.

[1409] 2. A means of storing the received information in a database

[1410] The received information is stored in a database, for example, using a relational database such as PostgreSQL.

[1411] 3. A means of analyzing user-submitted questions

[1412] Questions are parsed using natural language processing (NLP) techniques, using the Google Cloud Natural Language API to extract keywords and intent from the text.

[1413] 4. A means of automatically generating answers based on analysis results

[1414] Based on the analysis results, an appropriate response is generated, utilizing a pre-configured FAQ database and generative AI models.

[1415] 5. Means of converting audio data into text data

[1416] Passenger voice input is converted into text data using Google Cloud Speech-to-Text.

[1417] 6. How to convert business card image data into text data

[1418] When a passenger provides their business card information, OCR (optical character recognition) technology, such as Tesseract OCR, is used to extract text data from the image of the business card.

[1419] 7. A method for notifying the sales department of the converted business card information

[1420] Business card information is digitized and sent to the sales department via email or internal notification system.

[1421] 8. Means for receiving and analyzing passenger voice inputs in the vehicle to generate automated responses

[1422] Passenger voice commands are received through an in-car microphone, analyzed, and an appropriate response is generated, combining voice recognition and NLP technology.

[1423] 9. A means of recognizing passenger emotions and providing emotional responses

[1424] The system determines the passenger's emotions from their tone of voice and the content of their comments, and uses IBM Watson Emotion Analysis to generate responses based on those emotions.

[1425] 10. A means of digitizing and storing passenger business card information

[1426] The business card information is digitized, stored, and used to generate subsequent marketing messages.

[1427] Specific examples

[1428] Specific examples of voice input

[1429] A passenger says, "Please make the car's air conditioning a little cooler." This speech is collected through a microphone in the car and converted into text using Google Cloud Speech-to-Text: "Please make the car's air conditioning a little cooler." The system then analyzes this text using NLP technology and generates the response message, "The temperature has been adjusted."

[1430] Example prompts for generative AI models

[1431] "Please determine the emotion from the text below. Then suggest an appropriate response: \"How's the weather today?\""

[1432] This enables the system of the present invention to provide personalized services in autonomous vehicles and improve passenger satisfaction.

[1433] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1434] Step 1:

[1435] Users access the system through a touch panel in the car or a smartphone app. They input their destination, name, and other necessary information. For example, a user might input "Destination: Chuo Station" and "Name: Yamada Taro" into the touch panel.

[1436] Input: Destination, name, and other basic information

[1437] Output: The entered information is saved in JSON format on the device.

[1438] How it works: The device displays an input form, and once the user completes the form, it converts the information into JSON format and sends it to the server.

[1439] Step 2:

[1440] The terminal sends the entered information to a database. The server receives it and stores it in the database. A relational database such as PostgreSQL is used.

[1441] Input: User input information in JSON format

[1442] Output: User information stored in the database

[1443] How it works: The server receives the JSON data, generates the appropriate queries, and saves them to the database.

[1444] Step 3:

[1445] The user types a question into the touch panel or chat box of the app, for example, "Please lower the temperature inside the car."

[1446] Input: User's question

[1447] Output: The question sent to the server

[1448] Operation: The device sends the entered question to the server.

[1449] Step 4:

[1450] The server analyzes the user's question using natural language processing (NLP) techniques, including the Google Cloud Natural Language API, to extract keywords and intent from the question.

[1451] Input: User's question

[1452] Output: Parsed keywords and intent

[1453] How it works: The server uses an NLP engine to parse the incoming question and extract keywords (e.g., temperature) and intent (e.g., lower).

[1454] Step 5:

[1455] The server generates an automated response based on the analysis results, for example, "The temperature inside the car has been lowered."

[1456] Input: Parsed keywords and intent

[1457] Output: Auto-generated response message

[1458] How it works: The server references the FAQ database and generative AI models to generate the best response.

[1459] Step 6:

[1460] When a user speaks, the car's microphone records the speech and converts it into text using Google Cloud Speech-to-Text. For example, "Turn up the air conditioning" is converted into text as "Turn up the air conditioning."

[1461] Input: Audio data

[1462] Output: Text data

[1463] How it works: The device records audio and sends the audio data to the server, which uses a speech recognition engine to convert it into text.

[1464] Step 7:

[1465] When a user provides business card information, the system uses an in-car camera to take a picture of the business card and then uses OCR technology to extract the text data from the card. For example, the business card image is converted into "Yamada Taro, company name, job title."

[1466] Input: Business card image data

[1467] Output: Business card text data

[1468] How it works: The device scans a business card and sends the image data to the server, which uses Tesseract OCR to extract the text data.

[1469] Step 8:

[1470] The server stores the converted business card information in a database and notifies the sales department, for example, by sending an email to the sales department.

[1471] Input: Business card text data

[1472] Output: Business card information stored in the database, notification email

[1473] What it does: The server saves the business card information in a database and sends a notification to the specified email address.

[1474] Step 9:

[1475] The server recognizes the passenger's emotions and generates responses based on their emotions using IBM Watson Emotion Analysis. For example, if the user is angry, it generates a calmer response.

[1476] Input: What the user said

[1477] Output: Sentiment analysis results, tailored response message

[1478] How it works: The server analyzes the voice or text data using an emotion recognition engine and generates a response based on the user's emotional state.

[1479] Step 10:

[1480] The server uses the business card information to generate and send marketing messages, which allows for personalized marketing.

[1481] Input: Business card information, user data

[1482] Output: Marketing message

[1483] How it works: The server analyzes the business card information, generates marketing messages tailored to the specific user, and sends them via email or notification system.

[1484] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1485] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1486] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1487] [Third embodiment]

[1488] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1489] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1490] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1491] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1492] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1493] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1494] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1495] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1496] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1497] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1498] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1499] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1500] This invention is a system for improving the efficiency of call center operations and customer satisfaction. This system automatically responds to inquiries and requests from users, performs voice recognition, and collects business card information. It also effectively obtains information useful for sales activities and generates marketing messages.

[1501] Explanation of program processing

[1502] User reception process

[1503] 1. A user accesses the system

[1504] A user accesses the system from a terminal through a web browser or application. For example, the user accesses the system's URL in a browser.

[1505] 2. The user enters basic information

[1506] The user enters basic information such as "name," "email address," and "phone number" into the input form. Specifically, the user enters "Taro," "taro@example.com," "090-1234-5678," etc.

[1507] 3. The device sends the input information to the server

[1508] The terminal converts the input information into an appropriate format and sends it to the server.

[1509] 4. The server stores the information in a database

[1510] The server stores the received user information in the database. For example, it executes an SQL query like "INSERT INTO UserTable (Name, Email, Phone) VALUES ('Taro', 'taro@example.com', '090-1234-5678')".

[1511] Auto-Response Processing

[1512] 1. The user enters a question

[1513] The user types a question into the chat box on the device, for example, "What are your business hours?"

[1514] 2. The device sends a question to the server

[1515] The terminal sends the entered question to the server.

[1516] 3. The server parses the question

[1517] The server uses a natural language processing (NLP) engine to parse the question, in this example extracting the keyword "business hours."

[1518] 4. The server searches the FAQ database

[1519] The server queries the FAQ database based on the analysis results and retrieves the appropriate answer, for example, "Business hours are Monday through Friday, 9:00 AM to 6:00 PM."

[1520] 5. The server sends the answer to the device

[1521] The server sends the obtained answer to the terminal.

[1522] 6. The device displays the answer to the user

[1523] The device will display the response it receives in the chat box.

[1524] Voice-enabled processing

[1525] 1. The user selects a voice input

[1526] The user clicks the "Speech Input" button in the chat box.

[1527] 2. The device records the audio

[1528] The device activates the microphone and records the user's voice. For example, the user might say, "I'd like to know more about this product."

[1529] 3. The device sends the audio data to the server

[1530] The device sends the recorded audio data to the server.

[1531] 4. The server converts the speech to text

[1532] The server uses a speech recognition engine to convert the voice data into text. In this example, the voice "I want to know more about the product" is converted into the text "I want to know more about the product."

[1533] 5. The server generates the answer

[1534] The server analyzes the text information and generates an appropriate response, such as "This product was developed using the latest technology. Please visit our website for more information."

[1535] 6. The server sends the answer to the device

[1536] The server generates a response text and sends it to the terminal.

[1537] 7. The device displays the answer to the user

[1538] The device will display the response it receives in the chat box.

[1539] Business card information collection and processing

[1540] 1. The user provides business card information

[1541] The user takes a photo of the business card with a camera and uploads the image data to the terminal.

[1542] 2. The device sends the business card image to the server

[1543] The terminal transmits the business card image data to the server.

[1544] 3. The server converts the image into text

[1545] The server uses an OCR (Optical Character Recognition) engine to extract text information from the image. In this example, it extracts information such as "Yamada Taro," "ABC Co., Ltd.", and "Manager."

[1546] 4. The server saves the business card information in a database

[1547] The server saves the extracted business card information in the database. For example, it saves it as follows: INSERT INTO BusinessCardTable (Name, Company, Title) VALUES ('Yamada Taro', 'ABC Co., Ltd.', 'Manager').

[1548] 5. The server notifies the sales department

[1549] The server sends the stored information to the sales department staff via email.

[1550] DM policy processing

[1551] 1. The server analyzes the business card information

[1552] The server retrieves the business card information from the database and analyzes it, for example to generate a list of customers in a particular industry.

[1553] 2. The server generates the marketing message

[1554] Based on the analysis results, the server generates a marketing message tailored to each customer, such as "A special offer for Taro Yamada of ABC Co., Ltd."

[1555] 3. The server sends the DM

[1556] Sending server-generated marketing messages to targeted users via email and post.

[1557] As described above, this system responds to user inquiries and requests from multiple angles, thereby improving the efficiency of call center operations and customer satisfaction.

[1558] The processing flow will be explained below.

[1559] User reception process

[1560] Step 1:

[1561] A user accesses the system

[1562] A user accesses the system from a terminal through a web browser or application. For example, the user accesses the system's URL in a browser.

[1563] Step 2:

[1564] The user enters basic information

[1565] The user enters basic information such as "name," "email address," and "phone number" into the input form. Specifically, the user enters "Taro," "taro@example.com," "090-1234-5678," etc.

[1566] Step 3:

[1567] The device sends the input information to the server

[1568] The terminal converts the input information into an appropriate format and sends it to the server.

[1569] Step 4:

[1570] The server stores the information in a database

[1571] The server stores the received user information in the database. For example, it executes an SQL query like "INSERT INTO UserTable (Name, Email, Phone) VALUES ('Taro', 'taro@example.com', '090-1234-5678')".

[1572] Auto-Response Processing

[1573] Step 1:

[1574] The user enters a question

[1575] The user types a question into the chat box on the device, for example, "What are your business hours?"

[1576] Step 2:

[1577] The device sends a question to the server

[1578] The terminal sends the entered question to the server.

[1579] Step 3:

[1580] The server parses the question

[1581] The server uses a natural language processing (NLP) engine to parse the question, in this example extracting the keyword "business hours."

[1582] Step 4:

[1583] The server searches the FAQ database

[1584] The server queries the FAQ database based on the analysis results and retrieves the appropriate answer, for example, "Business hours are Monday through Friday, 9:00 AM to 6:00 PM."

[1585] Step 5:

[1586] The server sends the answer to the device

[1587] The server sends the obtained answer to the terminal.

[1588] Step 6:

[1589] The device displays the answer to the user

[1590] The device will display the response it receives in the chat box.

[1591] Voice-enabled processing

[1592] Step 1:

[1593] The user selects a voice input

[1594] The user clicks the "Speech Input" button in the chat box.

[1595] Step 2:

[1596] The device records audio

[1597] The device activates the microphone and records the user's voice. For example, the user might say, "I'd like to know more about this product."

[1598] Step 3:

[1599] The device sends the audio data to the server.

[1600] The device sends the recorded audio data to the server.

[1601] Step 4:

[1602] The server converts the speech to text

[1603] The server uses a speech recognition engine to convert the voice data into text. In this example, the voice "I want to know more about the product" is converted into the text "I want to know more about the product."

[1604] Step 5:

[1605] The server generates the answer

[1606] The server analyzes the text information and generates an appropriate response, such as "This product was developed using the latest technology. Please visit our website for more information."

[1607] Step 6:

[1608] The server sends the answer to the device

[1609] The server generates a response text and sends it to the terminal.

[1610] Step 7:

[1611] The device displays the answer to the user

[1612] The device will display the response it receives in the chat box.

[1613] Business card information collection and processing

[1614] Step 1:

[1615] User provides business card information

[1616] The user takes a photo of the business card with a camera and uploads the image data to the terminal.

[1617] Step 2:

[1618] The device sends the business card image to the server.

[1619] The terminal transmits the business card image data to the server.

[1620] Step 3:

[1621] The server converts the image into text

[1622] The server uses an OCR (Optical Character Recognition) engine to extract text information from the image. In this example, it extracts information such as "Yamada Taro," "ABC Co., Ltd.", and "Manager."

[1623] Step 4:

[1624] The server stores the business card information in a database

[1625] The server saves the extracted business card information in the database. For example, it saves it as follows: INSERT INTO BusinessCardTable (Name, Company, Title) VALUES ('Yamada Taro', 'ABC Co., Ltd.', 'Manager').

[1626] Step 5:

[1627] The server notifies the sales department

[1628] The server sends the stored information to the sales department staff via email.

[1629] DM policy processing

[1630] Step 1:

[1631] The server analyzes the business card information

[1632] The server retrieves the business card information from the database and analyzes it, for example to generate a list of customers in a particular industry.

[1633] Step 2:

[1634] The server generates the marketing message

[1635] Based on the analysis results, the server generates a marketing message tailored to each customer, such as "A special offer for Taro Yamada of ABC Co., Ltd."

[1636] Step 3:

[1637] The server sends a DM

[1638] Sending server-generated marketing messages to targeted users via email and post.

[1639] Example 1

[1640] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1641] Call center operations must efficiently respond to a large number of inquiries, which requires the rapid and accurate processing of data in various formats. However, conventional systems have difficulty integrating voice recognition, business card information collection, and natural language processing, resulting in problems such as reduced work efficiency and low customer satisfaction. Furthermore, the creation and transmission of marketing messages is also done manually, which is inefficient. There is a need to solve these problems, improve the efficiency of call center operations, and increase customer satisfaction.

[1642] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1643] In this invention, the server includes means for receiving information input by a user, means for storing the received information in a database, means for analyzing questions submitted by users, means for automatically generating answers based on the analysis results, means for converting voice data into text data, means for converting business card image data into text data, means for notifying the sales department of the converted business card information, means for analyzing questions using natural language processing technology, means for searching an FAQ database to obtain an appropriate answer, means for making voice input selectable, means for transmitting recorded voice data to the server, means for displaying answers to the user, means for formatting the input information and transmitting it to the server, means for analyzing the business card information and generating a marketing message, and means for transmitting the generated marketing message to the user. This enables efficient response to inquiries, voice recognition, automatic collection of business card information, and automatic generation and transmission of marketing messages.

[1644] "Means for receiving information input by a user" refers to an interface for receiving data input by a user.

[1645] "Means for saving received information to a database" refers to the process of saving the data entered by the user in a database for storing the data.

[1646] "Means for analyzing a question submitted by a user" refers to a processing technology for analyzing a question entered by a user and understanding its intent.

[1647] "Means for automatically generating answers based on the analysis results" refers to a process for automatically generating appropriate answers to the analyzed questions.

[1648] "Means for converting voice data into text data" refers to technology for converting information input by voice into text format.

[1649] "Means for converting business card image data into text data" refers to technology for converting business card photos or scanned data into text information.

[1650] "Means for notifying the sales department of the converted business card information" refers to the process of notifying the sales department of the data extracted from the business card.

[1651] "Means for analyzing questions using natural language processing technology" refers to technology for analyzing user questions using natural language processing technology.

[1652] "Means for searching the FAQ database to obtain an appropriate answer" refers to the process of searching the FAQ database to obtain an appropriate answer to a user's question.

[1653] "Means for enabling selection of voice input" refers to a technology that provides an interface that allows a user to select a voice input.

[1654] "Means for transmitting recorded voice data to a server" refers to a communication technology for transmitting recorded voice data to a server.

[1655] The "means for displaying the answer to the user" refers to an interface for displaying the generated answer on the user's terminal.

[1656] "Means for formatting input information and sending it to the server" refers to the process of converting the information input by the user into an appropriate format and sending it to the server.

[1657] "Means for analyzing business card information and generating a marketing message" refers to technology for analyzing information obtained from a business card and generating a specific marketing message.

[1658] "Means for sending generated marketing messages to users" refers to the process of sending generated marketing messages to target users.

[1659] This invention is a system aimed at improving the efficiency of call center operations and customer satisfaction. This system automatically responds to inquiries and requests from users, performs voice recognition, and collects business card information. It also effectively obtains information useful for sales activities and generates marketing messages.

[1660] The main components of the system are a terminal for users to input information, a server to process the input information, and a database to store the data. The specific hardware and software configurations are described below.

[1661] Hardware and software used

[1662] 1. Device:

[1663] A device used by a user, such as a smartphone, tablet, or PC. The device accesses the system through a web browser or a dedicated application.

[1664] 2. Server:

[1665] Data processing is performed using high-performance servers (e.g., AWS, Google Cloud) with a natural language processing engine (e.g., Google Natural Language API), a speech recognition engine (e.g., Google Speech-to-Text API), and an OCR engine (e.g., Tesseract OCR) installed.

[1666] 3. Database:

[1667] A relational database (e.g., MySQL, PostgreSQL) that stores user information, FAQ data, and business card information.

[1668] User reception process

[1669] User accesses and enters basic information: The user uses a terminal to access the system through a web browser or application. Then, the user enters basic information such as "name," "email address," and "phone number." For example, "Taro," "taro@example.com," and "090-1234-5678" are entered.

[1670] Sending and saving input information: The device sends the input information to the server, which stores it in a database.

[1671] Auto-Response Processing

[1672] User's question input and transmission: The user inputs a question into the chat box, and the terminal transmits the question to the server. For example, "What are your business hours?" is input.

[1673] Question analysis and answer generation: The server uses a natural language processing engine to analyze the question and search the FAQ database to get the appropriate answer, which is then sent to the device and displayed to the user.

[1674] Voice-enabled processing

[1675] Select and record voice input: The user clicks the voice input button in the chat box, and the device activates the microphone to record the voice. For example, "I would like to know more about this product."

[1676] Sending and converting voice data: The device sends the recorded voice data to the server, which then converts it into text using a voice recognition engine.

[1677] Generate and display an answer: The server analyzes the text information, generates an appropriate answer, sends it to the device, and displays it to the user. For example, "This product has been developed using the latest technology. Please visit our website for more information." is generated and displayed.

[1678] Business card information collection and processing

[1679] Providing and sending business card information: The user takes a photo of the business card with the camera and uploads it to the device, which then sends the image data to the server.

[1680] Image data conversion and storage: The server uses an OCR engine to extract text information from the image and store it in a database. It also sends the stored information to the sales department via email.

[1681] DM policy processing

[1682] Analyzing business card information and generating marketing messages: The server retrieves business card information from the database and generates a list of customers in a specific industry. The server then generates marketing messages tailored to each customer based on the analysis results.

[1683] Sending direct mail: Sending server-generated marketing messages to targeted users via email or post. For example, a message such as "We'd like to share a special offer with you" is generated.

[1684] With the above-described configuration and procedures, the system can improve the efficiency of call center operations in a variety of ways and increase customer satisfaction.

[1685] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1686] User reception process

[1687] Step 1:

[1688] A user accesses the system

[1689] A user starts a web browser or application on a device such as a PC or smartphone and accesses the system's URL. The input is the system's URL, and the output is the display of the system's web page.

[1690] Step 2:

[1691] The user enters basic information

[1692] A user enters basic information such as "name," "email address," and "phone number" into an input form displayed on the system. The input is the user's personal information (e.g., "Taro," "taro@example.com," "090-1234-5678"), and the output is that this information is displayed in the input fields.

[1693] Step 3:

[1694] The device sends the input information to the server

[1695] The device converts the information entered by the user into an appropriate format, such as JSON, and sends it to the server using an HTTP POST request. The input is the personal information entered by the user, and the output is the formatted data. As a concrete example, the following JSON data is sent from the device to the server:

[1696] json

[1697] {

[1698] "name": "Taro",

[1699] "email": "taro@example.com",

[1700] "phone": "090-1234-5678"

[1701] }

[1702] Step 4:

[1703] The server stores the information in a database

[1704] The server analyzes the received user information and executes SQL queries to store it in the database. The input is the personal information sent from the terminal, and the output is the data stored in the database. Specifically, the following SQL queries are executed:

[1705] sql

[1706] INSERT INTO UserTable (Name, Email, Phone) VALUES ('Taro', 'taro@example.com', '090-1234-5678')

[1707] Auto-Response Processing

[1708] Step 1:

[1709] The user enters a question

[1710] The user types a question into the chat box on the device and clicks the send button. The input is the user's question (e.g., "What are your business hours?"), and the output is the question displayed in the chat box.

[1711] Step 2:

[1712] The device sends a question to the server

[1713] The device sends the entered question to the server in an appropriate format (e.g. JSON format). The input is the question entered by the user, and the output is the data sent to the server. The content sent is as follows:

[1714] json

[1715] {

[1716] "question": "What are your opening hours?"

[1717] }

[1718] Step 3:

[1719] The server parses the question

[1720] The server uses a natural language processing engine to analyze the question and extract important keywords and intent. The input is the user's question data, and the output is the analyzed intent and keywords. For example, the keyword "business hours" is extracted.

[1721] Step 4:

[1722] The server searches the FAQ database

[1723] The server queries the FAQ database based on the analysis results to obtain the appropriate answer. The input is the analyzed keyword (e.g., "business hours"), and the output is the answer data from the FAQ database. Specifically, the following SQL query is executed:

[1724] sql

[1725] SELECT Answer FROM FAQTable WHERE Question LIKE '%Business Hours%'

[1726] And you find the answer, "Our business hours are Monday through Friday, 9:00 AM to 6:00 PM."

[1727] Step 5:

[1728] The server sends the answer to the device

[1729] The server converts the answer it gets into the appropriate format and sends it to the terminal as an HTTP response. The input is the answer from the FAQ database, and the output is the formatted response sent. An example response is as follows:

[1730] json

[1731] {

[1732] "answer": "Our business hours are Monday to Friday, 9am to 6pm."

[1733] }

[1734] Step 6:

[1735] The device displays the answer to the user

[1736] The terminal formats the answer received from the server and displays it in the chat box. The input is the answer data from the server, and the output is the display of the answer to the user.

[1737] Voice-enabled processing

[1738] Step 1:

[1739] The user selects a voice input

[1740] The user clicks the "Voice Input" button in the chat box. The input is an operation that enables voice input, and the output is an operation that starts voice recording.

[1741] Step 2:

[1742] The device records audio

[1743] The device activates the microphone and records the user's voice. The input is the user's voice (e.g., "I want to know more about this product"), and the output is the recorded voice data.

[1744] Step 3:

[1745] The device sends the audio data to the server.

[1746] The device sends the recorded audio data to the server in the appropriate format. The input is the recorded audio data, and the output is the data sent to the server.

[1747] Step 4:

[1748] The server converts the speech to text

[1749] The server uses a speech recognition engine to convert the voice data into text. The input is the voice data sent to the server, and the output is text data. For example, the text might be "I'd like to know more about this product."

[1750] Step 5:

[1751] The server generates the answer

[1752] The server analyzes the text information and generates an appropriate answer. The input is speech data converted into text, and the output is the generated answer text. For example, the answer generated might be, "This product was developed using the latest technology. Please visit our website for more information."

[1753] Step 6:

[1754] The server sends the answer to the device

[1755] The server generates a response and sends it to the terminal in text format. The input is the generated response text, and the output is the formatted response transmission.

[1756] Step 7:

[1757] The device displays the answer to the user

[1758] The terminal displays the answer received from the server in the chat box. The input is the answer data from the server, and the output is the display of the answer to the user.

[1759] Business card information collection and processing

[1760] Step 1:

[1761] User provides business card information

[1762] A user takes a photo of a business card with a camera and uploads it to a terminal as image data. The input is the image data of the business card, and the output is the image uploaded to the terminal.

[1763] Step 2:

[1764] The device sends the business card image to the server.

[1765] The terminal sends the business card image data to the server in the required format. The input is the business card image data, and the output is the data sent to the server.

[1766] Step 3:

[1767] The server converts the image into text

[1768] The server uses an OCR engine to extract text information from the business card image. The input is the business card image data, and the output is the extracted text data. For example, the extracted information is "Yamada Taro," "ABC Co., Ltd.", and "Manager."

[1769] Step 4:

[1770] The server stores the business card information in a database

[1771] The server saves the extracted business card information in a database. The input is the extracted business card information, and the output is the data stored in the database. Specifically, the following SQL query is executed:

[1772] sql

[1773] INSERT INTO BusinessCardTable (Name, Company, Title) VALUES ('Yamada Taro', 'ABC Co., Ltd.', 'Manager')

[1774] Step 5:

[1775] The server notifies the sales department

[1776] The server sends a notification email to the sales department based on the saved business card information. The input is the business card information saved in the database, and the output is a notification email to the sales department.

[1777] DM policy processing

[1778] Step 1:

[1779] The server analyzes the business card information

[1780] The server retrieves business card information from a database and analyzes it according to specific criteria. The input is the business card information retrieved from the database, and the output is the analysis result. For example, a customer list for a specific industry is generated.

[1781] Step 2:

[1782] The server generates the marketing message

[1783] The server generates a marketing message tailored to each customer based on the analysis results. The input is the analysis results, and the output is the generated marketing message. For example, a message such as "We'd like to share a special offer with you" is generated.

[1784] Step 3:

[1785] The server sends a DM

[1786] The server generates a marketing message and sends it to the target user via email or post. The input is the generated marketing message, and the output is the message sent to the target user.

[1787] By following these specific steps, the system will improve the efficiency of call center operations and customer satisfaction.

[1788] (Application example 1)

[1789] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1790] In food delivery services, efficient response to customer inquiries, ordering support, and the creation and transmission of marketing messages to specific customers requires the rapid and accurate processing of multiple pieces of information. However, with conventional systems, user input and responses to inquiries are often done manually, resulting in inefficiencies. Furthermore, managing voice data and business card information is cumbersome, and notifications to specific departments and the creation of marketing messages are not automated, limiting the ability to improve operational efficiency and customer satisfaction.

[1791] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1792] In this invention, the server includes means for receiving information input by a user, means for storing the received information in a database, means for analyzing questions submitted by the user, means for automatically generating answers based on the analysis results, means for converting voice data into text data, means for converting business card image data into text data, means for notifying a specific department of the converted business card information, means for receiving voice data, means for providing specific information based on the generated text data, means for analyzing input information received from the user and generating a message based on the specific information, and means for transmitting the generated message using a communication means. This allows inquiries, order support, and the generation and transmission of marketing messages in a food delivery service to be handled centrally and automatically, thereby significantly improving business efficiency and customer satisfaction.

[1793] The "means for receiving information input by the user" is a function for acquiring information input by the user into the application and incorporating it into the system.

[1794] The "means for saving received information in a database" is a function for storing acquired user information in a database so that it can be reused as needed.

[1795] The "means for analyzing questions submitted by users" is a function for interpreting the content of inquiries submitted by users and analyzing them to generate appropriate answers.

[1796] "Means for automatically generating an answer based on the analysis results" is a function that enables the system to automatically generate an appropriate response based on the analysis results of the question.

[1797] The "means for converting voice data into text data" is a function for converting voice data provided by the user into character data.

[1798] The "means for converting business card image data into text data" is a function for extracting character information from the image data of a business card provided by the user and saving it as text data.

[1799] The "means for notifying a specific department of the converted business card information" is a function for automatically notifying the relevant department of the extracted business card information.

[1800] The "means for receiving voice data" is a function for accepting voice input from the user.

[1801] The "means for providing specific information based on the generated text data" is a function for providing necessary information to the user based on information obtained from voice or character data.

[1802] The "means for analyzing input information received from a user and generating a message based on specific information" is a function for analyzing input information from a user and automatically creating a message based on specific conditions.

[1803] The "means for transmitting the generated message using a communication means" is a function for transmitting the automatically generated message to the user or the relevant department.

[1804] This invention is a system for improving the efficiency of customer service in food delivery services. The system is implemented using a smartphone and includes functions for receiving information entered by users, storing the received information in a database, converting voice input into text, converting business card images into text, and generating and sending marketing messages based on this information.

[1805] The server provides a dedicated application to the smartphone to receive the information entered by the user. This application has an interface where the user can enter basic information such as name, address, and phone number. The entered information is sent to the server in real time and stored in a database.

[1806] In the food delivery business, inquiries from customers occur frequently. When a customer enters a question via chat or voice through the application, the question is sent to the server. The server analyzes the question using natural language processing technology and automatically generates an appropriate answer from the FAQ database. At this time, the voice data is converted into text data using the Google Speech-to-Text API, and the text data is analyzed using the Google Cloud Natural Language API. Based on the analyzed data, an appropriate answer is extracted from the FAQ database and displayed to the customer.

[1807] Furthermore, if a customer provides image data of their business card, the application will activate the smartphone camera to take a picture of the business card. The captured image data will be sent to the server and converted into text data using the Google Cloud Vision API. The converted business card information will be stored in a database and notified to a specific department.

[1808] Regarding the generation of marketing messages, the server analyzes relevant data and automatically generates messages based on specific conditions. The generated messages are then sent to customers and related departments via communication means, which allows for targeting specific customers and improving the effectiveness of campaigns.

[1809] Here's an example of what happens when a customer uses voice input:

[1810] 1. User: "Launch the app."

[1811] 2. Application: "Hello, how can I help you?"

[1812] 3. User: "I'd like to order a pizza. Can you tell me what kind?"

[1813] 4. The application converts the speech to text and returns the appropriate information.

[1814] And here's an example of a prompt for the generative AI model:

[1815] A user asked, "What's today's special?" Find the right answer in our FAQ database.

[1816] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1817] Step 1:

[1818] The user launches the application and enters the required basic information (name, address, phone number, etc.).

[1819] Input: Basic information entered by the user.

[1820] Output: The device converts the basic information into an appropriate format and sends it to the server.

[1821] Step 2:

[1822] The server stores the received basic information in a database.

[1823] Input: Basic information sent from the device.

[1824] Output: Basic information is stored in the database. Specifically, the SQL query is executed and saved as follows: "INSERT INTO UserTable (Name, Address, Phone) VALUES ('Taro', '123 Main St', '090-1234-5678')".

[1825] Step 3:

[1826] Users use the chat box or voice input function within the application to make inquiries or orders.

[1827] Input: Text or voice input by the user.

[1828] Output: The device sends the entered text or voice data to the server.

[1829] Step 4:

[1830] The server uses a speech recognition engine to convert the voice data into text data.

[1831] Input: Audio data sent from the device.

[1832] Output: The audio data is converted to text using the Google Speech-to-Text API.

[1833] Step 5:

[1834] The server uses a natural language processing (NLP) engine to analyze the text data and generate appropriate answers.

[1835] Input: Text data.

[1836] Output: Answers based on the analysis results. Specifically, the text data is analyzed using the Google Cloud Natural Language API, and appropriate answers are extracted from the FAQ database.

[1837] Step 6:

[1838] The server generates a response and sends it to the terminal.

[1839] Input: The server-generated answer.

[1840] Output: The terminal displays the answer it received to the user, specifically, in the chat box.

[1841] Step 7:

[1842] The user provides a business card image through the application.

[1843] Input: Business card image data taken by the user with the smartphone camera.

[1844] Output: The terminal sends the business card image data to the server.

[1845] Step 8:

[1846] The server uses an OCR engine to convert the business card image data into text data.

[1847] Input: Business card image data sent from the device.

[1848] Output: The business card image data is converted to text data using the Google Cloud Vision API.

[1849] Step 9:

[1850] The server stores the converted business card information in a database and notifies a specific department.

[1851] Input: Business card information extracted by the OCR engine.

[1852] Output: The business card information is stored in a database and a notification is sent to a specific department. Specifically, an SQL query is executed, saved, and notified via email or the internal notification system.

[1853] Step 10:

[1854] A server generates a marketing message and transmits it using a communication means.

[1855] Input: Customer data and business card information stored in a database.

[1856] Output: The generated marketing messages are sent to the target users by analyzing the information in the database, generating messages based on the conditions, and sending them via email or in-app notifications.

[1857] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1858] This invention is a system for improving the efficiency of call center operations and customer satisfaction. This system automatically responds to inquiries and requests from users, performs voice recognition, collects business card information, and combines it with an emotion engine that recognizes the user's emotions to provide a more personalized response.

[1859] Explanation of program processing

[1860] User reception process

[1861] 1. A user accesses the system

[1862] A user accesses the system from a terminal through a web browser or application. For example, the user accesses the system's URL in a browser.

[1863] 2. The user enters basic information

[1864] The user enters basic information such as "name," "email address," and "phone number" into the input form. Specifically, the user enters "Taro," "taro@example.com," "090-1234-5678," etc.

[1865] 3. The device sends the input information to the server

[1866] The terminal converts the input information into an appropriate format and sends it to the server.

[1867] 4. The server stores the information in a database

[1868] The server stores the received user information in the database. For example, it executes an SQL query like "INSERT INTO UserTable (Name, Email, Phone) VALUES ('Taro', 'taro@example.com', '090-1234-5678')".

[1869] Auto-Response Processing

[1870] 1. The user enters a question

[1871] The user types a question into the chat box on the device, for example, "What are your business hours?"

[1872] 2. The device sends a question to the server

[1873] The terminal sends the entered question to the server.

[1874] 3. The server parses the question

[1875] The server uses a natural language processing (NLP) engine to parse the question, in this example extracting the keyword "business hours."

[1876] 4. The server searches the FAQ database

[1877] The server queries the FAQ database based on the analysis results and retrieves the appropriate answer, for example, "Business hours are Monday through Friday, 9:00 AM to 6:00 PM."

[1878] 5. The server sends the answer to the device

[1879] The server sends the obtained answer to the terminal.

[1880] 6. The device displays the answer to the user

[1881] The device will display the response it receives in the chat box.

[1882] Voice-enabled processing

[1883] 1. The user selects a voice input

[1884] The user clicks the "Speech Input" button in the chat box.

[1885] 2. The device records the audio

[1886] The device activates the microphone and records the user's voice. For example, the user might say, "I'd like to know more about this product."

[1887] 3. The device sends the audio data to the server

[1888] The device sends the recorded audio data to the server.

[1889] 4. The server converts the speech to text

[1890] The server uses a speech recognition engine to convert the voice data into text. In this example, the voice "I want to know more about the product" is converted into the text "I want to know more about the product."

[1891] 5. The server generates the answer

[1892] The server analyzes the text information and generates an appropriate response, such as "This product was developed using the latest technology. Please visit our website for more information."

[1893] 6. The server sends the answer to the device

[1894] The server generates a response text and sends it to the terminal.

[1895] 7. The device displays the answer to the user

[1896] The device will display the response it receives in the chat box.

[1897] Business card information collection and processing

[1898] 1. The user provides business card information

[1899] The user takes a photo of the business card with a camera and uploads the image data to the terminal.

[1900] 2. The device sends the business card image to the server

[1901] The terminal transmits the business card image data to the server.

[1902] 3. The server converts the image into text

[1903] The server uses an OCR (Optical Character Recognition) engine to extract text information from the image. In this example, it extracts information such as "Yamada Taro," "ABC Co., Ltd.", and "Manager."

[1904] 4. The server saves the business card information in a database

[1905] The server saves the extracted business card information in the database. For example, it saves it as follows: INSERT INTO BusinessCardTable (Name, Company, Title) VALUES ('Yamada Taro', 'ABC Co., Ltd.', 'Manager').

[1906] 5. The server notifies the sales department

[1907] The server sends the stored information to the sales department staff via email.

[1908] DM policy processing

[1909] 1. The server analyzes the business card information

[1910] The server retrieves the business card information from the database and analyzes it, for example to generate a list of customers in a particular industry.

[1911] 2. The server generates the marketing message

[1912] Based on the analysis results, the server generates a marketing message tailored to each customer, such as "A special offer for Taro Yamada of ABC Co., Ltd."

[1913] 3. The server sends the DM

[1914] Sending server-generated marketing messages to targeted users via email and post.

[1915] Emotion engine processing

[1916] 1. Recognize user emotions

[1917] The server uses an emotion engine to recognize emotions from user input data, for example, extracting emotions such as "anger" or "joy" from user speech or voice recordings.

[1918] 2. Tailor responses based on emotional data

[1919] The server automatically adjusts responses based on the recognized emotion data, for example, generating a calming tone if the user is angry.

[1920] 3. Save emotion data

[1921] The server stores the recognized emotion data in a database for later analysis, for example, to analyze the emotional tendencies of a particular user.

[1922] 4. Generating Marketing Messages Using Emotional Data

[1923] The server generates more personalized marketing messages based on the emotional data analyzed by the emotion engine. For example, it might suggest a "relaxing offer" to a user with a neutral emotion.

[1924] This allows the system to respond quickly and accurately to user inquiries and requests, and by combining voice recognition and emotion recognition, it will improve business efficiency and customer satisfaction.

[1925] The processing flow will be explained below.

[1926] User reception process

[1927] Step 1:

[1928] A user accesses the system

[1929] A user accesses the system from a terminal through a web browser or application. For example, the user accesses the system's URL in a browser.

[1930] Step 2:

[1931] The user enters basic information

[1932] The user enters basic information such as "name," "email address," and "phone number" into the input form. Specifically, the user enters "Taro," "taro@example.com," "090-1234-5678," etc.

[1933] Step 3:

[1934] The device sends the input information to the server

[1935] The terminal converts the input information into an appropriate format and sends it to the server.

[1936] Step 4:

[1937] The server stores the information in a database

[1938] The server stores the received user information in the database. For example, it executes an SQL query like "INSERT INTO UserTable (Name, Email, Phone) VALUES ('Taro', 'taro@example.com', '090-1234-5678')".

[1939] Auto-Response Processing

[1940] Step 1:

[1941] The user enters a question

[1942] The user types a question into the chat box on the device, for example, "What are your business hours?"

[1943] Step 2:

[1944] The device sends a question to the server

[1945] The terminal sends the entered question to the server.

[1946] Step 3:

[1947] The server parses the question

[1948] The server uses a natural language processing (NLP) engine to parse the question, in this example extracting the keyword "business hours."

[1949] Step 4:

[1950] The server searches the FAQ database

[1951] The server queries the FAQ database based on the analysis results and retrieves the appropriate answer, for example, "Business hours are Monday through Friday, 9:00 AM to 6:00 PM."

[1952] Step 5:

[1953] The server sends the answer to the device

[1954] The server sends the obtained answer to the terminal.

[1955] Step 6:

[1956] The device displays the answer to the user

[1957] The device will display the response it receives in the chat box.

[1958] Voice-enabled processing

[1959] Step 1:

[1960] The user selects a voice input

[1961] The user clicks the "Speech Input" button in the chat box.

[1962] Step 2:

[1963] The device records audio

[1964] The device activates the microphone and records the user's voice. For example, the user might say, "I'd like to know more about this product."

[1965] Step 3:

[1966] The device sends the audio data to the server.

[1967] The device sends the recorded audio data to the server.

[1968] Step 4:

[1969] The server converts the speech to text

[1970] The server uses a speech recognition engine to convert the voice data into text. In this example, the voice "I want to know more about the product" is converted into the text "I want to know more about the product."

[1971] Step 5:

[1972] The server generates the answer

[1973] The server analyzes the text information and generates an appropriate response, such as "This product was developed using the latest technology. Please visit our website for more information."

[1974] Step 6:

[1975] The server sends the answer to the device

[1976] The server generates a response text and sends it to the terminal.

[1977] Step 7:

[1978] The device displays the answer to the user

[1979] The device will display the response it receives in the chat box.

[1980] Business card information collection and processing

[1981] Step 1:

[1982] User provides business card information

[1983] The user takes a photo of the business card with a camera and uploads the image data to the terminal.

[1984] Step 2:

[1985] The device sends the business card image to the server.

[1986] The terminal transmits the business card image data to the server.

[1987] Step 3:

[1988] The server converts the image into text

[1989] The server uses an OCR (Optical Character Recognition) engine to extract text information from the image. In this example, it extracts information such as "Yamada Taro," "ABC Co., Ltd.", and "Manager."

[1990] Step 4:

[1991] The server stores the business card information in a database

[1992] The server saves the extracted business card information in the database. For example, it saves it as follows: INSERT INTO BusinessCardTable (Name, Company, Title) VALUES ('Yamada Taro', 'ABC Co., Ltd.', 'Manager').

[1993] Step 5:

[1994] The server notifies the sales department

[1995] The server sends the stored information to the sales department staff via email.

[1996] DM policy processing

[1997] Step 1:

[1998] The server analyzes the business card information

[1999] The server retrieves the business card information from the database and analyzes it, for example to generate a list of customers in a particular industry.

[2000] Step 2:

[2001] The server generates the marketing message

[2002] Based on the analysis results, the server generates a marketing message tailored to each customer, such as "A special offer for Taro Yamada of ABC Co., Ltd."

[2003] Step 3:

[2004] The server sends a DM

[2005] Sending server-generated marketing messages to targeted users via email and post.

[2006] Emotion engine processing

[2007] Step 1:

[2008] Recognize user emotions

[2009] The server uses an emotion engine to recognize emotions from user input data, for example, extracting emotions such as "anger" or "joy" from user speech or voice recordings.

[2010] Step 2:

[2011] Tailor responses based on emotional data

[2012] The server automatically adjusts responses based on the recognized emotion data, for example, generating a calming tone if the user is angry.

[2013] Step 3:

[2014] Storing emotion data

[2015] The server stores the recognized emotion data in a database for later analysis, for example, to analyze the emotional tendencies of a particular user.

[2016] Step 4:

[2017] Generating marketing messages using emotional data

[2018] The server generates more personalized marketing messages based on the emotional data analyzed by the emotion engine. For example, it might suggest a "relaxing offer" to a user with a neutral emotion.

[2019] The above is the specific processing flow of the system of the present invention, which enables quick and accurate responses to user inquiries and requests, and by combining speech recognition and emotion recognition, improves the efficiency of call center operations and customer satisfaction.

[2020] Example 2

[2021] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[2022] Conventional call center systems have limited capabilities for responding to user inquiries, making it difficult to provide personalized responses tailored to individual needs. Furthermore, there are insufficient means for improving customer satisfaction by utilizing advanced technologies such as voice recognition and emotion recognition. Furthermore, collecting business card information and generating marketing messages based on that information is time-consuming, requiring increased efficiency. To address these issues, the present invention provides a system that integrates the automation of inquiry responses, personalized responses using emotion recognition, and the collection and analysis of business card information.

[2023] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for receiving information input by a user, means for saving the received information in a database, means for analyzing a question submitted by a user, means for automatically generating an answer based on the analysis result, means for converting voice data into text data, means for converting business card image data into text data, means for notifying the sales department of the converted business card information, means for recognizing the user's emotion, and means for adjusting a response based on the recognized emotion. This improves the accuracy and speed of inquiry responses, enables personalized responses according to customer emotions, and further realizes more efficient marketing measures through the automatic collection and analysis of business card information.

[2024] "Means for receiving user-entered information" refers to the functionality of devices or software that capture data entered by users through forms or chat boxes and incorporate it into the system.

[2025] "Means for storing received information in a database" refers to a technology for permanently storing data received from a user via a database system such as SQL.

[2026] "Means for analyzing questions submitted by users" refers to software functionality that uses natural language processing (NLP) technology to understand the content of users' questions and extract keywords and intent.

[2027] "Means for automatically generating answers based on the analysis results" refers to the functions of algorithms and databases that automatically create appropriate answers based on the analyzed question content and provide them to users.

[2028] "Means for converting voice data into text data" refers to technology that uses a voice recognition engine to convert voice files or real-time voice input into text format.

[2029] The "means for converting business card image data into text data" is a software function for extracting text information from business card images using optical character recognition (OCR) technology.

[2030] "Means for notifying the sales department of the converted business card information" is a function for notifying the sales staff of the extracted business card information via email or a notification system.

[2031] "Means for recognizing user emotions" refers to technology that uses an emotion analysis engine to extract emotions from user input data and voice.

[2032] The "means for tailoring responses based on recognized emotions" is a software function for automatically generating and delivering responses with appropriate tone and content depending on the user's emotional state.

[2033] The present invention is a system that automatically responds to inquiries and requests from users, providing more personalized responses by performing voice recognition, collecting business card information, and even emotion recognition. This system operates using the following hardware and software.

[2034] Hardware

[2035] 1. Terminal: A device (e.g., computer, smartphone, tablet) through which a user enters information.

[2036] 2. Server: A computer system running hardware and software for processing and storing data.

[2037] 3. Database: A data storage system for persistently storing information (e.g., MySQL, PostgreSQL).

[2038] software

[2039] 1. Web Browser or Application: The interface through which the user accesses the system.

[2040] 2. Natural Language Processing (NLP) engine: Software for analyzing user questions (e.g., Google Dialogflow).

[2041] 3. Speech recognition engine: Software for converting voice data into text (e.g., Google Cloud Speech-to-Text).

[2042] 4. Optical Character Recognition (OCR) engine: Software for extracting text information from business card images (e.g., Tesseract OCR).

[2043] 5. Sentiment analysis engine: Software for recognizing user emotions (e.g., IBM Watson Tone Analyzer).

[2044] 6. Generative AI models: Software for tailoring responses based on user sentiment (e.g., GPT-3).

[2045] Specific examples

[2046] 1. User acceptance process

[2047] A user accesses the system's web page using a smartphone and enters their name (e.g., "Taro"), email address (e.g., "taro@example.com"), and phone number (e.g., "090-1234-5678"). The device sends this information to the server, which then stores the received information in a database.

[2048] 2. Auto-Response Processing

[2049] The user enters a question into the chat box (e.g., "What are your business hours?"). The device sends the question to the server, which analyzes the question using an NLP engine. Based on the analysis results, the server retrieves an appropriate answer from the FAQ database (e.g., "Business hours are Monday through Friday, 9 a.m. to 6 p.m.") and displays it to the user through the device.

[2050] 3. Voice-enabled processing

[2051] The user clicks the voice input button and says, "I'd like to know more about this product." The device records the voice and sends it to the server. The server uses a speech recognition engine to convert the voice data into text and generate an answer (e.g., "This product was developed using the latest technology. Please visit our website for more information."). The answer is returned to the device and displayed to the user.

[2052] 4. Business card information collection and processing

[2053] A user takes a photo of a business card with their smartphone and uploads it to the system. The device sends the business card image to the server, which uses an OCR engine to extract text information from the image. The extracted information (e.g., "Yamada Taro," "XYZ Co., Ltd.", "Manager") is stored in a database and notified to the sales department.

[2054] 5. Emotional Engine Processing

[2055] The emotion analysis engine recognizes emotional data (e.g., "anger" or "happiness") from the user's comments and questions, and the server generates a response that corresponds to that emotion. For example, if the user is angry, the server generates a response in a calming tone (e.g., "We apologize for any inconvenience this may have caused you"). The generated response is provided to the user via the device.

[2056] Prompt Sentence Examples

[2057] The prompts that users enter into the system include:

[2058] 1. "What are your opening hours?"

[2059] 2. "I want to know more about the product"

[2060] 3. "I want to know about new product release information."

[2061] As a result, this system responds quickly and accurately to user inquiries and requests, and by combining voice recognition and emotion recognition, it achieves improved business efficiency and customer satisfaction.

[2062] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2063] Program processing flow

[2064] User reception process

[2065] Step 1:

[2066] A user accesses the system. The user opens a web browser or application on the terminal and accesses the system's URL. Input: Access request is made by the URL output of the web browser. Output: Login screen or initial screen is displayed.

[2067] Step 2:

[2068] The user enters basic information. The user enters their name, email address, and phone number into the system form. Specific examples of input include "Taro," "taro@example.com," and "090-1234-5678." Input: Basic information of the user. Output: Entered information data.

[2069] Step 3:

[2070] The terminal sends the input information to the server. The terminal converts the input data into JSON format and sends it to the server via a POST request. Input: JSON format of user input information. Output: JSON data is sent to the server.

[2071] Step 4:

[2072] The server saves the information in the database. The server saves the received data in the database using an SQL query. Specifically, it executes the query "INSERT INTO UserTable (Name, Email, Phone) VALUES ('Taro', 'taro@example.com', '090-1234-5678')". Input: User information in JSON format. Output: Information saved in the database.

[2073] Auto-Response Processing

[2074] Step 1:

[2075] User types a question. User types a question into the chat box (e.g., "What are your business hours?"). Input: User's question text. Output: Question typed into the chat box.

[2076] Step 2:

[2077] The terminal sends the question to the server. The terminal converts the entered question into JSON format and sends it to the server. Input: JSON format of the question text. Output: JSON data sent to the server.

[2078] Step 3:

[2079] The server analyzes the question. The server uses a natural language processing engine to analyze the question and extract keywords. Input: The question text "What are your business hours?" Output: The keyword "business hours".

[2080] Step 4:

[2081] The server searches the FAQ database. The server queries the FAQ table based on the extracted keywords to get the appropriate answer. For example, it executes the query "SELECT Answer FROM FAQTable WHERE Question LIKE '%Business Hours%'". Input: Keyword "Business Hours". Output: Answer text from the FAQ database.

[2082] Step 5:

[2083] The server sends the answer to the terminal. The server converts the answer text to JSON format and sends it to the terminal. Input: Answer from the FAQ database. Output: Answer data in JSON format.

[2084] Step 6:

[2085] The terminal displays the answer to the user. The terminal displays the received answer in the chat box. Input: JSON data sent from the server. Output: Answer displayed in the chat box.

[2086] Voice-enabled processing

[2087] Step 1:

[2088] User selects audio input. User clicks "Audio Input" button in chat box. Input: Select audio input. Output: Activate microphone.

[2089] Step 2:

[2090] The device records the voice. The user speaks (e.g., "I'd like to know more about this product") and the device records the voice. Input: User's speech. Output: Recorded voice data.

[2091] Step 3:

[2092] The device sends audio data to the server. The device sends the recorded audio data to the server via a POST request. Input: Recorded audio data. Output: Audio data sent to the server.

[2093] Step 4:

[2094] The server converts speech to text. The server uses a speech recognition engine to convert the speech data into text format. Input: speech data. Output: converted text data.

[2095] Step 5:

[2096] The server generates the answer. The server analyzes the text question content and generates the corresponding answer. Input: Converted text data. Output: Generated answer text.

[2097] Step 6:

[2098] The server sends the answer to the terminal. The server converts the answer to JSON format and sends it to the terminal. Input: Generated answer text. Output: JSON format answer data.

[2099] Step 7:

[2100] The terminal displays the answer to the user. The terminal displays the received answer in the chat box. Input: JSON data sent from the server. Output: Answer displayed in the chat box.

[2101] Business card information collection and processing

[2102] Step 1:

[2103] The user provides business card information. The user takes a photo of the business card with the smartphone camera and uploads it to the system. Input: Business card image. Output: Uploaded image data.

[2104] Step 2:

[2105] The terminal sends the business card image to the server. The terminal sends the business card image data to the server. Input: Business card image data. Output: Image data sent to the server.

[2106] Step 3:

[2107] The server converts the image to text. The server uses an OCR engine to extract text information from the image data. Input: Business card image data. Output: Converted text information.

[2108] Step 4:

[2109] The server saves the business card information in the database. The server saves the extracted business card information in the database using an SQL query. For example, execute the query "INSERT INTO BusinessCardTable (Name, Company, Title) VALUES ('Yamada Taro', 'XYZ Co., Ltd.', 'Manager')". Input: Converted text information. Output: Information saved in the database.

[2110] Step 5:

[2111] The server notifies the sales department. The server sends a notification email to the sales department based on the saved information. Input: Business card information. Output: Notification email to the sales department.

[2112] Emotion engine processing

[2113] Step 1:

[2114] The server recognizes the user's emotions. The server uses an emotion analysis engine to extract emotions from the user's input data. Input: User's input data. Output: Extracted emotion data.

[2115] Step 2:

[2116] The server tailors the response based on the emotion data. The server tailors the response using a generative AI model based on the extracted emotion data. Input: Emotion data. Output: Tailored response text.

[2117] Step 3:

[2118] The server stores the emotion data. The server stores the recognized emotion data in a database and uses it for later analysis. Input: Emotion data. Output: Emotion data saved in the database.

[2119] Step 4:

[2120] The server generates a marketing message using the emotion data. The server generates a personalized marketing message for each user based on the emotion data. For example, a user with a calm emotion might be offered a "relaxing offer." Input: Emotion data. Output: Generated marketing message.

[2121] (Application example 2)

[2122] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[2123] Autonomous vehicles require rapid and appropriate responses to passenger inquiries and requests. However, conventional systems have struggled to recognize passenger emotions and provide personalized responses based on those emotions. They also lacked efficient means to digitize passenger business card information and generate marketing messages. The present invention aims to solve these problems and improve passenger service.

[2124] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[2125] In this invention, the server includes means for receiving information input by a user, means for storing the received information in a database, means for analyzing questions submitted by users, means for automatically generating answers based on the analysis results, means for converting voice data to text data, means for converting business card image data to text data, means for notifying the sales department of the converted business card information, means for receiving and analyzing voice input from passengers in the vehicle and generating an automatic response, means for recognizing passenger emotions and providing a response according to the emotions, and means for digitizing and storing passenger business card information, thereby making it possible to respond quickly and appropriately to passenger inquiries and requests and provide personalized services based on emotions.

[2126] "Information entered by the user" refers to data such as the name, email address, phone number, and questions that the user provides to the system.

[2127] "Means for storing in a database" refers to a method using a server or storage device to organize and efficiently store the received information.

[2128] "Means of analysis" refers to the process of using natural language processing technology to understand the content of questions and input data submitted by users and determine how to respond.

[2129] "Means for automatically generating answers" refers to algorithms or programs that generate optimal answers from analysis results or pre-set information and present them to the user.

[2130] "Means for converting voice data into text data" refers to the process of converting a user's voice input into text information using voice recognition technology.

[2131] The "means of converting business card image data into text data" is the process of extracting text information from business card images using OCR (optical character recognition) technology.

[2132] The "means of notifying the sales department" is a method of transmitting the extracted business card information and other important information to the sales department staff via email or a notification system.

[2133] "Means for receiving and analyzing passenger voice inputs within a vehicle and generating an automatic response" refers to a system for recognizing passenger voice instructions within an automated vehicle, analyzing the content of the instructions, and generating an appropriate response.

[2134] "Means for recognizing passenger emotions and providing emotionally appropriate responses" refers to a process for determining the passenger's emotional state from the passenger's tone of voice and content of speech, and generating a response with an appropriate tone and content.

[2135] The "means for digitizing and storing passenger business card information" refers to a method of capturing business cards provided by passengers as digital images, converting them into text data, and storing them in the system.

[2136] The present invention provides a system for streamlining passenger service and providing personalized responses in autonomous vehicles, which implements a series of processes for efficiently acquiring, analyzing, and responding to passenger input information.

[2137] System Overview

[2138] This system is composed of the following means:

[2139] 1. A means of receiving information entered by the user

[2140] Passengers enter information such as their name, destination, and questions using a smartphone app or a touch panel inside the vehicle, and send it to the system.

[2141] 2. A means of storing the received information in a database

[2142] The received information is stored in a database, for example, using a relational database such as PostgreSQL.

[2143] 3. A means of analyzing user-submitted questions

[2144] Questions are parsed using natural language processing (NLP) techniques, using the Google Cloud Natural Language API to extract keywords and intent from the text.

[2145] 4. A means of automatically generating answers based on analysis results

[2146] Based on the analysis results, an appropriate response is generated, utilizing a pre-configured FAQ database and generative AI models.

[2147] 5. Means of converting audio data into text data

[2148] Passenger voice input is converted into text data using Google Cloud Speech-to-Text.

[2149] 6. How to convert business card image data into text data

[2150] When a passenger provides their business card information, OCR (optical character recognition) technology, such as Tesseract OCR, is used to extract text data from the image of the business card.

[2151] 7. A method for notifying the sales department of the converted business card information

[2152] Business card information is digitized and sent to the sales department via email or internal notification system.

[2153] 8. Means for receiving and analyzing passenger voice inputs in the vehicle to generate automated responses

[2154] Passenger voice commands are received through an in-car microphone, analyzed, and an appropriate response is generated, combining voice recognition and NLP technology.

[2155] 9. A means of recognizing passenger emotions and providing emotional responses

[2156] The system determines the passenger's emotions from their tone of voice and the content of their comments, and uses IBM Watson Emotion Analysis to generate responses based on those emotions.

[2157] 10. A means of digitizing and storing passenger business card information

[2158] The business card information is digitized, stored, and used to generate subsequent marketing messages.

[2159] Specific examples

[2160] Specific examples of voice input

[2161] A passenger says, "Please make the car's air conditioning a little cooler." This speech is collected through a microphone in the car and converted into text using Google Cloud Speech-to-Text: "Please make the car's air conditioning a little cooler." The system then analyzes this text using NLP technology and generates the response message, "The temperature has been adjusted."

[2162] Example prompts for generative AI models

[2163] "Please determine the emotion from the text below. Then suggest an appropriate response: \"How's the weather today?\""

[2164] This enables the system of the present invention to provide personalized services in autonomous vehicles and improve passenger satisfaction.

[2165] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2166] Step 1:

[2167] Users access the system through a touch panel in the car or a smartphone app. They input their destination, name, and other necessary information. For example, a user might input "Destination: Chuo Station" and "Name: Yamada Taro" into the touch panel.

[2168] Input: Destination, name, and other basic information

[2169] Output: The entered information is saved in JSON format on the device.

[2170] How it works: The device displays an input form, and once the user completes the form, it converts the information into JSON format and sends it to the server.

[2171] Step 2:

[2172] The terminal sends the entered information to a database. The server receives it and stores it in the database. A relational database such as PostgreSQL is used.

[2173] Input: User input information in JSON format

[2174] Output: User information stored in the database

[2175] How it works: The server receives the JSON data, generates the appropriate queries, and saves them to the database.

[2176] Step 3:

[2177] The user types a question into the touch panel or chat box of the app, for example, "Please lower the temperature inside the car."

[2178] Input: User's question

[2179] Output: The question sent to the server

[2180] Operation: The device sends the entered question to the server.

[2181] Step 4:

[2182] The server analyzes the user's question using natural language processing (NLP) techniques, including the Google Cloud Natural Language API, to extract keywords and intent from the question.

[2183] Input: User's question

[2184] Output: Parsed keywords and intent

[2185] How it works: The server uses an NLP engine to parse the incoming question and extract keywords (e.g., temperature) and intent (e.g., lower).

[2186] Step 5:

[2187] The server generates an automated response based on the analysis results, for example, "The temperature inside the car has been lowered."

[2188] Input: Parsed keywords and intent

[2189] Output: Auto-generated response message

[2190] How it works: The server references the FAQ database and generative AI models to generate the best response.

[2191] Step 6:

[2192] When a user speaks, the car's microphone records the speech and converts it into text using Google Cloud Speech-to-Text. For example, "Turn up the air conditioning" is converted into text as "Turn up the air conditioning."

[2193] Input: Audio data

[2194] Output: Text data

[2195] How it works: The device records audio and sends the audio data to the server, which uses a speech recognition engine to convert it into text.

[2196] Step 7:

[2197] When a user provides business card information, the system uses an in-car camera to take a picture of the business card and then uses OCR technology to extract the text data from the card. For example, the business card image is converted into "Yamada Taro, company name, job title."

[2198] Input: Business card image data

[2199] Output: Business card text data

[2200] How it works: The device scans a business card and sends the image data to the server, which uses Tesseract OCR to extract the text data.

[2201] Step 8:

[2202] The server stores the converted business card information in a database and notifies the sales department, for example, by sending an email to the sales department.

[2203] Input: Business card text data

[2204] Output: Business card information stored in the database, notification email

[2205] What it does: The server saves the business card information in a database and sends a notification to the specified email address.

[2206] Step 9:

[2207] The server recognizes the passenger's emotions and generates responses based on their emotions using IBM Watson Emotion Analysis. For example, if the user is angry, it generates a calmer response.

[2208] Input: What the user said

[2209] Output: Sentiment analysis results, tailored response message

[2210] How it works: The server analyzes the voice or text data using an emotion recognition engine and generates a response based on the user's emotional state.

[2211] Step 10:

[2212] The server uses the business card information to generate and send marketing messages, which allows for personalized marketing.

[2213] Input: Business card information, user data

[2214] Output: Marketing message

[2215] How it works: The server analyzes the business card information, generates marketing messages tailored to the specific user, and sends them via email or notification system.

[2216] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[2217] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2218] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[2219] [Fourth embodiment]

[2220] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[2221] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[2222] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[2223] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[2224] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[2225] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[2226] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[2227] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[2228] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[2229] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[2230] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[2231] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[2232] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2233] This invention is a system for improving the efficiency of call center operations and customer satisfaction. This system automatically responds to inquiries and requests from users, performs voice recognition, and collects business card information. It also effectively obtains information useful for sales activities and generates marketing messages.

[2234] Explanation of program processing

[2235] User reception process

[2236] 1. A user accesses the system

[2237] A user accesses the system from a terminal through a web browser or application. For example, the user accesses the system's URL in a browser.

[2238] 2. The user enters basic information

[2239] The user enters basic information such as "name," "email address," and "phone number" into the input form. Specifically, the user enters "Taro," "taro@example.com," "090-1234-5678," etc.

[2240] 3. The device sends the input information to the server

[2241] The terminal converts the input information into an appropriate format and sends it to the server.

[2242] 4. The server stores the information in a database

[2243] The server stores the received user information in the database. For example, it executes an SQL query like "INSERT INTO UserTable (Name, Email, Phone) VALUES ('Taro', 'taro@example.com', '090-1234-5678')".

[2244] Auto-Response Processing

[2245] 1. The user enters a question

[2246] The user types a question into the chat box on the device, for example, "What are your business hours?"

[2247] 2. The device sends a question to the server

[2248] The terminal sends the entered question to the server.

[2249] 3. The server parses the question

[2250] The server uses a natural language processing (NLP) engine to parse the question, in this example extracting the keyword "business hours."

[2251] 4. The server searches the FAQ database

[2252] The server queries the FAQ database based on the analysis results and retrieves the appropriate answer, for example, "Business hours are Monday through Friday, 9:00 AM to 6:00 PM."

[2253] 5. The server sends the answer to the device

[2254] The server sends the obtained answer to the terminal.

[2255] 6. The device displays the answer to the user

[2256] The device will display the response it receives in the chat box.

[2257] Voice-enabled processing

[2258] 1. The user selects a voice input

[2259] The user clicks the "Speech Input" button in the chat box.

[2260] 2. The device records the audio

[2261] The device activates the microphone and records the user's voice. For example, the user might say, "I'd like to know more about this product."

[2262] 3. The device sends the audio data to the server

[2263] The device sends the recorded audio data to the server.

[2264] 4. The server converts the speech to text

[2265] The server uses a speech recognition engine to convert the voice data into text. In this example, the voice "I want to know more about the product" is converted into the text "I want to know more about the product."

[2266] 5. The server generates the answer

[2267] The server analyzes the text information and generates an appropriate response, such as "This product was developed using the latest technology. Please visit our website for more information."

[2268] 6. The server sends the answer to the device

[2269] The server generates a response text and sends it to the terminal.

[2270] 7. The device displays the answer to the user

[2271] The device will display the response it receives in the chat box.

[2272] Business card information collection and processing

[2273] 1. The user provides business card information

[2274] The user takes a photo of the business card with a camera and uploads the image data to the terminal.

[2275] 2. The device sends the business card image to the server

[2276] The terminal transmits the business card image data to the server.

[2277] 3. The server converts the image into text

[2278] The server uses an OCR (Optical Character Recognition) engine to extract text information from the image. In this example, it extracts information such as "Yamada Taro," "ABC Co., Ltd.", and "Manager."

[2279] 4. The server saves the business card information in a database

[2280] The server saves the extracted business card information in the database. For example, it saves it as follows: INSERT INTO BusinessCardTable (Name, Company, Title) VALUES ('Yamada Taro', 'ABC Co., Ltd.', 'Manager').

[2281] 5. The server notifies the sales department

[2282] The server sends the stored information to the sales department staff via email.

[2283] DM policy processing

[2284] 1. The server analyzes the business card information

[2285] The server retrieves the business card information from the database and analyzes it, for example to generate a list of customers in a particular industry.

[2286] 2. The server generates the marketing message

[2287] Based on the analysis results, the server generates a marketing message tailored to each customer, such as "A special offer for Taro Yamada of ABC Co., Ltd."

[2288] 3. The server sends the DM

[2289] Sending server-generated marketing messages to targeted users via email and post.

[2290] As described above, this system responds to user inquiries and requests from multiple angles, thereby improving the efficiency of call center operations and customer satisfaction.

[2291] The processing flow will be explained below.

[2292] User reception process

[2293] Step 1:

[2294] A user accesses the system

[2295] A user accesses the system from a terminal through a web browser or application. For example, the user accesses the system's URL in a browser.

[2296] Step 2:

[2297] The user enters basic information

[2298] The user enters basic information such as "name," "email address," and "phone number" into the input form. Specifically, the user enters "Taro," "taro@example.com," "090-1234-5678," etc.

[2299] Step 3:

[2300] The device sends the input information to the server

[2301] The terminal converts the input information into an appropriate format and sends it to the server.

[2302] Step 4:

[2303] The server stores the information in a database

[2304] The server stores the received user information in the database. For example, it executes an SQL query like "INSERT INTO UserTable (Name, Email, Phone) VALUES ('Taro', 'taro@example.com', '090-1234-5678')".

[2305] Auto-Response Processing

[2306] Step 1:

[2307] The user enters a question

[2308] The user types a question into the chat box on the device, for example, "What are your business hours?"

[2309] Step 2:

[2310] The device sends a question to the server

[2311] The terminal sends the entered question to the server.

[2312] Step 3:

[2313] The server parses the question

[2314] The server uses a natural language processing (NLP) engine to parse the question, in this example extracting the keyword "business hours."

[2315] Step 4:

[2316] The server searches the FAQ database

[2317] The server queries the FAQ database based on the analysis results and retrieves the appropriate answer, for example, "Business hours are Monday through Friday, 9:00 AM to 6:00 PM."

[2318] Step 5:

[2319] The server sends the answer to the device

[2320] The server sends the obtained answer to the terminal.

[2321] Step 6:

[2322] The device displays the answer to the user

[2323] The device will display the response it receives in the chat box.

[2324] Voice-enabled processing

[2325] Step 1:

[2326] The user selects a voice input

[2327] The user clicks the "Speech Input" button in the chat box.

[2328] Step 2:

[2329] The device records audio

[2330] The device activates the microphone and records the user's voice. For example, the user might say, "I'd like to know more about this product."

[2331] Step 3:

[2332] The device sends the audio data to the server.

[2333] The device sends the recorded audio data to the server.

[2334] Step 4:

[2335] The server converts the speech to text

[2336] The server uses a speech recognition engine to convert the voice data into text. In this example, the voice "I want to know more about the product" is converted into the text "I want to know more about the product."

[2337] Step 5:

[2338] The server generates the answer

[2339] The server analyzes the text information and generates an appropriate response, such as "This product was developed using the latest technology. Please visit our website for more information."

[2340] Step 6:

[2341] The server sends the answer to the device

[2342] The server generates a response text and sends it to the terminal.

[2343] Step 7:

[2344] The device displays the answer to the user

[2345] The device will display the response it receives in the chat box.

[2346] Business card information collection and processing

[2347] Step 1:

[2348] User provides business card information

[2349] The user takes a photo of the business card with a camera and uploads the image data to the terminal.

[2350] Step 2:

[2351] The device sends the business card image to the server.

[2352] The terminal transmits the business card image data to the server.

[2353] Step 3:

[2354] The server converts the image into text

[2355] The server uses an OCR (Optical Character Recognition) engine to extract text information from the image. In this example, it extracts information such as "Yamada Taro," "ABC Co., Ltd.", and "Manager."

[2356] Step 4:

[2357] The server stores the business card information in a database

[2358] The server saves the extracted business card information in the database. For example, it saves it as follows: INSERT INTO BusinessCardTable (Name, Company, Title) VALUES ('Yamada Taro', 'ABC Co., Ltd.', 'Manager').

[2359] Step 5:

[2360] The server notifies the sales department

[2361] The server sends the stored information to the sales department staff via email.

[2362] DM policy processing

[2363] Step 1:

[2364] The server analyzes the business card information

[2365] The server retrieves the business card information from the database and analyzes it, for example to generate a list of customers in a particular industry.

[2366] Step 2:

[2367] The server generates the marketing message

[2368] Based on the analysis results, the server generates a marketing message tailored to each customer, such as "A special offer for Taro Yamada of ABC Co., Ltd."

[2369] Step 3:

[2370] The server sends a DM

[2371] Sending server-generated marketing messages to targeted users via email and post.

[2372] Example 1

[2373] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2374] Call center operations must efficiently respond to a large number of inquiries, which requires the rapid and accurate processing of data in various formats. However, conventional systems have difficulty integrating voice recognition, business card information collection, and natural language processing, resulting in problems such as reduced work efficiency and low customer satisfaction. Furthermore, the creation and transmission of marketing messages is also done manually, which is inefficient. There is a need to solve these problems, improve the efficiency of call center operations, and increase customer satisfaction.

[2375] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[2376] In this invention, the server includes means for receiving information input by a user, means for storing the received information in a database, means for analyzing questions submitted by users, means for automatically generating answers based on the analysis results, means for converting voice data into text data, means for converting business card image data into text data, means for notifying the sales department of the converted business card information, means for analyzing questions using natural language processing technology, means for searching an FAQ database to obtain an appropriate answer, means for making voice input selectable, means for transmitting recorded voice data to the server, means for displaying answers to the user, means for formatting the input information and transmitting it to the server, means for analyzing the business card information and generating a marketing message, and means for transmitting the generated marketing message to the user. This enables efficient response to inquiries, voice recognition, automatic collection of business card information, and automatic generation and transmission of marketing messages.

[2377] "Means for receiving information input by a user" refers to an interface for receiving data input by a user.

[2378] "Means for saving received information to a database" refers to the process of saving the data entered by the user in a database for storing the data.

[2379] "Means for analyzing a question submitted by a user" refers to a processing technology for analyzing a question entered by a user and understanding its intent.

[2380] "Means for automatically generating answers based on the analysis results" refers to a process for automatically generating appropriate answers to the analyzed questions.

[2381] "Means for converting voice data into text data" refers to technology for converting information input by voice into text format.

[2382] "Means for converting business card image data into text data" refers to technology for converting business card photos or scanned data into text information.

[2383] "Means for notifying the sales department of the converted business card information" refers to the process of notifying the sales department of the data extracted from the business card.

[2384] "Means for analyzing questions using natural language processing technology" refers to technology for analyzing user questions using natural language processing technology.

[2385] "Means for searching the FAQ database to obtain an appropriate answer" refers to the process of searching the FAQ database to obtain an appropriate answer to a user's question.

[2386] "Means for enabling selection of voice input" refers to a technology that provides an interface that allows a user to select a voice input.

[2387] "Means for transmitting recorded voice data to a server" refers to a communication technology for transmitting recorded voice data to a server.

[2388] The "means for displaying the answer to the user" refers to an interface for displaying the generated answer on the user's terminal.

[2389] "Means for formatting input information and sending it to the server" refers to the process of converting the information input by the user into an appropriate format and sending it to the server.

[2390] "Means for analyzing business card information and generating a marketing message" refers to technology for analyzing information obtained from a business card and generating a specific marketing message.

[2391] "Means for sending generated marketing messages to users" refers to the process of sending generated marketing messages to target users.

[2392] This invention is a system aimed at improving the efficiency of call center operations and customer satisfaction. This system automatically responds to inquiries and requests from users, performs voice recognition, and collects business card information. It also effectively obtains information useful for sales activities and generates marketing messages.

[2393] The main components of the system are a terminal for users to input information, a server to process the input information, and a database to store the data. The specific hardware and software configurations are described below.

[2394] Hardware and software used

[2395] 1. Device:

[2396] A device used by a user, such as a smartphone, tablet, or PC. The device accesses the system through a web browser or a dedicated application.

[2397] 2. Server:

[2398] Data processing is performed using high-performance servers (e.g., AWS, Google Cloud) with a natural language processing engine (e.g., Google Natural Language API), a speech recognition engine (e.g., Google Speech-to-Text API), and an OCR engine (e.g., Tesseract OCR) installed.

[2399] 3. Database:

[2400] A relational database (e.g., MySQL, PostgreSQL) that stores user information, FAQ data, and business card information.

[2401] User reception process

[2402] User accesses and enters basic information: The user uses a terminal to access the system through a web browser or application. Then, the user enters basic information such as "name," "email address," and "phone number." For example, "Taro," "taro@example.com," and "090-1234-5678" are entered.

[2403] Sending and saving input information: The device sends the input information to the server, which stores it in a database.

[2404] Auto-Response Processing

[2405] User's question input and transmission: The user inputs a question into the chat box, and the terminal transmits the question to the server. For example, "What are your business hours?" is input.

[2406] Question analysis and answer generation: The server uses a natural language processing engine to analyze the question and search the FAQ database to get the appropriate answer, which is then sent to the device and displayed to the user.

[2407] Voice-enabled processing

[2408] Select and record voice input: The user clicks the voice input button in the chat box, and the device activates the microphone to record the voice. For example, "I would like to know more about this product."

[2409] Sending and converting voice data: The device sends the recorded voice data to the server, which then converts it into text using a voice recognition engine.

[2410] Generate and display an answer: The server analyzes the text information, generates an appropriate answer, sends it to the device, and displays it to the user. For example, "This product has been developed using the latest technology. Please visit our website for more information." is generated and displayed.

[2411] Business card information collection and processing

[2412] Providing and sending business card information: The user takes a photo of the business card with the camera and uploads it to the device, which then sends the image data to the server.

[2413] Image data conversion and storage: The server uses an OCR engine to extract text information from the image and store it in a database. It also sends the stored information to the sales department via email.

[2414] DM policy processing

[2415] Analyzing business card information and generating marketing messages: The server retrieves business card information from the database and generates a list of customers in a specific industry. The server then generates marketing messages tailored to each customer based on the analysis results.

[2416] Sending direct mail: Sending server-generated marketing messages to targeted users via email or post. For example, a message such as "We'd like to share a special offer with you" is generated.

[2417] With the above-described configuration and procedures, the system can improve the efficiency of call center operations in a variety of ways and increase customer satisfaction.

[2418] The flow of the identification process in the first embodiment will be described with reference to FIG.

[2419] User reception process

[2420] Step 1:

[2421] A user accesses the system

[2422] A user starts a web browser or application on a device such as a PC or smartphone and accesses the system's URL. The input is the system's URL, and the output is the display of the system's web page.

[2423] Step 2:

[2424] The user enters basic information

[2425] A user enters basic information such as "name," "email address," and "phone number" into an input form displayed on the system. The input is the user's personal information (e.g., "Taro," "taro@example.com," "090-1234-5678"), and the output is that this information is displayed in the input fields.

[2426] Step 3:

[2427] The device sends the input information to the server

[2428] The device converts the information entered by the user into an appropriate format, such as JSON, and sends it to the server using an HTTP POST request. The input is the personal information entered by the user, and the output is the formatted data. As a concrete example, the following JSON data is sent from the device to the server:

[2429] json

[2430] {

[2431] "name": "Taro",

[2432] "email": "taro@example.com",

[2433] "phone": "090-1234-5678"

[2434] }

[2435] Step 4:

[2436] The server stores the information in a database

[2437] The server analyzes the received user information and executes SQL queries to store it in the database. The input is the personal information sent from the terminal, and the output is the data stored in the database. Specifically, the following SQL queries are executed:

[2438] sql

[2439] INSERT INTO UserTable (Name, Email, Phone) VALUES ('Taro', 'taro@example.com', '090-1234-5678')

[2440] Auto-Response Processing

[2441] Step 1:

[2442] The user enters a question

[2443] The user types a question into the chat box on the device and clicks the send button. The input is the user's question (e.g., "What are your business hours?"), and the output is the question displayed in the chat box.

[2444] Step 2:

[2445] The device sends a question to the server

[2446] The device sends the entered question to the server in an appropriate format (e.g. JSON format). The input is the question entered by the user, and the output is the data sent to the server. The content sent is as follows:

[2447] json

[2448] {

[2449] "question": "What are your opening hours?"

[2450] }

[2451] Step 3:

[2452] The server parses the question

[2453] The server uses a natural language processing engine to analyze the question and extract important keywords and intent. The input is the user's question data, and the output is the analyzed intent and keywords. For example, the keyword "business hours" is extracted.

[2454] Step 4:

[2455] The server searches the FAQ database

[2456] The server queries the FAQ database based on the analysis results to obtain the ap...

Claims

1. means for receiving user input information; a means for storing the received information in a database; and means for analyzing questions submitted by users; a means for automatically generating an answer based on the analysis results; means for converting voice data into text data; A means for converting image data of a business card into text data; The system includes a means for notifying the sales department of the converted business card information.

2. 10. The system of claim 1, further comprising means for analyzing the input information using natural language processing techniques.

3. 10. The system of claim 1, further comprising means for analyzing the business card information and generating a marketing message.

Images