System

Abstract

To provide systems.SOLUTION: A system includes the means of: comprehensively analyzing a customer profile, schedule management information, customer service voice data, and a taste of the customer or a degree of interest in an own company service; inputting a specific prompt sentence to a generative AI model, and thereby analyzing the taste of the customer and the degree of interest in the own company service on the basis of the customer profile and the customer service voice data; converting the customer service voice data to text data; estimating an emotion state of the customer from the customer service voice data; inputting a characteristic extracted from the customer profile and the customer service voice data to an artificial intelligence model, and thereby analyzing a degree of interest relative to the taste of the customer and the own company service; and providing a service personalized for each customer on the basis of an analysis result by the analysis means.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Currently, it is difficult to determine which customer is being served from customer service voice data, and it is also a challenge to efficiently analyze customer preferences and interest in the company's services and provide personalized services based on that analysis. [Means for solving the problem]

[0005] As a means for solving this problem, the system disclosed herein includes: means for comprehensively analyzing customer profiles, schedule management information, customer response voice data, and customer preferences and interest in the company's services; means for identifying information about customers visiting the store today using schedule management information and customer response voice data; means for analyzing customer preferences and interest in the company's services based on the customer profiles and customer response voice data by inputting specific prompt sentences into a generative AI model; means for converting the customer response voice data into text data using a voice recognition system; means for estimating the customer's emotional state from the customer response voice data using an emotion engine; analysis means for analyzing customer preferences and interest in the company's services by inputting features extracted from the customer profiles and the customer response voice data into the artificial intelligence model; and means for providing personalized services for each customer based on the analysis results by the analysis means. [Brief explanation of the drawings]

[0006] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 2 is a sequence diagram showing a flow of processing in the data processing system according to the first embodiment of the first form example. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1 of Embodiment 1. [Figure 13] FIG. 10 is a sequence diagram showing a processing flow of a data processing system in a second embodiment of the second form example. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 of Embodiment Example 2. [Figure 15] FIG. 10 is a sequence diagram showing the flow of processing in a data processing system according to a third embodiment of the third embodiment. [Figure 16] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 3 of Embodiment 3. [Figure 17] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the first embodiment of the first form example when an emotion engine is combined. [Figure 18] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1 of Form Example 1 when an emotion engine is combined. [Figure 19] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the second embodiment of the second form example when an emotion engine is combined. [Figure 20] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 of Form Example 2 when an emotion engine is combined. [Figure 21] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the third embodiment of the third form example when an emotion engine is combined. [Figure 22] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 3 of Form Example 3 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0007] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0008] First, the terms used in the following description will be explained.

[0009] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be one arithmetic unit or a combination of multiple arithmetic units.

[0010] Furthermore, the processor may be one type of computing device or a combination of multiple types of computing devices. Examples of computing devices include a central processing unit (CPU), a graphics processing unit (GPU), a general-purpose computing on graphics processing units (GPGPU), an accelerated processing unit (APU), or a tensor processing unit (TPU (registered trademark)).

[0011] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0012] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0013] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0014] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0015] [First embodiment]

[0016] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0017] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0018] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0019] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0020] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0021] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0022] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0023] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0024] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0025] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0026] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0027] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0028] "Example 1"

[0029] In one embodiment of the present invention, a speech recognition system is provided for acquiring customer-facing speech data. The speech recognition system acquires speech data input from a microphone installed in the store in real time and converts the data into text data. The text data includes what the customer is saying and other speech information.

[0030] "Example 2"

[0031] Furthermore, a system is provided that uses schedule management information and customer interaction voice data to identify customer information for that day. Specifically, a list of customers scheduled to visit the store that day is created based on schedule management information, and the list is compared with customer interaction voice data to identify which customer corresponds to which voice data.

[0032] "Example 3"

[0033] The company also has a system that uses AI to analyze customer preferences and interest in its services. This system analyzes customer preferences and interest in its services based on features extracted from customer interaction voice data and customer profiles. Specifically, the AI ​​learns what products and services customers are interested in and what purchasing behavior they tend to exhibit, and provides personalized services for each customer based on that information.

[0034] The processing flow of each embodiment will be described below.

[0035] "Example 1"

[0036] Step 1: Acquire audio data in real time from microphones installed in the store.

[0037] Step 2: Convert the acquired audio data into text data.

[0038] Step 3: Save the converted text data and use it for further processing.

[0039] "Example 2"

[0040] Step 1: Create a list of customers who are scheduled to visit the store that day based on scheduling information.

[0041] Step 2: Compare the list you created with customer interaction voice data.

[0042] Step 3: From the comparison results, identify which customer corresponds to which voice data.

[0043] "Example 3"

[0044] Step 1: Extract features from customer interaction voice data and customer profiles.

[0045] Step 2: Based on the extracted features, the AI ​​analyzes the customer's preferences and level of interest in the company's services.

[0046] Step 3: Provide personalized services to each customer based on the results of AI analysis.

[0047] Example 1

[0048] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0049] Conventional customer support systems have difficulty accurately grasping customer comments and preferences in real time, making it difficult to respond to customer needs promptly. Furthermore, there is a lack of efficient means for managing and analyzing voice data, making it difficult to provide personalized services based on customer preferences and interests.

[0050] The identification process by the identification processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means. In this invention, the server includes: a means for comprehensively analyzing customer profiles, schedule management information, customer response voice data, and customer preferences and interest in the company's services; a means for identifying information about customers visiting the store today using the schedule management information and customer response voice data; a means for analyzing customer preferences and interest in the company's services using artificial intelligence; a means for acquiring voice data in real time from a voice input device installed in the store; a means for converting the acquired voice data into text data using voice recognition technology; a means for storing the converted text data in a database; and a means for analyzing the stored text data using natural language processing technology. This makes it possible to accurately grasp the content of customer comments and preferences in real time and respond quickly.

[0051] A "customer profile" is a collection of data that includes basic information about a customer, past purchase history, preferences, interests, etc.

[0052] "Schedule management information" is data for managing customer visit schedules, reservation information, past visit history, and the like.

[0053] "Customer interaction voice data" is voice data including conversations between customers and store clerks and the content of what customers say.

[0054] "Customer preferences" refers to information such as the products and services that customers prefer and the areas in which they are interested.

[0055] "Interest in our company's services" is an indicator that shows how interested customers are in the products and services that our company offers.

[0056] "Artificial intelligence" is a technology that allows computers to learn and reason by imitating human intelligence.

[0057] An "audio input device" is a device for acquiring audio as digital data, and generally refers to a microphone.

[0058] "Speech recognition technology" is a technology for converting voice data into text data.

[0059] "Text data" is data expressed as character information.

[0060] A "database" is a system for efficiently storing, managing, and retrieving data.

[0061] "Natural language processing technology" is a technology that allows computers to understand, interpret, and generate human language.

[0062] This invention relates to a system for acquiring and analyzing customer interaction voice data. Specifically, this system acquires voice data in real time from a voice input device installed in a store, converts it into text data using voice recognition technology, stores the text data in a database, and analyzes it using natural language processing technology.

[0063] Hardware and software used

[0064] The server uses the following hardware and software:

[0065] Voice input device: Uses microphones installed in the store to capture high-quality conversations between customers and store staff.

[0066] Speech recognition technology: The acquired voice data is converted into text data using the Google (registered trademark) Cloud Speech-to-Text API.

[0067] Database: A MySQL® database is used to store the converted text data.

[0068] Natural language processing techniques: Analyze stored text data using natural language processing toolkits such as NLTK and SpaCy.

[0069] Specific operation of the system

[0070] The server collects voice data from multiple microphones installed in various areas of the store. These microphones capture conversations between customers and store staff in real time. For example, microphones installed near the cash register and shelves pick up customer questions and requests.

[0071] The collected voice data is sent to the Google Cloud Speech-to-Text API, which uses advanced speech recognition algorithms to convert voice data into text. For example, if a customer says, "I'd like to know more about this product," the API will convert this speech into text: "I'd like to know more about this product."

[0072] The converted text data is stored in a MySQL database, which is designed to efficiently manage customer comments and other voice information. For example, if a customer says, "I'd like to know more about this product," the text data is stored in a specific table in the database.

[0073] The stored text data is analyzed using natural language processing toolkits (e.g., NLTK or SpaCy). This analysis can identify customer needs and problems. For example, if multiple customers say, "I want to know more about this product," the server aggregates that information and suggests that there is a lack of information about a particular product.

[0074] Specific examples

[0075] For example, if a customer says, "I'd like to know more about this product," the server captures this speech through a microphone and uses the Google Cloud Speech-to-Text API to convert it into text data that says, "I'd like to know more about this product." This text data is then stored in a MySQL database and analyzed using natural language processing tools. The analysis may reveal that information about a particular product is lacking, and the store can take steps to provide more information about that product.

[0076] Prompt Sentence Examples

[0077] Examples of prompts to be input to a generative AI model include:

[0078] "Design a system to capture customer-facing voice data in real time and convert it into text data. Specifically, voice data will be captured from microphones installed in stores and converted into text data using the Google Cloud Speech-to-Text API. The converted text data will be stored in a MySQL database and analyzed using natural language processing tools."

[0079] By inputting this prompt into a generative AI model, specific advice and code examples can be obtained on system design and implementation.

[0080] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0081] Step 1: Acquire audio data

[0082] The server collects real-time audio data from multiple microphones installed in the store. These microphones are placed in various areas of the store and capture conversations between customers and store staff in high quality. The input is the audio signal from the microphones, and the output is digital audio data. For example, microphones installed near the cash register or shelves pick up customer questions and requests.

[0083] Step 2: Convert the audio data

[0084] The server sends the captured voice data to the Google Cloud Speech-to-Text API. This API uses advanced speech recognition algorithms to convert voice data into text data. The input is digital voice data, and the output is text data. For example, if a customer says, "I'd like to know more about this product," the API converts this speech into text: "I'd like to know more about this product."

[0085] Step 3: Save the text data

[0086] The server stores the converted text data in a MySQL database. This database is designed to efficiently manage customer utterances and other voice information. The input is text data, and the output is a record stored in the database. For example, if a customer utters, "I'd like to know more about this product," the text data is stored in a specific table in the database.

[0087] Step 4: Analyze the data

[0088] The server analyzes the stored text data using a natural language processing toolkit (e.g., NLTK or SpaCy). This analysis can identify customer needs and problems. The input is the text data stored in the database, and the output is the analysis results. For example, if multiple customers say, "I want to know more about this product," the server aggregates that information and suggests that there is a lack of information about a particular product.

[0089] Step 5: Personalized service delivery

[0090] Based on the analysis results, the server provides personalized services based on the customer's preferences and interest in the store's services. The input is the analysis results, and the output is the proposal or provision of personalized services. For example, if it is determined that information about a specific product is lacking, the store can take measures to provide detailed information about that product.

[0091] (Application example 1)

[0092] Next, a description will be given of Application Example 1 of Embodiment Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0093] Conventional customer service systems have had the problem of being unable to grasp customer comments in real time and provide appropriate responses quickly. Furthermore, personalized service based on customer preferences and interests was not provided adequately, making it difficult to improve customer satisfaction. Furthermore, there was a lack of support for store staff to respond quickly and appropriately to customers.

[0094] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0095] In this invention, the server includes: a means for comprehensively analyzing customer profiles, schedule management information, customer response voice data, and customer preferences and interest in the company's services; a means for identifying today's customer information using schedule management information and customer response voice data; a means for analyzing customer preferences and interest in the company's services using AI; a means for converting voice data into text data in real time; a means for analyzing the text data and generating appropriate responses to customer questions and requests; and a means for displaying the generated responses to support rapid customer responses. This enables the content of customer comments to be understood in real time and appropriate responses to be provided promptly. This also enables personalized service based on customer preferences and interests, which is expected to improve customer satisfaction. Furthermore, this system provides enhanced support for store staff to quickly provide appropriate responses when serving customers.

[0096] A "customer profile" is data that includes basic information about a customer, their past purchasing history, preferences, etc.

[0097] "Schedule management information" is data related to time, such as customer visit schedules and reservation information.

[0098] "Customer interaction voice data" is voice data including the content of the conversation between the customer and the store clerk.

[0099] "Customer preferences and interest in our company's services" is data that indicates the products and services that customers prefer and the level of interest in them.

[0100] A "voice recognition system" is a system that converts voice data into text data.

[0101] "AI analytics" is the process of using artificial intelligence to analyze data and extract patterns and trends.

[0102] "Real-time" means that data is processed as it is generated.

[0103] "Text data" is character information converted from voice data.

[0104] "Analysis" is the process of examining data in detail to understand its meaning and structure.

[0105] "Appropriate response" refers to the most appropriate response or action to a customer's question or request.

[0106] "Personalized service provision" means providing customized services based on the individual preferences and interests of customers.

[0107] "Quick customer response" refers to responding quickly to customer requests.

[0108] The system for carrying out this invention includes a means for comprehensively analyzing customer profiles, schedule management information, customer interaction voice data, and customer preferences and interest in the company's services. The system is realized using the following hardware and software.

[0109] Hardware

[0110] Smartphone: A device used by store associates to capture and display voice data.

[0111] Microphone: A device installed in the store to collect customer comments.

[0112] software

[0113] Python: A programming language used to implement programs.

[0114] SpeechRecognition library: A library for converting voice data into text data.

[0115] OpenAI® API: An artificial intelligence model for analyzing text data and generating appropriate responses.

[0116] Data processing and calculation

[0117] 1. Voice data capture: A microphone collects what the customer says and sends it to a smartphone.

[0118] 2. Converting speech data to text: Use the SpeechRecognition library to convert speech data to text in real time.

[0119] 3. Text data analysis: Use OpenAI API to analyze text data and generate appropriate responses to customer questions and requests.

[0120] 4. Display of response: The generated response is displayed on the smartphone, allowing the store clerk to quickly respond to the customer.

[0121] Specific examples

[0122] For example, if a customer says, "Tell me more about this product," the system operates as follows:

[0123] 1. A microphone collects what the customer says and sends it to a smartphone.

[0124] 2. The SpeechRecognition library converts the audio data into text data.

[0125] 3. The OpenAI API analyzes the text data and generates a response like this:

[0126] "This product uses the latest technology and is particularly durable. For more information, please see this brochure."

[0127] 4. The generated response is displayed on the smartphone, and the store clerk responds promptly to the customer.

[0128] Prompt Sentence Examples

[0129] Customers say: Tell me more about this product

[0130] Please suggest how the store clerk should respond.

[0131] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0132] Step 1:

[0133] Acquiring audio data

[0134] The server collects customer speech from microphones installed in the store. The microphones capture voice data in real time and send it to a smartphone. The input is the customer's speech, and the output is voice data. This voice data is converted into text data in a subsequent processing step.

[0135] Step 2:

[0136] Converting audio data to text

[0137] The device (smartphone) uses the SpeechRecognition library to convert the received voice data into text data in real time. The input is voice data and the output is text data. In this conversion process, voice recognition technology is used to convert the voice content into text information.

[0138] Step 3:

[0139] Text data analysis

[0140] The server uses the OpenAI API to analyze the text data. The input is text data, and the output is the analysis results. Specifically, it understands customer questions and requests based on the text data and generates appropriate responses. This analysis process uses a generative AI model to perform natural language processing.

[0141] Step 4:

[0142] Generate correspondence

[0143] The server generates an appropriate response to the customer's question or request based on the analysis results. The input is the analysis results, and the output is the generated response. The generated response includes a specific reply or instruction for action based on the content of the customer's statement.

[0144] Step 5:

[0145] Display of correspondence

[0146] The terminal (smartphone) displays the generated response to the store clerk. The input is the generated response, and the output is text displayed on the smartphone screen. The store clerk can check this display and respond to the customer quickly.

[0147] Specific examples

[0148] For example, if a customer says, "Tell me more about this product," the system operates as follows:

[0149] 1. Step 1: The microphone captures the customer's utterance, "Tell me more about this product," and sends it to the smartphone.

[0150] 2. Step 2: The smartphone converts the voice data into text data: "Tell me more about this product."

[0151] 3. Step 3: The server analyzes the text data and generates an appropriate response to the customer's question.

[0152] 4. Step 4: The server generates a response saying, "This product uses the latest technology and is particularly durable. If you need more information, please see this brochure."

[0153] 5. Step 5: The smartphone displays the generated response to the store associate, who then responds promptly to the customer.

[0154] Prompt Sentence Examples

[0155] Customers say: Tell me more about this product

[0156] Please suggest how the store clerk should respond.

[0157] Example 2

[0158] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0159] With conventional customer management systems, it was difficult to effectively integrate schedule management information and customer interaction voice data to identify customer information. Furthermore, there was a lack of means to accurately understand customer preferences and interest in the company's services, and to provide personalized services. This hindered the efficiency of customer interactions and the improvement of service quality.

[0160] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0161] In this invention, the server includes means for acquiring schedule management information, means for acquiring voice data, means for converting voice data into text data, means for comparing the acquired schedule management information with the converted text data and identifying customer information, means for saving and notifying the identified customer information, and means for analyzing customer preferences and interest in the company's services using artificial intelligence. This makes it possible to integrate schedule management information and customer interaction voice data to identify customer information and further analyze customer preferences and interest to provide personalized services.

[0162] A "customer profile" is a collection of data that includes basic information about a customer, past transaction history, preferences, etc.

[0163] "Schedule management information" is a collection of data including customer visit schedules and reservation information.

[0164] "Customer response voice data" refers to voice recordings of conversations and inquiries with customers.

[0165] "Customer preferences" refers to information that indicates the types and characteristics of products and services that customers prefer.

[0166] The "degree of interest in our company's services" is information indicating the degree of interest that customers have in the services provided by our company.

[0167] "Artificial intelligence" is a technology that uses machine learning and data analysis to imitate human intellectual tasks.

[0168] A "voice recognition system" is a technology that analyzes voice data and converts it into text data.

[0169] "Personalized services" are customized services provided based on a customer's individual preferences and interests.

[0170] A "server" is a computer system that processes, stores, and communicates data.

[0171] "Text data" refers to data obtained by converting voice data into character information.

[0172] A "database" is a system for efficiently storing, searching, and managing data.

[0173] "Notification" is the act of notifying a user or other system of specific information.

[0174] This invention is a system for comprehensively analyzing customer profiles, schedule management information, customer interaction voice data, and customer preferences and interest in the company's services. A specific embodiment of this system is described below.

[0175] Hardware and software used

[0176] Scheduling software: Google Calendar

[0177] Audio data storage database: Amazon S3

[0178] Speech recognition software: Google Cloud Speech-to-Text

[0179] Database: MySQL

[0180] Server Processing

[0181] 1. Acquiring schedule management information

[0182] The server uses the Google Calendar API to obtain a list of customers scheduled to visit the store that day.

[0183] Specifically, the server sends a request to the Google Calendar API and receives JSON-formatted data as a response.

[0184] 2. Acquisition of customer interaction voice data

[0185] The server downloads customer response voice data from Amazon S3.

[0186] Specifically, the server accesses an Amazon S3 bucket and retrieves the specified audio file.

[0187] 3. Converting Audio Data to Text

[0188] The server converts the audio data into text data using the Google Cloud Speech-to-Text API.

[0189] Specifically, the server sends the audio file to the Google Cloud Speech-to-Text API and receives text data in response.

[0190] 4. Compare customer lists with text data

[0191] The server compares the acquired customer list with the converted text data and identifies which customer corresponds to which voice data.

[0192] Specifically, the server analyzes keywords and phrases in the text data and compares them with information in the customer list.

[0193] 5. Storage of Identified Customer Information and Notification

[0194] The server stores the identified customer information in a MySQL database and notifies the terminal.

[0195] Specifically, the server executes an INSERT query in the MySQL database and notifies the terminal by sending an HTTP request.

[0196] Terminal handling

[0197] 1. Receiving customer list

[0198] The terminal receives the list of customers scheduled to visit the store on that day sent from the server.

[0199] Specifically, the terminal receives an HTTP response from the server and parses the JSON-formatted data.

[0200] 2. Receiving and playing audio data

[0201] The terminal receives the customer response voice data transmitted from the server and plays back the voice data.

[0202] Specifically, the device downloads the audio file and plays it using the built-in media player.

[0203] 3. Display of identified customer information

[0204] The terminal displays the identified customer information sent from the server so that the user can confirm it.

[0205] Specifically, the terminal displays the received data on a GUI (graphical user interface).

[0206] User operations

[0207] 1. Check your customer list

[0208] The user checks the list of customers scheduled to visit the store that day displayed on the terminal.

[0209] Specifically, the user looks at the screen of the terminal and checks the customer information displayed in the list.

[0210] 2. Check the audio data

[0211] The user listens to the customer service voice data reproduced from the terminal.

[0212] Specifically, the user listens to the audio data through the terminal's speaker or headphones.

[0213] 3. Confirmation and response of identified customer information

[0214] The user checks the specified customer information displayed on the terminal and takes action as necessary.

[0215] Specifically, the user responds to the customer based on the displayed information.

[0216] Specific examples

[0217] Prompt Sentence Examples

[0218] "Compare the list of customers scheduled to visit obtained from the schedule management information with the customer response voice data stored in the voice data storage database, and identify which customer corresponds to which voice data."

[0219] In this way, the system uses schedule management information and customer interaction voice data to identify information about customers visiting the store that day, and further analyzes their preferences and interests to provide personalized services.

[0220] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0221] Step 1:

[0222] Obtaining schedule management information

[0223] The server uses the schedule management software's API to obtain a list of customers scheduled to visit the store that day. Specifically, the server sends a request to the API and receives JSON-formatted data as a response.

[0224] Input: API request

[0225] Output: A list of customers in JSON format

[0226] Specific operation: The server sends an HTTP GET request to the API endpoint and parses the JSON data received as a response to extract the customer list.

[0227] Step 2:

[0228] Acquisition of customer interaction voice data

[0229] The server downloads the customer response voice data from the voice data storage database. Specifically, the server accesses a bucket in the database and acquires a specified voice file.

[0230] Input: Database bucket information

[0231] Output: Audio file

[0232] Specific operation: The server obtains a list of audio files using the database API and downloads the necessary audio files.

[0233] Step 3:

[0234] Converting audio data to text

[0235] The server uses speech recognition software to convert the audio data into text data. Specifically, the server sends the audio file to a speech recognition API and receives the text data in response.

[0236] Input: Audio file

[0237] Output: Text data

[0238] Specific operation: The server uploads the audio file to the API, performs speech recognition processing, and obtains text data.

[0239] Step 4:

[0240] Comparing customer lists and text data

[0241] The server compares the acquired customer list with the converted text data to identify which customer corresponds to which voice data. Specifically, the server analyzes keywords and phrases in the text data and matches them with the information in the customer list.

[0242] Input: Customer list, text data

[0243] Output: Identified customer information

[0244] Specific operation: The server analyzes the text data to identify parts that match names and reservation information in the customer list, and extracts the corresponding customer information.

[0245] Step 5:

[0246] Retention of identified customer information and notification

[0247] The server stores the identified customer information in a database and notifies the terminal. Specifically, the server executes an INSERT query in the database and notifies the terminal by sending an HTTP request.

[0248] Input: Identified customer information

[0249] Output: Save to database, notify to terminal

[0250] Specific operations: The server stores the identified customer information in a database, and generates and sends an HTTP request to send a notification to the terminal.

[0251] Step 6:

[0252] Receive customer list

[0253] The terminal receives the list of customers scheduled to visit the store that day sent from the server. Specifically, the terminal receives the HTTP response from the server and parses the JSON format data.

[0254] Input: HTTP response from the server

[0255] Output: Parsed customer list

[0256] Specific operation: The terminal analyzes the received HTTP response, extracts the customer list, and displays it.

[0257] Step 7:

[0258] Receiving and playing audio data

[0259] The terminal receives the customer response voice data transmitted from the server and plays the voice data. Specifically, the terminal downloads the voice file and plays it on a built-in media player.

[0260] Input: Audio file

[0261] Output: The audio being played

[0262] Specific operations: The device downloads the audio file, launches the media player, and plays the audio.

[0263] Step 8:

[0264] View identified customer information

[0265] The terminal displays the specified customer information sent from the server so that the user can check it. Specifically, the terminal displays the received data on a GUI.

[0266] Input: Identified customer information

[0267] Output: Displayed customer information

[0268] Specific operation: The terminal displays the identified customer information on the GUI so that the user can check it.

[0269] Step 9:

[0270] Check the customer list

[0271] The user checks the list of customers scheduled to visit the store that day, which is displayed on the terminal. Specifically, the user looks at the screen of the terminal and checks the customer information displayed in the list.

[0272] Input: Displayed customer list

[0273] Output: Confirmed customer information

[0274] Specific operations: The user checks the customer list on the terminal screen and takes action as necessary.

[0275] Step 10:

[0276] Checking audio data

[0277] The user listens to the customer service voice data reproduced from the terminal, specifically, through the terminal's speaker or headphones.

[0278] Input: Audio to be played

[0279] Output: Confirmed audio data

[0280] Specific operation: The user listens to the audio data through the device's speaker or headphones and checks the content.

[0281] Step 11:

[0282] Confirmation and response of identified customer information

[0283] The user checks the specified customer information displayed on the terminal and responds as necessary. Specifically, the user responds to the customer based on the displayed information.

[0284] Input: Displayed customer information

[0285] Output: Customer information handled

[0286] Specific operations: The user checks the identified customer information displayed on the terminal and takes action as necessary to assist the customer.

[0287] (Application example 2)

[0288] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0289] Conventional customer management systems required a lot of manual work to identify customers who were planning to visit the store, which led to problems with inefficiency. It was also difficult to provide personalized services based on customer preferences and interests, making it difficult to improve customer satisfaction.

[0290] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for comprehensively analyzing customer profiles, schedule management information, customer response voice data, and customer preferences and interest in the company's services, means for identifying information about customers visiting today using the schedule management information and customer response voice data, means for analyzing customer preferences and interest in the company's services using AI, means for converting voice data to text, means for comparing the converted text data with the schedule management information to identify customers visiting the store, and means for displaying the identified customer information. This makes it possible to automatically identify information about customers who are scheduled to visit the store and provide personalized services based on the customers' preferences and interests.

[0291] A "customer profile" is data that includes basic information about a customer, past purchase history, preferences, etc.

[0292] "Schedule management information" is data including customer visit schedules and reservation information.

[0293] "Customer response voice data" refers to voice recordings of conversations with customers and inquiries.

[0294] "Customer preferences" refers to information that indicates the types and characteristics of products and services that customers prefer.

[0295] "Interest in our company's services" is an indicator that shows how interested customers are in the services provided by our company.

[0296] "Comprehensive analysis methods" are methods that integrate and analyze multiple data sets to understand customer behavior and preferences.

[0297] "Means of identification" refers to a method for identifying and clarifying specific information or data.

[0298] "AI analysis" refers to the use of artificial intelligence technology to analyze data and find patterns and trends.

[0299] "Means for converting voice data into text" refers to a method for converting voice into text information using voice recognition technology.

[0300] A "comparison tool" is a method for comparing different data and finding commonalities and differences.

[0301] The "display means" refers to a method for visually presenting the analysis results and the identified information.

[0302] The system for carrying out the present invention operates in cooperation with three entities: a server, a terminal, and a user. A specific embodiment of the system will be described below.

[0303] Server Roles

[0304] The server has the ability to comprehensively analyze customer profiles, schedule management information, customer interaction voice data, and customer preferences and interest in the company's services. The server integrates this data and performs analysis using AI technology. Specifically, it uses a voice recognition system to convert voice data into text, and compares the converted text data with schedule management information to identify visiting customers.

[0305] Device Role

[0306] The terminal (e.g., a smartphone) displays the identified customer information sent from the server. The terminal is designed so that the user can check customer information in real time, enabling the user to respond to the customer promptly and appropriately.

[0307] User Roles

[0308] The user can check the customer information identified through the terminal and provide personalized service based on the customer's preferences and interest in the company's services.

[0309] Hardware and software used

[0310] Speech recognition system: Converts voice data into text using the Python speech_recognition library.

[0311] AI technology: Uses machine learning models to analyze customer preferences and interest in your services.

[0312] Database: We use a SQL database to store customer profiles and scheduling information.

[0313] Device: A mobile device such as a smartphone or tablet.

[0314] Specific examples

[0315] For example, based on the schedule management information for a certain day, the server creates a list of customers scheduled to visit that day. Next, the server converts customer interaction voice data into text using a voice recognition system, and identifies the customers by comparing the converted text data with the schedule management information. The identified customer information is sent to the terminal, allowing the user to check it in real time.

[0316] Example prompt sentence:

[0317] "Convert an audio file to text. The path to the audio file is 'path / to / audio / file.wav'."

[0318] In this way, by linking the server, terminal, and user, it becomes possible to automatically identify customer information about customers who are planning to visit the store and provide personalized services based on the customer's preferences and interest.

[0319] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0320] Step 1:

[0321] The server obtains schedule management information. As input, it obtains a list of customers scheduled to visit the store that day from the database. As output, it generates a list of customers scheduled to visit the store. This list includes information such as customer ID, name, and scheduled visit time.

[0322] Step 2:

[0323] The server acquires customer response voice data. As input, the path of the voice file is specified. As output, voice data is acquired. This voice data is a record of conversations and inquiries with customers.

[0324] Step 3:

[0325] The server uses a speech recognition system to convert the speech data into text. It uses the acquired speech data as input. It generates text data as output. This text data represents the contents of the speech data as character information.

[0326] Step 4:

[0327] The server compares the converted text data with the schedule management information to identify customers. It uses the text data and a list of customers who are scheduled to visit as input. It generates information about identified customers as output. This information includes customer IDs, names, and scheduled visit times.

[0328] Step 5:

[0329] The server sends the identified customer information to the terminal. It uses the identified customer information as input and generates customer information that is sent to the terminal as output. This information is converted into a format for display on the terminal.

[0330] Step 6:

[0331] The terminal receives and displays the customer information sent from the server. As input, it uses the customer information sent from the server. As output, it displays the information in a format that can be viewed by the user. This allows the user to view the customer information in real time.

[0332] Step 7:

[0333] The user checks the customer information identified through the terminal and responds to the customer. The customer information displayed on the terminal is used as input. The output is to respond appropriately to the customer. This response includes providing personalized services based on the customer's preferences and level of interest in the company's services.

[0334] Example 3

[0335] Next, a description will be given of a third embodiment of the third embodiment. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0336] With conventional customer service systems, it was difficult to accurately grasp customer preferences and interests and provide personalized services. Furthermore, there was a lack of means to effectively utilize customer service voice data, making it impossible to respond quickly to customer needs. This made it difficult to improve customer satisfaction and service quality.

[0337] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[0338] In this invention, the server includes means for comprehensively analyzing customer profiles, schedule management information, customer response voice data, and customer preferences and interest in the company's services, means for identifying information about customers visiting the store today using the schedule management information and customer response voice data, means for converting the customer response voice data into text data using voice recognition software, means for analyzing customer preferences and interest in the company's services using AI based on features extracted from the customer profiles and voice data, and means for providing personalized services for each customer. This makes it possible to accurately grasp customer preferences and interest and quickly provide personalized services.

[0339] A "customer profile" is data that includes basic information about a customer, past purchasing history, preferences, and interest levels.

[0340] "Schedule management information" is data related to customer visit plans, reservation information, and schedules.

[0341] "Customer interaction voice data" is voice data of a conversation between a customer and a user.

[0342] "Speech recognition software" is software for converting voice data into text data.

[0343] "Features" are data extracted from voice data and customer profiles that indicate customer preferences and interests.

[0344] "AI" is an artificial intelligence technology that uses machine learning models and natural language processing tools to analyze data and make predictions and classifications.

[0345] "Personalized service" refers to a service that is provided individually based on a customer's preferences and interests.

[0346] "Visiting customer information" is information about customers who are planning to visit the store today.

[0347] This invention relates to a system that uses AI to comprehensively analyze customer profiles, schedule management information, and customer interaction voice data to analyze customer preferences and interest in the company's services. Specific embodiments of this system are described below.

[0348] Hardware and software used

[0349] Hardware: Servers, terminals (PCs, smartphones)

[0350] Software: speech recognition software (e.g., Google Cloud Speech-to-Text API), database management systems (DBMS), machine learning models (e.g., TENSORFLOW® or PyTorch), natural language processing (NLP) tools (e.g., spaCy or NLTK)

[0351] System Overview

[0352] 1. Collection of audio data

[0353] When a user interacts with a customer, voice data is collected through a microphone connected to the terminal.

[0354] The device transmits the collected voice data to the server in real time.

[0355] 2. Converting voice data to text

[0356] The server converts the received voice data into text data using voice recognition software.

[0357] 3. Obtaining customer profiles

[0358] The server uses a database management system (DBMS) to obtain profile data such as basic customer information and past purchasing history.

[0359] 4. Feature Extraction

[0360] The server uses natural language processing (NLP) tools to extract features from the text extracted from the voice data and the customer profile data.

[0361] 5. AI-powered analysis

[0362] The server uses a machine learning model to analyze customer preferences and interest in the company's services based on the extracted features.

[0363] 6. Providing Results

[0364] The server sends the analysis results to the terminal and makes personalized service proposals to the user.

[0365] Specific examples

[0366] Example 1: If customer A has purchased many health foods in the past, the AI ​​will determine that customer A is health-conscious and suggest new health foods and related services.

[0367] Prompt: "Analyze Customer A's interest in healthy foods based on their past purchase history and voice data."

[0368] Example 2: If Customer B frequently asks travel-related questions, the AI ​​determines that Customer B is interested in travel and suggests travel-related products and services.

[0369] Prompt: "Extract travel interests from customer B's voice data and suggest related services."

[0370] In this way, the system uses AI to analyze customer preferences and interests based on customer interaction voice data and customer profiles, and provides personalized services. The flow of the identification process in Example 3 will be described with reference to Figure 15.

[0371] Step 1:

[0372] Audio data collection

[0373] When a user interacts with a customer, voice data is collected through a microphone connected to the terminal.

[0374] Input: Audio of conversation with customer

[0375] Output: Collected audio data

[0376] What it does: When a user starts a conversation with a customer, the device's microphone captures the audio and streams that data to the server.

[0377] Step 2:

[0378] Converting audio data to text

[0379] The server converts the received voice data into text data using voice recognition software.

[0380] Input: Collected audio data

[0381] Output: Converted text data

[0382] Specific operation: The server inputs the voice data into speech recognition software (e.g., Google Cloud Speech-to-Text API) and obtains the converted text data.

[0383] Step 3:

[0384] Retrieving a customer profile

[0385] The server uses a database management system (DBMS) to obtain profile data such as basic customer information and past purchasing history.

[0386] Input: Customer ID

[0387] Output: Customer profile data

[0388] What happens: The server queries the database using the customer ID as a key to retrieve relevant customer profile data.

[0389] Step 4:

[0390] Feature Extraction

[0391] The server uses natural language processing (NLP) tools to extract features from the text extracted from the voice data and the customer profile data.

[0392] Input: Text data, customer profile data

[0393] Output: Extracted feature data

[0394] What it does: The server feeds the text data into an NLP tool (e.g., spaCy or NLTK) for keyword extraction and sentiment analysis, and also extracts relevant features from customer profile data.

[0395] Step 5:

[0396] AI-powered analysis

[0397] The server uses a machine learning model to analyze customer preferences and interest in the company's services based on the extracted features.

[0398] Input: Extracted feature data

[0399] Output: Analysis results

[0400] How it works: The server inputs the feature data into a machine learning model (e.g., TensorFlow or PyTorch) to obtain prediction results. The model learns from past data and generates personalized service proposals for each customer.

[0401] Step 6:

[0402] Providing results

[0403] The server sends the analysis results to the terminal and makes personalized service proposals to the user.

[0404] Input: Analysis results

[0405] Output: Personalized service offers

[0406] Specific operation: The server sends the analysis results in JSON format to the terminal, which then displays them to the user. For example, it can suggest new products or services based on products that a specific customer has shown interest in in the past.

[0407] (Application example 3)

[0408] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0409] With conventional customer service systems, it was difficult to accurately grasp customer preferences and interests and provide personalized services. Furthermore, there was a lack of means to effectively utilize customer voice data, which meant that product proposals that met customer needs could not be made. This created the problem of being unable to improve customer satisfaction or increase sales.

[0410] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[0411] In this invention, the server includes means for comprehensively analyzing customer profiles, schedule management information, customer response voice data, and customer preferences and interest in the company's services, means for identifying information about customers visiting the store today using the schedule management information and customer response voice data, means for analyzing customer preferences and interest in the company's services using AI, means for analyzing customer preferences and interest based on the customer's voice data and profile data and proposing optimal products, and means for converting voice data into text using a voice recognition system and vectorizing and analyzing the text data. This makes it possible to accurately grasp customer preferences and interest and propose personalized products.

[0412] A "customer profile" is data that includes a customer's basic information, purchasing history, preferences, etc.

[0413] "Schedule management information" is data related to schedules such as customer visit plans and reservation information.

[0414] "Customer response voice data" is voice recording data relating to conversations with customers and inquiries.

[0415] "Customer preferences" refers to information about the types and characteristics of products and services that customers prefer.

[0416] "Interest in our company's services" is an indicator that shows how interested customers are in the products and services that our company offers.

[0417] "Comprehensive analysis means" refers to a method of integrating and analyzing multiple data sets to understand customer preferences and interests.

[0418] The "means for identifying information on customers visiting the store today" is a method for identifying customers visiting the store that day using schedule management information and customer response voice data.

[0419] "AI analysis" refers to a method of analyzing data using artificial intelligence to evaluate customer preferences and interests.

[0420] "Means for proposing optimal products" refers to a method for selecting and proposing the most suitable products to customers based on their preferences and interests.

[0421] A "voice recognition system" is a technology for converting voice data into text data.

[0422] "Means for vectorizing and analyzing text data" refers to a method for converting text data into numerical vectors and analyzing the data using those vectors.

[0423] The system for implementing this invention includes a means for comprehensively analyzing customer profiles, schedule management information, customer response voice data, and customer preferences and interest in the company's services. The server integrates and analyzes this data to understand customer preferences and interest. Furthermore, the system includes a means for identifying customer information for the day using schedule management information and customer response voice data. This allows the system to identify customers visiting the store that day and provide appropriate service.

[0424] The server uses AI to analyze customer preferences and interest in the company's services based on their voice data and profile data, and then proposes optimal products. Voice data is converted into text using a voice recognition system, and the text data is vectorized and analyzed. This process enables product proposals tailored to customer needs.

[0425] Specifically, the server processes the data using software such as Python, pandas, scikit-learn, and speech_recognition. Customer profile data is read from a CSV file, and customer-interacted voice data is converted to text using a speech recognition library. The text data is then vectorized using TF-IDF and analyzed for customer preferences and interests using the Nearest Neighbors algorithm.

[0426] For example, if a customer says, "I'm looking for a new smartphone," the server analyzes the voice data and compares it with their past purchase and browsing history to suggest the smartphone that's best suited to them. In this way, personalized product suggestions tailored to the customer's needs are realized.

[0427] An example of a prompt to input to a generative AI model is as follows:

[0428] Create a Python program that analyzes customer preferences and interests based on their voice data and profile data, and suggests the most suitable products. The customer's voice data will be converted into text using a voice recognition library, and the text data will be converted into TF-IDF vectors for analysis.

[0429] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[0430] Step 1:

[0431] The server reads customer profile data from a CSV file. The input is a CSV file of customer profile data, and the output is customer profile data in data frame format. Specifically, the server uses the Python pandas library to read the CSV file and convert it to a data frame.

[0432] Step 2:

[0433] The server converts customer-facing voice data into text using a speech recognition system. The input is an audio file, and the output is text data. Specifically, the server reads the audio file using the speech_recognition library and converts it into text using Google's speech recognition API.

[0434] Step 3:

[0435] The server combines customer profile data and audio data. The input is customer profile data and text data in data frame format, and the output is the combined data frame. Specifically, the server uses the pandas library to add audio data to customer profile data.

[0436] Step 4:

[0437] The server performs TF-IDF vectorization on the text data. The input is the combined data frame text data, and the output is a TF-IDF vector. Specifically, it uses scikit-learn's TfidfVectorizer to convert the text data into a numeric vector.

[0438] Step 5:

[0439] The server analyzes customer preferences and interests using the NearestNeighbors algorithm based on the vectorized data. The input is a TF-IDF vector, and the output is an analysis result based on the customer preferences and interests. Specifically, the data is analyzed using NearestNeighbors from scikit-learn.

[0440] Step 6:

[0441] The server proposes optimal products based on the customer's preferences and interests. The input is the analysis results, and the output is a list of products to be proposed to the customer. Specifically, the server selects the most suitable products for the customer based on the analysis results and generates a list of products to be proposed.

[0442] Step 7:

[0443] The terminal displays the product list received from the server to the user. The input is the product list sent from the server, and the output is the product list displayed on the user's smartphone screen. Specifically, the product list is visually presented to the user using a smartphone application.

[0444] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0445] "Example 1"

[0446] In one embodiment of the present invention, AI analysis uses an emotion engine that recognizes user emotions to analyze the emotional state of a customer. Specifically, the customer's emotional state is estimated from customer interaction voice data, and the estimation results are used in AI analysis. For example, if a customer is pleased with a service, that information can be used to improve the service or propose new services.

[0447] "Example 2"

[0448] Furthermore, AI analysis will be used to provide personalized services based on the customer's emotional state. Specifically, services will be provided that take into account the customer's emotional state. For example, if a customer is dissatisfied, that information will be used to improve the service. Also, if a customer is happy, that information will be used to suggest new services.

[0449] "Example 3"

[0450] Furthermore, the emotion engine estimates the emotional state of the customer from the customer-interacting voice data acquired by the voice recognition system.

[0451] Specifically, the emotional state of the customer is estimated from the voice data acquired by the voice recognition system, for example, from the tone, speed, and volume of the voice.

[0452] The processing flow of each embodiment will be described below.

[0453] "Example 1"

[0454] Step 1: In AI analysis, an emotion engine that recognizes user emotions is used to analyze the customer's emotional state.

[0455] Step 2: Estimate the customer's emotional state from customer interaction voice data and use the estimation results for AI analysis.

[0456] Step 3: For example, if a customer is pleased with a service, use that information to improve the service or propose new services.

[0457] "Example 2"

[0458] Step 1: Use AI analysis to provide personalized service based on the customer's emotional state.

[0459] Step 2: Specifically, provide services that take into account the customer's emotional state.

[0460] Step 3: For example, if a customer is dissatisfied, improve the service based on that information. Also, if a customer is happy, propose a new service based on that information. "Example 3"

[0461] Step 1: The emotion engine estimates the emotional state of the customer from the customer-facing voice data acquired by the voice recognition system.

[0462] Step 2: Specifically, the customer's emotional state is estimated from the voice data acquired by the voice recognition system.

[0463] Step 3: Infer the customer's emotional state, for example, from the tone, speed, and volume of the voice.

[0464] Example 1

[0465] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0466] With conventional customer service systems, it was difficult to accurately grasp customers' emotional states and preferences, resulting in a lack of concrete data to improve service quality. Furthermore, it was difficult to respond to customers in real time, limiting the improvement of customer satisfaction. This made it difficult to respond to customer needs quickly, potentially resulting in a loss of competitiveness.

[0467] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0468] In this invention, the server includes means for comprehensively analyzing customer profiles, schedule management information, customer response voice data, and customer preferences and interest in the company's services, means for identifying information about customers visiting the store today using the schedule management information and customer response voice data, means for analyzing customer preferences and interest in the company's services using artificial intelligence, means for acquiring voice data in real time from a voice input device installed in the store, means for converting the acquired voice data into text data using voice recognition technology, means for analyzing the emotional state of the customer based on the converted text data, and means for improving services or proposing new services based on the analysis of the emotional state. This makes it possible to grasp the emotional state and preferences of customers in real time and provide services quickly and accurately.

[0469] A "customer profile" is data that includes basic information about a customer, past purchase history, preferences, etc.

[0470] "Schedule management information" is data for managing customer visit schedules and reservation information.

[0471] "Customer interaction voice data" is voice data including conversations between customers and staff.

[0472] "Preferences" refer to the preferences and tendencies that customers have toward particular products or services.

[0473] "Interest in our company's services" refers to the degree of interest or concern that customers have in the services provided by our company.

[0474] "Artificial intelligence" is a technology that allows computers to learn and reason by imitating human intelligence.

[0475] An "audio input device" is a device for converting audio into digital data.

[0476] "Speech recognition technology" is a technology that converts voice data into text data.

[0477] "Text data" is character information converted using voice recognition technology.

[0478] "Emotional state" refers to the type and intensity of emotions a customer has.

[0479] "Improvement of services" refers to improving the content of services in order to increase customer satisfaction.

[0480] "Proposing new services" refers to providing new services in response to customer needs.

[0481] MODE FOR CARRYING OUT THE INVENTION

[0482] The present invention relates to a system for acquiring customer interaction voice data and analyzing the emotional state and preferences of the customer. Specific embodiments of this system will be described below.

[0483] Acquiring audio data

[0484] The server receives real-time voice data from a voice input device (microphone) installed in the store. This voice input device captures all voices, including conversations between customers and staff. For example, when a customer says, "Tell me about this product," the voice is sent to the server via the microphone.

[0485] Converting audio data to text

[0486] The server converts the acquired voice data into text data using voice recognition technology (for example, Google Cloud Speech-to-Text API). This API provides highly accurate voice recognition technology and converts voice into text quickly and accurately. For example, voice data such as "Tell me about this product" is converted into text data such as "Tell me about this product."

[0487] Emotional state analysis

[0488] The server inputs the converted text data into an emotion engine (e.g., IBM Watson (registered trademark) Tone Analyzer) to analyze the customer's emotional state. This emotion engine infers the customer's emotion (joy, anger, sadness, etc.) from the text data. For example, from the text data "Tell me about this product," it infers the customer's interest.

[0489] Use of analysis results

[0490] The server improves services and proposes new services based on the estimated emotional state obtained from the emotion engine. For example, if a customer says, "This service is really great!", the server uses that information to develop a strategy for providing similar services to other customers.

[0491] Specific examples

[0492] Example 1: Customer Delight Analysis

[0493] 1. The server receives voice data from the in-store microphone saying, "This service is truly amazing!"

[0494] 2. The server converts this voice data into text data using the Google Cloud Speech-to-Text API. The converted text data becomes "This service is really great!"

[0495] 3. The server inputs the converted text data into the IBM Watson Tone Analyzer and estimates that the customer is feeling "joy." The emotion engine analyzes emotions based on the context of the text data and word choice.

[0496] 4. The server uses this information to develop strategies for providing similar service to other customers, such as offering a special bonus to the customer the next time they visit.

[0497] Prompt Sentence Examples

[0498] "Please explain the system that converts voice data acquired from microphones in stores into text data in real time and analyzes the emotional state of customers. Please also specify the names of any specific hardware or software."

[0499] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0500] Step 1:

[0501] The server acquires voice data in real time from voice input devices (microphones) installed in the store. Specifically, it captures all voice, including conversations between customers and staff. For example, when a customer says, "Tell me about this product," that voice is sent to the server via the microphone. The input is voice data, and the output is voice data stored on the server.

[0502] Step 2:

[0503] The server converts the acquired voice data into text data using voice recognition technology (for example, Google Cloud Speech-to-Text API). Specifically, the voice data is input into the API, and the voice recognition technology quickly and accurately converts the voice into text. For example, voice data saying "Tell me about this product" is converted into text data saying "Tell me about this product." The input is voice data, and the output is text data.

[0504] Step 3:

[0505] The server inputs the converted text data into an emotion engine (for example, IBM Watson Tone Analyzer) to analyze the customer's emotional state. Specifically, the text data is input into the emotion engine, which infers the customer's emotion (joy, anger, sadness, etc.) from the context of the text data and the choice of words. For example, from the text data "Tell me about this product," it infers the customer's interest. The input is text data, and the output is the analysis result of the emotional state.

[0506] Step 4:

[0507] The server improves services and proposes new services based on the estimated emotional state obtained from the emotion engine. Specifically, it considers measures to improve customer satisfaction based on the results of the emotional state analysis. For example, if a customer says, "This service is truly wonderful!", it uses that information to develop a strategy for providing similar services to other customers. The input is the results of the emotional state analysis, and the output is specific measures for improving the service.

[0508] (Application example 1)

[0509] Next, a description will be given of Application Example 1 of Embodiment Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0510] Conventional customer service systems have difficulty understanding a customer's emotional state in real time, making it difficult to respond appropriately to immediately improve customer satisfaction. Furthermore, there is a lack of means to improve services or propose new services based on the customer's emotional state. This can lead to a decline in the quality of the customer experience, leading to a decrease in customer repeat rates and satisfaction.

[0511] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0512] In this invention, the server includes means for comprehensively analyzing customer profiles, schedule management information, customer interaction voice data, and customer preferences and interest in the company's services, means for identifying information about customers visiting the store today using the schedule management information and customer interaction voice data, means for analyzing customer preferences and interest in the company's services using AI, means for converting customer interaction voice data into text data in real time, means for analyzing the customer's emotional state based on the converted text data, and means for improving services or proposing new services based on the customer's emotional state. This makes it possible to grasp the customer's emotional state in real time and take appropriate action.

[0513] A "customer profile" is data that includes basic information about a customer, past purchase history, preferences, etc.

[0514] "Schedule management information" is data related to time, such as customer visit schedules and reservation information.

[0515] "Customer interaction voice data" refers to voice data of conversations between customers and staff.

[0516] "Customer preferences" refers to information about products and services that customers prefer.

[0517] "Interest in our company's services" is an indicator that shows how interested customers are in the services provided by our company.

[0518] A "voice recognition system" is a technology that converts voice data into text data.

[0519] "AI analysis" is a technique that uses artificial intelligence to analyze data and find specific patterns and trends.

[0520] "Real-time" means that data is processed as it is generated.

[0521] "Text data" is data expressed as character information.

[0522] An "emotional state" is a state that indicates the emotions such as joy, anger, sadness, etc. that a customer is feeling.

[0523] "Improvement of Services" means making changes or additions to an existing Service to make it better.

[0524] "New service proposals" means providing new services based on the customer's needs and emotional state.

[0525] The following system configuration will be described as an embodiment of the present invention.

[0526] System Configuration

[0527] The system consists of a server, terminals, and users. The server has the function of comprehensively analyzing customer profiles, schedule management information, customer response voice data, and customer preferences and interest in the company's services. Terminals are devices such as smartphones and tablets used by store staff, and users are customers who visit the store.

[0528] Program processing

[0529] The server performs the following process.

[0530] 1. Acquiring and converting audio data

[0531] A microphone connected to the terminal is used to capture customer-facing voice data in real time. The captured voice data is converted into text data using a voice recognition system (e.g., the speech_recognition library).

[0532] 2. Emotional state analysis

[0533] The converted text data is sent to a server where a generative AI model (e.g., a sentiment analysis model from the transformers library) is used to analyze the customer's emotional state, which can include joy, anger, sadness, etc.

[0534] 3. Service Improvements and Suggestions

[0535] Based on the analysis results, the server evaluates the customer's emotional state and proposes improvements or new services. For example, if the customer is feeling happy, the server will use that information to make suggestions to further increase satisfaction.

[0536] Hardware and software used

[0537] Hardware: Microphones, smartphones, tablets

[0538] Software: speech_recognition library, transformers library

[0539] Specific examples

[0540] For example, if a customer says, "I really like this product!", the device's microphone picks up the voice data and the voice recognition system converts it into text data. The converted text data is sent to the server, where the sentiment analysis model recognizes it as "delighted." This allows the server to understand in real time that the customer is satisfied and suggest actions to the store staff to provide even better service.

[0541] Prompt Sentence Examples

[0542] "Capture customer voice data in real time and convert it into text data. Then, input the text data into a sentiment analysis model to analyze the customer's emotional state. For example, if the customer says, 'I really like this product!', recognize the emotional state as 'Delighted.'"

[0543] In this way, the system can understand the customer's emotional state in real time and respond appropriately.

[0544] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0545] Step 1:

[0546] The terminal uses a microphone to acquire customer-facing voice data in real time. The input is the customer's voice, and the output is voice data. The terminal captures the customer's voice with the microphone and saves it as digital voice data.

[0547] Step 2:

[0548] The device converts the acquired voice data into text data using a voice recognition system (for example, the speech_recognition library). The input is voice data and the output is text data. The device starts the voice recognition system, analyzes the voice data, and generates corresponding text data.

[0549] Step 3:

[0550] The terminal transmits the converted text data to the server. The input is the text data, and the output is data transmission to the server. The terminal transmits the text data to the server via the network.

[0551] Step 4:

[0552] The server inputs the received text data into a generative AI model (for example, a sentiment analysis model from the Transformers library) to analyze the customer's emotional state. The input is text data, and the output is the analysis result of the emotional state. The server passes the text data to the sentiment analysis model, which analyzes the sentiment of the text and returns the result.

[0553] Step 5:

[0554] The server improves services or proposes new services based on the results of the emotional state analysis. The input is the analysis result of the emotional state, and the output is proposals for service improvements or new services. The server evaluates the analysis results and generates optimal proposals for service improvements or new services according to the customer's emotional state.

[0555] Step 6:

[0556] The server sends the generated service improvement proposals and new service proposals to the terminal. The input is the service improvement proposals and new service proposals, and the output is data transmission to the terminal. The server sends the proposals to the terminal via the network.

[0557] Step 7:

[0558] The terminal notifies the store staff of the received proposals for service improvements or new services. The input is the proposal for service improvements or new services, and the output is a notification to the staff. The terminal uses its notification function to display the proposal to the staff and encourage them to take appropriate action.

[0559] In this way, the system can understand the customer's emotional state in real time and respond appropriately.

[0560] Example 2

[0561] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0562] Conventional customer management systems have difficulty effectively integrating customer visit schedule information and customer interaction voice data to provide personalized service based on the customer's emotional state. Furthermore, there is a lack of means to accurately grasp the customer's emotional state and provide appropriate service based on that. This has prevented companies from fully improving customer satisfaction and service quality.

[0563] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0564] In this invention, the server includes means for comprehensively analyzing customer profiles, schedule management information, customer interaction voice data, and customer preferences and interest in the company's services, means for creating a list of customers scheduled to visit the store that day using the schedule management information, means for converting the customer interaction voice data into text data using voice recognition technology, means for comparing the created customer list with the converted text data to identify which customer corresponds to which voice data, means for using emotion analysis technology to analyze the emotional state of the customer from the voice data, and means for generating instructions for providing personalized service based on the identified emotional state. This makes it possible to integrate customer visit schedule information and customer interaction voice data and provide personalized service based on the customer's emotional state.

[0565] A "customer profile" is data that includes a customer's basic information, preferences, purchasing history, etc.

[0566] "Schedule management information" is data for managing customer visit schedules and reservation information.

[0567] "Customer response voice data" refers to voice data recorded during conversations and responses with customers.

[0568] "Customer preferences" refers to data that indicates the types and characteristics of products and services that customers prefer.

[0569] "Interest in our company's services" is data that indicates how interested customers are in the services provided by our company.

[0570] "Speech recognition technology" is a technology for converting voice data into text data.

[0571] "Emotion analysis technology" is a technology for analyzing a customer's emotional state from voice data and text data.

[0572] "Personalized service" refers to customized services provided based on the customer's individual preferences and emotional state.

[0573] The "means for generating instructions" refers to a means for instructing specific actions or responses based on the analysis results.

[0574] MODE FOR CARRYING OUT THE INVENTION

[0575] This invention is a system that comprehensively analyzes customer profiles, schedule management information, customer interaction voice data, and customer preferences and interest in the company's services, and provides personalized services based on the customer's emotional state. Specific embodiments of this system are described below.

[0576] Hardware and software used

[0577] The server uses the following hardware and software:

[0578] Scheduling software: Google Calendar API

[0579] Speech recognition software: Google Cloud Speech-to-Text API

[0580] Sentiment analysis software: IBM Watson Tone Analyzer

[0581] Database Management System: A typical relational database (e.g., MySQL)

[0582] Data processing and calculation

[0583] 1. The server uses the Google Calendar API to get a list of customers who are scheduled to visit the store that day, and saves the list in a database.

[0584] 2. The server retrieves the customer-facing voice data from Google Cloud Storage and converts it to text using the Google Cloud Speech-to-Text API, which is then stored in the database.

[0585] 3. The server compares the text of the voice data with the customer list stored in the database to identify which customer corresponds to which voice data.

[0586] 4. The server uses IBM Watson Tone Analyzer to analyze the customer's emotional state from the voice data, and the analysis results are stored in a database.

[0587] 5. The server generates instructions for providing personalized services based on the identified emotional state.

[0588] Specific examples

[0589] For example, the server uses the Google Calendar API to obtain a list of customers scheduled to visit the store on October 10, 2023. Next, it obtains customer interaction voice data for October 10, 2023 from Google Cloud Storage and converts the voice data into text using the Google Cloud Speech-to-Text API. The converted text data is saved in a database and compared with the customer list to identify that "Customer A corresponds to voice data 1."

[0590] Furthermore, the server analyzes "voice data 1" using IBM Watson Tone Analyzer and determines that "Customer A is dissatisfied." Finally, based on the identified emotional state, it generates "specific instructions for improving service," and the device follows the instructions and provides "a special discount coupon" to Customer A.

[0591] Prompt Sentence Examples

[0592] Example prompts to input to a generative AI model:

[0593] "Get a list of customers scheduled to visit the store on October 10, 2023 from the Google Calendar API, convert the customer-facing voice data into text using the Google Cloud Speech-to-Text API, and identify which customer corresponds to which voice data."

[0594] "Use IBM Watson Tone Analyzer to analyze customer voice data and generate instructions to provide personalized service based on the customer's emotional state."

[0595] In this way, it is possible to integrate customer visit schedule information with customer interaction voice data and provide personalized service based on the customer's emotional state.

[0596] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0597] Step 1:

[0598] The server uses schedule management software (e.g., Google Calendar API) to obtain a list of customers who are scheduled to visit the store on that day.

[0599] Input: Schedule data from the Google Calendar API

[0600] Data processing: Analyze the schedule data obtained from the API and extract a list of customers scheduled to visit the store.

[0601] Output: List of customers who plan to visit the store

[0602] Specific operation: The server accesses the Google Calendar API and obtains the "list of customers scheduled to visit the store on October 10, 2023." The obtained list is saved in the database.

[0603] Step 2:

[0604] The server converts customer-facing voice data into text data using voice recognition software (e.g., Google Cloud Speech-to-Text API).

[0605] Input: Customer-facing voice data from Google Cloud Storage

[0606] Data processing: Speech recognition processing to convert voice data into text data

[0607] Output: Customer interaction voice data converted into text data

[0608] Specific operation: The server retrieves the "customer interaction voice data for October 10, 2023" from Google Cloud Storage and converts the voice data to text using the Google Cloud Speech-to-Text API. The converted text data is stored in a database.

[0609] Step 3:

[0610] The server compares the text of the voice data with a customer list stored in a database to identify which customer corresponds to which voice data.

[0611] Input: Customer list and text data stored in a database

[0612] Data processing: Comparison of customer list and text data

[0613] Output: Correspondence between customer and voice data

[0614] Specific operation: The server retrieves the list of "Customer A, Customer B, Customer C" and the text "Audio data 1: 'Hello, this is Customer A.'" from the database, analyzes the text data, and determines that "Customer A corresponds to audio data 1."

[0615] Step 4:

[0616] The server uses sentiment analysis software (e.g., IBM Watson Tone Analyzer) to analyze the customer's emotional state from the voice data.

[0617] Input: Customer interaction voice data converted to text data

[0618] Data processing: Sentiment analysis processing

[0619] Output: Customer's emotional state

[0620] Specific operation: The server analyzes "Voice Data 1" using IBM Watson Tone Analyzer and identifies that "Customer A is dissatisfied." The analysis results are stored in the database.

[0621] Step 5:

[0622] The server generates instructions for providing personalized services based on the identified emotional state.

[0623] Input: Customer's emotional state

[0624] Data processing: Instruction generation for personalized service delivery

[0625] Output: Specific instructions for providing the service

[0626] Specific operation: The server generates "specific instructions for improving the service" based on the information that "Customer A is dissatisfied with." The terminal follows the instructions and provides Customer A with a "special discount coupon."

[0627] (Application example 2)

[0628] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0629] While conventional customer service systems can grasp customer visit information, preferences, and service interest, it is difficult to analyze the customer's emotional state in real time and provide personalized service based on that. This has led to the issue of not being able to sufficiently improve customer satisfaction and service quality.

[0630] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0631] In this invention, the server includes means for comprehensively analyzing customer profiles, schedule management information, customer interaction voice data, and customer preferences and interest in the company's services, means for identifying information about customers visiting the store today using the schedule management information and customer interaction voice data, means for analyzing customer preferences and interest in the company's services using AI, and means for analyzing the customer's emotional state and providing personalized services, thereby making it possible to provide personalized services based on the customer's emotional state.

[0632] A "customer profile" is a collection of data that includes a customer's basic information, preferences, past purchase history, etc.

[0633] "Schedule management information" is a collection of data including customer visit schedules and reservation information.

[0634] "Customer response voice data" refers to voice recordings of conversations and inquiries with customers.

[0635] "Customer preferences" refers to information about products and services that customers prefer.

[0636] The "degree of interest in our company's services" is information indicating the degree of interest that customers have in the services provided by our company.

[0637] "AI analytics" is the process of using artificial intelligence to analyze data and extract specific patterns and trends.

[0638] An "emotional state" is a psychological state such as joy, dissatisfaction, or excitement felt by a customer.

[0639] "Personalized service" is service that is customized based on the preferences and emotional state of each individual customer.

[0640] A system for implementing this invention includes means for comprehensively analyzing customer profiles, schedule management information, customer response voice data, and customer preferences and interest in the company's services, means for identifying information about customers visiting the store today using the schedule management information and customer response voice data, means for analyzing customer preferences and interest in the company's services using AI, and means for analyzing the customer's emotional state and providing personalized services.

[0641] System Configuration

[0642] The server uses the following hardware and software:

[0643] Hardware: A server with high-performance CPU, memory, and storage

[0644] Software: Python, SpeechRecognition library, Transformers library

[0645] Data processing and calculation

[0646] The server first obtains a list of customers scheduled to visit that day from the schedule management information. Next, the customer interaction voice data is converted into text using speech recognition technology. This process uses the SpeechRecognition library. The converted text data is then analyzed using AI with the Transformers library to determine the customer's emotional state.

[0647] Specific examples

[0648] For example, if customer A is planning to visit a store and voice data indicates dissatisfaction, the server will use that information to suggest service improvements. Similarly, if customer B is planning to visit a store and voice data indicates happiness, the server will suggest a new service.

[0649] Prompt Sentence Examples

[0650] Analyze customer interaction voice data to identify the emotional state of your customers. Based on their emotional state, offer personalized service.

[0651] In this way, the server can provide personalized service based on the emotional state of the customer.

[0652] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0653] Step 1:

[0654] The server acquires a list of customers scheduled to visit the store on that day from the schedule management information.

[0655] Input: Schedule management information

[0656] Output: List of customers scheduled to visit on the day

[0657] Specific operation: The server queries the database for the day's schedule information and generates a list of customers who are scheduled to visit the store.

[0658] Step 2:

[0659] The server converts the customer-interacting voice data into text using voice recognition technology.

[0660] Input: Customer response voice data

[0661] Output: Text data

[0662] What happens: The server loads the audio file and converts the audio data to text using the SpeechRecognition library.

[0663] Step 3:

[0664] The server then uses AI analysis of the converted text data to analyze the customer's emotional state.

[0665] Input: Text data

[0666] Output: Emotional state

[0667] How it works: The server uses the Transformers library to input text data into a sentiment analysis model to analyze the emotional state.

[0668] Step 4:

[0669] The server provides personalized services based on the analyzed emotional state.

[0670] Input: Emotional state, customer profile, preference information, interest in our company's services

[0671] Output: Personalized service offers

[0672] Specific operation: The server takes into account the customer profile, preference information, and level of interest in the company's services, and generates suggestions for service improvements and new services according to the customer's emotional state.

[0673] Step 5:

[0674] The server transmits the generated service proposal to the terminal and notifies the user.

[0675] Input: Personalized service offers

[0676] Output: User notification

[0677] Specific operation: The server sends the generated service proposal to the terminal and calls an API to notify the user.

[0678] Example 3

[0679] Next, a description will be given of a third embodiment of the third embodiment. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0680] With conventional customer service systems, it was difficult to accurately grasp customer preferences and their level of interest in the company's services, making it difficult to provide personalized services. Furthermore, there was a lack of means to grasp the customer's emotional state in real time, making it difficult to improve customer satisfaction.

[0681] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[0682] In this invention, the server includes means for comprehensively analyzing customer profiles, schedule management information, customer interaction voice data, and customer preferences and interest in the company's services, means for identifying information about customers visiting the store today using the schedule management information and customer interaction voice data, means for analyzing customer preferences and interest in the company's services using artificial intelligence, means for converting voice data into text data using a voice recognition system, means for estimating the customer's emotional state from the voice data using an emotion engine, means for inputting features extracted from the customer profile information and voice data into an artificial intelligence model to analyze the customer's preferences and interest in the company's services, and means for providing personalized services for each customer based on the analysis results.This makes it possible to accurately grasp customer preferences and interest and estimate the emotional state in real time.

[0683] A "customer profile" is data that includes personal information such as a customer's age, gender, and past purchasing history.

[0684] "Schedule management information" is data related to schedules, such as customer visit schedules and reservation information.

[0685] "Customer response voice data" refers to voice data obtained from call center call records and customer interaction logs.

[0686] "Preferences" are the preferences and interests that customers have for particular products and services.

[0687] "Interest in our company's services" refers to the degree of interest customers have in the products and services offered by our company.

[0688] "Artificial intelligence" is a technology that uses machine learning and data analysis to mimic human intelligence.

[0689] A "voice recognition system" is a technology that converts voice data into text data.

[0690] "Emotion engine" is a technology that estimates a customer's emotional state from voice data.

[0691] An "artificial intelligence model" is a model that uses machine learning algorithms to analyze data and make predictions and classifications.

[0692] "Personalized services" are customized services provided based on a customer's individual preferences and interests.

[0693] This invention is a system that uses artificial intelligence to analyze customer preferences and interest in the company's services, and provides personalized services. Specific embodiments of this system are described below.

[0694] Hardware and software used

[0695] Hardware: Servers, devices (PCs, smartphones, etc.)

[0696] Software: speech recognition systems (e.g., Google Speech-to-Text API), emotion engines (e.g., IBM Watson Tone Analyzer), artificial intelligence models (e.g., TensorFlow, PyTorch)

[0697] Data collection

[0698] The server collects customer interaction voice data from call center call records and customer interaction logs. For example, when a call ends, the voice data is automatically saved on the server.

[0699] The terminal sends the customer profile information entered by the user to the server, which includes the customer's age, gender, past purchase history, etc.

[0700] Audio data processing

[0701] The server sends the collected voice data to a voice recognition system (e.g., Google Speech-to-Text API) and converts it into text data. For example, you can upload the voice file to the API and save the returned text data.

[0702] The server inputs the converted text data into an emotion engine (e.g., IBM Watson Tone Analyzer) to estimate the customer's emotional state. For example, the server sends the text data to the emotion engine and stores the returned emotion analysis results.

[0703] Data analysis

[0704] The server inputs customer profile information and features extracted from the voice data into an AI model (e.g., TensorFlow, PyTorch) to analyze customer preferences and interest in the company's services. For example, the profile information and sentiment analysis results can be input into an AI model to predict customer preferences and interest.

[0705] AI models learn what products and services customers are interested in and what purchasing behaviors they tend to exhibit, for example by training the model using past data and then making predictions on new data.

[0706] Providing results

[0707] The server then uses the analysis results to provide personalized services for each customer, such as generating a list of recommended products and notifying the user.

[0708] The user checks the personalized service provided by the server and takes appropriate action for the customer, for example, presenting a special offer to the customer.

[0709] Specific examples

[0710] For example, suppose a customer makes an inquiry to a call center. The voice data from that conversation is collected and converted into text data using a voice recognition system. Next, the customer's emotional state is estimated using an emotion engine, and the customer's preferences and interest levels are analyzed using an artificial intelligence model. Finally, products that may be of interest to the customer are recommended.

[0711] Prompt Sentence Examples

[0712] "Build an artificial intelligence model that analyzes customer preferences and interest in your services based on customer interaction voice data and customer profile information. Also, use an emotion engine that infers the customer's emotional state from the voice data."

[0713] The above is a specific embodiment for carrying out the present invention. The flow of the identification process in the third embodiment will be described with reference to FIG.

[0714] Step 1:

[0715] Data collection

[0716] The server collects customer interaction voice data from call center call records and customer interaction logs. For example, when a call ends, the voice data is automatically saved on the server.

[0717] Input: Call center call records, customer interaction logs

[0718] Output: Customer interaction voice data (audio file)

[0719] Specific operation: The server automatically collects voice data at the end of the call and stores it in a database.

[0720] Step 2:

[0721] Entering Customer Profile Information

[0722] The terminal sends the customer profile information entered by the user to the server, which includes the customer's age, gender, past purchase history, etc.

[0723] Input: Customer profile information entered by the user

[0724] Output: Customer profile information sent to the server

[0725] Specific operation: When the terminal enters data into the input form, it immediately sends it to the server.

[0726] Step 3:

[0727] Converting audio data to text

[0728] The server sends the collected voice data to a voice recognition system (e.g., Google Speech-to-Text API) and converts it into text data.

[0729] Input: Customer response voice data (audio file)

[0730] Output: Text data

[0731] Specific operation: The server uploads the audio file to the API and stores the returned text data in the database.

[0732] Step 4:

[0733] Emotional state estimation

[0734] The server inputs the converted text data into an emotion engine (e.g., IBM Watson Tone Analyzer) to estimate the customer's emotional state.

[0735] Input: Text data

[0736] Output: Sentiment analysis results

[0737] Specific operation: The server sends text data to the emotion engine and stores the returned emotion analysis results in a database.

[0738] Step 5:

[0739] Data analysis

[0740] The server inputs customer profile information and features extracted from voice data into an artificial intelligence model (e.g., TensorFlow, PyTorch) to analyze customer preferences and interest in the company's services.

[0741] Input: Customer profile information, sentiment analysis results

[0742] Output: Analysis of preferences and interest levels

[0743] Specific operation: The server inputs profile information and sentiment analysis results into an artificial intelligence model to predict customer preferences and interests.

[0744] Step 6:

[0745] Providing results

[0746] Based on the analysis results, the server provides personalized services for each customer.

[0747] Input: Analysis results of preferences and interests

[0748] Output: Providing personalized service

[0749] Specific operation: The server generates a list to recommend specific products and notifies the user.

[0750] Step 7:

[0751] User response

[0752] The user checks the personalized service provided by the server and responds appropriately to the customer.

[0753] Input: Personalized service offerings

[0754] Output: Customer response

[0755] Specific Action: The user presents a special offer to a customer.

[0756] (Application example 3)

[0757] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0758] With conventional customer service systems, it was difficult to fully understand customer preferences and emotional states, making it difficult to provide personalized services. Furthermore, there was a lack of means to effectively utilize customer voice data, limiting the improvement of customer satisfaction. This meant that customers' purchasing motivation could not be maximized, and sales did not increase.

[0759] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[0760] In this invention, the server includes means for comprehensively analyzing customer profiles, schedule management information, customer interaction voice data, and customer preferences and interest in the company's services, means for identifying information about customers visiting the store today using the schedule management information and customer interaction voice data, means for analyzing customer preferences and interest in the company's services using AI, a voice recognition system for converting customer voice input into text, an emotion engine for estimating the customer's emotional state from the voice data, and means for personalized product recommendations based on the customer's preferences and emotional state. This makes it possible to grasp the customer's preferences and emotional state in real time and recommend optimal products.

[0761] A "customer profile" is data that includes basic information about a customer, their past purchasing history, preferences, etc.

[0762] "Schedule management information" is data related to time, such as customer visit schedules and reservation information.

[0763] "Customer response voice data" refers to voice recordings of conversations and inquiries with customers.

[0764] "Preferences" are the preferences and interests that customers have for particular products and services.

[0765] "Interest in our company's services" refers to the degree of interest that customers have in the products and services that our company offers.

[0766] A "voice recognition system" is a technology that converts voice data into text.

[0767] "Emotion engine" is a technology that estimates a customer's emotional state from voice data.

[0768] "Personalized product recommendation" means suggesting the most suitable products to each customer based on their preferences and emotional state.

[0769] "AI analysis" refers to the use of artificial intelligence to analyze data and evaluate customer preferences and interests.

[0770] A system for implementing this invention includes a means for comprehensively analyzing customer profiles, schedule management information, customer interaction voice data, and customer preferences and interest in the company's services. A server collects this data and analyzes the customer's preferences and interest using AI. Furthermore, a voice recognition system converts the customer's voice input into text, and an emotion engine estimates the customer's emotional state from the voice data. This allows personalized product recommendations to be made based on the customer's preferences and emotional state.

[0771] Hardware and software used

[0772] Hardware: Servers, user devices (smartphones, tablets, PCs, etc.)

[0773] Software: speech recognition systems (e.g., Google Speech Recognition API), sentiment analysis models (e.g., sentiment analysis models from the Transformers library), database management systems (e.g., MySQL)

[0774] Data processing and calculation

[0775] 1. Speech recognition: Customer voice data collected on the user terminal is sent to the server and converted into text using a voice recognition system.

[0776] 2. Sentiment analysis: The converted text data is fed into a sentiment engine to estimate the customer's emotional state.

[0777] 3. Preference analysis: The server uses AI to analyze customer preferences based on customer profiles, past purchase history, and browsing history.

[0778] 4. Product Recommendation: Based on the analysis results, recommend the most suitable products to the customer.

[0779] Specific examples

[0780] For example, if a customer says, "I've been wanting to buy some smartphone accessories lately," the speech recognition system converts this speech into text. The emotion engine then uses this text to estimate the customer's emotional state as "positive." The server then recommends accessories such as smartphone cases, chargers, and earphones based on the customer's past purchase and browsing history.

[0781] Prompt Sentence Examples

[0782] Enter the customer: I've recently been wanting some accessories for my smartphone.

[0783] In this way, it becomes possible to understand the customer's preferences and emotional state in real time and recommend the most suitable products.

[0784] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[0785] Step 1:

[0786] The user inputs voice into the terminal. For example, the user might say, "Recently, I've been thinking about buying some accessories for my smartphone." The terminal collects this voice data and sends it to the server.

[0787] Input: User's voice data

[0788] Output: Audio data sent to the server

[0789] Step 2:

[0790] The server converts the received voice data into text using a voice recognition system (e.g., Google Speech Recognition API), which analyzes the voice data and generates corresponding text data.

[0791] Input: Audio data

[0792] Output: Text data

[0793] Step 3:

[0794] The server inputs the generated text data into an emotion engine to estimate the customer's emotional state. The emotion engine (e.g., a sentiment analysis model from the Transformers library) analyzes the text data and outputs an emotional state (e.g., positive, negative, or neutral).

[0795] Input: Text data

[0796] Output: Emotional state

[0797] Step 4:

[0798] The server uses AI to analyze customer preferences based on customer profiles, past purchase history, and browsing history. The necessary information is retrieved from the customer profile database, and the AI ​​model analyzes this data to identify customer preferences.

[0799] Input: Customer profile, purchase history, browsing history

[0800] Output: Customer preference data

[0801] Step 5:

[0802] The server makes personalized product recommendations based on the customer's preference data and emotional state. The AI ​​model considers the customer's preferences and emotional state to select the most suitable products and generate a recommendation list.

[0803] Input: Customer preference data, emotional state

[0804] Output: Recommended products list

[0805] Step 6:

[0806] The server transmits the generated recommended product list to the user terminal, which displays the received recommended product list and proposes personalized products to the user.

[0807] Input: Recommended product list

[0808] Output: Recommended product list displayed on the user's device

[0809] In this way, it becomes possible to understand the customer's preferences and emotional state in real time and recommend the most suitable products.

[0810] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0811] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0812] Another example of generative AI is Gemini (registered trademark) (Internet search engine). <url: https: gemini.google.com ?hl="ja">) are mentioned.

[0813] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0814] [Second embodiment]

[0815] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0816] As shown in FIG. 3, the data processing system 210 includes the data processing device 12 and smart glasses 214 .

[0817] An example of the data processing device 12 is a server.

[0818] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0819] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0820] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0821] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0822] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0823] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0824] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0825] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0826] In the smart glasses 214, the processor 46 performs reception and output processing.

[0827] A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0828] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0829] "Example 1"

[0830] In one embodiment of the present invention, a speech recognition system is provided for acquiring customer-facing speech data. The speech recognition system acquires speech data input from a microphone installed in the store in real time and converts the data into text data. The text data includes what the customer is saying and other speech information.

[0831] "Example 2"

[0832] Furthermore, a system is provided that uses schedule management information and customer interaction voice data to identify customer information for that day. Specifically, a list of customers scheduled to visit the store that day is created based on schedule management information, and the list is compared with customer interaction voice data to identify which customer corresponds to which voice data.

[0833] "Example 3"

[0834] The company also has a system that uses AI to analyze customer preferences and interest in its services. This system analyzes customer preferences and interest in its services based on features extracted from customer interaction voice data and customer profiles. Specifically, the AI ​​learns what products and services customers are interested in and what purchasing behavior they tend to exhibit, and provides personalized services for each customer based on that information.

[0835] The processing flow of each embodiment will be described below.

[0836] "Example 1"

[0837] Step 1: Acquire audio data in real time from microphones installed in the store.

[0838] Step 2: Convert the acquired audio data into text data.

[0839] Step 3: Save the converted text data and use it for further processing.

[0840] "Example 2"

[0841] Step 1: Create a list of customers who are scheduled to visit the store that day based on scheduling information.

[0842] Step 2: Compare the list you created with customer interaction voice data.

[0843] Step 3: From the comparison results, identify which customer corresponds to which voice data.

[0844] "Example 3"

[0845] Step 1: Extract features from customer interaction voice data and customer profiles.

[0846] Step 2: Based on the extracted features, the AI ​​analyzes the customer's preferences and level of interest in the company's services.

[0847] Step 3: Provide personalized services to each customer based on the results of AI analysis.

[0848] Example 1

[0849] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0850] Conventional customer support systems have difficulty accurately grasping customer comments and preferences in real time, making it difficult to respond to customer needs promptly. Furthermore, there is a lack of efficient means for managing and analyzing voice data, making it difficult to provide personalized services based on customer preferences and interests.

[0851] The identification process by the identification processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means. In this invention, the server includes: a means for comprehensively analyzing customer profiles, schedule management information, customer response voice data, and customer preferences and interest in the company's services; a means for identifying information about customers visiting the store today using the schedule management information and customer response voice data; a means for analyzing customer preferences and interest in the company's services using artificial intelligence; a means for acquiring voice data in real time from a voice input device installed in the store; a means for converting the acquired voice data into text data using voice recognition technology; a means for storing the converted text data in a database; and a means for analyzing the stored text data using natural language processing technology. This makes it possible to accurately grasp the content of customer comments and preferences in real time and respond quickly.

[0852] A "customer profile" is a collection of data that includes basic information about a customer, past purchase history, preferences, interests, etc.

[0853] "Schedule management information" is data for managing customer visit schedules, reservation information, past visit history, and the like.

[0854] "Customer interaction voice data" is voice data including conversations between customers and store clerks and the content of what customers say.

[0855] "Customer preferences" refers to information such as the products and services that customers prefer and the areas in which they are interested.

[0856] "Interest in our company's services" is an indicator that shows how interested customers are in the products and services that our company offers.

[0857] "Artificial intelligence" is a technology that allows computers to learn and reason by imitating human intelligence.

[0858] An "audio input device" is a device for acquiring audio as digital data, and generally refers to a microphone.

[0859] "Speech recognition technology" is a technology for converting voice data into text data.

[0860] "Text data" is data expressed as character information.

[0861] A "database" is a system for efficiently storing, managing, and retrieving data.

[0862] "Natural language processing technology" is a technology that allows computers to understand, interpret, and generate human language.

[0863] This invention relates to a system for acquiring and analyzing customer interaction voice data. Specifically, this system acquires voice data in real time from a voice input device installed in a store, converts it into text data using voice recognition technology, stores the text data in a database, and analyzes it using natural language processing technology.

[0864] Hardware and software used

[0865] The server uses the following hardware and software:

[0866] Voice input device: Uses microphones installed in the store to capture high-quality conversations between customers and store staff.

[0867] Speech recognition technology: The Google Cloud Speech-to-Text API is used to convert captured voice data into text data.

[0868] Database: A MySQL database is used to store the converted text data.

[0869] Natural language processing techniques: Analyze stored text data using natural language processing toolkits such as NLTK and SpaCy.

[0870] Specific operation of the system

[0871] The server collects voice data from multiple microphones installed in various areas of the store. These microphones capture conversations between customers and store staff in real time. For example, microphones installed near the cash register and shelves pick up customer questions and requests.

[0872] The collected voice data is sent to the Google Cloud Speech-to-Text API, which uses advanced speech recognition algorithms to convert voice data into text. For example, if a customer says, "I'd like to know more about this product," the API will convert this speech into text: "I'd like to know more about this product."

[0873] The converted text data is stored in a MySQL database, which is designed to efficiently manage customer comments and other voice information. For example, if a customer says, "I'd like to know more about this product," the text data is stored in a specific table in the database.

[0874] The stored text data is analyzed using natural language processing toolkits (e.g., NLTK or SpaCy). This analysis can identify customer needs and problems. For example, if multiple customers say, "I want to know more about this product," the server aggregates that information and suggests that there is a lack of information about a particular product.

[0875] Specific examples

[0876] For example, if a customer says, "I'd like to know more about this product," the server captures this speech through a microphone and uses the Google Cloud Speech-to-Text API to convert it into text data that says, "I'd like to know more about this product." This text data is then stored in a MySQL database and analyzed using natural language processing tools. The analysis may reveal that information about a particular product is lacking, and the store can take steps to provide more information about that product.

[0877] Prompt Sentence Examples

[0878] Examples of prompts to be input to a generative AI model include:

[0879] "Design a system to capture customer-facing voice data in real time and convert it into text data. Specifically, voice data will be captured from microphones installed in stores and converted into text data using the Google Cloud Speech-to-Text API. The converted text data will be stored in a MySQL database and analyzed using natural language processing tools."

[0880] By inputting this prompt into a generative AI model, specific advice and code examples can be obtained on system design and implementation.

[0881] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0882] Step 1: Acquire audio data

[0883] The server collects real-time audio data from multiple microphones installed in the store. These microphones are placed in various areas of the store and capture conversations between customers and store staff in high quality. The input is the audio signal from the microphones, and the output is digital audio data. For example, microphones installed near the cash register or shelves pick up customer questions and requests.

[0884] Step 2: Convert the audio data

[0885] The server sends the captured voice data to the Google Cloud Speech-to-Text API. This API uses advanced speech recognition algorithms to convert voice data into text data. The input is digital voice data, and the output is text data. For example, if a customer says, "I'd like to know more about this product," the API converts this speech into text: "I'd like to know more about this product."

[0886] Step 3: Save the text data

[0887] The server stores the converted text data in a MySQL database. This database is designed to efficiently manage customer utterances and other voice information. The input is text data, and the output is a record stored in the database. For example, if a customer utters, "I'd like to know more about this product," the text data is stored in a specific table in the database.

[0888] Step 4: Analyze the data

[0889] The server analyzes the stored text data using a natural language processing toolkit (e.g., NLTK or SpaCy). This analysis can identify customer needs and problems. The input is the text data stored in the database, and the output is the analysis results. For example, if multiple customers say, "I want to know more about this product," the server aggregates that information and suggests that there is a lack of information about a particular product.

[0890] Step 5: Personalized service delivery

[0891] Based on the analysis results, the server provides personalized services based on the customer's preferences and interest in the store's services. The input is the analysis results, and the output is the proposal or provision of personalized services. For example, if it is determined that information about a specific product is lacking, the store can take measures to provide detailed information about that product.

[0892] (Application example 1)

[0893] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0894] Conventional customer service systems have had the problem of being unable to grasp customer comments in real time and provide appropriate responses quickly. Furthermore, personalized service based on customer preferences and interests was not provided adequately, making it difficult to improve customer satisfaction. Furthermore, there was a lack of support for store staff to respond quickly and appropriately to customers.

[0895] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0896] In this invention, the server includes: a means for comprehensively analyzing customer profiles, schedule management information, customer response voice data, and customer preferences and interest in the company's services; a means for identifying today's customer information using schedule management information and customer response voice data; a means for analyzing customer preferences and interest in the company's services using AI; a means for converting voice data into text data in real time; a means for analyzing the text data and generating appropriate responses to customer questions and requests; and a means for displaying the generated responses to support rapid customer responses. This enables the content of customer comments to be understood in real time and appropriate responses to be provided promptly. This also enables personalized service based on customer preferences and interests, which is expected to improve customer satisfaction. Furthermore, this system provides enhanced support for store staff to quickly provide appropriate responses when serving customers.

[0897] A "customer profile" is data that includes basic information about a customer, their past purchasing history, preferences, etc.

[0898] "Schedule management information" is data related to time, such as customer visit schedules and reservation information.

[0899] "Customer interaction voice data" is voice data including the content of the conversation between the customer and the store clerk.

[0900] "Customer preferences and interest in our company's services" is data that indicates the products and services that customers prefer and the level of interest in them.

[0901] A "voice recognition system" is a system that converts voice data into text data.

[0902] "AI analytics" is the process of using artificial intelligence to analyze data and extract patterns and trends.

[0903] "Real-time" means that data is processed as it is generated.

[0904] "Text data" is character information converted from voice data.

[0905] "Analysis" is the process of examining data in detail to understand its meaning and structure.

[0906] "Appropriate response" refers to the most appropriate response or action to a customer's question or request.

[0907] "Personalized service provision" means providing customized services based on the individual preferences and interests of customers.

[0908] "Quick customer response" refers to responding quickly to customer requests.

[0909] The system for carrying out this invention includes a means for comprehensively analyzing customer profiles, schedule management information, customer interaction voice data, and customer preferences and interest in the company's services. The system is realized using the following hardware and software.

[0910] Hardware

[0911] Smartphone: A device used by store associates to capture and display voice data.

[0912] Microphone: A device installed in the store to collect customer comments.

[0913] software

[0914] Python: A programming language used to implement programs.

[0915] SpeechRecognition library: A library for converting voice data into text data.

[0916] OpenAI API: An artificial intelligence model for analyzing text data and generating appropriate responses.

[0917] Data processing and calculation

[0918] 1. Voice data capture: A microphone collects what the customer says and sends it to a smartphone.

[0919] 2. Converting speech data to text: Use the SpeechRecognition library to convert speech data to text in real time.

[0920] 3. Text data analysis: Use OpenAI API to analyze text data and generate appropriate responses to customer questions and requests.

[0921] 4. Display of response: The generated response is displayed on the smartphone, allowing the store clerk to quickly respond to the customer.

[0922] Specific examples

[0923] For example, if a customer says, "Tell me more about this product," the system operates as follows:

[0924] 1. A microphone collects what the customer says and sends it to a smartphone.

[0925] 2. The SpeechRecognition library converts the audio data into text data.

[0926] 3. The OpenAI API analyzes the text data and generates a response like this:

[0927] "This product uses the latest technology and is particularly durable. For more information, please see this brochure."

[0928] 4. The generated response is displayed on the smartphone, and the store clerk responds promptly to the customer.

[0929] Prompt Sentence Examples

[0930] Customers say: Tell me more about this product

[0931] Please suggest how the store clerk should respond.

[0932] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0933] Step 1:

[0934] Acquiring audio data

[0935] The server collects customer speech from microphones installed in the store. The microphones capture voice data in real time and send it to a smartphone. The input is the customer's speech, and the output is voice data. This voice data is converted into text data in a subsequent processing step.

[0936] Step 2:

[0937] Converting audio data to text

[0938] The device (smartphone) uses the SpeechRecognition library to convert the received voice data into text data in real time. The input is voice data and the output is text data. In this conversion process, voice recognition technology is used to convert the voice content into text information.

[0939] Step 3:

[0940] Text data analysis

[0941] The server uses the OpenAI API to analyze the text data. The input is text data, and the output is the analysis results. Specifically, it understands customer questions and requests based on the text data and generates appropriate responses. This analysis process uses a generative AI model to perform natural language processing.

[0942] Step 4:

[0943] Generate correspondence

[0944] The server generates an appropriate response to the customer's question or request based on the analysis results. The input is the analysis results, and the output is the generated response. The generated response includes a specific reply or instruction for action based on the content of the customer's statement.

[0945] Step 5:

[0946] Display of correspondence

[0947] The terminal (smartphone) displays the generated response to the store clerk. The input is the generated response, and the output is the text displayed on the smartphone screen. The store clerk can check this display and respond to the customer quickly.

[0948] Specific examples

[0949] For example, if a customer says, "Tell me more about this product," the system operates as follows:

[0950] 1. Step 1: The microphone captures the customer's utterance, "Tell me more about this product," and sends it to the smartphone.

[0951] 2. Step 2: The smartphone converts the voice data into text data: "Tell me more about this product."

[0952] 3. Step 3: The server analyzes the text data and generates an appropriate response to the customer's question.

[0953] 4. Step 4: The server generates a response saying, "This product uses the latest technology and is particularly durable. If you need more information, please see this brochure."

[0954] 5. Step 5: The smartphone displays the generated response to the store associate, who then responds promptly to the customer.

[0955] Prompt Sentence Examples

[0956] Customers say: Tell me more about this product

[0957] Please suggest how the store clerk should respond.

[0958] Example 2

[0959] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0960] With conventional customer management systems, it was difficult to effectively integrate schedule management information and customer interaction voice data to identify customer information. Furthermore, there was a lack of means to accurately understand customer preferences and interest in the company's services, and to provide personalized services. This hindered the efficiency of customer interactions and the improvement of service quality.

[0961] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0962] In this invention, the server includes means for acquiring schedule management information, means for acquiring voice data, means for converting voice data into text data, means for comparing the acquired schedule management information with the converted text data and identifying customer information, means for saving and notifying the identified customer information, and means for analyzing customer preferences and interest in the company's services using artificial intelligence. This makes it possible to integrate schedule management information and customer interaction voice data to identify customer information and further analyze customer preferences and interest to provide personalized services.

[0963] A "customer profile" is a collection of data that includes basic information about a customer, past transaction history, preferences, etc.

[0964] "Schedule management information" is a collection of data including customer visit schedules and reservation information.

[0965] "Customer response voice data" refers to voice recordings of conversations and inquiries with customers.

[0966] "Customer preferences" refers to information that indicates the types and characteristics of products and services that customers prefer.

[0967] The "degree of interest in our company's services" is information indicating the degree of interest that customers have in the services provided by our company.

[0968] "Artificial intelligence" is a technology that uses machine learning and data analysis to imitate human intellectual tasks.

[0969] A "voice recognition system" is a technology that analyzes voice data and converts it into text data.

[0970] "Personalized services" are customized services provided based on a customer's individual preferences and interests.

[0971] A "server" is a computer system that processes, stores, and communicates data.

[0972] "Text data" refers to data obtained by converting voice data into character information.

[0973] A "database" is a system for efficiently storing, searching, and managing data.

[0974] "Notification" is the act of notifying a user or other system of specific information.

[0975] This invention is a system for comprehensively analyzing customer profiles, schedule management information, customer interaction voice data, and customer preferences and interest in the company's services. A specific embodiment of this system is described below.

[0976] Hardware and software used

[0977] Scheduling software: Google Calendar

[0978] Audio data storage database: Amazon S3

[0979] Speech recognition software: Google Cloud Speech-to-Text

[0980] Database: MySQL

[0981] Server Processing

[0982] 1. Acquiring schedule management information

[0983] The server uses the Google Calendar API to obtain a list of customers scheduled to visit the store that day.

[0984] Specifically, the server sends a request to the Google Calendar API and receives JSON-formatted data as a response.

[0985] 2. Acquisition of customer interaction voice data

[0986] The server downloads customer response voice data from Amazon S3.

[0987] Specifically, the server accesses an Amazon S3 bucket and retrieves the specified audio file.

[0988] 3. Converting Audio Data to Text

[0989] The server converts the audio data into text data using the Google Cloud Speech-to-Text API.

[0990] Specifically, the server sends the audio file to the Google Cloud Speech-to-Text API and receives text data in response.

[0991] 4. Compare customer lists with text data

[0992] The server compares the acquired customer list with the converted text data and identifies which customer corresponds to which voice data.

[0993] Specifically, the server analyzes keywords and phrases in the text data and compares them with information in the customer list.

[0994] 5. Storage of Identified Customer Information and Notification

[0995] The server stores the identified customer information in a MySQL database and notifies the terminal.

[0996] Specifically, the server executes an INSERT query in the MySQL database and notifies the terminal by sending an HTTP request.

[0997] Terminal handling

[0998] 1. Receiving customer list

[0999] The terminal receives the list of customers scheduled to visit the store on that day sent from the server.

[1000] Specifically, the terminal receives an HTTP response from the server and parses the JSON-formatted data.

[1001] 2. Receiving and playing audio data

[1002] The terminal receives the customer response voice data transmitted from the server and plays back the voice data.

[1003] Specifically, the device downloads the audio file and plays it using the built-in media player.

[1004] 3. Display of identified customer information

[1005] The terminal displays the identified customer information sent from the server so that the user can confirm it.

[1006] Specifically, the terminal displays the received data on a GUI (graphical user interface).

[1007] User operations

[1008] 1. Check your customer list

[1009] The user checks the list of customers scheduled to visit the store that day displayed on the terminal.

[1010] Specifically, the user looks at the screen of the terminal and checks the customer information displayed in the list.

[1011] 2. Check the audio data

[1012] The user listens to the customer service voice data reproduced from the terminal.

[1013] Specifically, the user listens to the audio data through the terminal's speaker or headphones.

[1014] 3. Confirmation and response of identified customer information

[1015] The user checks the specified customer information displayed on the terminal and takes action as necessary.

[1016] Specifically, the user responds to the customer based on the displayed information.

[1017] Specific examples

[1018] Prompt Sentence Examples

[1019] "Compare the list of customers scheduled to visit obtained from the schedule management information with the customer response voice data stored in the voice data storage database, and identify which customer corresponds to which voice data."

[1020] In this way, the system uses schedule management information and customer interaction voice data to identify information about customers visiting the store that day, and further analyzes their preferences and interests to provide personalized services.

[1021] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1022] Step 1:

[1023] Obtaining schedule management information

[1024] The server uses the schedule management software's API to obtain a list of customers scheduled to visit the store that day. Specifically, the server sends a request to the API and receives JSON-formatted data as a response.

[1025] Input: API request

[1026] Output: A list of customers in JSON format

[1027] Specific operation: The server sends an HTTP GET request to the API endpoint and parses the JSON data received as a response to extract the customer list.

[1028] Step 2:

[1029] Acquisition of customer interaction voice data

[1030] The server downloads the customer response voice data from the voice data storage database. Specifically, the server accesses a bucket in the database and acquires a specified voice file.

[1031] Input: Database bucket information

[1032] Output: Audio file

[1033] Specific operation: The server obtains a list of audio files using the database API and downloads the necessary audio files.

[1034] Step 3:

[1035] Converting audio data to text

[1036] The server uses speech recognition software to convert the audio data into text data. Specifically, the server sends the audio file to a speech recognition API and receives the text data in response.

[1037] Input: Audio file

[1038] Output: Text data

[1039] Specific operation: The server uploads the audio file to the API, performs speech recognition processing, and obtains text data.

[1040] Step 4:

[1041] Comparing customer lists and text data

[1042] The server compares the acquired customer list with the converted text data to identify which customer corresponds to which voice data. Specifically, the server analyzes keywords and phrases in the text data and matches them with the information in the customer list.

[1043] Input: Customer list, text data

[1044] Output: Identified customer information

[1045] Specific operation: The server analyzes the text data to identify parts that match names and reservation information in the customer list, and extracts the corresponding customer information.

[1046] Step 5:

[1047] Retention of identified customer information and notification

[1048] The server stores the identified customer information in a database and notifies the terminal. Specifically, the server executes an INSERT query in the database and notifies the terminal by sending an HTTP request.

[1049] Input: Identified customer information

[1050] Output: Save to database, notify to terminal

[1051] Specific operations: The server stores the identified customer information in a database, and generates and sends an HTTP request to send a notification to the terminal.

[1052] Step 6:

[1053] Receive customer list

[1054] The terminal receives the list of customers scheduled to visit the store that day sent from the server. Specifically, the terminal receives the HTTP response from the server and parses the JSON format data.

[1055] Input: HTTP response from the server

[1056] Output: Parsed customer list

[1057] Specific operation: The terminal analyzes the received HTTP response, extracts the customer list, and displays it.

[1058] Step 7:

[1059] Receiving and playing audio data

[1060] The terminal receives the customer response voice data transmitted from the server and plays the voice data. Specifically, the terminal downloads the voice file and plays it on a built-in media player.

[1061] Input: Audio file

[1062] Output: The audio being played

[1063] Specific operations: The device downloads the audio file, launches the media player, and plays the audio.

[1064] Step 8:

[1065] View identified customer information

[1066] The terminal displays the specified customer information sent from the server so that the user can check it. Specifically, the terminal displays the received data on a GUI.

[1067] Input: Identified customer information

[1068] Output: Displayed customer information

[1069] Specific operation: The terminal displays the identified customer information on the GUI so that the user can check it.

[1070] Step 9:

[1071] Check the customer list

[1072] The user checks the list of customers scheduled to visit the store that day, which is displayed on the terminal. Specifically, the user looks at the screen of the terminal and checks the customer information displayed in the list.

[1073] Input: Displayed customer list

[1074] Output: Confirmed customer information

[1075] Specific operations: The user checks the customer list on the terminal screen and takes action as necessary.

[1076] Step 10:

[1077] Checking audio data

[1078] The user listens to the customer service voice data reproduced from the terminal, specifically, through the terminal's speaker or headphones.

[1079] Input: Audio to be played

[1080] Output: Confirmed audio data

[1081] Specific operation: The user listens to the audio data through the device's speaker or headphones and checks the content.

[1082] Step 11:

[1083] Confirmation and response of identified customer information

[1084] The user checks the specified customer information displayed on the terminal and responds as necessary. Specifically, the user responds to the customer based on the displayed information.

[1085] Input: Displayed customer information

[1086] Output: Customer information handled

[1087] Specific operations: The user checks the identified customer information displayed on the terminal and takes action as necessary to assist the customer.

[1088] (Application example 2)

[1089] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1090] Conventional customer management systems required a lot of manual work to identify customers who were planning to visit the store, which led to problems with inefficiency. It was also difficult to provide personalized services based on customer preferences and interests, making it difficult to improve customer satisfaction.

[1091] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for comprehensively analyzing customer profiles, schedule management information, customer response voice data, and customer preferences and interest in the company's services, means for identifying information about customers visiting today using the schedule management information and customer response voice data, means for analyzing customer preferences and interest in the company's services using AI, means for converting voice data to text, means for comparing the converted text data with the schedule management information to identify customers visiting the store, and means for displaying the identified customer information. This makes it possible to automatically identify information about customers who are scheduled to visit the store and provide personalized services based on the customers' preferences and interests.

[1092] A "customer profile" is data that includes basic information about a customer, past purchase history, preferences, etc.

[1093] "Schedule management information" is data including customer visit schedules and reservation information.

[1094] "Customer response voice data" refers to voice recordings of conversations with customers and inquiries.

[1095] "Customer preferences" refers to information that indicates the types and characteristics of products and services that customers prefer.

[1096] "Interest in our company's services" is an indicator that shows how interested customers are in the services provided by our company.

[1097] "Comprehensive analysis methods" are methods that integrate and analyze multiple data sets to understand customer behavior and preferences.

[1098] "Means of identification" refers to a method for identifying and clarifying specific information or data.

[1099] "AI analysis" refers to the use of artificial intelligence technology to analyze data and find patterns and trends.

[1100] "Means for converting voice data into text" refers to a method for converting voice into text information using voice recognition technology.

[1101] A "comparison tool" is a method for comparing different data and finding commonalities and differences.

[1102] The "display means" refers to a method for visually presenting the analysis results and the identified information.

[1103] The system for carrying out the present invention operates in cooperation with three entities: a server, a terminal, and a user. A specific embodiment of the system will be described below.

[1104] Server Roles

[1105] The server has the ability to comprehensively analyze customer profiles, schedule management information, customer interaction voice data, and customer preferences and interest in the company's services. The server integrates this data and performs analysis using AI technology. Specifically, it uses a voice recognition system to convert voice data into text, and compares the converted text data with schedule management information to identify visiting customers.

[1106] Device Role

[1107] The terminal (e.g., a smartphone) displays the identified customer information sent from the server. The terminal is designed so that the user can check customer information in real time, enabling the user to respond to the customer promptly and appropriately.

[1108] User Roles

[1109] The user can check the customer information identified through the terminal and provide personalized service based on the customer's preferences and interest in the company's services.

[1110] Hardware and software used

[1111] Speech recognition system: Converts voice data into text using the Python speech_recognition library.

[1112] AI technology: Uses machine learning models to analyze customer preferences and interest in your services.

[1113] Database: We use a SQL database to store customer profiles and scheduling information.

[1114] Device: A mobile device such as a smartphone or tablet.

[1115] Specific examples

[1116] For example, based on the schedule management information for a certain day, the server creates a list of customers scheduled to visit that day. Next, the server converts customer interaction voice data into text using a voice recognition system, and identifies the customers by comparing the converted text data with the schedule management information. The identified customer information is sent to the terminal, allowing the user to check it in real time.

[1117] Example prompt sentence:

[1118] "Convert an audio file to text. The path to the audio file is 'path / to / audio / file.wav'."

[1119] In this way, by linking the server, terminal, and user, it becomes possible to automatically identify customer information about customers who are planning to visit the store and provide personalized services based on the customer's preferences and interest.

[1120] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1121] Step 1:

[1122] The server obtains schedule management information. As input, it obtains a list of customers scheduled to visit the store that day from the database. As output, it generates a list of customers scheduled to visit the store. This list includes information such as customer ID, name, and scheduled visit time.

[1123] Step 2:

[1124] The server acquires customer response voice data. As input, the path of the voice file is specified. As output, voice data is acquired. This voice data is a record of conversations and inquiries with customers.

[1125] Step 3:

[1126] The server uses a speech recognition system to convert the speech data into text. The acquired speech data is used as input. As output, text data is generated. This text data is a representation of the content of the speech data as character information.

[1127] Step 4:

[1128] The server compares the converted text data with the schedule management information to identify customers. It uses the text data and a list of customers who are scheduled to visit as input. It generates information about identified customers as output. This information includes customer IDs, names, and scheduled visit times.

[1129] Step 5:

[1130] The server sends the identified customer information to the terminal. It uses the identified customer information as input and generates customer information that is sent to the terminal as output. This information is converted into a format for display on the terminal.

[1131] Step 6:

[1132] The terminal receives and displays the customer information sent from the server. As input, it uses the customer information sent from the server. As output, it displays the information in a format that can be viewed by the user. This allows the user to view the customer information in real time.

[1133] Step 7:

[1134] The user checks the customer information identified through the terminal and responds to the customer. The customer information displayed on the terminal is used as input. The output is to respond appropriately to the customer. This response includes providing personalized services based on the customer's preferences and level of interest in the company's services.

[1135] Example 3

[1136] Next, a description will be given of Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1137] With conventional customer service systems, it was difficult to accurately grasp customer preferences and interests and provide personalized services. Furthermore, there was a lack of means to effectively utilize customer service voice data, making it impossible to respond quickly to customer needs. This made it difficult to improve customer satisfaction and service quality.

[1138] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[1139] In this invention, the server includes means for comprehensively analyzing customer profiles, schedule management information, customer response voice data, and customer preferences and interest in the company's services, means for identifying information about customers visiting the store today using the schedule management information and customer response voice data, means for converting the customer response voice data into text data using voice recognition software, means for analyzing customer preferences and interest in the company's services using AI based on features extracted from the customer profiles and voice data, and means for providing personalized services for each customer. This makes it possible to accurately grasp customer preferences and interest and quickly provide personalized services.

[1140] A "customer profile" is data that includes basic information about a customer, past purchasing history, preferences, and interest levels.

[1141] "Schedule management information" is data related to customer visit plans, reservation information, and schedules.

[1142] "Customer interaction voice data" is voice data of a conversation between a customer and a user.

[1143] "Speech recognition software" is software for converting voice data into text data.

[1144] "Features" are data extracted from voice data and customer profiles that indicate customer preferences and interests.

[1145] "AI" is an artificial intelligence technology that uses machine learning models and natural language processing tools to analyze data and make predictions and classifications.

[1146] "Personalized service" refers to a service that is provided individually based on a customer's preferences and interests.

[1147] "Visiting customer information" is information about customers who are planning to visit the store today.

[1148] This invention relates to a system that uses AI to comprehensively analyze customer profiles, schedule management information, and customer interaction voice data to analyze customer preferences and interest in the company's services. Specific embodiments of this system are described below.

[1149] Hardware and software used

[1150] Hardware: Servers, terminals (PCs, smartphones)

[1151] Software: speech recognition software (e.g., Google Cloud Speech-to-Text API), database management systems (DBMS), machine learning models (e.g., TensorFlow or PyTorch), natural language processing (NLP) tools (e.g., spaCy or NLTK)

[1152] System Overview

[1153] 1. Collection of audio data

[1154] When a user interacts with a customer, voice data is collected through a microphone connected to the terminal.

[1155] The device transmits the collected voice data to the server in real time.

[1156] 2. Converting voice data to text

[1157] The server converts the received voice data into text data using voice recognition software.

[1158] 3. Obtaining customer profiles

[1159] The server uses a database management system (DBMS) to obtain profile data such as basic customer information and past purchasing history.

[1160] 4. Feature Extraction

[1161] The server uses natural language processing (NLP) tools to extract features from the text extracted from the voice data and the customer profile data.

[1162] 5. AI-powered analysis

[1163] The server uses a machine learning model to analyze customer preferences and interest in the company's services based on the extracted features.

[1164] 6. Providing Results

[1165] The server sends the analysis results to the terminal and makes personalized service proposals to the user.

[1166] Specific examples

[1167] Example 1: If customer A has purchased many health foods in the past, the AI ​​will determine that customer A is health-conscious and suggest new health foods and related services.

[1168] Prompt: "Analyze Customer A's interest in healthy foods based on their past purchase history and voice data."

[1169] Example 2: If Customer B frequently asks travel-related questions, the AI ​​determines that Customer B is interested in travel and suggests travel-related products and services.

[1170] Prompt: "Extract travel interests from customer B's voice data and suggest related services."

[1171] In this way, the system uses AI to analyze customer preferences and interests based on customer interaction voice data and customer profiles, and provides personalized services. The flow of the identification process in Example 3 will be described with reference to Figure 15.

[1172] Step 1:

[1173] Audio data collection

[1174] When a user interacts with a customer, voice data is collected through a microphone connected to the terminal.

[1175] Input: Audio of conversation with customer

[1176] Output: Collected audio data

[1177] What it does: When a user starts a conversation with a customer, the device's microphone captures the audio and streams that data to the server.

[1178] Step 2:

[1179] Converting audio data to text

[1180] The server converts the received voice data into text data using voice recognition software.

[1181] Input: Collected audio data

[1182] Output: Converted text data

[1183] Specific operation: The server inputs the voice data into speech recognition software (e.g., Google Cloud Speech-to-Text API) and obtains the converted text data.

[1184] Step 3:

[1185] Retrieving a customer profile

[1186] The server uses a database management system (DBMS) to obtain profile data such as basic customer information and past purchasing history.

[1187] Input: Customer ID

[1188] Output: Customer profile data

[1189] What happens: The server queries the database using the customer ID as a key to retrieve relevant customer profile data.

[1190] Step 4:

[1191] Feature Extraction

[1192] The server uses natural language processing (NLP) tools to extract features from the text extracted from the voice data and the customer profile data.

[1193] Input: Text data, customer profile data

[1194] Output: Extracted feature data

[1195] What it does: The server feeds the text data into an NLP tool (e.g., spaCy or NLTK) for keyword extraction and sentiment analysis, and also extracts relevant features from customer profile data.

[1196] Step 5:

[1197] AI-powered analysis

[1198] The server uses a machine learning model to analyze customer preferences and interest in the company's services based on the extracted features.

[1199] Input: Extracted feature data

[1200] Output: Analysis results

[1201] How it works: The server inputs the feature data into a machine learning model (e.g., TensorFlow or PyTorch) to obtain prediction results. The model learns from past data and generates personalized service proposals for each customer.

[1202] Step 6:

[1203] Providing results

[1204] The server sends the analysis results to the terminal and makes personalized service proposals to the user.

[1205] Input: Analysis results

[1206] Output: Personalized service offers

[1207] Specific operation: The server sends the analysis results in JSON format to the device, which then displays them to the user. For example, it can suggest new products or services based on products that a specific customer has shown interest in in the past.

[1208] (Application example 3)

[1209] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1210] With conventional customer service systems, it was difficult to accurately grasp customer preferences and interests and provide personalized services. Furthermore, there was a lack of means to effectively utilize customer voice data, which meant that product proposals that met customer needs could not be made. This created the problem of being unable to improve customer satisfaction or increase sales.

[1211] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[1212] In this invention, the server includes means for comprehensively analyzing customer profiles, schedule management information, customer response voice data, and customer preferences and interest in the company's services, means for identifying information about customers visiting the store today using the schedule management information and customer response voice data, means for analyzing customer preferences and interest in the company's services using AI, means for analyzing customer preferences and interest based on the customer's voice data and profile data and proposing optimal products, and means for converting voice data into text using a voice recognition system and vectorizing and analyzing the text data. This makes it possible to accurately grasp customer preferences and interest and propose personalized products.

[1213] A "customer profile" is data that includes a customer's basic information, purchasing history, preferences, etc.

[1214] "Schedule management information" is data related to schedules such as customer visit plans and reservation information.

[1215] "Customer response voice data" is voice recording data relating to conversations with customers and inquiries.

[1216] "Customer preferences" refers to information about the types and characteristics of products and services that customers prefer.

[1217] "Interest in our company's services" is an indicator that shows how interested customers are in the products and services that our company offers.

[1218] "Comprehensive analysis means" refers to a method of integrating and analyzing multiple data sets to understand customer preferences and interests.

[1219] The "means for identifying information on customers visiting the store today" is a method for identifying customers visiting the store that day using schedule management information and customer response voice data.

[1220] "AI analysis" refers to a method of analyzing data using artificial intelligence to evaluate customer preferences and interests.

[1221] "Means for proposing optimal products" refers to a method for selecting and proposing the most suitable products to customers based on their preferences and interests.

[1222] A "voice recognition system" is a technology for converting voice data into text data.

[1223] "Means for vectorizing and analyzing text data" refers to a method for converting text data into numerical vectors and analyzing the data using those vectors.

[1224] The system for implementing this invention includes a means for comprehensively analyzing customer profiles, schedule management information, customer response voice data, and customer preferences and interest in the company's services. The server integrates and analyzes this data to understand customer preferences and interest. Furthermore, the system includes a means for identifying customer information for the day using schedule management information and customer response voice data. This allows the system to identify customers visiting the store that day and provide appropriate service.

[1225] The server uses AI to analyze customer preferences and interest in the company's services based on their voice data and profile data, and then proposes optimal products. Voice data is converted into text using a voice recognition system, and the text data is vectorized and analyzed. This process enables product proposals tailored to customer needs.

[1226] Specifically, the server processes the data using software such as Python, pandas, scikit-learn, and speech_recognition. Customer profile data is read from a CSV file, and customer-interacted voice data is converted to text using a speech recognition library. The text data is then vectorized using TF-IDF and analyzed for customer preferences and interests using the Nearest Neighbors algorithm.

[1227] For example, if a customer says, "I'm looking for a new smartphone," the server analyzes the voice data and compares it with their past purchase and browsing history to suggest the smartphone that's best suited to them. In this way, personalized product suggestions tailored to the customer's needs are realized.

[1228] An example of a prompt to input to a generative AI model is as follows:

[1229] Create a Python program that analyzes customer preferences and interests based on their voice data and profile data, and suggests the most suitable products. The customer's voice data will be converted into text using a voice recognition library, and the text data will be converted into TF-IDF vectors for analysis.

[1230] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[1231] Step 1:

[1232] The server reads customer profile data from a CSV file. The input is a CSV file of customer profile data, and the output is customer profile data in data frame format. Specifically, the server uses the Python pandas library to read the CSV file and convert it to a data frame.

[1233] Step 2:

[1234] The server converts customer-facing voice data into text using a speech recognition system. The input is an audio file, and the output is text data. Specifically, the server reads the audio file using the speech_recognition library and converts it into text using Google's speech recognition API.

[1235] Step 3:

[1236] The server combines customer profile data and audio data. The input is customer profile data and text data in data frame format, and the output is the combined data frame. Specifically, the server uses the pandas library to add audio data to customer profile data.

[1237] Step 4:

[1238] The server performs TF-IDF vectorization on the text data. The input is the combined data frame text data, and the output is a TF-IDF vector. Specifically, it uses scikit-learn's TfidfVectorizer to convert the text data into a numeric vector.

[1239] Step 5:

[1240] The server analyzes customer preferences and interests using the NearestNeighbors algorithm based on the vectorized data. The input is a TF-IDF vector, and the output is an analysis result based on the customer preferences and interests. Specifically, the data is analyzed using NearestNeighbors from scikit-learn.

[1241] Step 6:

[1242] The server proposes optimal products based on the customer's preferences and interests. The input is the analysis results, and the output is a list of products to be proposed to the customer. Specifically, the server selects the most suitable products for the customer based on the analysis results and generates a list of products to be proposed.

[1243] Step 7:

[1244] The terminal displays the product list received from the server to the user. The input is the product list sent from the server, and the output is the product list displayed on the user's smartphone screen. Specifically, the product list is visually presented to the user using a smartphone application.

[1245] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1246] "Example 1"

[1247] In one embodiment of the present invention, the AI ​​analysis uses an emotion engine that recognizes the user's emotions to analyze the customer's emotional state.

[1248] Specifically, the system estimates the customer's emotional state from customer interaction voice data and uses the results of this estimation in AI analysis. For example, if a customer is pleased with a service, that information can be used to improve the service or propose new services.

[1249] "Example 2"

[1250] Furthermore, AI analysis will be used to provide personalized services based on the customer's emotional state. Specifically, services will be provided that take into account the customer's emotional state. For example, if a customer is dissatisfied, that information will be used to improve the service. Also, if a customer is happy, that information will be used to suggest new services.

[1251] "Example 3"

[1252] Furthermore, the emotion engine estimates the emotional state of the customer from the customer interaction voice data acquired by the voice recognition system. Specifically, the emotion engine estimates the emotional state of the customer from the voice data acquired by the voice recognition system. For example, the emotion engine estimates the emotional state of the customer from the tone, speed, volume, etc. of the voice.

[1253] The processing flow of each embodiment will be described below.

[1254] "Example 1"

[1255] Step 1: In AI analysis, an emotion engine that recognizes user emotions is used to analyze the customer's emotional state.

[1256] Step 2: Estimate the customer's emotional state from customer interaction voice data, and use the estimation results for AI analysis.

[1257] Step 3: For example, if a customer is pleased with a service, use that information to improve the service or propose new services.

[1258] "Example 2"

[1259] Step 1: Use AI analysis to provide personalized service based on the customer's emotional state.

[1260] Step 2: Specifically, provide services that take into account the customer's emotional state.

[1261] Step 3: For example, if a customer is dissatisfied, improve the service based on that information. Also, if a customer is happy, propose a new service based on that information. "Example 3"

[1262] Step 1: The emotion engine estimates the emotional state of the customer from the customer-facing voice data acquired by the voice recognition system.

[1263] Step 2: Specifically, the customer's emotional state is estimated from the voice data acquired by the voice recognition system.

[1264] Step 3: Infer the customer's emotional state, for example, from the tone, speed, and volume of the voice.

[1265] Example 1

[1266] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1267] With conventional customer service systems, it was difficult to accurately grasp customers' emotional states and preferences, resulting in a lack of concrete data to improve service quality. Furthermore, it was difficult to respond to customers in real time, limiting the improvement of customer satisfaction. This made it difficult to respond to customer needs quickly, potentially resulting in a loss of competitiveness.

[1268] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1269] In this invention, the server includes means for comprehensively analyzing customer profiles, schedule management information, customer response voice data, and customer preferences and interest in the company's services, means for identifying information about customers visiting the store today using the schedule management information and customer response voice data, means for analyzing customer preferences and interest in the company's services using artificial intelligence, means for acquiring voice data in real time from a voice input device installed in the store, means for converting the acquired voice data into text data using voice recognition technology, means for analyzing the emotional state of the customer based on the converted text data, and means for improving services or proposing new services based on the analysis of the emotional state. This makes it possible to grasp the emotional state and preferences of customers in real time and provide services quickly and accurately.

[1270] A "customer profile" is data that includes basic information about a customer, past purchase history, preferences, etc.

[1271] "Schedule management information" is data for managing customer visit schedules and reservation information.

[1272] "Customer interaction voice data" is voice data including conversations between customers and staff.

[1273] "Preferences" refer to the preferences and tendencies that customers have toward particular products or services.

[1274] "Interest in our company's services" refers to the degree of interest or concern that customers have in the services provided by our company.

[1275] "Artificial intelligence" is a technology that allows computers to learn and reason by imitating human intelligence.

[1276] An "audio input device" is a device for converting audio into digital data.

[1277] "Speech recognition technology" is a technology that converts voice data into text data.

[1278] "Text data" is character information converted using voice recognition technology.

[1279] "Emotional state" refers to the type and intensity of emotions a customer has.

[1280] "Improvement of services" refers to improving the content of services in order to increase customer satisfaction.

[1281] "Proposing new services" refers to providing new services in response to customer needs.

[1282] MODE FOR CARRYING OUT THE INVENTION

[1283] The present invention relates to a system for acquiring customer interaction voice data and analyzing the emotional state and preferences of the customer. Specific embodiments of this system will be described below.

[1284] Acquiring audio data

[1285] The server receives real-time voice data from a voice input device (microphone) installed in the store. This voice input device captures all voices, including conversations between customers and staff. For example, when a customer says, "Tell me about this product," the voice is sent to the server via the microphone.

[1286] Converting audio data to text

[1287] The server converts the acquired voice data into text data using voice recognition technology (for example, Google Cloud Speech-to-Text API). This API provides highly accurate voice recognition technology and converts voice into text quickly and accurately. For example, voice data such as "Tell me about this product" is converted into text data such as "Tell me about this product."

[1288] Emotional state analysis

[1289] The server inputs the converted text data into an emotion engine (e.g., IBM Watson Tone Analyzer) to analyze the customer's emotional state. This emotion engine infers the customer's emotion (joy, anger, sadness, etc.) from the text data. For example, it infers the customer's interest from the text data "Tell me about this product."

[1290] Use of analysis results

[1291] The server improves services and proposes new services based on the estimated emotional state obtained from the emotion engine. For example, if a customer says, "This service is really great!", the server uses that information to develop a strategy for providing similar services to other customers.

[1292] Specific examples

[1293] Example 1: Customer Delight Analysis

[1294] 1. The server receives voice data from the in-store microphone saying, "This service is truly amazing!"

[1295] 2. The server converts this voice data into text data using the Google Cloud Speech-to-Text API. The converted text data becomes "This service is really great!"

[1296] 3. The server inputs the converted text data into the IBM Watson Tone Analyzer and estimates that the customer is feeling "joy." The emotion engine analyzes emotions based on the context of the text data and word choice.

[1297] 4. The server uses this information to develop strategies for providing similar service to other customers, such as offering a special bonus to the customer the next time they visit.

[1298] Prompt Sentence Examples

[1299] "Please explain the system that converts voice data acquired from microphones in stores into text data in real time and analyzes the emotional state of customers. Please also specify the names of any specific hardware or software."

[1300] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1301] Step 1:

[1302] The server acquires voice data in real time from voice input devices (microphones) installed in the store. Specifically, it captures all voice, including conversations between customers and staff. For example, when a customer says, "Tell me about this product," that voice is sent to the server via the microphone. The input is voice data, and the output is voice data stored on the server.

[1303] Step 2:

[1304] The server converts the acquired voice data into text data using voice recognition technology (for example, Google Cloud Speech-to-Text API). Specifically, the voice data is input into the API, and the voice recognition technology quickly and accurately converts the voice into text. For example, voice data saying "Tell me about this product" is converted into text data saying "Tell me about this product." The input is voice data, and the output is text data.

[1305] Step 3:

[1306] The server inputs the converted text data into an emotion engine (for example, IBM Watson Tone Analyzer) to analyze the customer's emotional state. Specifically, the text data is input into the emotion engine, which infers the customer's emotion (joy, anger, sadness, etc.) from the context of the text data and the choice of words. For example, from the text data "Tell me about this product," it infers the customer's interest. The input is text data, and the output is the analysis result of the emotional state.

[1307] Step 4:

[1308] The server improves services and proposes new services based on the estimated emotional state obtained from the emotion engine. Specifically, it considers measures to improve customer satisfaction based on the results of the emotional state analysis. For example, if a customer says, "This service is truly wonderful!", it uses that information to develop a strategy for providing similar services to other customers. The input is the results of the emotional state analysis, and the output is specific measures for improving the service.

[1309] (Application example 1)

[1310] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1311] Conventional customer service systems have difficulty understanding a customer's emotional state in real time, making it difficult to respond appropriately to immediately improve customer satisfaction. Furthermore, there is a lack of means to improve services or propose new services based on the customer's emotional state. This can lead to a decline in the quality of the customer experience, leading to a decrease in customer repeat rates and satisfaction.

[1312] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1313] In this invention, the server includes means for comprehensively analyzing customer profiles, schedule management information, customer interaction voice data, and customer preferences and interest in the company's services, means for identifying information about customers visiting the store today using the schedule management information and customer interaction voice data, means for analyzing customer preferences and interest in the company's services using AI, means for converting customer interaction voice data into text data in real time, means for analyzing the customer's emotional state based on the converted text data, and means for improving services or proposing new services based on the customer's emotional state. This makes it possible to grasp the customer's emotional state in real time and take appropriate action.

[1314] A "customer profile" is data that includes basic information about a customer, past purchase history, preferences, etc.

[1315] "Schedule management information" is data related to time, such as customer visit schedules and reservation information.

[1316] "Customer interaction voice data" refers to voice data of conversations between customers and staff.

[1317] "Customer preferences" refers to information about products and services that customers prefer.

[1318] "Interest in our company's services" is an indicator that shows how interested customers are in the services provided by our company.

[1319] A "voice recognition system" is a technology that converts voice data into text data.

[1320] "AI analysis" is a technique that uses artificial intelligence to analyze data and find specific patterns and trends.

[1321] "Real-time" means that data is processed as it is generated.

[1322] "Text data" is data expressed as character information.

[1323] An "emotional state" is a state that indicates the emotions such as joy, anger, sadness, etc. that a customer is feeling.

[1324] "Improvement of Services" means making changes or additions to an existing Service to make it better.

[1325] "New service proposals" means providing new services based on the customer's needs and emotional state.

[1326] The following system configuration will be described as an embodiment of the present invention.

[1327] System Configuration

[1328] The system consists of a server, terminals, and users. The server has the function of comprehensively analyzing customer profiles, schedule management information, customer response voice data, and customer preferences and interest in the company's services. Terminals are devices such as smartphones and tablets used by store staff, and users are customers who visit the store.

[1329] Program processing

[1330] The server performs the following process.

[1331] 1. Acquiring and converting audio data

[1332] A microphone connected to the terminal is used to capture customer-facing voice data in real time. The captured voice data is converted into text data using a voice recognition system (e.g., the speech_recognition library).

[1333] 2. Emotional state analysis

[1334] The converted text data is sent to a server where a generative AI model (e.g., a sentiment analysis model from the transformers library) is used to analyze the customer's emotional state, which can include joy, anger, sadness, etc.

[1335] 3. Service Improvements and Suggestions

[1336] Based on the analysis results, the server evaluates the customer's emotional state and proposes improvements or new services. For example, if the customer is feeling happy, the server will use that information to make suggestions to further increase satisfaction.

[1337] Hardware and software used

[1338] Hardware: Microphones, smartphones, tablets

[1339] Software: speech_recognition library, transformers library

[1340] Specific examples

[1341] For example, if a customer says, "I really like this product!", the device's microphone picks up the voice data and the voice recognition system converts it into text data. The converted text data is sent to the server, where the sentiment analysis model recognizes it as "delighted." This allows the server to understand in real time that the customer is satisfied and suggest actions to the store staff to provide even better service.

[1342] Prompt Sentence Examples

[1343] "Capture customer voice data in real time and convert it into text data. Then, input the text data into a sentiment analysis model to analyze the customer's emotional state. For example, if the customer says, 'I really like this product!', recognize the emotional state as 'Delighted.'"

[1344] In this way, the system can understand the customer's emotional state in real time and respond appropriately.

[1345] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1346] Step 1:

[1347] The terminal uses a microphone to acquire customer-facing voice data in real time. The input is the customer's voice, and the output is voice data. The terminal captures the customer's voice with the microphone and saves it as digital voice data.

[1348] Step 2:

[1349] The device converts the acquired voice data into text data using a voice recognition system (for example, the speech_recognition library). The input is voice data and the output is text data. The device starts the voice recognition system, analyzes the voice data, and generates corresponding text data.

[1350] Step 3:

[1351] The terminal transmits the converted text data to the server. The input is the text data, and the output is data transmission to the server. The terminal transmits the text data to the server via the network.

[1352] Step 4:

[1353] The server inputs the received text data into a generative AI model (for example, a sentiment analysis model from the Transformers library) to analyze the customer's emotional state. The input is text data, and the output is the analysis result of the emotional state. The server passes the text data to the sentiment analysis model, which analyzes the sentiment of the text and returns the result.

[1354] Step 5:

[1355] The server improves services or proposes new services based on the results of the emotional state analysis. The input is the analysis result of the emotional state, and the output is proposals for service improvements or new services. The server evaluates the analysis results and generates optimal proposals for service improvements or new services according to the customer's emotional state.

[1356] Step 6:

[1357] The server sends the generated service improvement proposals and new service proposals to the terminal. The input is the service improvement proposals and new service proposals, and the output is data transmission to the terminal. The server sends the proposals to the terminal via the network.

[1358] Step 7:

[1359] The terminal notifies the store staff of the received proposals for service improvements or new services. The input is the proposal for service improvements or new services, and the output is a notification to the staff. The terminal uses its notification function to display the proposal to the staff and encourage them to take appropriate action.

[1360] In this way, the system can understand the customer's emotional state in real time and respond appropriately.

[1361] Example 2

[1362] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1363] Conventional customer management systems have difficulty effectively integrating customer visit schedule information and customer interaction voice data to provide personalized service based on the customer's emotional state. Furthermore, there is a lack of means to accurately grasp the customer's emotional state and provide appropriate service based on that. This has prevented companies from fully improving customer satisfaction and service quality.

[1364] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1365] In this invention, the server includes means for comprehensively analyzing customer profiles, schedule management information, customer interaction voice data, and customer preferences and interest in the company's services, means for creating a list of customers scheduled to visit the store that day using the schedule management information, means for converting the customer interaction voice data into text data using voice recognition technology, means for comparing the created customer list with the converted text data to identify which customer corresponds to which voice data, means for using emotion analysis technology to analyze the emotional state of the customer from the voice data, and means for generating instructions for providing personalized service based on the identified emotional state. This makes it possible to integrate customer visit schedule information and customer interaction voice data and provide personalized service based on the customer's emotional state.

[1366] A "customer profile" is data that includes a customer's basic information, preferences, purchasing history, etc.

[1367] "Schedule management information" is data for managing customer visit schedules and reservation information.

[1368] "Customer response voice data" refers to voice data recorded during conversations and responses with customers.

[1369] "Customer preferences" refers to data that indicates the types and characteristics of products and services that customers prefer.

[1370] "Interest in our company's services" is data that indicates how interested customers are in the services provided by our company.

[1371] "Speech recognition technology" is a technology for converting voice data into text data.

[1372] "Emotion analysis technology" is a technology for analyzing a customer's emotional state from voice data and text data.

[1373] "Personalized service" refers to customized services provided based on the customer's individual preferences and emotional state.

[1374] The "means for generating instructions" refers to a means for instructing specific actions or responses based on the analysis results.

[1375] MODE FOR CARRYING OUT THE INVENTION

[1376] This invention is a system that comprehensively analyzes customer profiles, schedule management information, customer interaction voice data, and customer preferences and interest in the company's services, and provides personalized services based on the customer's emotional state. Specific embodiments of this system are described below.

[1377] Hardware and software used

[1378] The server uses the following hardware and software:

[1379] Scheduling software: Google Calendar API

[1380] Speech recognition software: Google Cloud Speech-to-Text API

[1381] Sentiment analysis software: IBM Watson Tone Analyzer

[1382] Database Management System: A typical relational database (e.g., MySQL)

[1383] Data processing and calculation

[1384] 1. The server uses the Google Calendar API to get a list of customers who are scheduled to visit the store that day, and saves the list in a database.

[1385] 2. The server retrieves the customer-facing voice data from Google Cloud Storage and converts it to text using the Google Cloud Speech-to-Text API, which is then stored in the database.

[1386] 3. The server compares the text of the voice data with the customer list stored in the database to identify which customer corresponds to which voice data.

[1387] 4. The server uses IBM Watson Tone Analyzer to analyze the customer's emotional state from the voice data, and the analysis results are stored in a database.

[1388] 5. The server generates instructions for providing personalized services based on the identified emotional state.

[1389] Specific examples

[1390] For example, the server uses the Google Calendar API to obtain a list of customers scheduled to visit the store on October 10, 2023. Next, it obtains customer interaction voice data for October 10, 2023 from Google Cloud Storage and converts the voice data into text using the Google Cloud Speech-to-Text API. The converted text data is saved in a database and compared with the customer list to identify that "Customer A corresponds to voice data 1."

[1391] Furthermore, the server analyzes "voice data 1" using IBM Watson Tone Analyzer and determines that "Customer A is dissatisfied." Finally, based on the identified emotional state, it generates "specific instructions for improving service," and the device follows the instructions and provides "a special discount coupon" to Customer A.

[1392] Prompt Sentence Examples

[1393] Example prompts to input to a generative AI model:

[1394] "Get a list of customers scheduled to visit the store on October 10, 2023 from the Google Calendar API, convert the customer-facing voice data into text using the Google Cloud Speech-to-Text API, and identify which customer corresponds to which voice data."

[1395] "Use IBM Watson Tone Analyzer to analyze customer voice data and generate instructions to provide personalized service based on the customer's emotional state."

[1396] In this way, it is possible to integrate customer visit schedule information with customer interaction voice data and provide personalized services based on the customer's emotional state.

[1397] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1398] Step 1:

[1399] The server uses schedule management software (e.g., Google Calendar API) to obtain a list of customers who are scheduled to visit the store on that day.

[1400] Input: Schedule data from the Google Calendar API

[1401] Data processing: Analyze the schedule data obtained from the API and extract a list of customers scheduled to visit the store.

[1402] Output: List of customers who will visit the store

[1403] Specific operation: The server accesses the Google Calendar API and obtains the "list of customers scheduled to visit the store on October 10, 2023." The obtained list is saved in the database.

[1404] Step 2:

[1405] The server converts customer-facing voice data into text data using voice recognition software (e.g., Google Cloud Speech-to-Text API).

[1406] Input: Customer-facing voice data from Google Cloud Storage

[1407] Data processing: Speech recognition processing to convert voice data into text data

[1408] Output: Customer interaction voice data converted into text data

[1409] Specific operation: The server retrieves the "customer interaction voice data for October 10, 2023" from Google Cloud Storage and converts the voice data to text using the Google Cloud Speech-to-Text API. The converted text data is stored in a database.

[1410] Step 3:

[1411] The server compares the text of the voice data with a customer list stored in a database to identify which customer corresponds to which voice data.

[1412] Input: Customer list and text data stored in a database

[1413] Data processing: Comparison of customer list and text data

[1414] Output: Correspondence between customer and voice data

[1415] Specific operation: The server retrieves the list of "Customer A, Customer B, Customer C" and the text "Audio data 1: 'Hello, this is Customer A.'" from the database, analyzes the text data, and determines that "Customer A corresponds to audio data 1."

[1416] Step 4:

[1417] The server uses sentiment analysis software (e.g., IBM Watson Tone Analyzer) to analyze the customer's emotional state from the voice data.

[1418] Input: Customer interaction voice data converted to text data

[1419] Data processing: Sentiment analysis processing

[1420] Output: Customer's emotional state

[1421] Specific operation: The server analyzes "Voice Data 1" using IBM Watson Tone Analyzer and identifies that "Customer A is dissatisfied." The analysis results are stored in the database.

[1422] Step 5:

[1423] The server generates instructions for providing personalized services based on the identified emotional state.

[1424] Input: Customer's emotional state

[1425] Data processing: Instruction generation for personalized service delivery

[1426] Output: Specific instructions for providing the service

[1427] Specific operation: The server generates "specific instructions for improving the service" based on the information that "Customer A is dissatisfied with." The terminal follows the instructions and provides Customer A with a "special discount coupon."

[1428] (Application example 2)

[1429] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1430] While conventional customer service systems can grasp customer visit information, preferences, and service interest, it is difficult to analyze the customer's emotional state in real time and provide personalized service based on that. This has led to the issue of not being able to sufficiently improve customer satisfaction and service quality.

[1431] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1432] In this invention, the server includes means for comprehensively analyzing customer profiles, schedule management information, customer interaction voice data, and customer preferences and interest in the company's services, means for identifying information about customers visiting the store today using the schedule management information and customer interaction voice data, means for analyzing customer preferences and interest in the company's services using AI, and means for analyzing the customer's emotional state and providing personalized services, thereby making it possible to provide personalized services based on the customer's emotional state.

[1433] A "customer profile" is a collection of data that includes a customer's basic information, preferences, past purchase history, etc.

[1434] "Schedule management information" is a collection of data including customer visit schedules and reservation information.

[1435] "Customer response voice data" refers to voice recordings of conversations and inquiries with customers.

[1436] "Customer preferences" refers to information about products and services that customers prefer.

[1437] The "degree of interest in our company's services" is information indicating the degree of interest that customers have in the services provided by our company.

[1438] "AI analytics" is the process of using artificial intelligence to analyze data and extract specific patterns and trends.

[1439] An "emotional state" is a psychological state such as joy, dissatisfaction, or excitement felt by a customer.

[1440] "Personalized service" is service that is customized based on the preferences and emotional state of each individual customer.

[1441] A system for implementing this invention includes means for comprehensively analyzing customer profiles, schedule management information, customer response voice data, and customer preferences and interest in the company's services, means for identifying information about customers visiting the store today using the schedule management information and customer response voice data, means for analyzing customer preferences and interest in the company's services using AI, and means for analyzing the customer's emotional state and providing personalized services.

[1442] System Configuration

[1443] The server uses the following hardware and software:

[1444] Hardware: A server with high-performance CPU, memory, and storage

[1445] Software: Python, SpeechRecognition library, Transformers library

[1446] Data processing and calculation

[1447] The server first obtains a list of customers scheduled to visit that day from the schedule management information. Next, the customer interaction voice data is converted into text using speech recognition technology. This process uses the SpeechRecognition library. The converted text data is then analyzed using AI with the Transformers library to determine the customer's emotional state.

[1448] Specific examples

[1449] For example, if customer A is planning to visit a store and voice data indicates dissatisfaction, the server will use that information to suggest service improvements. Similarly, if customer B is planning to visit a store and voice data indicates enjoyment, the server will suggest a new service.

[1450] Prompt Sentence Examples

[1451] Analyze customer interaction voice data to identify the emotional state of your customers. Based on their emotional state, offer personalized service.

[1452] In this way, the server can provide personalized service based on the emotional state of the customer.

[1453] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1454] Step 1:

[1455] The server acquires a list of customers scheduled to visit the store on that day from the schedule management information.

[1456] Input: Schedule management information

[1457] Output: List of customers scheduled to visit on the day

[1458] Specific operation: The server queries the database for the day's schedule information and generates a list of customers who are scheduled to visit the store.

[1459] Step 2:

[1460] The server converts the customer-interacting voice data into text using voice recognition technology.

[1461] Input: Customer response voice data

[1462] Output: Text data

[1463] What happens: The server loads the audio file and converts the audio data to text using the SpeechRecognition library.

[1464] Step 3:

[1465] The server then uses AI analysis of the converted text data to analyze the customer's emotional state.

[1466] Input: Text data

[1467] Output: Emotional state

[1468] How it works: The server uses the Transformers library to input text data into a sentiment analysis model to analyze the emotional state.

[1469] Step 4:

[1470] The server provides personalized services based on the analyzed emotional state.

[1471] Input: Emotional state, customer profile, preference information, interest in our company's services

[1472] Output: Personalized service offers

[1473] Specific operation: The server takes into account the customer profile, preference information, and level of interest in the company's services, and generates suggestions for service improvements and new services according to the customer's emotional state.

[1474] Step 5:

[1475] The server transmits the generated service proposal to the terminal and notifies the user.

[1476] Input: Personalized service offers

[1477] Output: User notification

[1478] Specific operation: The server sends the generated service proposal to the terminal and calls an API to notify the user.

[1479] Example 3

[1480] Next, a description will be given of Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1481] With conventional customer service systems, it was difficult to accurately grasp customer preferences and their level of interest in the company's services, making it difficult to provide personalized services. Furthermore, there was a lack of means to grasp the customer's emotional state in real time, making it difficult to improve customer satisfaction.

[1482] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[1483] In this invention, the server includes means for comprehensively analyzing customer profiles, schedule management information, customer interaction voice data, and customer preferences and interest in the company's services, means for identifying information about customers visiting the store today using the schedule management information and customer interaction voice data, means for analyzing customer preferences and interest in the company's services using artificial intelligence, means for converting voice data into text data using a voice recognition system, means for estimating the customer's emotional state from the voice data using an emotion engine, means for inputting features extracted from the customer profile information and voice data into an artificial intelligence model to analyze the customer's preferences and interest in the company's services, and means for providing personalized services for each customer based on the analysis results.This makes it possible to accurately grasp customer preferences and interest and estimate the emotional state in real time.

[1484] A "customer profile" is data that includes personal information such as a customer's age, gender, and past purchasing history.

[1485] "Schedule management information" is data related to schedules, such as customer visit schedules and reservation information.

[1486] "Customer response voice data" refers to voice data obtained from call center call records and customer interaction logs.

[1487] "Preferences" are the preferences and interests that customers have for particular products and services.

[1488] "Interest in our company's services" refers to the degree of interest customers have in the products and services offered by our company.

[1489] "Artificial intelligence" is a technology that uses machine learning and data analysis to mimic human intelligence.

[1490] A "voice recognition system" is a technology that converts voice data into text data.

[1491] "Emotion engine" is a technology that estimates a customer's emotional state from voice data.

[1492] An "artificial intelligence model" is a model that uses machine learning algorithms to analyze data and make predictions and classifications.

[1493] "Personalized services" are customized services provided based on a customer's individual preferences and interests.

[1494] This invention is a system that uses artificial intelligence to analyze customer preferences and interest in the company's services, and provides personalized services. Specific embodiments of this system are described below.

[1495] Hardware and software used

[1496] Hardware: Servers, devices (PCs, smartphones, etc.)

[1497] Software: speech recognition systems (e.g., Google Speech-to-Text API), emotion engines (e.g., IBM Watson Tone Analyzer), artificial intelligence models (e.g., TensorFlow, PyTorch)

[1498] Data collection

[1499] The server collects customer interaction voice data from call center call records and customer interaction logs. For example, when a call ends, the voice data is automatically saved on the server.

[1500] The terminal sends the customer profile information entered by the user to the server, which includes the customer's age, gender, past purchase history, etc.

[1501] Audio data processing

[1502] The server sends the collected voice data to a voice recognition system (e.g., Google Speech-to-Text API) and converts it into text data. For example, you can upload the voice file to the API and save the returned text data.

[1503] The server inputs the converted text data into an emotion engine (e.g., IBM Watson Tone Analyzer) to estimate the customer's emotional state. For example, the server sends the text data to the emotion engine and stores the returned emotion analysis results.

[1504] Data analysis

[1505] The server inputs customer profile information and features extracted from the voice data into an AI model (e.g., TensorFlow, PyTorch) to analyze customer preferences and interest in the company's services. For example, the profile information and sentiment analysis results can be input into an AI model to predict customer preferences and interest.

[1506] AI models learn what products and services customers are interested in and what purchasing behaviors they tend to exhibit, for example by training the model using past data and then making predictions on new data.

[1507] Providing results

[1508] The server then uses the analysis results to provide personalized services for each customer, such as generating a list of recommended products and notifying the user.

[1509] The user checks the personalized service provided by the server and takes appropriate action for the customer, for example, presenting a special offer to the customer.

[1510] Specific examples

[1511] For example, suppose a customer makes an inquiry to a call center. The voice data from that conversation is collected and converted into text data using a voice recognition system. Next, the customer's emotional state is estimated using an emotion engine, and the customer's preferences and interest levels are analyzed using an artificial intelligence model. Finally, products that may be of interest to the customer are recommended.

[1512] Prompt Sentence Examples

[1513] "Build an artificial intelligence model that analyzes customer preferences and interest in your services based on customer interaction voice data and customer profile information. Also, use an emotion engine that infers the customer's emotional state from the voice data."

[1514] The above is a specific embodiment for carrying out the present invention. The flow of the identification process in the third embodiment will be described with reference to FIG.

[1515] Step 1:

[1516] Data collection

[1517] The server collects customer interaction voice data from call center call records and customer interaction logs. For example, when a call ends, the voice data is automatically saved on the server.

[1518] Input: Call center call records, customer interaction logs

[1519] Output: Customer interaction voice data (audio file)

[1520] Specific operation: The server automatically collects voice data at the end of the call and stores it in a database.

[1521] Step 2:

[1522] Entering Customer Profile Information

[1523] The terminal sends the customer profile information entered by the user to the server, which includes the customer's age, gender, past purchase history, etc.

[1524] Input: Customer profile information entered by the user

[1525] Output: Customer profile information sent to the server

[1526] Specific operation: When the terminal enters data into the input form, it immediately sends it to the server.

[1527] Step 3:

[1528] Converting audio data to text

[1529] The server sends the collected voice data to a voice recognition system (e.g., Google Speech-to-Text API) and converts it into text data.

[1530] Input: Customer response voice data (audio file)

[1531] Output: Text data

[1532] Specific operation: The server uploads the audio file to the API and stores the returned text data in the database.

[1533] Step 4:

[1534] Emotional state estimation

[1535] The server inputs the converted text data into an emotion engine (e.g., IBM Watson Tone Analyzer) to estimate the customer's emotional state.

[1536] Input: Text data

[1537] Output: Sentiment analysis results

[1538] Specific operation: The server sends text data to the emotion engine and stores the returned emotion analysis results in a database.

[1539] Step 5:

[1540] Data analysis

[1541] The server inputs customer profile information and features extracted from voice data into an artificial intelligence model (e.g., TensorFlow, PyTorch) to analyze customer preferences and interest in the company's services.

[1542] Input: Customer profile information, sentiment analysis results

[1543] Output: Analysis of preferences and interest levels

[1544] Specific operation: The server inputs profile information and sentiment analysis results into an artificial intelligence model to predict customer preferences and interests.

[1545] Step 6:

[1546] Providing results

[1547] Based on the analysis results, the server provides personalized services for each customer.

[1548] Input: Analysis results of preferences and interests

[1549] Output: Providing personalized service

[1550] Specific operation: The server generates a list to recommend specific products and notifies the user.

[1551] Step 7:

[1552] User response

[1553] The user checks the personalized service provided by the server and responds appropriately to the customer.

[1554] Input: Personalized service offerings

[1555] Output: Customer response

[1556] Specific Action: The user presents a special offer to a customer.

[1557] (Application example 3)

[1558] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1559] With conventional customer service systems, it was difficult to fully understand customer preferences and emotional states, making it difficult to provide personalized services. Furthermore, there was a lack of means to effectively utilize customer voice data, limiting the improvement of customer satisfaction. This meant that customers' purchasing motivation could not be maximized, and sales did not increase.

[1560] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[1561] In this invention, the server includes means for comprehensively analyzing customer profiles, schedule management information, customer interaction voice data, and customer preferences and interest in the company's services, means for identifying information about customers visiting the store today using the schedule management information and customer interaction voice data, means for analyzing customer preferences and interest in the company's services using AI, a voice recognition system for converting customer voice input into text, an emotion engine for estimating the customer's emotional state from the voice data, and means for personalized product recommendations based on the customer's preferences and emotional state. This makes it possible to grasp the customer's preferences and emotional state in real time and recommend optimal products.

[1562] A "customer profile" is data that includes basic information about a customer, their past purchasing history, preferences, etc.

[1563] "Schedule management information" is data related to time, such as customer visit schedules and reservation information.

[1564] "Customer response voice data" refers to voice recordings of conversations and inquiries with customers.

[1565] "Preferences" are the preferences and interests that customers have for particular products and services.

[1566] "Interest in our company's services" refers to the degree of interest that customers have in the products and services that our company offers.

[1567] A "voice recognition system" is a technology that converts voice data into text.

[1568] "Emotion engine" is a technology that estimates a customer's emotional state from voice data.

[1569] "Personalized product recommendation" means suggesting the most suitable products to each customer based on their preferences and emotional state.

[1570] "AI analysis" refers to the use of artificial intelligence to analyze data and evaluate customer preferences and interests.

[1571] A system for implementing this invention includes a means for comprehensively analyzing customer profiles, schedule management information, customer interaction voice data, and customer preferences and interest in the company's services. A server collects this data and analyzes the customer's preferences and interest using AI. Furthermore, a voice recognition system converts the customer's voice input into text, and an emotion engine estimates the customer's emotional state from the voice data. This allows personalized product recommendations to be made based on the customer's preferences and emotional state.

[1572] Hardware and software used

[1573] Hardware: Servers, user devices (smartphones, tablets, PCs, etc.)

[1574] Software: speech recognition systems (e.g., Google Speech Recognition API), sentiment analysis models (e.g., sentiment analysis models from the Transformers library), database management systems (e.g., MySQL)

[1575] Data processing and calculation

[1576] 1. Speech recognition: Customer voice data collected on the user terminal is sent to the server and converted into text using a voice recognition system.

[1577] 2. Sentiment analysis: The converted text data is fed into a sentiment engine to estimate the customer's emotional state.

[1578] 3. Preference analysis: The server uses AI to analyze customer preferences based on customer profiles, past purchase history, and browsing history.

[1579] 4. Product Recommendation: Based on the analysis results, recommend the most suitable products to the customer.

[1580] Specific examples

[1581] For example, if a customer says, "I've been wanting to buy some smartphone accessories lately," the speech recognition system converts this speech into text. The emotion engine then uses this text to estimate the customer's emotional state as "positive." The server then recommends accessories such as smartphone cases, chargers, and earphones based on the customer's past purchase and browsing history.

[1582] Prompt Sentence Examples

[1583] Enter the customer: I've recently been wanting some accessories for my smartphone.

[1584] In this way, it becomes possible to understand the customer's preferences and emotional state in real time and recommend the most suitable products.

[1585] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[1586] Step 1:

[1587] The user inputs voice into the terminal. For example, the user might say, "Recently, I've been thinking about buying some accessories for my smartphone." The terminal collects this voice data and sends it to the server.

[1588] Input: User's voice data

[1589] Output: Audio data sent to the server

[1590] Step 2:

[1591] The server converts the received voice data into text using a voice recognition system (e.g., Google Speech Recognition API), which analyzes the voice data and generates corresponding text data.

[1592] Input: Audio data

[1593] Output: Text data

[1594] Step 3:

[1595] The server inputs the generated text data into an emotion engine to estimate the customer's emotional state. The emotion engine (e.g., a sentiment analysis model from the Transformers library) analyzes the text data and outputs an emotional state (e.g., positive, negative, or neutral).

[1596] Input: Text data

[1597] Output: Emotional state

[1598] Step 4:

[1599] The server uses AI to analyze customer preferences based on customer profiles, past purchase history, and browsing history. The necessary information is retrieved from the customer profile database, and the AI ​​model analyzes this data to identify customer preferences.

[1600] Input: Customer profile, purchase history, browsing history

[1601] Output: Customer preference data

[1602] Step 5:

[1603] The server makes personalized product recommendations based on the customer's preference data and emotional state. The AI ​​model considers the customer's preferences and emotional state to select the most suitable products and generate a recommendation list.

[1604] Input: Customer preference data, emotional state

[1605] Output: Recommended products list

[1606] Step 6:

[1607] The server transmits the generated recommended product list to the user terminal, which displays the received recommended product list and proposes personalized products to the user.

[1608] Input: Recommended product list

[1609] Output: Recommended product list displayed on the user's device

[1610] In this way, it becomes possible to understand the customer's preferences and emotional state in real time and recommend the most suitable products.

[1611] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1612] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in the form of data such as voice data and text data.

[1613] Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1614] Another example of generative AI is Gemini (internet search engine). <url: https: gemini.google.com ?hl="ja">) are mentioned.

[1615] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1616] [Third embodiment]

[1617] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1618] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1619] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1620] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1621] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1622] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1623] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1624] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1625] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1626] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1627] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1628] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[1629] "Example 1"

[1630] In one embodiment of the present invention, a speech recognition system is provided for acquiring customer-facing speech data. The speech recognition system acquires speech data input from a microphone installed in the store in real time and converts the data into text data. The text data includes what the customer is saying and other speech information.

[1631] "Example 2"

[1632] Furthermore, a system is provided that uses schedule management information and customer interaction voice data to identify customer information for that day. Specifically, a list of customers scheduled to visit the store that day is created based on schedule management information, and the list is compared with customer interaction voice data to identify which customer corresponds to which voice data.

[1633] "Example 3"

[1634] The company also has a system that uses AI to analyze customer preferences and interest in its services. This system analyzes customer preferences and interest in its services based on features extracted from customer interaction voice data and customer profiles. Specifically, the AI ​​learns what products and services customers are interested in and what purchasing behavior they tend to exhibit, and provides personalized services for each customer based on that information.

[1635] The processing flow of each embodiment will be described below.

[1636] "Example 1"

[1637] Step 1: Acquire audio data in real time from microphones installed in the store.

[1638] Step 2: Convert the acquired audio data into text data.

[1639] Step 3: Save the converted text data and use it for further processing.

[1640] "Example 2"

[1641] Step 1: Create a list of customers who are scheduled to visit the store that day based on scheduling information.

[1642] Step 2: Compare the list you created with customer interaction voice data.

[1643] Step 3: From the comparison results, identify which customer corresponds to which voice data.

[1644] "Example 3"

[1645] Step 1: Extract features from customer interaction voice data and customer profiles.

[1646] Step 2: Based on the extracted features, the AI ​​analyzes the customer's preferences and level of interest in the company's services.

[1647] Step 3: Provide personalized services to each customer based on the results of AI analysis.

[1648] Example 1

[1649] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1650] Conventional customer support systems have difficulty accurately grasping customer comments and preferences in real time, making it difficult to respond to customer needs promptly. Furthermore, there is a lack of efficient means for managing and analyzing voice data, making it difficult to provide personalized services based on customer preferences and interests.

[1651] The identification process by the identification processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means. In this invention, the server includes: a means for comprehensively analyzing customer profiles, schedule management information, customer response voice data, and customer preferences and interest in the company's services; a means for identifying information about customers visiting the store today using the schedule management information and customer response voice data; a means for analyzing customer preferences and interest in the company's services using artificial intelligence; a means for acquiring voice data in real time from a voice input device installed in the store; a means for converting the acquired voice data into text data using voice recognition technology; a means for storing the converted text data in a database; and a means for analyzing the stored text data using natural language processing technology. This makes it possible to accurately grasp the content of customer comments and preferences in real time and respond quickly.

[1652] A "customer profile" is a collection of data that includes basic information about a customer, past purchase history, preferences, interests, etc.

[1653] "Schedule management information" is data for managing customer visit schedules, reservation information, past visit history, and the like.

[1654] "Customer interaction voice data" is voice data including conversations between customers and store clerks and the content of what customers say.

[1655] "Customer preferences" refers to information such as the products and services that customers prefer and the areas in which they are interested.

[1656] "Interest in our company's services" is an indicator that shows how interested customers are in the products and services that our company offers.

[1657] "Artificial intelligence" is a technology that allows computers to learn and reason by imitating human intelligence.

[1658] An "audio input device" is a device for acquiring audio as digital data, and generally refers to a microphone.

[1659] "Speech recognition technology" is a technology for converting voice data into text data.

[1660] "Text data" is data expressed as character information.

[1661] A "database" is a system for efficiently storing, managing, and retrieving data.

[1662] "Natural language processing technology" is a technology that allows computers to understand, interpret, and generate human language.

[1663] This invention relates to a system for acquiring and analyzing customer interaction voice data. Specifically, this system acquires voice data in real time from a voice input device installed in a store, converts it into text data using voice recognition technology, stores the text data in a database, and analyzes it using natural language processing technology.

[1664] Hardware and software used

[1665] The server uses the following hardware and software:

[1666] Voice input device: Uses microphones installed in the store to capture high-quality conversations between customers and store staff.

[1667] Speech recognition technology: The Google Cloud Speech-to-Text API is used to convert captured voice data into text data.

[1668] Database: A MySQL database is used to store the converted text data.

[1669] Natural language processing techniques: Analyze stored text data using natural language processing toolkits such as NLTK and SpaCy.

[1670] Specific operation of the system

[1671] The server collects voice data from multiple microphones installed in various areas of the store. These microphones capture conversations between customers and store staff in real time. For example, microphones installed near the cash register and shelves pick up customer questions and requests.

[1672] The collected voice data is sent to the Google Cloud Speech-to-Text API, which uses advanced speech recognition algorithms to convert voice data into text. For example, if a customer says, "I'd like to know more about this product," the API will convert this speech into text: "I'd like to know more about this product."

[1673] The converted text data is stored in a MySQL database, which is designed to efficiently manage customer comments and other voice information. For example, if a customer says, "I'd like to know more about this product," the text data is stored in a specific table in the database.

[1674] The stored text data is analyzed using natural language processing toolkits (e.g., NLTK or SpaCy). This analysis can identify customer needs and problems. For example, if multiple customers say, "I want to know more about this product," the server aggregates that information and suggests that there is a lack of information about a particular product.

[1675] Specific examples

[1676] For example, if a customer says, "I'd like to know more about this product," the server captures this speech through a microphone and uses the Google Cloud Speech-to-Text API to convert it into text data that says, "I'd like to know more about this product." This text data is then stored in a MySQL database and analyzed using natural language processing tools. The analysis may reveal that information about a particular product is lacking, and the store can take steps to provide more information about that product.

[1677] Prompt Sentence Examples

[1678] Examples of prompts to be input to a generative AI model include:

[1679] "Design a system to capture customer-facing voice data in real time and convert it into text data. Specifically, voice data will be captured from microphones installed in stores and converted into text data using the Google Cloud Speech-to-Text API. The converted text data will be stored in a MySQL database and analyzed using natural language processing tools."

[1680] By inputting this prompt into a generative AI model, specific advice and code examples can be obtained on system design and implementation.

[1681] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1682] Step 1: Acquire audio data

[1683] The server collects real-time audio data from multiple microphones installed in the store. These microphones are placed in various areas of the store and capture conversations between customers and store staff in high quality. The input is the audio signal from the microphones, and the output is digital audio data. For example, microphones installed near the cash register or shelves pick up customer questions and requests.

[1684] Step 2: Convert the audio data

[1685] The server sends the captured voice data to the Google Cloud Speech-to-Text API. This API uses advanced speech recognition algorithms to convert voice data into text data. The input is digital voice data, and the output is text data. For example, if a customer says, "I'd like to know more about this product," the API converts this speech into text: "I'd like to know more about this product."

[1686] Step 3: Save the text data

[1687] The server stores the converted text data in a MySQL database. This database is designed to efficiently manage customer utterances and other voice information. The input is text data, and the output is a record stored in the database. For example, if a customer utters, "I'd like to know more about this product," the text data is stored in a specific table in the database.

[1688] Step 4: Analyze the data

[1689] The server analyzes the stored text data using a natural language processing toolkit (e.g., NLTK or SpaCy). This analysis can identify customer needs and problems. The input is the text data stored in the database, and the output is the analysis results. For example, if multiple customers say, "I want to know more about this product," the server aggregates that information and suggests that there is a lack of information about a particular product.

[1690] Step 5: Personalized service delivery

[1691] Based on the analysis results, the server provides personalized services based on the customer's preferences and interest in the store's services. The input is the analysis results, and the output is the proposal or provision of personalized services. For example, if it is determined that information about a specific product is lacking, the store can take measures to provide detailed information about that product.

[1692] (Application example 1)

[1693] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1694] Conventional customer service systems have had the problem of being unable to grasp customer comments in real time and provide appropriate responses quickly. Furthermore, personalized service based on customer preferences and interests was not provided adequately, making it difficult to improve customer satisfaction. Furthermore, there was a lack of support for store staff to respond quickly and appropriately to customers.

[1695] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1696] In this invention, the server includes: a means for comprehensively analyzing customer profiles, schedule management information, customer response voice data, and customer preferences and interest in the company's services; a means for identifying today's customer information using schedule management information and customer response voice data; a means for analyzing customer preferences and interest in the company's services using AI; a means for converting voice data into text data in real time; a means for analyzing the text data and generating appropriate responses to customer questions and requests; and a means for displaying the generated responses to support rapid customer responses. This enables the content of customer comments to be understood in real time and appropriate responses to be provided promptly. This also enables personalized service based on customer preferences and interests, which is expected to improve customer satisfaction. Furthermore, this system provides enhanced support for store staff to quickly provide appropriate responses when serving customers.

[1697] A "customer profile" is data that includes basic information about a customer, their past purchasing history, preferences, etc.

[1698] "Schedule management information" is data related to time, such as customer visit schedules and reservation information.

[1699] "Customer interaction voice data" is voice data including the content of the conversation between the customer and the store clerk.

[1700] "Customer preferences and interest in our company's services" is data that indicates the products and services that customers prefer and the level of interest in them.

[1701] A "voice recognition system" is a system that converts voice data into text data.

[1702] "AI analytics" is the process of using artificial intelligence to analyze data and extract patterns and trends.

[1703] "Real-time" means that data is processed as it is generated.

[1704] "Text data" is character information converted from voice data.

[1705] "Analysis" is the process of examining data in detail to understand its meaning and structure.

[1706] "Appropriate response" refers to the most appropriate response or action to a customer's question or request.

[1707] "Personalized service provision" means providing customized services based on the individual preferences and interests of customers.

[1708] "Quick customer response" refers to responding quickly to customer requests.

[1709] The system for carrying out this invention includes a means for comprehensively analyzing customer profiles, schedule management information, customer interaction voice data, and customer preferences and interest in the company's services. The system is realized using the following hardware and software.

[1710] Hardware

[1711] Smartphone: A device used by store associates to capture and display voice data.

[1712] Microphone: A device installed in the store to collect customer comments.

[1713] software

[1714] Python: A programming language used to implement programs.

[1715] SpeechRecognition library: A library for converting voice data into text data.

[1716] OpenAI API: An artificial intelligence model for analyzing text data and generating appropriate responses.

[1717] Data processing and calculation

[1718] 1. Voice data capture: A microphone collects what the customer says and sends it to a smartphone.

[1719] 2. Converting speech data to text: Use the SpeechRecognition library to convert speech data to text in real time.

[1720] 3. Text data analysis: Use OpenAI API to analyze text data and generate appropriate responses to customer questions and requests.

[1721] 4. Display of response: The generated response is displayed on the smartphone, allowing the store clerk to quickly respond to the customer.

[1722] Specific examples

[1723] For example, if a customer says, "Tell me more about this product," the system operates as follows:

[1724] 1. A microphone collects what the customer says and sends it to a smartphone.

[1725] 2. The SpeechRecognition library converts the audio data into text data.

[1726] 3. The OpenAI API analyzes the text data and generates a response like this:

[1727] "This product uses the latest technology and is particularly durable. For more information, please see this brochure."

[1728] 4. The generated response is displayed on the smartphone, and the store clerk responds promptly to the customer.

[1729] Prompt Sentence Examples

[1730] Customers say: Tell me more about this product

[1731] Please suggest how the store clerk should respond.

[1732] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1733] Step 1:

[1734] Acquiring audio data

[1735] The server collects customer speech from microphones installed in the store. The microphones capture voice data in real time and send it to a smartphone. The input is the customer's speech, and the output is voice data. This voice data is converted into text data in a subsequent processing step.

[1736] Step 2:

[1737] Converting audio data to text

[1738] The device (smartphone) uses the SpeechRecognition library to convert the received voice data into text data in real time. The input is voice data and the output is text data. In this conversion process, voice recognition technology is used to convert the voice content into text information.

[1739] Step 3:

[1740] Text data analysis

[1741] The server uses the OpenAI API to analyze the text data. The input is text data, and the output is the analysis results. Specifically, it understands customer questions and requests based on the text data and generates appropriate responses. This analysis process uses a generative AI model to perform natural language processing.

[1742] Step 4:

[1743] Generate correspondence

[1744] The server generates an appropriate response to the customer's question or request based on the analysis results. The input is the analysis results, and the output is the generated response. The generated response includes a specific reply or instruction for action based on the content of the customer's statement.

[1745] Step 5:

[1746] Display of correspondence

[1747] The terminal (smartphone) displays the generated response to the store clerk. The input is the generated response, and the output is the text displayed on the smartphone screen. The store clerk can check this display and respond to the customer quickly.

[1748] Specific examples

[1749] For example, if a customer says, "Tell me more about this product," the system operates as follows:

[1750] 1. Step 1: The microphone captures the customer's utterance, "Tell me more about this product," and sends it to the smartphone.

[1751] 2. Step 2: The smartphone converts the voice data into text data: "Tell me more about this product."

[1752] 3. Step 3: The server analyzes the text data and generates an appropriate response to the customer's question.

[1753] 4. Step 4: The server generates a response saying, "This product uses the latest technology and is particularly durable. If you need more information, please see this brochure."

[1754] 5. Step 5: The smartphone displays the generated response to the store associate, who then responds promptly to the customer.

[1755] Prompt Sentence Examples

[1756] Customers say: Tell me more about this product

[1757] Please suggest how the store clerk should respond.

[1758] Example 2

[1759] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1760] With conventional customer management systems, it was difficult to effectively integrate schedule management information and customer interaction voice data to identify customer information. Furthermore, there was a lack of means to accurately understand customer preferences and interest in the company's services, and to provide personalized services. This hindered the efficiency of customer interactions and the improvement of service quality.

[1761] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1762] In this invention, the server includes means for acquiring schedule management information, means for acquiring voice data, means for converting voice data into text data, means for comparing the acquired schedule management information with the converted text data and identifying customer information, means for saving and notifying the identified customer information, and means for analyzing customer preferences and interest in the company's services using artificial intelligence. This makes it possible to integrate schedule management information and customer interaction voice data to identify customer information and further analyze customer preferences and interest to provide personalized services.

[1763] A "customer profile" is a collection of data that includes basic information about a customer, past transaction history, preferences, etc.

[1764] "Schedule management information" is a collection of data including customer visit schedules and reservation information.

[1765] "Customer response voice data" refers to voice recordings of conversations and inquiries with customers.

[1766] "Customer preferences" refers to information that indicates the types and characteristics of products and services that customers prefer.

[1767] The "degree of interest in our company's services" is information indicating the degree of interest that customers have in the services provided by our company.

[1768] "Artificial intelligence" is a technology that uses machine learning and data analysis to imitate human intellectual tasks.

[1769] A "voice recognition system" is a technology that analyzes voice data and converts it into text data.

[1770] "Personalized services" are customized services provided based on a customer's individual preferences and interests.

[1771] A "server" is a computer system that processes, stores, and communicates data.

[1772] "Text data" refers to data obtained by converting voice data into character information.

[1773] A "database" is a system for efficiently storing, searching, and managing data.

[1774] "Notification" is the act of notifying a user or other system of specific information.

[1775] This invention is a system for comprehensively analyzing customer profiles, schedule management information, customer interaction voice data, and customer preferences and interest in the company's services. A specific embodiment of this system is described below.

[1776] Hardware and software used

[1777] Scheduling software: Google Calendar

[1778] Audio data storage database: Amazon S3

[1779] Speech recognition software: Google Cloud Speech-to-Text

[1780] Database: MySQL

[1781] Server Processing

[1782] 1. Acquiring schedule management information

[1783] The server uses the Google Calendar API to obtain a list of customers scheduled to visit the store that day.

[1784] Specifically, the server sends a request to the Google Calendar API and receives JSON-formatted data as a response.

[1785] 2. Acquisition of customer interaction voice data

[1786] The server downloads customer response voice data from Amazon S3.

[1787] Specifically, the server accesses an Amazon S3 bucket and retrieves the specified audio file.

[1788] 3. Converting voice data to text

[1789] The server converts the audio data into text data using the Google Cloud Speech-to-Text API.

[1790] Specifically, the server sends the audio file to the Google Cloud Speech-to-Text API and receives text data in response.

[1791] 4. Compare customer lists with text data

[1792] The server compares the acquired customer list with the converted text data and identifies which customer corresponds to which voice data.

[1793] Specifically, the server analyzes keywords and phrases in the text data and compares them with information in the customer list.

[1794] 5. Storage of Identified Customer Information and Notification

[1795] The server stores the identified customer information in a MySQL database and notifies the terminal.

[1796] Specifically, the server executes an INSERT query in the MySQL database and notifies the terminal by sending an HTTP request.

[1797] Terminal handling

[1798] 1. Receiving customer list

[1799] The terminal receives the list of customers scheduled to visit the store on that day sent from the server.

[1800] Specifically, the terminal receives an HTTP response from the server and parses the JSON-formatted data.

[1801] 2. Receiving and playing audio data

[1802] The terminal receives the customer response voice data transmitted from the server and plays back the voice data.

[1803] Specifically, the device downloads the audio file and plays it using the built-in media player.

[1804] 3. Display of identified customer information

[1805] The terminal displays the identified customer information sent from the server so that the user can confirm it.

[1806] Specifically, the terminal displays the received data on a GUI (graphical user interface).

[1807] User operations

[1808] 1. Check your customer list

[1809] The user checks the list of customers scheduled to visit the store that day displayed on the terminal.

[1810] Specifically, the user looks at the screen of the terminal and checks the customer information displayed in the list.

[1811] 2. Check the audio data

[1812] The user listens to the customer service voice data reproduced from the terminal.

[1813] Specifically, the user listens to the audio data through the terminal's speaker or headphones.

[1814] 3. Confirmation and response of identified customer information

[1815] The user checks the specified customer information displayed on the terminal and takes action as necessary.

[1816] Specifically, the user responds to the customer based on the displayed information.

[1817] Specific examples

[1818] Prompt Sentence Examples

[1819] "Compare the list of customers scheduled to visit obtained from the schedule management information with the customer response voice data stored in the voice data storage database, and identify which customer corresponds to which voice data."

[1820] In this way, the system uses schedule management information and customer interaction voice data to identify information about customers visiting the store that day, and further analyzes their preferences and interests to provide personalized services.

[1821] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1822] Step 1:

[1823] Obtaining schedule management information

[1824] The server uses the schedule management software's API to obtain a list of customers scheduled to visit the store that day. Specifically, the server sends a request to the API and receives JSON-formatted data as a response.

[1825] Input: API request

[1826] Output: A list of customers in JSON format

[1827] Specific operation: The server sends an HTTP GET request to the API endpoint and parses the JSON data received as a response to extract the customer list.

[1828] Step 2:

[1829] Acquisition of customer interaction voice data

[1830] The server downloads the customer response voice data from the voice data storage database. Specifically, the server accesses a bucket in the database and acquires a specified voice file.

[1831] Input: Database bucket information

[1832] Output: Audio file

[1833] Specific operation: The server obtains a list of audio files using the database API and downloads the necessary audio files.

[1834] Step 3:

[1835] Converting audio data to text

[1836] The server uses speech recognition software to convert the audio data into text data. Specifically, the server sends the audio file to a speech recognition API and receives the text data in response.

[1837] Input: Audio file

[1838] Output: Text data

[1839] Specific operation: The server uploads the audio file to the API, performs speech recognition processing, and obtains text data.

[1840] Step 4:

[1841] Comparing customer lists and text data

[1842] The server compares the acquired customer list with the converted text data to identify which customer corresponds to which voice data. Specifically, the server analyzes keywords and phrases in the text data and matches them with the information in the customer list.

[1843] Input: Customer list, text data

[1844] Output: Identified customer information

[1845] Specific operation: The server analyzes the text data to identify parts that match names and reservation information in the customer list, and extracts the corresponding customer information.

[1846] Step 5:

[1847] Retention of identified customer information and notification

[1848] The server stores the identified customer information in a database and notifies the terminal. Specifically, the server executes an INSERT query in the database and notifies the terminal by sending an HTTP request.

[1849] Input: Identified customer information

[1850] Output: Save to database, notify to terminal

[1851] Specific operations: The server stores the identified customer information in a database, and generates and sends an HTTP request to send a notification to the terminal.

[1852] Step 6:

[1853] Receive customer list

[1854] The terminal receives the list of customers scheduled to visit the store that day sent from the server. Specifically, the terminal receives the HTTP response from the server and parses the JSON format data.

[1855] Input: HTTP response from the server

[1856] Output: Parsed customer list

[1857] Specific operation: The terminal analyzes the received HTTP response, extracts the customer list, and displays it.

[1858] Step 7:

[1859] Receiving and playing audio data

[1860] The terminal receives the customer response voice data transmitted from the server and plays the voice data. Specifically, the terminal downloads the voice file and plays it on a built-in media player.

[1861] Input: Audio file

[1862] Output: The audio being played

[1863] Specific operations: The device downloads the audio file, launches the media player, and plays the audio.

[1864] Step 8:

[1865] View identified customer information

[1866] The terminal displays the specified customer information sent from the server so that the user can check it. Specifically, the terminal displays the received data on a GUI.

[1867] Input: Identified customer information

[1868] Output: Displayed customer information

[1869] Specific operation: The terminal displays the identified customer information on the GUI so that the user can check it.

[1870] Step 9:

[1871] Check the customer list

[1872] The user checks the list of customers scheduled to visit the store that day, which is displayed on the terminal. Specifically, the user looks at the screen of the terminal and checks the customer information displayed in the list.

[1873] Input: Displayed customer list

[1874] Output: Confirmed customer information

[1875] Specific operations: The user checks the customer list on the terminal screen and takes action as necessary.

[1876] Step 10:

[1877] Checking audio data

[1878] The user listens to the customer service voice data reproduced from the terminal, specifically, through the terminal's speaker or headphones.

[1879] Input: Audio to be played

[1880] Output: Confirmed audio data

[1881] Specific operation: The user listens to the audio data through the device's speaker or headphones and checks the content.

[1882] Step 11:

[1883] Confirmation and response of identified customer information

[1884] The user checks the specified customer information displayed on the terminal and responds as necessary. Specifically, the user responds to the customer based on the displayed information.

[1885] Input: Displayed customer information

[1886] Output: Customer information handled

[1887] Specific operations: The user checks the identified customer information displayed on the terminal and takes action as necessary to assist the customer.

[1888] (Application example 2)

[1889] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1890] Conventional customer management systems required a lot of manual work to identify customers who were planning to visit the store, which led to problems with inefficiency. It was also difficult to provide personalized services based on customer preferences and interests, making it difficult to improve customer satisfaction.

[1891] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for comprehensively analyzing customer profiles, schedule management information, customer response voice data, and customer preferences and interest in the company's services, means for identifying information about customers visiting today using the schedule management information and customer response voice data, means for analyzing customer preferences and interest in the company's services using AI, means for converting voice data to text, means for comparing the converted text data with the schedule management information to identify customers visiting the store, and means for displaying the identified customer information. This makes it possible to automatically identify information about customers who are scheduled to visit the store and provide personalized services based on the customers' preferences and interests.

[1892] A "customer profile" is data that includes basic information about a customer, past purchase history, preferences, etc.

[1893] "Schedule management information" is data including customer visit schedules and reservation information.

[1894] "Customer response voice data" refers to voice recordings of conversations with customers and inquiries.

[1895] "Customer preferences" refers to information that indicates the types and characteristics of products and services that customers prefer.

[1896] "Interest in our company's services" is an indicator that shows how interested customers are in the services provided by our company.

[1897] "Comprehensive analysis methods" are methods that integrate and analyze multiple data sets to understand customer behavior and preferences.

[1898] "Means of identification" refers to a method for identifying and clarifying specific information or data.

[1899] "AI analysis" refers to the use of artificial intelligence technology to analyze data and find patterns and trends.

[1900] "Means for converting voice data into text" refers to a method for converting voice into text information using voice recognition technology.

[1901] A "comparison tool" is a method for comparing different data and finding commonalities and differences.

[1902] The "display means" refers to a method for visually presenting the analysis results and the identified information.

[1903] The system for carrying out the present invention operates in cooperation with three entities: a server, a terminal, and a user. A specific embodiment of the system will be described below.

[1904] Server Roles

[1905] The server has the ability to comprehensively analyze customer profiles, schedule management information, customer interaction voice data, and customer preferences and interest in the company's services. The server integrates this data and performs analysis using AI technology. Specifically, it uses a voice recognition system to convert voice data into text, and compares the converted text data with schedule management information to identify visiting customers.

[1906] Device Role

[1907] The terminal (e.g., a smartphone) displays the identified customer information sent from the server. The terminal is designed so that the user can check customer information in real time, enabling the user to respond to the customer promptly and appropriately.

[1908] User Roles

[1909] The user can check the customer information identified through the terminal and provide personalized service based on the customer's preferences and interest in the company's services.

[1910] Hardware and software used

[1911] Speech recognition system: Converts voice data into text using the Python speech_recognition library.

[1912] AI technology: Uses machine learning models to analyze customer preferences and interest in your services.

[1913] Database: We use a SQL database to store customer profiles and scheduling information.

[1914] Device: A mobile device such as a smartphone or tablet.

[1915] Specific examples

[1916] For example, based on the schedule management information for a certain day, the server creates a list of customers scheduled to visit that day. Next, the server converts customer interaction voice data into text using a voice recognition system, and identifies the customers by comparing the converted text data with the schedule management information. The identified customer information is sent to the terminal, allowing the user to check it in real time.

[1917] Example prompt sentence:

[1918] "Convert an audio file to text. The path to the audio file is 'path / to / audio / file.wav'."

[1919] In this way, by linking the server, terminal, and user, it becomes possible to automatically identify customer information about customers who are planning to visit the store and provide personalized services based on the customer's preferences and interest.

[1920] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1921] Step 1:

[1922] The server obtains schedule management information. As input, it obtains a list of customers scheduled to visit the store that day from the database. As output, it generates a list of customers scheduled to visit the store. This list includes information such as customer ID, name, and scheduled visit time.

[1923] Step 2:

[1924] The server acquires customer response voice data. As input, the path of the voice file is specified. As output, voice data is acquired. This voice data is a record of conversations and inquiries with customers.

[1925] Step 3:

[1926] The server uses a speech recognition system to convert the speech data into text. The acquired speech data is used as input. As output, text data is generated. This text data is a representation of the content of the speech data as character information.

[1927] Step 4:

[1928] The server compares the converted text data with the schedule management information to identify customers. It uses the text data and a list of customers who are scheduled to visit as input. It generates information about identified customers as output. This information includes customer IDs, names, and scheduled visit times.

[1929] Step 5:

[1930] The server sends the identified customer information to the terminal. It uses the identified customer information as input and generates customer information that is sent to the terminal as output. This information is converted into a format for display on the terminal.

[1931] Step 6:

[1932] The terminal receives and displays the customer information sent from the server. As input, it uses the customer information sent from the server. As output, it displays the information in a format that can be viewed by the user. This allows the user to view the customer information in real time.

[1933] Step 7:

[1934] The user checks the customer information identified through the terminal and responds to the customer. The customer information displayed on the terminal is used as input. The output is to respond appropriately to the customer. This response includes providing personalized services based on the customer's preferences and level of interest in the company's services.

[1935] Example 3

[1936] Next, a third embodiment of the third embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1937] With conventional customer service systems, it was difficult to accurately grasp customer preferences and interests and provide personalized services. Furthermore, there was a lack of means to effectively utilize customer service voice data, making it impossible to respond quickly to customer needs. This made it difficult to improve customer satisfaction and service quality.

[1938] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[1939] In this invention, the server includes means for comprehensively analyzing customer profiles, schedule management information, customer response voice data, and customer preferences and interest in the company's services, means for identifying information about customers visiting the store today using the schedule management information and customer response voice data, means for converting the customer response voice data into text data using voice recognition software, means for analyzing customer preferences and interest in the company's services using AI based on features extracted from the customer profiles and voice data, and means for providing personalized services for each customer. This makes it possible to accurately grasp customer preferences and interest and quickly provide personalized services.

[1940] A "customer profile" is data that includes basic information about a customer, past purchasing history, preferences, and interest levels.

[1941] "Schedule management information" is data related to customer visit plans, reservation information, and schedules.

[1942] "Customer interaction voice data" is voice data of a conversation between a customer and a user.

[1943] "Speech recognition software" is software for converting voice data into text data.

[1944] "Features" are data extracted from voice data and customer profiles that indicate customer preferences and interests.

[1945] "AI" is an artificial intelligence technology that uses machine learning models and natural language processing tools to analyze data and make predictions and classifications.

[1946] "Personalized service" refers to a service that is provided individually based on a customer's preferences and interests.

[1947] "Visiting customer information" is information about customers who are planning to visit the store today.

[1948] This invention relates to a system that uses AI to comprehensively analyze customer profiles, schedule management information, and customer interaction voice data to analyze customer preferences and interest in the company's services. Specific embodiments of this system are described below.

[1949] Hardware and software used

[1950] Hardware: Servers, terminals (PCs, smartphones)

[1951] Software: speech recognition software (e.g., Google Cloud Speech-to-Text API), database management systems (DBMS), machine learning models (e.g., TensorFlow or PyTorch), natural language processing (NLP) tools (e.g., spaCy or NLTK)

[1952] System Overview

[1953] 1. Collection of audio data

[1954] When a user interacts with a customer, voice data is collected through a microphone connected to the terminal.

[1955] The device transmits the collected voice data to the server in real time.

[1956] 2. Converting voice data to text

[1957] The server converts the received voice data into text data using voice recognition software.

[1958] 3. Obtaining customer profiles

[1959] The server uses a database management system (DBMS) to obtain profile data such as basic customer information and past purchasing history.

[1960] 4. Feature Extraction

[1961] The server uses natural language processing (NLP) tools to extract features from the text extracted from the voice data and the customer profile data.

[1962] 5. AI-powered analysis

[1963] The server uses a machine learning model to analyze customer preferences and interest in the company's services based on the extracted features.

[1964] 6. Providing Results

[1965] The server sends the analysis results to the terminal and makes personalized service proposals to the user.

[1966] Specific examples

[1967] Example 1: If customer A has purchased many health foods in the past, the AI ​​will determine that customer A is health-conscious and suggest new health foods and related services.

[1968] Prompt: "Analyze Customer A's interest in healthy foods based on their past purchase history and voice data."

[1969] Example 2: If Customer B frequently asks travel-related questions, the AI ​​determines that Customer B is interested in travel and suggests travel-related products and services.

[1970] Prompt: "Extract travel interests from customer B's voice data and suggest related services."

[1971] In this way, the system uses AI to analyze customer preferences and interests based on customer interaction voice data and customer profiles, and provides personalized services. The flow of the identification process in Example 3 will be described with reference to Figure 15.

[1972] Step 1:

[1973] Audio data collection

[1974] When a user interacts with a customer, voice data is collected through a microphone connected to the terminal.

[1975] Input: Audio of conversation with customer

[1976] Output: Collected audio data

[1977] What it does: When a user starts a conversation with a customer, the device's microphone captures the audio and streams that data to the server.

[1978] Step 2:

[1979] Converting audio data to text

[1980] The server converts the received voice data into text data using voice recognition software.

[1981] Input: Collected audio data

[1982] Output: Converted text data

[1983] Specific operation: The server inputs the voice data into speech recognition software (e.g., Google Cloud Speech-to-Text API) and obtains the converted text data.

[1984] Step 3:

[1985] Retrieving a customer profile

[1986] The server uses a database management system (DBMS) to obtain profile data such as basic customer information and past purchasing history.

[1987] Input: Customer ID

[1988] Output: Customer profile data

[1989] What happens: The server queries the database using the customer ID as a key to retrieve relevant customer profile data.

[1990] Step 4:

[1991] Feature Extraction

[1992] The server uses natural language processing (NLP) tools to extract features from the text extracted from the voice data and the customer profile data.

[1993] Input: Text data, customer profile data

[1994] Output: Extracted feature data

[1995] What it does: The server feeds the text data into an NLP tool (e.g., spaCy or NLTK) for keyword extraction and sentiment analysis, and also extracts relevant features from customer profile data.

[1996] Step 5:

[1997] AI-powered analysis

[1998] The server uses a machine learning model to analyze customer preferences and interest in the company's services based on the extracted features.

[1999] Input: Extracted feature data

[2000] Output: Analysis results

[2001] How it works: The server inputs the feature data into a machine learning model (e.g., TensorFlow or PyTorch) to obtain prediction results. The model learns from past data and generates personalized service proposals for each customer.

[2002] Step 6:

[2003] Providing results

[2004] The server sends the analysis results to the terminal and makes personalized service proposals to the user.

[2005] Input: Analysis results

[2006] Output: Personalized service offers

[2007] Specific operation: The server sends the analysis results in JSON format to the device, which then displays them to the user...

Claims

1. A means of comprehensively analyzing customer profiles, schedule management information, customer response voice data, and customer preferences and interest in the company's services; A means for identifying information of customers visiting the store today using schedule management information and customer response voice data; A means for analyzing customer preferences and interest in the company's services based on the customer profile and customer response voice data by inputting a specific prompt sentence into a generation AI model; A means for converting the customer response voice data into text data using a voice recognition system; A means for estimating an emotional state of a customer from the customer-interaction voice data using an emotion engine; an analysis means for inputting the customer profile and features extracted from the customer response voice data into an artificial intelligence model to analyze the customer's preferences and the degree of interest in the company's services; A means for providing a personalized service for each customer based on the analysis result by the analysis means; A system including:

2. Based on the analysis result by the analysis means, The system according to claim 1 , further comprising means for improving services and proposing new services based on the emotional state.

3. The system according to claim 1 , further comprising a means for identifying visiting customers by comparing the text data with schedule management information.

Images