system

Abstract

Provide a system. A means for recording customer operations; a means of recording customer usage; a means for recording customer inquiries; means for pre-processing the recorded data; means for converting the preprocessed data into multimodal data; means for analyzing the converted multimodal data to generate product proposals; The system includes a means for notifying the generated product proposal.

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Conventional methods have the problem of requiring a significant amount of man-hours to effectively collect customer operations, usage status, and inquiries on customer portal sites and to use this information to make product proposals. This reduces the efficiency of sales activities and makes it difficult to respond quickly to customer needs. Furthermore, there is a lack of systems that can centrally manage various data and generate accurate product proposals. The objective of this project is to solve this problem. [Means for solving the problem]

[0005] To solve this problem, the present invention provides the following means: A system is constructed that includes a means for recording customer operation details, a means for recording customer usage status, and a means for recording customer inquiries. Furthermore, the system is provided with a means for preprocessing the recorded data, a means for converting the preprocessed data into multimodal data, a means for analyzing the converted multimodal data to generate product proposals, and a means for notifying the generated product proposals. This makes it possible to quickly and accurately grasp customer needs and improve the efficiency of sales activities.

[0006] "Operation content" refers to the series of actions or behaviors performed by a customer on the portal site.

[0007] "Usage" refers to data about how customers use products and services.

[0008] "Inquiry Content" refers to the content of questions or support requests made by customers through the portal site.

[0009] "Preprocessing" refers to the filtering and transformation of collected data to prepare it in an analyzable format.

[0010] "Multimodal data" refers to composite data that integrates different types of data, such as text data, numerical data, and time data.

[0011] "Analysis" refers to the process of clarifying customer needs and behavioral patterns based on collected data.

[0012] "Product proposal" refers to specific recommendations regarding products or services to be provided to customers.

[0013] "Notification" refers to the act of communicating the generated product proposal to the sales department. [Brief explanation of the drawings]

[0014] [Figure 1]1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0015] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0016] First, the terms used in the following description will be explained.

[0017] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0018] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0019] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0020] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0021] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0022] [First embodiment]

[0023] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0024] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0025] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0026] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0027] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0028] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0029] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0030] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0031] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0032] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0033] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0034] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0035] This invention is a system that effectively collects operation details, usage status, and inquiry details on a customer portal site, generates product proposals based on this data, and notifies the sales department of the proposals.

[0036] System Configuration

[0037] 1. Data Collection

[0038] The server has the function of recording the series of operations that customers perform on the portal site. For example, it collects click information when customers use a certain function and information on the browsing of specific pages. It also records how customers use the product (usage status) and the content of their inquiries.

[0039] 2. Data Preprocessing

[0040] The server filters the collected data to remove unnecessary and duplicated data, such as repeated logs of the same operation and noise data, and normalizes the data into a standard format to facilitate subsequent analysis.

[0041] 3. Data Conversion

[0042] The server converts the preprocessed data into multimodal data, specifically integrating text data (inquiry content), numerical data (usage amount), and time data (operation timing) into a single dataset.

[0043] 4. AI analysis

[0044] The server feeds multimodal data into AI models to analyze customer behavior patterns and needs, for example using machine learning algorithms to predict whether a customer is interested in a particular product feature.

[0045] 5. Generate product proposals

[0046] Based on the results of the AI ​​analysis, the server generates optimal product proposals for customers, such as upgrades and new services related to features that customers frequently inquire about.

[0047] 6. Notification

[0048] The server notifies the sales department of the generated product proposals via an internal email system, a notification dashboard, or by sending a direct message to a specific sales representative.

[0049] Specific examples

[0050] 1. Data Collection

[0051] The server collects operation logs when Customer A starts using the "new subscription service."

[0052] The server also records the inquiries made by Customer A regarding "how to set up this service."

[0053] 2. Data Preprocessing

[0054] The server removes duplicate entries and unnecessary noise from the operation log.

[0055] The server formats the normal log data into a standard format.

[0056] 3. Data Conversion

[0057] The server integrates the inquiry content as text data, the usage amount as numerical data, and the service start date and time as time data.

[0058] 4. AI analysis

[0059] The server inputs this integrated data into an AI model and analyzes Customer A's behavioral patterns.

[0060] The server proposes an upgrade plan with a new setup guide because Customer A frequently asks about "how to set it up."

[0061] 5. Generate product proposals

[0062] The server generates a guided configuration upgrade plan based on AI analysis.

[0063] 6. Notification

[0064] The server notifies the sales representative of the generated proposal by email, and the sales representative contacts Customer A.

[0065] In this way, this system significantly improves the efficiency of sales activities by making efficient and precise product proposals based on customer operations and inquiries.

[0066] The processing flow will be explained below.

[0067] Step 1: Data collection

[0068] The server records the operations performed by customers on the portal site in real time. Specifically, it saves click information and page viewing information when customers use certain functions in a log file. It also records customer usage status (e.g., connection time and data consumption) and inquiry content (e.g., support messages and FAQ search history).

[0069] Step 2: Data Preprocessing

[0070] The server filters the collected log data to remove duplicates and noise, such as duplicate entries of the same operation repeated multiple times and meaningless text (noise). It also converts the data into a standard format that makes it easier to analyze.

[0071] Step 3: Data conversion

[0072] The server converts the preprocessed data into multimodal data. For example, it combines the inquiry content as text data, the usage amount as numerical data, and the operation date and time as time data. This makes it possible to handle different types of data in a unified manner.

[0073] Step 4: AI analysis

[0074] The server then feeds the converted multimodal data into an AI algorithm to analyze customer behavior patterns and needs. Specifically, it uses machine learning models to predict whether a customer wants a specific feature. For example, if a customer frequently inquires about a new feature, it will recommend products or services that include that feature.

[0075] Step 5: Generate product proposals

[0076] The server generates product proposals based on the results of AI analysis. For example, it proposes upgrade plans or new services related to features that customers frequently inquire about. The proposals include the products and services offered, their benefits, and how to use them.

[0077] Step 6: Notification

[0078] The server notifies the sales department of the generated product proposal. Specifically, the proposal details are communicated to the sales representative via the company's internal email system and notification dashboard. The sales representative then follows up with the customer based on this notification.

[0079] These are the specific processing steps of this system, which makes it possible to effectively utilize customer operation and inquiry data and improve the efficiency of sales activities.

[0080] Example 1

[0081] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0082] Conventional customer support systems have difficulty effectively collecting customer operations, usage status, and inquiry details, and quickly and efficiently generating appropriate product proposals based on this data. This has led to problems such as reduced sales efficiency and difficulty in improving customer satisfaction. Furthermore, a lack of technology for precisely analyzing customer behavior patterns has made it difficult to provide accurate proposals. A new system that can solve these issues is needed.

[0083] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0084] In this invention, the server includes means for recording customer operation details, means for recording customer usage status, means for recording customer inquiry details, means for preprocessing the recorded data, means for converting the preprocessed data into multimodal data, means for inputting the converted multimodal data into a generative AI model to analyze customer behavior patterns, means for generating product proposals based on the analysis results, and means for notifying the sales department of the generated product proposals. This makes it possible to effectively collect and preprocess customer data and analyze it using an AI model, thereby enabling accurate and efficient generation of product proposals and notification to the sales department.

[0085] The "means for recording customer operations" is a function for recording operations such as clicks and page views that customers perform on the web portal site.

[0086] "Means for recording customer usage" refers to a function for recording how and to what extent customers use products and services.

[0087] "Means for recording customer inquiries" is a function for recording the inquiries made by customers to support or the help desk in text format.

[0088] "Means for preprocessing recorded data" refers to a function for removing noise and redundant data from collected data and preparing the data in a format suitable for analysis.

[0089] The "means for converting preprocessed data into multimodal data" is a function for integrating preprocessed data into multiple formats, such as text data, numerical data, and time data, to create a single data set.

[0090] "Means for inputting converted multimodal data into a generative artificial intelligence model to analyze customer behavior patterns" refers to a function for inputting multimodal data into an AI model to analyze customer behavior patterns and needs.

[0091] "Means for generating product proposals based on analysis results" refers to a function that automatically creates optimal product and service proposals for customers based on the results of analysis by AI.

[0092] The "means for notifying the sales department of the generated product proposal" is a function for communicating the generated product proposal to the sales department using an appropriate notification means.

[0093] MODE FOR CARRYING OUT THE INVENTION

[0094] This invention is a system that effectively collects operation details, usage status, and inquiry details on a customer portal site, generates product proposals based on this data, and notifies the sales department of the proposals. The configuration and operation of this system will be specifically described.

[0095] 1. Data Collection

[0096] The server has the function of recording a series of operations performed by customers on the portal site. For example, it collects information on customers clicking specific buttons and logs the specific pages they view. It also records how customers use products and services (usage status) and the content of their inquiries. For this purpose, the server uses a log collection module and database.

[0097] Examples:

[0098] The server collects operation logs when Customer A starts using the "new subscription service."

[0099] The server also records the inquiries made by Customer A regarding "how to set up this service."

[0100] 2. Data Preprocessing

[0101] The server filters the collected data to remove unnecessary or redundant data, such as repeated logs of the same operation or noise data, and normalizes the data into a standard format to facilitate subsequent analysis.

[0102] Examples:

[0103] The server removes duplicate entries and unnecessary noise from the operation log.

[0104] The server formats the normal log data into a standard format.

[0105] 3. Data Conversion

[0106] The server converts the preprocessed data into multimodal data, specifically integrating text data (inquiry content), numerical data (usage amount), and time data (operation timing) into a single dataset.

[0107] Examples:

[0108] The server integrates the inquiry content as text data, the usage amount as numerical data, and the service start date and time as time data.

[0109] 4. Analysis using AI models

[0110] The server feeds multimodal data into generative AI models to analyze customer behavior patterns and needs. For example, it uses machine learning algorithms to predict whether a customer is interested in a particular product feature. To do this, the server uses machine learning libraries and AI models.

[0111] Examples:

[0112] The server inputs this integrated data into an AI model and analyzes Customer A's behavioral patterns.

[0113] The server detects that customer A frequently asks about "how to set things up" and proposes a new guided setup upgrade plan.

[0114] 5. Generate product proposals

[0115] The server generates optimal product proposals for customers based on the analysis results of the AI ​​model, such as proposing upgrades or new services related to features that customers frequently inquire about.

[0116] Examples:

[0117] The server generates a guided upgrade plan based on the output of the AI ​​model.

[0118] 6. Notification of Proposal

[0119] The server notifies the sales department of the generated product proposals using the company's internal email system, a notification dashboard, or direct messaging to specific sales representatives.

[0120] Examples:

[0121] The server notifies the sales representative of the generated proposal via email, and the sales representative contacts Customer A and proposes a guided configuration upgrade plan.

[0122] Prompt Sentence Examples

[0123] "Design a system that generates optimal product proposals based on a customer's operation log and inquiry details, and notifies the sales department."

[0124] This system significantly improves the efficiency of sales activities by making efficient and precise product proposals based on customer operations and inquiries.

[0125] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0126] Divide the program's processing flow into processing steps

[0127] Step 1: Data collection

[0128] The server records the series of operations and inquiries that the user makes on the portal site.

[0129] Input: User clicks, page views, inquiries

[0130] Specific operation: When a customer clicks a specific button, the click information is recorded as a log along with a timestamp, and the content submitted in the inquiry form is saved as text data.

[0131] Output: Operation log and inquiry data

[0132] Step 2: Data Preprocessing

[0133] The server filters the collected data, removing unnecessary and redundant data, and normalizes the data into a standard format.

[0134] Input: Operation log and inquiry data

[0135] What it does: It finds and removes duplicate entries and noise data, and converts the remaining data into a standard format such as JSON.

[0136] Output: Preprocessed operation log and query data

[0137] Step 3: Data Integration and Transformation

[0138] The server converts the pre-processed data into multimodal data.

[0139] Input: Preprocessed operation log and query data

[0140] Specific operation: Text data (e.g., inquiry content), numerical data (e.g., usage amount), and time data (e.g., operation timing) are integrated into a single data set.

[0141] Output: Multimodal dataset

[0142] Step 4: Analysis by AI model

[0143] The server feeds the multimodal dataset into a generative AI model to analyze customer behavior patterns.

[0144] Input: Multimodal dataset

[0145] What it does: Multimodal data is fed into an AI model and analyzed using machine learning algorithms.

[0146] Output: Analysis of customer behavior patterns and needs

[0147] Step 5: Generate product proposals

[0148] The server generates optimal product proposals for customers based on the analysis results of the AI ​​model.

[0149] Input: Analysis results of customer behavior patterns and needs

[0150] What it does: Based on the analysis results, it generates recommendations for new services and additional features that customers may be interested in.

[0151] Output: Product proposal data

[0152] Step 6: Proposal Notification

[0153] The server notifies the sales department of the generated product proposal.

[0154] Input: Product proposal data

[0155] What happens next? Communicate proposals to the sales team via email and notification dashboards, sending messages directly to specific salespeople as needed.

[0156] Output: Notification data to the sales department

[0157] (Application example 1)

[0158] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0159] There is a need for a system that can effectively collect and analyze customer behavior data and generate appropriate product proposals to improve the efficiency of sales activities and customer satisfaction. However, current systems do not fully utilize customer behavior data, limiting the accuracy and degree of personalization of product proposals. In particular, there is a lack of a mechanism for integrating and analyzing various customer operations and inquiry data, making it difficult to provide proposals that are optimized for each individual customer.

[0160] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0161] In this invention, the server includes means for recording customer operation details, means for recording customer usage status, means for recording customer inquiry details, means for preprocessing the recorded data, means for converting the preprocessed data into multimodal data, means for analyzing the converted multimodal data to generate product proposals, means for notifying the generated product proposals, analysis means for clustering customer behavior, means for generating personalized product proposals based on the clustering results, and means for notifying the customer of the generated product proposals by email. This makes it possible to comprehensively analyze a variety of customer behavior data and provide optimal product proposals for each individual customer.

[0162] "Means for recording customer operations" refers to means for collecting operation logs such as clicks and page transitions that customers make on websites and applications.

[0163] "Means for recording customer usage" means means for monitoring and recording the extent to which customers use specific features or services.

[0164] The "means for recording customer inquiries" refers to a means for collecting questions and problem reports made by customers to customer support as text data.

[0165] "Means for pre-processing recorded data" refers to means for filtering and normalizing the collected raw data to prepare it for easier subsequent analysis.

[0166] A "means for transforming preprocessed data into multimodal data" is a means for integrating data of different formats (e.g., textual data, numerical data, temporal data) into a single analyzable data set.

[0167] "Means for analyzing converted multimodal data and generating product proposals" refers to means for analyzing data using techniques such as machine learning and generating proposals for products and services that are optimal for customers.

[0168] The "means for notifying the generated product proposal" refers to a means for notifying the customer or the sales department of the generated proposal using a method such as email or a dashboard.

[0169] The "analysis means for clustering customer behavior" is a means for analyzing collected customer behavior data and dividing them into groups of customers with common characteristics.

[0170] The "means for generating personalized product proposals based on clustering results" is a means for proposing individually optimized products and services based on the characteristics of each customer group identified by clustering.

[0171] The "means for notifying the customer of the generated product proposal by email" is a means for directly notifying the customer of the content of the proposal by sending the generated proposal to the customer's email address.

[0172] This invention is a system that collects and analyzes customer behavior data, and generates and notifies individually optimized product proposals. A specific embodiment of this system is described below.

[0173] System Overview

[0174] The system consists of the following main components:

[0175] 1. Data Collection Module

[0176] 2. Data Preprocessing Module

[0177] 3. Data Conversion Module

[0178] 4. AI analysis module

[0179] 5. Product proposal generation module

[0180] 6. Notification Module

[0181] Hardware and Software Used

[0182] Hardware: The system operates using a server or cloud environment, while data collection and notifications are performed using the user's smartphone, tablet, or PC.

[0183] Software: Python and Pandas are used for data processing, and machine learning libraries such as Scikit-learn and TENSORFLOW (registered trademark) are used for analysis. A library (smtplib) that sends emails using the SMTP protocol is used for notifications.

[0184] Program processing explanation

[0185] Data Collection Module

[0186] The server collects user operations, usage status, and inquiries in real time, for example, by recording website click information, page browsing history, and product inquiries.

[0187] Data Preprocessing Module

[0188] The collected data is pre-processed on the server, which removes noise and duplicate data and arranges all data into a standard format. For example, if the same click information is recorded multiple times, it will be combined into one.

[0189] Data Conversion Module

[0190] The preprocessed data is converted into multimodal data, a process that combines text data (e.g., inquiry content), numerical data (e.g., product usage), and temporal data (e.g., operation duration) into a single dataset.

[0191] AI Analysis Module

[0192] The server then inputs the converted multimodal data into an AI model to analyze the user's behavioral patterns and needs, using a clustering algorithm to classify user behavior into groups with common characteristics.

[0193] Product proposal generation module

[0194] Based on the clustering results, the server generates product recommendations optimized for each user group. For example, if a user frequently inquires about how to use a particular product, the server can suggest new accessories related to that product.

[0195] Notification Module

[0196] The generated product proposal will be notified by email. The server will send the generated proposal to the user's registered email address. The email is sent using the SMTP protocol.

[0197] Specific examples

[0198] For example, if a user does the following:

[0199] Viewing a specific product page multiple times

[0200] Make an inquiry about "How to use the product"

[0201] Browse additional related products

[0202] The server collects this data, pre-processes and analyzes it, and then sends the following email to the user based on the analysis results:

[0203] Subject: Personalized product recommendations

[0204] Main text:

[0205] Hello,

[0206] Get recommended products related to your recently viewed products.

[0207] 1. Upgrade kit for product X

[0208] 2. Accessories related to product X

[0209] Please check it out.

[0210] An example prompt for a generative AI model would be:

[0211] Generate optimal product recommendations based on user behavior data, including:

[0212] Click information: / home, / product / 3, / product / 5

[0213] Inquiry: How to return product? Warranty information

[0214] In this way, personalized product proposals based on user-specific data can be made, thereby improving the user experience and streamlining sales activities.

[0215] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0216] Step 1:

[0217] The server collects user operations, usage, and inquiries in real time, for example, logging the links users click on websites, the pages they view, the search terms they use, and the information they enter into inquiry forms. The collected data is stored as raw data.

[0218] Input: User operation logs, usage data, and inquiry details

[0219] Output: Raw data collected

[0220] Step 2:

[0221] The server pre-processes the collected raw data, removing unnecessary and duplicate data and arranging the data into a standard format, specifically combining multiple records of the same click and removing noise data (e.g., error logs).

[0222] Input: Raw data collected

[0223] Output: Preprocessed data

[0224] Step 3:

[0225] The server converts the preprocessed data into multimodal data, a process that integrates data of different formats (e.g., textual, numerical, and temporal data) into a single dataset that facilitates subsequent analysis.

[0226] Input: Preprocessed data

[0227] Output: Multimodal data

[0228] Step 4:

[0229] The server inputs the multimodal data into an AI model to analyze user behavior patterns and needs. A clustering algorithm is used for the analysis, classifying user behavior into groups with common characteristics. For example, KMeans clustering is used to classify customers into multiple groups.

[0230] Input: Multimodal data

[0231] Output: Clustered user groups

[0232] Step 5:

[0233] The server generates personalized product recommendations based on the clustering results. This process involves suggesting the best products and services for each user group. For example, if a user is interested in a particular product, it will suggest accessories and upgrades related to that product.

[0234] Input: Clustered user groups

[0235] Output: Personalized product proposals

[0236] Step 6:

[0237] The server notifies the user of the generated product proposal via email. During this process, the product proposal is sent to the user's registered email address. The email containing the proposal is sent using the SMTP protocol.

[0238] Input: Personalized product suggestions

[0239] Output: Product proposal email sent to user

[0240] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0241] This invention combines a system that effectively collects customer operations, usage status, and inquiries on a customer portal site, generates product proposals based on that data, and notifies the sales department of the proposals with an emotion engine that recognizes user emotions.

[0242] System Configuration

[0243] 1. Data Collection

[0244] The server records in real time the series of operations that customers perform on the portal site. For example, it saves click information and page view information when customers use a certain function in a log file. It also records customer usage status (e.g., connection time and data consumption) and inquiry content (e.g., support messages and FAQ search history).

[0245] 2. Data Preprocessing

[0246] The server filters the collected log data to remove unnecessary and duplicate data, such as duplicate entries of the same operation repeated multiple times and meaningless text (noise), and converts the data into a standard format that makes it easier to analyze.

[0247] 3. Data Conversion

[0248] The server converts the preprocessed data into multimodal data. For example, it combines the inquiry content as text data, the usage amount as numerical data, and the operation date and time as time data. This makes it possible to handle different types of data in a unified manner.

[0249] 4. Emotion Recognition by Emotion Engine

[0250] The server analyzes the user's inquiries and operations to recognize their emotions. For example, it determines whether the user is frustrated or excited based on the content of the text and the frequency of operations. This emotional data is also included in the multimodal data.

[0251] 5. AI analysis

[0252] The server then feeds the converted multimodal data into an AI algorithm to analyze customer behavior patterns and needs. Specifically, it uses machine learning models to predict whether a customer wants a specific feature. For example, if a customer frequently inquires about a new feature, it will recommend products or services that include that feature.

[0253] 6. Product proposal generation taking into account emotional data

[0254] The server generates product proposals based on the results of the AI ​​analysis and also takes into account the user's emotional data. For example, if a customer expresses dissatisfaction, it will make proposals to resolve that dissatisfaction (e.g., expanding support services).

[0255] 7. Notification

[0256] The server notifies the sales department of the generated product proposal. Specifically, the proposal details are communicated to the sales representative via the company's internal email system and notification dashboard. The sales representative then follows up with the customer based on this notification.

[0257] Specific examples

[0258] 1. Data Collection

[0259] The server collects operation logs when Customer A starts using the "Subscription Service."

[0260] The server records the details of the inquiry made by Customer A regarding "how to set up this service."

[0261] 2. Data Preprocessing

[0262] The server removes duplicate entries and unnecessary noise from the operations log.

[0263] The server formats the normal log data into a standard format.

[0264] 3. Data Conversion

[0265] The server integrates the inquiry content as text data, the usage amount as numerical data, and the service start date and time as time data.

[0266] 4. Emotion Recognition by Emotion Engine

[0267] The server analyzes the content of the inquiry and the frequency of operations and detects that customer A is dissatisfied.

[0268] 5. AI analysis

[0269] The server inputs the integrated data into an AI model and analyzes customer A's behavioral patterns.

[0270] The server proposes an upgrade plan with a new setup guide because Customer A frequently asks about "how to set it up."

[0271] 6. Product proposal generation taking into account emotional data

[0272] To resolve Customer A's dissatisfaction, the server generates an upgrade plan with a setup guide as well as a proposal to expand the support content.

[0273] 7. Notification

[0274] The server notifies the sales representative of the generated proposal by email, and the sales representative contacts Customer A based on the proposal.

[0275] In this way, by combining this system with an emotion engine, it is possible to make efficient and precise product proposals that take customer emotions into consideration, thereby improving the efficiency of sales activities and customer satisfaction.

[0276] The processing flow will be explained below.

[0277] Step 1: Data collection

[0278] The server records in real time the operations performed by customers on the portal site. For example, it stores click information and page browsing history in a log when a customer starts a "subscription service." It also records connection time and data consumption while the customer is using the service. It also records the content of inquiries made by customers regarding "how to set up this service."

[0279] Step 2: Data Preprocessing

[0280] The server filters the collected log data to remove unnecessary and duplicate data, such as duplicate entries of the same operation repeated multiple times and meaningless text (noise), and converts the data into a standard format to ensure consistency.

[0281] Step 3: Data conversion

[0282] The server converts the preprocessed data into multimodal data. Specifically, it integrates the inquiry content as text data, the usage amount as numerical data, and the operation date and time as time data. This makes it possible to handle different types of data in a unified manner.

[0283] Step 4: Emotion Recognition with the Emotion Engine

[0284] The server analyzes the user's inquiries and operations to recognize the user's emotions. For example, it can determine whether the user is happy or dissatisfied from the text of the inquiry. It can also estimate the user's emotions from the frequency and patterns of operations. This emotional data is included in the multimodal data.

[0285] Step 5: AI analysis

[0286] The server then feeds the converted multimodal data into an AI algorithm to analyze customer behavior patterns and needs. Specifically, it uses machine learning models to predict whether a customer wants a specific feature. For example, if a customer frequently inquires about a new feature, it will recommend products or services that include that feature.

[0287] Step 6: Generate product recommendations taking into account emotional data

[0288] The server generates product proposals based on the results of the AI ​​analysis and also takes into account the user's emotional data. For example, if a customer expresses dissatisfaction, it makes proposals to resolve that dissatisfaction (e.g., an upgrade plan with setup guides and additional support). Based on the emotional data, it adjusts the proposal content to increase customer satisfaction.

[0289] Step 7: Notification

[0290] The server notifies the sales department of the generated product proposal. Specifically, the proposal details are communicated to the sales representative via the company's internal email system or notification dashboard. The sales representative then follows up with the customer based on this notification. For example, they may contact Customer A to propose a new upgrade plan with a setup guide and additional support.

[0291] Example 2

[0292] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0293] Conventional customer portal sites were able to collect customer operations, usage status, and inquiries, but simply recording and analyzing this data made it difficult to make appropriate product proposals based on the customer's true needs and emotions. Furthermore, the collected data sometimes contained noise or duplicate data, which reduced the accuracy of the analysis. Furthermore, even if product proposals were generated, they were sometimes not efficiently communicated to the sales department. An effective solution to these issues was needed.

[0294] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0295] In this invention, the server includes means for recording customer operation details, means for recording customer usage status, means for recording customer inquiry details, means for preprocessing the recorded data, means for converting the preprocessed data into multimodal data, means for analyzing the converted multimodal data to generate product proposals, means for recognizing user emotions, means for generating product proposals taking into account the emotion data, and means for notifying the generated product proposals. This enables precise product proposals that also take emotions into account based on the customer's operation history and inquiry details. Furthermore, preprocessing the collected data removes noise and duplicate data, enabling highly accurate analysis and enabling efficient notification to the sales department.

[0296] "Customer operations" refers to a series of operations performed by customers on the portal site, such as click information and page viewing information.

[0297] "Customer usage" refers to the usage history, such as connection time and data consumption, when a customer uses a portal site.

[0298] "Customer inquiry content" refers to inquiry records such as support messages and FAQ search history made by customers when using the portal site.

[0299] "Recorded data" refers to all data stored on the server regarding customer operations, usage, and inquiries.

[0300] "Preprocessing" refers to the process of removing unnecessary noise and redundant data from recorded data and adjusting the data format.

[0301] "Multimodal data" refers to a format that integrates data of different formats (text data, numerical data, time data, etc.).

[0302] "Means for recognizing user emotions" refers to an emotion engine that analyzes the content of customer inquiries and operations and determines the customer's emotions (dissatisfaction, excitement, etc.).

[0303] "Means for generating product proposals" refers to means for predicting customer needs using AI analysis and proposing optimal products and services.

[0304] The "means for generating a product proposal taking into consideration emotional data" refers to a means for generating a more precise product proposal based on the emotional data of the user.

[0305] The "means for notifying the generated product proposal" refers to a means for effectively communicating the generated product proposal to the sales department.

[0306] This invention combines a system that effectively collects customer operations, usage status, and inquiries on a customer portal site, generates product proposals based on that data, and notifies the sales department of the proposals, with an emotion engine that recognizes user emotions. Specific embodiments of the present invention are described below.

[0307] System Configuration

[0308] The system is primarily composed of a server and terminals, and generates product proposals by recording and analyzing the operations, usage status, and inquiries that occur when customers (users) use the portal site, and notifies these proposals to the sales department.

[0309] Hardware and software used

[0310] Server: The central location for data collection, pre-processing, analysis, proposal generation, and notifications.

[0311] Terminal: The device (PC, smartphone, tablet, etc.) that users use to access the portal site.

[0312] Emotion engine: Software that analyzes user emotions and generates emotion data.

[0313] Data collection

[0314] The server collects the operations performed by customers on the portal site in real time. Specifically, click information, page view information, and inquiry details are instantly saved in a log file. When a user starts a subscription service, the operation log is immediately transferred to the server.

[0315] Examples:

[0316] The operation log generated by the terminal is sent to the server.

[0317] The server saves click information in a log file when Customer A starts using the "Subscription Service."

[0318] Data Preprocessing

[0319] The collected log data is filtered by the server to remove unnecessary noise and duplicate data, and through this process the data is organized into a standard format suitable for subsequent analysis.

[0320] Examples:

[0321] The server removes duplicate entries and meaningless data and formats the normal log data into a standard format.

[0322] Data Conversion

[0323] The preprocessed data is converted into multimodal data by the server, which integrates different types of data (text data, numerical data, time data, etc.) into one and allows for unified handling.

[0324] Examples:

[0325] The server saves the inquiry content as text data, converts the usage amount into numerical data, and integrates the operation date and time as time data.

[0326] Emotion recognition by emotion engine

[0327] The emotion engine analyzes the user's inquiries and operations to recognize their emotions. For example, it determines whether the user is dissatisfied or excited based on the content of the text and the frequency of operations, and generates emotion data.

[0328] Examples:

[0329] The server performs text analysis of the inquiry, detects keywords such as "dissatisfaction" and "irritation," and determines the user's emotions.

[0330] Product proposal generation and notification

[0331] The server uses an AI model based on multimodal data to analyze customer behavior patterns and needs. It then generates optimal product proposals taking into account emotional data and notifies the sales department of these proposals. Notifications are sent via the company's internal email system and notification dashboard.

[0332] Examples:

[0333] The server detects that Customer A frequently inquires about new features and suggests an upgrade plan with a guided setup.

[0334] The server notifies the sales department staff of this proposal by email, and the staff contacts Customer A based on the proposal.

[0335] Prompt Sentence Examples

[0336] Below are examples of prompt sentences to efficiently generate product proposals based on an AI model.

[0337] "Based on this customer's operation history and inquiry details, predict the products and services that the customer may be interested in, and generate product recommendations that take into account usage history and sentiment data."

[0338] This system makes it possible to make precise and effective product proposals based on the customer's operation history and inquiry details, as well as taking into account the user's emotions, dramatically improving the efficiency of sales activities and customer satisfaction.

[0339] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0340] Step 1:

[0341] Data collection

[0342] The server collects the operations performed by customers on the portal site in real time. Specifically, click information and page view information generated by the device are sent to the server and saved in a log file. In addition, connection time, data consumption, and inquiry content are also recorded.

[0343] Input: Customer operation details, usage status, inquiry details

[0344] Output: Operation logs, usage data, and query records saved in log files

[0345] Specific behavior:

[0346] The terminal transmits the user's click information on the portal site to the server.

[0347] The server records click information and page view information in a log file.

[0348] Step 2:

[0349] Data Preprocessing

[0350] The server filters the collected log data, removing unnecessary and duplicate data, and formats the data for subsequent analysis.

[0351] Input: Recorded log data

[0352] Output: filtered and standard formatted data

[0353] Specific behavior:

[0354] The server removes duplicate entries where the same operation has been performed multiple times.

[0355] Remove unnecessary noise data.

[0356] Convert the data into a standard format.

[0357] Step 3:

[0358] Data Conversion

[0359] The server converts the preprocessed data into multimodal data, integrating different types of data (text data, numerical data, time data, etc.) and handling them in a unified manner.

[0360] Input: Preprocessed log data

[0361] Output: Multimodal data

[0362] Specific behavior:

[0363] The server saves the inquiry content as text data.

[0364] Convert usage into numerical data.

[0365] The operation date and time are recorded as time data.

[0366] Integrate all data as multimodal data.

[0367] Step 4:

[0368] Emotion recognition by emotion engine

[0369] The server analyzes the user's inquiries and operations, recognizes the user's emotions, and adds the emotion information to the multimodal data.

[0370] Input: Inquiry details, operation frequency data

[0371] Output: Multimodal data including emotion data

[0372] Specific behavior:

[0373] The server performs text analysis of the inquiry content and extracts keywords that indicate emotions.

[0374] The frequency of operations is analyzed to determine the user's emotions.

[0375] Adding emotion data to multimodal data.

[0376] Step 5:

[0377] AI analysis

[0378] The server inputs multimodal data into an AI algorithm to analyze customer behavior patterns and needs, generating product recommendations that customers may be interested in.

[0379] Input: Multimodal data including emotion data

[0380] Output: Product proposals based on customer needs

[0381] Specific behavior:

[0382] The server inputs multimodal data, including emotional data, into the AI ​​model.

[0383] Analyze behavioral patterns and needs to generate product proposals.

[0384] Step 6:

[0385] Product proposal generation taking into account emotional data

[0386] The server generates more sophisticated product proposals based on the results of AI analysis and emotional data.

[0387] Input: AI analysis results, emotional data

[0388] Output: Product proposals that take emotions into account

[0389] Specific behavior:

[0390] If the customer is dissatisfied, the server generates a proposal to resolve the dissatisfaction.

[0391] Reflect emotional data in proposal content.

[0392] Step 7:

[0393] notification

[0394] The server notifies the sales department of the generated product proposals, using an email system and a notification dashboard to communicate the proposal details in real time.

[0395] Input: Generated product proposal

[0396] Output: Product proposal notified to the sales department

[0397] Specific behavior:

[0398] The server sends the generated product proposal to the sales representative via an email system.

[0399] Sales representatives receive the proposal and follow up with the customer.

[0400] (Application example 2)

[0401] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0402] With conventional portal sites, it was difficult to effectively collect customer operations, usage status, and inquiry details and generate product proposals based on that information. Furthermore, proposals were made without taking into account customer emotions, making it difficult to improve customer satisfaction. Therefore, there is a need for a system that can accurately grasp customer emotions and needs and make optimal product proposals.

[0403] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for recording customer operation details, means for recording customer usage status, means for recording customer inquiry details, means for preprocessing the recorded data, means for converting the preprocessed data into multimodal data, means for analyzing the converted multimodal data to generate product proposals, means for generating user emotion data using an emotion engine that recognizes user emotions and including the emotion data in the multimodal data, and means for notifying the generated product proposals. This makes it possible to comprehensively analyze customer operations, usage status, and inquiry details, and to make precise product proposals that take emotions into consideration.

[0404] 1. "Means for recording customer operations" refers to means for saving information about operations performed by customers on electronic devices or websites as logs.

[0405] 2. "Means for recording customer usage" means means for recording usage data such as the time and amount consumed when a customer uses a service or product.

[0406] 3. "Means for recording customer inquiries" refers to means for saving the details and history of support requests and inquiries made by customers.

[0407] 4. "Means for pre-processing recorded data" means means for removing unnecessary information and redundant data from collected data and converting it into a format that can be analyzed.

[0408] 5. "Means for converting preprocessed data into multimodal data" refers to means for converting data of different formats (text data, numerical data, time data, etc.) so that they can be handled in a unified manner.

[0409] 6. "Means for analyzing converted multimodal data and generating product proposals" refers to means for analyzing customer needs and behavioral patterns based on converted data and creating appropriate product proposals.

[0410] 7. "Means for generating user emotion data using an emotion engine that recognizes user emotions and including the emotion data in multimodal data" refers to means for analyzing user emotions, saving them as numerical data, and integrating them with other data.

[0411] 8. "Means for notifying the generated product proposal" refers to the means for notifying the customer or sales representative of the product proposal by means such as email or push notification.

[0412] This invention combines a system that effectively collects customer operations, usage status, and inquiries on a customer portal site, and generates and notifies product proposals based on that data with an emotion engine that recognizes user emotions.

[0413] First, the server records the operations performed by customers when they use the portal site or smartphone app in real time. The operations include click information and page browsing history. The server also records the customer's usage status (e.g., connection time and data consumption) and inquiries (e.g., support messages and FAQ search history).

[0414] The server then preprocesses the collected log data, using pandas to remove unnecessary and duplicate data and convert it into a standard format for easier analysis. This preprocessing ensures that only meaningful data remains.

[0415] The server then converts the preprocessed data into multimodal data, integrating the query content as text data, the usage amount as numerical data, and the operation date and time as time data, making it possible to handle different types of data in a unified manner.

[0416] Furthermore, the server uses an emotion engine to recognize the user's emotions. Specifically, it uses the SentimentIntensityAnalyzer in the nltk library to determine the user's emotions from the content of the text and the frequency of operations. This emotion data is also included in the multimodal data.

[0417] The server uses the converted multimodal data to analyze customer behavior patterns and needs using AI algorithms. Specifically, it quantifies the text data using TfidfVectorizer, scales it using StandardScaler, and performs cluster analysis using KMeans. This analysis predicts whether the customer is interested in a specific product or service.

[0418] The server then generates product recommendations based on the results of the AI ​​analysis and the emotional data. For example, if the emotion engine indicates that the user is dissatisfied, it can make suggestions to resolve that dissatisfaction (e.g., expanding support services).

[0419] Finally, the server notifies the customer and the sales department of the generated product proposal via email or push notification, allowing the sales representative to follow up on the customer's needs.

[0420] Specific examples

[0421] 1. If Customer A views Product B multiple times and posts a question about how to set it up, the emotion engine determines that they are dissatisfied. In this case, the server will suggest a guided upgrade plan for Product B.

[0422] 2. If Customer B expresses dissatisfaction with the service, a proposal to resolve the dissatisfaction by expanding the support content is generated and notified to Customer B.

[0423] Prompt Sentence Examples

[0424] "Generate a list of recommended products related to product B, which this customer has recently viewed frequently."

[0425] "Please tell us the best product or support you can offer to resolve the issue for Customer B, who is dissatisfied with your service."

[0426] This makes it possible to comprehensively analyze customer operations and inquiries, and to make precise product proposals that take emotions into consideration.

[0427] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0428] Step 1:

[0429] The server records customer operations in real time. Every time a customer views or clicks on a product page, that information is saved in a log. For example, an operation log is generated stating, "Customer clicked on product A." The input is the customer's operation, and the output is the recorded operation log.

[0430] Step 2:

[0431] The server records the customer's usage, which includes information such as the time the customer uses the service and the amount of data consumed. For example, a usage log is generated stating that "the customer used the service for 30 minutes." The input is the customer's service usage, and the output is the recorded usage log.

[0432] Step 3:

[0433] The server records customer inquiries. Every time a customer sends a support message or searches an FAQ, the content is saved in a log. For example, an inquiry log is generated that reads, "A customer inquired about how to set up product B." The input is the customer's inquiry, and the output is the recorded inquiry log.

[0434] Step 4:

[0435] The server preprocesses the collected log data. It uses pandas to remove unnecessary and duplicate data and convert it into a standard format. For example, it removes duplicate click information and meaningless text. The input is the recorded operation log, usage log, and query log, and the output is the preprocessed data.

[0436] Step 5:

[0437] The server converts the preprocessed data into multimodal data, which allows different types of data to be handled in a unified manner. For example, query content can be integrated as text data, usage amount as numerical data, and operation date and time as time data. The input is preprocessed data, and the output is multimodal data.

[0438] Step 6:

[0439] The server uses an emotion engine to recognize the user's emotions. Specifically, it uses the SentimentIntensityAnalyzer from the nltk library. It determines the user's emotions (e.g., dissatisfaction or excitement) from the text content and frequency of operations. The input is preprocessed text data, and the output is emotion data.

[0440] Step 7:

[0441] The server integrates the emotion data into multimodal data, which allows the emotion data to be analyzed together with other data. The input is emotion data and multimodal data, and the output is the integrated multimodal data.

[0442] Step 8:

[0443] The server runs an AI algorithm using the converted multimodal data to analyze customer behavior patterns and needs. The text data is quantified using TfidfVectorizer, scaled using StandardScaler, and subjected to cluster analysis using KMeans. The input is the integrated multimodal data, and the output is customer cluster information and analysis results.

[0444] Step 9:

[0445] The server generates product proposals based on the results of AI analysis and emotional data. For example, if a customer expresses dissatisfaction, it makes proposals to resolve the dissatisfaction (e.g., expanding support services). The input is the customer's cluster information and emotional data, and the output is the generated product proposal.

[0446] Step 10:

[0447] The server notifies the customer and the sales department of the generated product proposal. Notifications are sent by email, push notification, or other methods. The input is the generated product proposal, and the output is the notified product proposal.

[0448] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0449] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0450] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0451] [Second embodiment]

[0452] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0453] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0454] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0455] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0456] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0457] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0458] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0459] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0460] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0461] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0462] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0463] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0464] This invention is a system that effectively collects operation details, usage status, and inquiry details on a customer portal site, generates product proposals based on this data, and notifies the sales department of the proposals.

[0465] System Configuration

[0466] 1. Data Collection

[0467] The server has the function of recording the series of operations that customers perform on the portal site. For example, it collects click information when customers use a certain function and information on the browsing of specific pages. It also records how customers use the product (usage status) and the content of their inquiries.

[0468] 2. Data Preprocessing

[0469] The server filters the collected data to remove unnecessary and duplicated data, such as repeated logs of the same operation and noise data, and normalizes the data into a standard format to facilitate subsequent analysis.

[0470] 3. Data Conversion

[0471] The server converts the preprocessed data into multimodal data, specifically integrating text data (inquiry content), numerical data (usage amount), and time data (operation timing) into a single dataset.

[0472] 4. AI analysis

[0473] The server feeds multimodal data into AI models to analyze customer behavior patterns and needs, for example using machine learning algorithms to predict whether a customer is interested in a particular product feature.

[0474] 5. Generate product proposals

[0475] Based on the results of the AI ​​analysis, the server generates optimal product proposals for customers, such as upgrades and new services related to features that customers frequently inquire about.

[0476] 6. Notification

[0477] The server notifies the sales department of the generated product proposals via an internal email system, a notification dashboard, or by sending a direct message to a specific sales representative.

[0478] Specific examples

[0479] 1. Data Collection

[0480] The server collects operation logs when Customer A starts using the "new subscription service."

[0481] The server also records the inquiries made by Customer A regarding "how to set up this service."

[0482] 2. Data Preprocessing

[0483] The server removes duplicate entries and unnecessary noise from the operation log.

[0484] The server formats the normal log data into a standard format.

[0485] 3. Data Conversion

[0486] The server integrates the inquiry content as text data, the usage amount as numerical data, and the service start date and time as time data.

[0487] 4. AI analysis

[0488] The server inputs this integrated data into an AI model and analyzes Customer A's behavioral patterns.

[0489] The server proposes an upgrade plan with a new setup guide because Customer A frequently asks about "how to set it up."

[0490] 5. Generate product proposals

[0491] The server generates a guided configuration upgrade plan based on AI analysis.

[0492] 6. Notification

[0493] The server notifies the sales representative of the generated proposal by email, and the sales representative contacts Customer A.

[0494] In this way, this system significantly improves the efficiency of sales activities by making efficient and precise product proposals based on customer operations and inquiries.

[0495] The processing flow will be explained below.

[0496] Step 1: Data collection

[0497] The server records the operations performed by customers on the portal site in real time. Specifically, it saves click information and page viewing information when customers use certain functions in a log file. It also records customer usage status (e.g., connection time and data consumption) and inquiry content (e.g., support messages and FAQ search history).

[0498] Step 2: Data Preprocessing

[0499] The server filters the collected log data to remove duplicates and noise, such as duplicate entries of the same operation repeated multiple times and meaningless text (noise). It also converts the data into a standard format that makes it easier to analyze.

[0500] Step 3: Data conversion

[0501] The server converts the preprocessed data into multimodal data. For example, it combines the inquiry content as text data, the usage amount as numerical data, and the operation date and time as time data. This makes it possible to handle different types of data in a unified manner.

[0502] Step 4: AI analysis

[0503] The server then feeds the converted multimodal data into an AI algorithm to analyze customer behavior patterns and needs. Specifically, it uses machine learning models to predict whether a customer wants a specific feature. For example, if a customer frequently inquires about a new feature, it will recommend products or services that include that feature.

[0504] Step 5: Generate product proposals

[0505] The server generates product proposals based on the results of AI analysis. For example, it proposes upgrade plans or new services related to features that customers frequently inquire about. The proposals include the products and services offered, their benefits, and how to use them.

[0506] Step 6: Notification

[0507] The server notifies the sales department of the generated product proposal. Specifically, the proposal details are communicated to the sales representative via the company's internal email system and notification dashboard. The sales representative then follows up with the customer based on this notification.

[0508] These are the specific processing steps of this system, which makes it possible to effectively utilize customer operation and inquiry data and improve the efficiency of sales activities.

[0509] Example 1

[0510] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0511] Conventional customer support systems have difficulty effectively collecting customer operations, usage status, and inquiry details, and quickly and efficiently generating appropriate product proposals based on this data. This has led to problems such as reduced sales efficiency and difficulty in improving customer satisfaction. Furthermore, a lack of technology for precisely analyzing customer behavior patterns has made it difficult to provide accurate proposals. A new system that can solve these issues is needed.

[0512] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0513] In this invention, the server includes means for recording customer operation details, means for recording customer usage status, means for recording customer inquiry details, means for preprocessing the recorded data, means for converting the preprocessed data into multimodal data, means for inputting the converted multimodal data into a generative AI model to analyze customer behavior patterns, means for generating product proposals based on the analysis results, and means for notifying the sales department of the generated product proposals. This makes it possible to effectively collect and preprocess customer data and analyze it using an AI model, thereby enabling accurate and efficient generation of product proposals and notification to the sales department.

[0514] The "means for recording customer operations" is a function for recording operations such as clicks and page views that customers perform on the web portal site.

[0515] "Means for recording customer usage" refers to a function for recording how and to what extent customers use products and services.

[0516] "Means for recording customer inquiries" is a function for recording the inquiries made by customers to support or the help desk in text format.

[0517] "Means for preprocessing recorded data" refers to a function for removing noise and redundant data from collected data and preparing the data in a format suitable for analysis.

[0518] The "means for converting preprocessed data into multimodal data" is a function for integrating preprocessed data into multiple formats, such as text data, numerical data, and time data, to create a single data set.

[0519] "Means for inputting converted multimodal data into a generative artificial intelligence model to analyze customer behavior patterns" refers to a function for inputting multimodal data into an AI model to analyze customer behavior patterns and needs.

[0520] "Means for generating product proposals based on analysis results" refers to a function that automatically creates optimal product and service proposals for customers based on the results of analysis by AI.

[0521] The "means for notifying the sales department of the generated product proposal" is a function for communicating the generated product proposal to the sales department using an appropriate notification means.

[0522] MODE FOR CARRYING OUT THE INVENTION

[0523] This invention is a system that effectively collects operation details, usage status, and inquiry details on a customer portal site, generates product proposals based on this data, and notifies the sales department of the proposals. The configuration and operation of this system will be specifically described.

[0524] 1. Data Collection

[0525] The server has the function of recording a series of operations performed by customers on the portal site. For example, it collects information on customers clicking specific buttons and logs the specific pages they view. It also records how customers use products and services (usage status) and the content of their inquiries. For this purpose, the server uses a log collection module and database.

[0526] Examples:

[0527] The server collects operation logs when Customer A starts using the "new subscription service."

[0528] The server also records the inquiries made by Customer A regarding "how to set up this service."

[0529] 2. Data Preprocessing

[0530] The server filters the collected data to remove unnecessary or redundant data, such as repeated logs of the same operation or noise data, and normalizes the data into a standard format to facilitate subsequent analysis.

[0531] Examples:

[0532] The server removes duplicate entries and unnecessary noise from the operation log.

[0533] The server formats the normal log data into a standard format.

[0534] 3. Data Conversion

[0535] The server converts the preprocessed data into multimodal data, specifically integrating text data (inquiry content), numerical data (usage amount), and time data (operation timing) into a single dataset.

[0536] Examples:

[0537] The server integrates the inquiry content as text data, the usage amount as numerical data, and the service start date and time as time data.

[0538] 4. Analysis using AI models

[0539] The server feeds multimodal data into generative AI models to analyze customer behavior patterns and needs. For example, it uses machine learning algorithms to predict whether a customer is interested in a particular product feature. To do this, the server uses machine learning libraries and AI models.

[0540] Examples:

[0541] The server inputs this integrated data into an AI model and analyzes Customer A's behavioral patterns.

[0542] The server detects that customer A frequently asks about "how to set things up" and proposes a new guided setup upgrade plan.

[0543] 5. Generate product proposals

[0544] The server generates optimal product proposals for customers based on the analysis results of the AI ​​model, such as proposing upgrades or new services related to features that customers frequently inquire about.

[0545] Examples:

[0546] The server generates a guided upgrade plan based on the output of the AI ​​model.

[0547] 6. Notification of Proposal

[0548] The server notifies the sales department of the generated product proposals using the company's internal email system, a notification dashboard, or direct messaging to specific sales representatives.

[0549] Examples:

[0550] The server notifies the sales representative of the generated proposal via email, and the sales representative contacts Customer A and proposes a guided configuration upgrade plan.

[0551] Prompt Sentence Examples

[0552] "Design a system that generates optimal product proposals based on a customer's operation log and inquiry details, and notifies the sales department."

[0553] This system significantly improves the efficiency of sales activities by making efficient and precise product proposals based on customer operations and inquiries.

[0554] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0555] Divide the program's processing flow into processing steps

[0556] Step 1: Data collection

[0557] The server records the series of operations and inquiries that the user makes on the portal site.

[0558] Input: User clicks, page views, inquiries

[0559] Specific operation: When a customer clicks a specific button, the click information is recorded as a log along with a timestamp, and the content submitted in the inquiry form is saved as text data.

[0560] Output: Operation log and inquiry data

[0561] Step 2: Data Preprocessing

[0562] The server filters the collected data, removing unnecessary and redundant data, and normalizes the data into a standard format.

[0563] Input: Operation log and inquiry data

[0564] What it does: It finds and removes duplicate entries and noise data, and converts the remaining data into a standard format such as JSON.

[0565] Output: Preprocessed operation log and query data

[0566] Step 3: Data Integration and Transformation

[0567] The server converts the pre-processed data into multimodal data.

[0568] Input: Preprocessed operation log and query data

[0569] Specific operation: Text data (e.g., inquiry content), numerical data (e.g., usage amount), and time data (e.g., operation timing) are integrated into a single data set.

[0570] Output: Multimodal dataset

[0571] Step 4: Analysis by AI model

[0572] The server feeds the multimodal dataset into a generative AI model to analyze customer behavior patterns.

[0573] Input: Multimodal dataset

[0574] What it does: Multimodal data is fed into an AI model and analyzed using machine learning algorithms.

[0575] Output: Analysis of customer behavior patterns and needs

[0576] Step 5: Generate product proposals

[0577] The server generates optimal product proposals for customers based on the analysis results of the AI ​​model.

[0578] Input: Analysis results of customer behavior patterns and needs

[0579] What it does: Based on the analysis results, it generates recommendations for new services and additional features that customers may be interested in.

[0580] Output: Product proposal data

[0581] Step 6: Proposal Notification

[0582] The server notifies the sales department of the generated product proposal.

[0583] Input: Product proposal data

[0584] What happens next? Communicate proposals to the sales team via email and notification dashboards, sending messages directly to specific salespeople as needed.

[0585] Output: Notification data to the sales department

[0586] (Application example 1)

[0587] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0588] There is a need for a system that can effectively collect and analyze customer behavior data and generate appropriate product proposals to improve the efficiency of sales activities and customer satisfaction. However, current systems do not fully utilize customer behavior data, limiting the accuracy and degree of personalization of product proposals. In particular, there is a lack of a mechanism for integrating and analyzing various customer operations and inquiry data, making it difficult to provide proposals that are optimized for each individual customer.

[0589] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0590] In this invention, the server includes means for recording customer operation details, means for recording customer usage status, means for recording customer inquiry details, means for preprocessing the recorded data, means for converting the preprocessed data into multimodal data, means for analyzing the converted multimodal data to generate product proposals, means for notifying the generated product proposals, analysis means for clustering customer behavior, means for generating personalized product proposals based on the clustering results, and means for notifying the customer of the generated product proposals by email. This makes it possible to comprehensively analyze a variety of customer behavior data and provide optimal product proposals for each individual customer.

[0591] "Means for recording customer operations" refers to means for collecting operation logs such as clicks and page transitions that customers make on websites and applications.

[0592] "Means for recording customer usage" means means for monitoring and recording the extent to which customers use specific features or services.

[0593] The "means for recording customer inquiries" refers to a means for collecting questions and problem reports made by customers to customer support as text data.

[0594] "Means for pre-processing recorded data" refers to means for filtering and normalizing the collected raw data to prepare it for easier subsequent analysis.

[0595] A "means for transforming preprocessed data into multimodal data" is a means for integrating data of different formats (e.g., textual data, numerical data, temporal data) into a single analyzable data set.

[0596] "Means for analyzing converted multimodal data and generating product proposals" refers to means for analyzing data using techniques such as machine learning and generating proposals for products and services that are optimal for customers.

[0597] The "means for notifying the generated product proposal" refers to a means for notifying the customer or the sales department of the generated proposal using a method such as email or a dashboard.

[0598] The "analysis means for clustering customer behavior" is a means for analyzing collected customer behavior data and dividing them into groups of customers with common characteristics.

[0599] The "means for generating personalized product proposals based on clustering results" is a means for proposing individually optimized products and services based on the characteristics of each customer group identified by clustering.

[0600] The "means for notifying the customer of the generated product proposal by email" is a means for directly notifying the customer of the content of the proposal by sending the generated proposal to the customer's email address.

[0601] This invention is a system that collects and analyzes customer behavior data, and generates and notifies individually optimized product proposals. A specific embodiment of this system is described below.

[0602] System Overview

[0603] The system consists of the following main components:

[0604] 1. Data Collection Module

[0605] 2. Data Preprocessing Module

[0606] 3. Data Conversion Module

[0607] 4. AI analysis module

[0608] 5. Product proposal generation module

[0609] 6. Notification Module

[0610] Hardware and Software Used

[0611] Hardware: The system operates using a server or cloud environment, while data collection and notifications are performed using the user's smartphone, tablet, or PC.

[0612] Software: Python and Pandas are used for data processing, and machine learning libraries such as Scikit-learn and TensorFlow are used for analysis. A library (smtplib) that sends emails using the SMTP protocol is used for notifications.

[0613] Program processing explanation

[0614] Data Collection Module

[0615] The server collects user operations, usage status, and inquiries in real time, for example, by recording website click information, page browsing history, and product inquiries.

[0616] Data Preprocessing Module

[0617] The collected data is pre-processed on the server, which removes noise and duplicate data and arranges all data into a standard format. For example, if the same click information is recorded multiple times, it will be combined into one.

[0618] Data Conversion Module

[0619] The preprocessed data is converted into multimodal data, a process that combines text data (e.g., inquiry content), numerical data (e.g., product usage), and temporal data (e.g., operation duration) into a single dataset.

[0620] AI Analysis Module

[0621] The server then inputs the converted multimodal data into an AI model to analyze the user's behavioral patterns and needs, using a clustering algorithm to classify user behavior into groups with common characteristics.

[0622] Product proposal generation module

[0623] Based on the clustering results, the server generates product recommendations optimized for each user group. For example, if a user frequently inquires about how to use a particular product, the server can suggest new accessories related to that product.

[0624] Notification Module

[0625] The generated product proposal will be notified by email. The server will send the generated proposal to the user's registered email address. The email is sent using the SMTP protocol.

[0626] Specific examples

[0627] For example, if a user does the following:

[0628] Viewing a specific product page multiple times

[0629] Make an inquiry about "How to use the product"

[0630] Browse additional related products

[0631] The server collects this data, pre-processes and analyzes it, and then sends the following email to the user based on the analysis results:

[0632] Subject: Personalized product recommendations

[0633] Main text:

[0634] Hello,

[0635] Get recommended products related to your recently viewed products.

[0636] 1. Upgrade kit for product X

[0637] 2. Accessories related to product X

[0638] Please check it out.

[0639] An example prompt for a generative AI model would be:

[0640] Generate optimal product recommendations based on user behavior data, including:

[0641] Click information: / home, / product / 3, / product / 5

[0642] Inquiry: How to return product? Warranty information

[0643] In this way, personalized product proposals based on user-specific data can be made, thereby improving the user experience and streamlining sales activities.

[0644] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0645] Step 1:

[0646] The server collects user operations, usage, and inquiries in real time, for example, logging the links users click on websites, the pages they view, the search terms they use, and the information they enter into inquiry forms. The collected data is stored as raw data.

[0647] Input: User operation logs, usage data, and inquiry details

[0648] Output: Raw data collected

[0649] Step 2:

[0650] The server pre-processes the collected raw data, removing unnecessary and duplicate data and arranging the data into a standard format, specifically combining multiple records of the same click and removing noise data (e.g., error logs).

[0651] Input: Raw data collected

[0652] Output: Preprocessed data

[0653] Step 3:

[0654] The server converts the preprocessed data into multimodal data, a process that integrates data of different formats (e.g., textual, numerical, and temporal data) into a single dataset that facilitates subsequent analysis.

[0655] Input: Preprocessed data

[0656] Output: Multimodal data

[0657] Step 4:

[0658] The server inputs the multimodal data into an AI model to analyze user behavior patterns and needs. A clustering algorithm is used for the analysis, classifying user behavior into groups with common characteristics. For example, KMeans clustering is used to classify customers into multiple groups.

[0659] Input: Multimodal data

[0660] Output: Clustered user groups

[0661] Step 5:

[0662] The server generates personalized product recommendations based on the clustering results. This process involves suggesting the best products and services for each user group. For example, if a user is interested in a particular product, it will suggest accessories and upgrades related to that product.

[0663] Input: Clustered user groups

[0664] Output: Personalized product proposals

[0665] Step 6:

[0666] The server notifies the user of the generated product proposal via email. During this process, the product proposal is sent to the user's registered email address. The email containing the proposal is sent using the SMTP protocol.

[0667] Input: Personalized product suggestions

[0668] Output: Product proposal email sent to user

[0669] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0670] This invention combines a system that effectively collects customer operations, usage status, and inquiries on a customer portal site, generates product proposals based on that data, and notifies the sales department of the proposals with an emotion engine that recognizes user emotions.

[0671] System Configuration

[0672] 1. Data Collection

[0673] The server records in real time the series of operations that customers perform on the portal site. For example, it saves click information and page view information when customers use a certain function in a log file. It also records customer usage status (e.g., connection time and data consumption) and inquiry content (e.g., support messages and FAQ search history).

[0674] 2. Data Preprocessing

[0675] The server filters the collected log data to remove unnecessary and duplicate data, such as duplicate entries of the same operation repeated multiple times and meaningless text (noise), and converts the data into a standard format that makes it easier to analyze.

[0676] 3. Data Conversion

[0677] The server converts the preprocessed data into multimodal data. For example, it combines the inquiry content as text data, the usage amount as numerical data, and the operation date and time as time data. This makes it possible to handle different types of data in a unified manner.

[0678] 4. Emotion Recognition by Emotion Engine

[0679] The server analyzes the user's inquiries and operations to recognize their emotions. For example, it determines whether the user is frustrated or excited based on the content of the text and the frequency of operations. This emotional data is also included in the multimodal data.

[0680] 5. AI analysis

[0681] The server then feeds the converted multimodal data into an AI algorithm to analyze customer behavior patterns and needs. Specifically, it uses machine learning models to predict whether a customer wants a specific feature. For example, if a customer frequently inquires about a new feature, it will recommend products or services that include that feature.

[0682] 6. Product proposal generation taking into account emotional data

[0683] The server generates product proposals based on the results of the AI ​​analysis and also takes into account the user's emotional data. For example, if a customer expresses dissatisfaction, it will make proposals to resolve that dissatisfaction (e.g., expanding support services).

[0684] 7. Notification

[0685] The server notifies the sales department of the generated product proposal. Specifically, the proposal details are communicated to the sales representative via the company's internal email system and notification dashboard. The sales representative then follows up with the customer based on this notification.

[0686] Specific examples

[0687] 1. Data Collection

[0688] The server collects operation logs when Customer A starts using the "Subscription Service."

[0689] The server records the details of the inquiry made by Customer A regarding "how to set up this service."

[0690] 2. Data Preprocessing

[0691] The server removes duplicate entries and unnecessary noise from the operations log.

[0692] The server formats the normal log data into a standard format.

[0693] 3. Data Conversion

[0694] The server integrates the inquiry content as text data, the usage amount as numerical data, and the service start date and time as time data.

[0695] 4. Emotion Recognition by Emotion Engine

[0696] The server analyzes the content of the inquiry and the frequency of operations and detects that customer A is dissatisfied.

[0697] 5. AI analysis

[0698] The server inputs the integrated data into an AI model and analyzes customer A's behavioral patterns.

[0699] The server proposes an upgrade plan with a new setup guide because Customer A frequently asks about "how to set it up."

[0700] 6. Product proposal generation taking into account emotional data

[0701] To resolve Customer A's dissatisfaction, the server generates an upgrade plan with a setup guide as well as a proposal to expand the support content.

[0702] 7. Notification

[0703] The server notifies the sales representative of the generated proposal by email, and the sales representative contacts Customer A based on the proposal.

[0704] In this way, by combining this system with an emotion engine, it is possible to make efficient and precise product proposals that take customer emotions into consideration, thereby improving the efficiency of sales activities and customer satisfaction.

[0705] The processing flow will be explained below.

[0706] Step 1: Data collection

[0707] The server records in real time the operations performed by customers on the portal site. For example, it stores click information and page browsing history in a log when a customer starts a "subscription service." It also records connection time and data consumption while the customer is using the service. It also records the content of inquiries made by customers regarding "how to set up this service."

[0708] Step 2: Data Preprocessing

[0709] The server filters the collected log data to remove unnecessary and duplicate data, such as duplicate entries of the same operation repeated multiple times and meaningless text (noise), and converts the data into a standard format to ensure consistency.

[0710] Step 3: Data conversion

[0711] The server converts the preprocessed data into multimodal data. Specifically, it integrates the inquiry content as text data, the usage amount as numerical data, and the operation date and time as time data. This makes it possible to handle different types of data in a unified manner.

[0712] Step 4: Emotion Recognition with the Emotion Engine

[0713] The server analyzes the user's inquiries and operations to recognize the user's emotions. For example, it can determine whether the user is happy or dissatisfied from the text of the inquiry. It can also estimate the user's emotions from the frequency and patterns of operations. This emotional data is included in the multimodal data.

[0714] Step 5: AI analysis

[0715] The server then feeds the converted multimodal data into an AI algorithm to analyze customer behavior patterns and needs. Specifically, it uses machine learning models to predict whether a customer wants a specific feature. For example, if a customer frequently inquires about a new feature, it will recommend products or services that include that feature.

[0716] Step 6: Generate product recommendations taking into account emotional data

[0717] The server generates product proposals based on the results of the AI ​​analysis and also takes into account the user's emotional data. For example, if a customer expresses dissatisfaction, it makes proposals to resolve that dissatisfaction (e.g., an upgrade plan with setup guides and additional support). Based on the emotional data, it adjusts the proposal content to increase customer satisfaction.

[0718] Step 7: Notification

[0719] The server notifies the sales department of the generated product proposal. Specifically, the proposal details are communicated to the sales representative via the company's internal email system or notification dashboard. The sales representative then follows up with the customer based on this notification. For example, they may contact Customer A to propose a new upgrade plan with a setup guide and additional support.

[0720] Example 2

[0721] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0722] Conventional customer portal sites were able to collect customer operations, usage status, and inquiries, but simply recording and analyzing this data made it difficult to make appropriate product proposals based on the customer's true needs and emotions. Furthermore, the collected data sometimes contained noise or duplicate data, which reduced the accuracy of the analysis. Furthermore, even if product proposals were generated, they were sometimes not efficiently communicated to the sales department. An effective solution to these issues was needed.

[0723] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0724] In this invention, the server includes means for recording customer operation details, means for recording customer usage status, means for recording customer inquiry details, means for preprocessing the recorded data, means for converting the preprocessed data into multimodal data, means for analyzing the converted multimodal data to generate product proposals, means for recognizing user emotions, means for generating product proposals taking into account the emotion data, and means for notifying the generated product proposals. This enables precise product proposals that also take emotions into account based on the customer's operation history and inquiry details. Furthermore, preprocessing the collected data removes noise and duplicate data, enabling highly accurate analysis and enabling efficient notification to the sales department.

[0725] "Customer operations" refers to a series of operations performed by customers on the portal site, such as click information and page viewing information.

[0726] "Customer usage" refers to the usage history, such as connection time and data consumption, when a customer uses a portal site.

[0727] "Customer inquiry content" refers to inquiry records such as support messages and FAQ search history made by customers when using the portal site.

[0728] "Recorded data" refers to all data stored on the server regarding customer operations, usage, and inquiries.

[0729] "Preprocessing" refers to the process of removing unnecessary noise and redundant data from recorded data and adjusting the data format.

[0730] "Multimodal data" refers to a format that integrates data of different formats (text data, numerical data, time data, etc.).

[0731] "Means for recognizing user emotions" refers to an emotion engine that analyzes the content of customer inquiries and operations and determines the customer's emotions (dissatisfaction, excitement, etc.).

[0732] "Means for generating product proposals" refers to means for predicting customer needs using AI analysis and proposing optimal products and services.

[0733] The "means for generating a product proposal taking into consideration emotional data" refers to a means for generating a more precise product proposal based on the emotional data of the user.

[0734] The "means for notifying the generated product proposal" refers to a means for effectively communicating the generated product proposal to the sales department.

[0735] This invention combines a system that effectively collects customer operations, usage status, and inquiries on a customer portal site, generates product proposals based on that data, and notifies the sales department of the proposals, with an emotion engine that recognizes user emotions. Specific embodiments of the present invention are described below.

[0736] System Configuration

[0737] The system is primarily composed of a server and terminals, and generates product proposals by recording and analyzing the operations, usage status, and inquiries that occur when customers (users) use the portal site, and notifies these proposals to the sales department.

[0738] Hardware and software used

[0739] Server: The central location for data collection, pre-processing, analysis, proposal generation, and notifications.

[0740] Terminal: The device (PC, smartphone, tablet, etc.) that users use to access the portal site.

[0741] Emotion engine: Software that analyzes user emotions and generates emotion data.

[0742] Data collection

[0743] The server collects the operations performed by customers on the portal site in real time. Specifically, click information, page view information, and inquiry details are instantly saved in a log file. When a user starts a subscription service, the operation log is immediately transferred to the server.

[0744] Examples:

[0745] The operation log generated by the terminal is sent to the server.

[0746] The server saves click information in a log file when Customer A starts using the "Subscription Service."

[0747] Data Preprocessing

[0748] The collected log data is filtered by the server to remove unnecessary noise and duplicate data, and through this process the data is organized into a standard format suitable for subsequent analysis.

[0749] Examples:

[0750] The server removes duplicate entries and meaningless data and formats the normal log data into a standard format.

[0751] Data Conversion

[0752] The preprocessed data is converted into multimodal data by the server, which integrates different types of data (text data, numerical data, time data, etc.) into one and allows for unified handling.

[0753] Examples:

[0754] The server saves the inquiry content as text data, converts the usage amount into numerical data, and integrates the operation date and time as time data.

[0755] Emotion recognition by emotion engine

[0756] The emotion engine analyzes the user's inquiries and operations to recognize their emotions. For example, it determines whether the user is dissatisfied or excited based on the content of the text and the frequency of operations, and generates emotion data.

[0757] Examples:

[0758] The server performs text analysis of the inquiry, detects keywords such as "dissatisfaction" and "irritation," and determines the user's emotions.

[0759] Product proposal generation and notification

[0760] The server uses an AI model based on multimodal data to analyze customer behavior patterns and needs. It then generates optimal product proposals taking into account emotional data and notifies the sales department of these proposals. Notifications are sent via the company's internal email system and notification dashboard.

[0761] Examples:

[0762] The server detects that Customer A frequently inquires about new features and suggests an upgrade plan with a guided setup.

[0763] The server notifies the sales department staff of this proposal by email, and the staff contacts Customer A based on the proposal.

[0764] Prompt Sentence Examples

[0765] Below are examples of prompt sentences to efficiently generate product proposals based on an AI model.

[0766] "Based on this customer's operation history and inquiry details, predict the products and services that the customer may be interested in, and generate product recommendations that take into account usage history and sentiment data."

[0767] This system makes it possible to make precise and effective product proposals based on the customer's operation history and inquiry details, as well as taking into account the user's emotions, dramatically improving the efficiency of sales activities and customer satisfaction.

[0768] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0769] Step 1:

[0770] Data collection

[0771] The server collects the operations performed by customers on the portal site in real time. Specifically, click information and page view information generated by the device are sent to the server and saved in a log file. In addition, connection time, data consumption, and inquiry content are also recorded.

[0772] Input: Customer operation details, usage status, inquiry details

[0773] Output: Operation logs, usage data, and query records saved in log files

[0774] Specific behavior:

[0775] The terminal transmits the user's click information on the portal site to the server.

[0776] The server records click information and page view information in a log file.

[0777] Step 2:

[0778] Data Preprocessing

[0779] The server filters the collected log data, removing unnecessary and duplicate data, and formats the data for subsequent analysis.

[0780] Input: Recorded log data

[0781] Output: filtered and standard formatted data

[0782] Specific behavior:

[0783] The server removes duplicate entries where the same operation has been performed multiple times.

[0784] Remove unnecessary noise data.

[0785] Convert the data into a standard format.

[0786] Step 3:

[0787] Data Conversion

[0788] The server converts the preprocessed data into multimodal data, integrating different types of data (text data, numerical data, time data, etc.) and handling them in a unified manner.

[0789] Input: Preprocessed log data

[0790] Output: Multimodal data

[0791] Specific behavior:

[0792] The server saves the inquiry content as text data.

[0793] Convert usage into numerical data.

[0794] The operation date and time are recorded as time data.

[0795] Integrate all data as multimodal data.

[0796] Step 4:

[0797] Emotion recognition by emotion engine

[0798] The server analyzes the user's inquiries and operations, recognizes the user's emotions, and adds the emotion information to the multimodal data.

[0799] Input: Inquiry details, operation frequency data

[0800] Output: Multimodal data including emotion data

[0801] Specific behavior:

[0802] The server performs text analysis of the inquiry content and extracts keywords that indicate emotions.

[0803] The frequency of operations is analyzed to determine the user's emotions.

[0804] Adding emotion data to multimodal data.

[0805] Step 5:

[0806] AI analysis

[0807] The server inputs multimodal data into an AI algorithm to analyze customer behavior patterns and needs, generating product recommendations that customers may be interested in.

[0808] Input: Multimodal data including emotion data

[0809] Output: Product proposals based on customer needs

[0810] Specific behavior:

[0811] The server inputs multimodal data, including emotional data, into the AI ​​model.

[0812] Analyze behavioral patterns and needs to generate product proposals.

[0813] Step 6:

[0814] Product proposal generation taking into account emotional data

[0815] The server generates more sophisticated product proposals based on the results of AI analysis and emotional data.

[0816] Input: AI analysis results, emotional data

[0817] Output: Product proposals that take emotions into account

[0818] Specific behavior:

[0819] If the customer is dissatisfied, the server generates a proposal to resolve the dissatisfaction.

[0820] Reflect emotional data in proposal content.

[0821] Step 7:

[0822] notification

[0823] The server notifies the sales department of the generated product proposals, using an email system and a notification dashboard to communicate the proposal details in real time.

[0824] Input: Generated product proposal

[0825] Output: Product proposal notified to the sales department

[0826] Specific behavior:

[0827] The server sends the generated product proposal to the sales representative via an email system.

[0828] Sales representatives receive the proposal and follow up with the customer.

[0829] (Application example 2)

[0830] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0831] With conventional portal sites, it was difficult to effectively collect customer operations, usage status, and inquiry details and generate product proposals based on that information. Furthermore, proposals were made without taking into account customer emotions, making it difficult to improve customer satisfaction. Therefore, there is a need for a system that can accurately grasp customer emotions and needs and make optimal product proposals.

[0832] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for recording customer operation details, means for recording customer usage status, means for recording customer inquiry details, means for preprocessing the recorded data, means for converting the preprocessed data into multimodal data, means for analyzing the converted multimodal data to generate product proposals, means for generating user emotion data using an emotion engine that recognizes user emotions and including the emotion data in the multimodal data, and means for notifying the generated product proposals. This makes it possible to comprehensively analyze customer operations, usage status, and inquiry details, and to make precise product proposals that take emotions into consideration.

[0833] 1. "Means for recording customer operations" refers to means for saving information about operations performed by customers on electronic devices or websites as logs.

[0834] 2. "Means for recording customer usage" means means for recording usage data such as the time and amount consumed when a customer uses a service or product.

[0835] 3. "Means for recording customer inquiries" refers to means for saving the details and history of support requests and inquiries made by customers.

[0836] 4. "Means for pre-processing recorded data" means means for removing unnecessary information and redundant data from collected data and converting it into a format that can be analyzed.

[0837] 5. "Means for converting preprocessed data into multimodal data" refers to means for converting data of different formats (text data, numerical data, time data, etc.) so that they can be handled in a unified manner.

[0838] 6. "Means for analyzing converted multimodal data and generating product proposals" refers to means for analyzing customer needs and behavioral patterns based on converted data and creating appropriate product proposals.

[0839] 7. "Means for generating user emotion data using an emotion engine that recognizes user emotions and including the emotion data in multimodal data" refers to means for analyzing user emotions, saving them as numerical data, and integrating them with other data.

[0840] 8. "Means for notifying the generated product proposal" refers to the means for notifying the customer or sales representative of the product proposal by means such as email or push notification.

[0841] This invention combines a system that effectively collects customer operations, usage status, and inquiries on a customer portal site, and generates and notifies product proposals based on that data with an emotion engine that recognizes user emotions.

[0842] First, the server records the operations performed by customers when they use the portal site or smartphone app in real time. The operations include click information and page browsing history. The server also records the customer's usage status (e.g., connection time and data consumption) and inquiries (e.g., support messages and FAQ search history).

[0843] The server then preprocesses the collected log data, using pandas to remove unnecessary and duplicate data and convert it into a standard format for easier analysis. This preprocessing ensures that only meaningful data remains.

[0844] The server then converts the preprocessed data into multimodal data, integrating the query content as text data, the usage amount as numerical data, and the operation date and time as time data, making it possible to handle different types of data in a unified manner.

[0845] Furthermore, the server uses an emotion engine to recognize the user's emotions. Specifically, it uses the SentimentIntensityAnalyzer in the nltk library to determine the user's emotions from the content of the text and the frequency of operations. This emotion data is also included in the multimodal data.

[0846] The server uses the converted multimodal data to analyze customer behavior patterns and needs using AI algorithms. Specifically, it quantifies the text data using TfidfVectorizer, scales it using StandardScaler, and performs cluster analysis using KMeans. This analysis predicts whether the customer is interested in a specific product or service.

[0847] The server then generates product recommendations based on the results of the AI ​​analysis and the emotional data. For example, if the emotion engine indicates that the user is dissatisfied, it can make suggestions to resolve that dissatisfaction (e.g., expanding support services).

[0848] Finally, the server notifies the customer and the sales department of the generated product proposal via email or push notification, allowing the sales representative to follow up on the customer's needs.

[0849] Specific examples

[0850] 1. If Customer A views Product B multiple times and posts a question about how to set it up, the emotion engine determines that they are dissatisfied. In this case, the server will suggest a guided upgrade plan for Product B.

[0851] 2. If Customer B expresses dissatisfaction with the service, a proposal to resolve the dissatisfaction by expanding the support content is generated and notified to Customer B.

[0852] Prompt Sentence Examples

[0853] "Generate a list of recommended products related to product B, which this customer has recently viewed frequently."

[0854] "Please tell us the best product or support you can offer to resolve the issue for Customer B, who is dissatisfied with your service."

[0855] This makes it possible to comprehensively analyze customer operations and inquiries, and to make precise product proposals that take emotions into consideration.

[0856] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0857] Step 1:

[0858] The server records customer operations in real time. Every time a customer views or clicks on a product page, that information is saved in a log. For example, an operation log is generated stating, "Customer clicked on product A." The input is the customer's operation, and the output is the recorded operation log.

[0859] Step 2:

[0860] The server records the customer's usage, which includes information such as the time the customer uses the service and the amount of data consumed. For example, a usage log is generated stating that "the customer used the service for 30 minutes." The input is the customer's service usage, and the output is the recorded usage log.

[0861] Step 3:

[0862] The server records customer inquiries. Every time a customer sends a support message or searches an FAQ, the content is saved in a log. For example, an inquiry log is generated that reads, "A customer inquired about how to set up product B." The input is the customer's inquiry, and the output is the recorded inquiry log.

[0863] Step 4:

[0864] The server preprocesses the collected log data. It uses pandas to remove unnecessary and duplicate data and convert it into a standard format. For example, it removes duplicate click information and meaningless text. The input is the recorded operation log, usage log, and query log, and the output is the preprocessed data.

[0865] Step 5:

[0866] The server converts the preprocessed data into multimodal data, which allows different types of data to be handled in a unified manner. For example, query content can be integrated as text data, usage amount as numerical data, and operation date and time as time data. The input is preprocessed data, and the output is multimodal data.

[0867] Step 6:

[0868] The server uses an emotion engine to recognize the user's emotions. Specifically, it uses the SentimentIntensityAnalyzer from the nltk library. It determines the user's emotions (e.g., dissatisfaction or excitement) from the text content and frequency of operations. The input is preprocessed text data, and the output is emotion data.

[0869] Step 7:

[0870] The server integrates the emotion data into multimodal data, which allows the emotion data to be analyzed together with other data. The input is emotion data and multimodal data, and the output is the integrated multimodal data.

[0871] Step 8:

[0872] The server runs an AI algorithm using the converted multimodal data to analyze customer behavior patterns and needs. The text data is quantified using TfidfVectorizer, scaled using StandardScaler, and subjected to cluster analysis using KMeans. The input is the integrated multimodal data, and the output is customer cluster information and analysis results.

[0873] Step 9:

[0874] The server generates product proposals based on the results of AI analysis and emotional data. For example, if a customer expresses dissatisfaction, it makes proposals to resolve the dissatisfaction (e.g., expanding support services). The input is the customer's cluster information and emotional data, and the output is the generated product proposal.

[0875] Step 10:

[0876] The server notifies the customer and the sales department of the generated product proposal. Notifications are sent by email, push notification, or other methods. The input is the generated product proposal, and the output is the notified product proposal.

[0877] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0878] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0879] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0880] [Third embodiment]

[0881] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0882] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0883] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0884] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0885] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0886] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0887] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0888] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0889] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0890] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0891] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0892] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0893] This invention is a system that effectively collects operation details, usage status, and inquiry details on a customer portal site, generates product proposals based on this data, and notifies the sales department of the proposals.

[0894] System Configuration

[0895] 1. Data Collection

[0896] The server has the function of recording the series of operations that customers perform on the portal site. For example, it collects click information when customers use a certain function and information on the browsing of specific pages. It also records how customers use the product (usage status) and the content of their inquiries.

[0897] 2. Data Preprocessing

[0898] The server filters the collected data to remove unnecessary and duplicated data, such as repeated logs of the same operation and noise data, and normalizes the data into a standard format to facilitate subsequent analysis.

[0899] 3. Data Conversion

[0900] The server converts the preprocessed data into multimodal data, specifically integrating text data (inquiry content), numerical data (usage amount), and time data (operation timing) into a single dataset.

[0901] 4. AI analysis

[0902] The server feeds multimodal data into AI models to analyze customer behavior patterns and needs, for example using machine learning algorithms to predict whether a customer is interested in a particular product feature.

[0903] 5. Generate product proposals

[0904] Based on the results of the AI ​​analysis, the server generates optimal product proposals for customers, such as upgrades and new services related to features that customers frequently inquire about.

[0905] 6. Notification

[0906] The server notifies the sales department of the generated product proposals via an internal email system, a notification dashboard, or by sending a direct message to a specific sales representative.

[0907] Specific examples

[0908] 1. Data Collection

[0909] The server collects operation logs when Customer A starts using the "new subscription service."

[0910] The server also records the inquiries made by Customer A regarding "how to set up this service."

[0911] 2. Data Preprocessing

[0912] The server removes duplicate entries and unnecessary noise from the operation log.

[0913] The server formats the normal log data into a standard format.

[0914] 3. Data Conversion

[0915] The server integrates the inquiry content as text data, the usage amount as numerical data, and the service start date and time as time data.

[0916] 4. AI analysis

[0917] The server inputs this integrated data into an AI model and analyzes Customer A's behavioral patterns.

[0918] The server proposes an upgrade plan with a new setup guide because Customer A frequently asks about "how to set it up."

[0919] 5. Generate product proposals

[0920] The server generates a guided configuration upgrade plan based on AI analysis.

[0921] 6. Notification

[0922] The server notifies the sales representative of the generated proposal by email, and the sales representative contacts Customer A.

[0923] In this way, this system significantly improves the efficiency of sales activities by making efficient and precise product proposals based on customer operations and inquiries.

[0924] The processing flow will be explained below.

[0925] Step 1: Data collection

[0926] The server records the operations performed by customers on the portal site in real time. Specifically, it saves click information and page viewing information when customers use certain functions in a log file. It also records customer usage status (e.g., connection time and data consumption) and inquiry content (e.g., support messages and FAQ search history).

[0927] Step 2: Data Preprocessing

[0928] The server filters the collected log data to remove duplicates and noise, such as duplicate entries of the same operation repeated multiple times and meaningless text (noise). It also converts the data into a standard format that makes it easier to analyze.

[0929] Step 3: Data conversion

[0930] The server converts the preprocessed data into multimodal data. For example, it combines the inquiry content as text data, the usage amount as numerical data, and the operation date and time as time data. This makes it possible to handle different types of data in a unified manner.

[0931] Step 4: AI analysis

[0932] The server then feeds the converted multimodal data into an AI algorithm to analyze customer behavior patterns and needs. Specifically, it uses machine learning models to predict whether a customer wants a specific feature. For example, if a customer frequently inquires about a new feature, it will recommend products or services that include that feature.

[0933] Step 5: Generate product proposals

[0934] The server generates product proposals based on the results of AI analysis. For example, it proposes upgrade plans or new services related to features that customers frequently inquire about. The proposals include the products and services offered, their benefits, and how to use them.

[0935] Step 6: Notification

[0936] The server notifies the sales department of the generated product proposal. Specifically, the proposal details are communicated to the sales representative via the company's internal email system and notification dashboard. The sales representative then follows up with the customer based on this notification.

[0937] These are the specific processing steps of this system, which makes it possible to effectively utilize customer operation and inquiry data and improve the efficiency of sales activities.

[0938] Example 1

[0939] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0940] Conventional customer support systems have difficulty effectively collecting customer operations, usage status, and inquiry details, and quickly and efficiently generating appropriate product proposals based on this data. This has led to problems such as reduced sales efficiency and difficulty in improving customer satisfaction. Furthermore, a lack of technology for precisely analyzing customer behavior patterns has made it difficult to provide accurate proposals. A new system that can solve these issues is needed.

[0941] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0942] In this invention, the server includes means for recording customer operation details, means for recording customer usage status, means for recording customer inquiry details, means for preprocessing the recorded data, means for converting the preprocessed data into multimodal data, means for inputting the converted multimodal data into a generative AI model to analyze customer behavior patterns, means for generating product proposals based on the analysis results, and means for notifying the sales department of the generated product proposals. This makes it possible to effectively collect and preprocess customer data and analyze it using an AI model, thereby enabling accurate and efficient generation of product proposals and notification to the sales department.

[0943] The "means for recording customer operations" is a function for recording operations such as clicks and page views that customers perform on the web portal site.

[0944] "Means for recording customer usage" refers to a function for recording how and to what extent customers use products and services.

[0945] "Means for recording customer inquiries" is a function for recording the inquiries made by customers to support or the help desk in text format.

[0946] "Means for preprocessing recorded data" refers to a function for removing noise and redundant data from collected data and preparing the data in a format suitable for analysis.

[0947] The "means for converting preprocessed data into multimodal data" is a function for integrating preprocessed data into multiple formats, such as text data, numerical data, and time data, to create a single data set.

[0948] "Means for inputting converted multimodal data into a generative artificial intelligence model to analyze customer behavior patterns" refers to a function for inputting multimodal data into an AI model to analyze customer behavior patterns and needs.

[0949] "Means for generating product proposals based on analysis results" refers to a function that automatically creates optimal product and service proposals for customers based on the results of analysis by AI.

[0950] The "means for notifying the sales department of the generated product proposal" is a function for communicating the generated product proposal to the sales department using an appropriate notification means.

[0951] MODE FOR CARRYING OUT THE INVENTION

[0952] This invention is a system that effectively collects operation details, usage status, and inquiry details on a customer portal site, generates product proposals based on this data, and notifies the sales department of the proposals. The configuration and operation of this system will be specifically described.

[0953] 1. Data Collection

[0954] The server has the function of recording a series of operations performed by customers on the portal site. For example, it collects information on customers clicking specific buttons and logs the specific pages they view. It also records how customers use products and services (usage status) and the content of their inquiries. For this purpose, the server uses a log collection module and database.

[0955] Examples:

[0956] The server collects operation logs when Customer A starts using the "new subscription service."

[0957] The server also records the inquiries made by Customer A regarding "how to set up this service."

[0958] 2. Data Preprocessing

[0959] The server filters the collected data to remove unnecessary or redundant data, such as repeated logs of the same operation or noise data, and normalizes the data into a standard format to facilitate subsequent analysis.

[0960] Examples:

[0961] The server removes duplicate entries and unnecessary noise from the operation log.

[0962] The server formats the normal log data into a standard format.

[0963] 3. Data Conversion

[0964] The server converts the preprocessed data into multimodal data, specifically integrating text data (inquiry content), numerical data (usage amount), and time data (operation timing) into a single dataset.

[0965] Examples:

[0966] The server integrates the inquiry content as text data, the usage amount as numerical data, and the service start date and time as time data.

[0967] 4. Analysis using AI models

[0968] The server feeds multimodal data into generative AI models to analyze customer behavior patterns and needs. For example, it uses machine learning algorithms to predict whether a customer is interested in a particular product feature. To do this, the server uses machine learning libraries and AI models.

[0969] Examples:

[0970] The server inputs this integrated data into an AI model and analyzes Customer A's behavioral patterns.

[0971] The server detects that customer A frequently asks about "how to set things up" and proposes a new guided setup upgrade plan.

[0972] 5. Generate product proposals

[0973] The server generates optimal product proposals for customers based on the analysis results of the AI ​​model, such as proposing upgrades or new services related to features that customers frequently inquire about.

[0974] Examples:

[0975] The server generates a guided upgrade plan based on the output of the AI ​​model.

[0976] 6. Notification of Proposal

[0977] The server notifies the sales department of the generated product proposals using the company's internal email system, a notification dashboard, or direct messaging to specific sales representatives.

[0978] Examples:

[0979] The server notifies the sales representative of the generated proposal via email, and the sales representative contacts Customer A and proposes a guided configuration upgrade plan.

[0980] Prompt Sentence Examples

[0981] "Design a system that generates optimal product proposals based on a customer's operation log and inquiry details, and notifies the sales department."

[0982] This system significantly improves the efficiency of sales activities by making efficient and precise product proposals based on customer operations and inquiries.

[0983] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0984] Divide the program's processing flow into processing steps

[0985] Step 1: Data collection

[0986] The server records the series of operations and inquiries that the user makes on the portal site.

[0987] Input: User clicks, page views, inquiries

[0988] Specific operation: When a customer clicks a specific button, the click information is recorded as a log along with a timestamp, and the content submitted in the inquiry form is saved as text data.

[0989] Output: Operation log and inquiry data

[0990] Step 2: Data Preprocessing

[0991] The server filters the collected data, removing unnecessary and redundant data, and normalizes the data into a standard format.

[0992] Input: Operation log and inquiry data

[0993] What it does: It finds and removes duplicate entries and noise data, and converts the remaining data into a standard format such as JSON.

[0994] Output: Preprocessed operation log and query data

[0995] Step 3: Data Integration and Transformation

[0996] The server converts the pre-processed data into multimodal data.

[0997] Input: Preprocessed operation log and query data

[0998] Specific operation: Text data (e.g., inquiry content), numerical data (e.g., usage amount), and time data (e.g., operation timing) are integrated into a single data set.

[0999] Output: Multimodal dataset

[1000] Step 4: Analysis by AI model

[1001] The server feeds the multimodal dataset into a generative AI model to analyze customer behavior patterns.

[1002] Input: Multimodal dataset

[1003] What it does: Multimodal data is fed into an AI model and analyzed using machine learning algorithms.

[1004] Output: Analysis of customer behavior patterns and needs

[1005] Step 5: Generate product proposals

[1006] The server generates optimal product proposals for customers based on the analysis results of the AI ​​model.

[1007] Input: Analysis results of customer behavior patterns and needs

[1008] What it does: Based on the analysis results, it generates recommendations for new services and additional features that customers may be interested in.

[1009] Output: Product proposal data

[1010] Step 6: Proposal Notification

[1011] The server notifies the sales department of the generated product proposal.

[1012] Input: Product proposal data

[1013] What happens next? Communicate proposals to the sales team via email and notification dashboards, sending messages directly to specific salespeople as needed.

[1014] Output: Notification data to the sales department

[1015] (Application example 1)

[1016] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1017] There is a need for a system that can effectively collect and analyze customer behavior data and generate appropriate product proposals to improve the efficiency of sales activities and customer satisfaction. However, current systems do not fully utilize customer behavior data, limiting the accuracy and degree of personalization of product proposals. In particular, there is a lack of a mechanism for integrating and analyzing various customer operations and inquiry data, making it difficult to provide proposals that are optimized for each individual customer.

[1018] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1019] In this invention, the server includes means for recording customer operation details, means for recording customer usage status, means for recording customer inquiry details, means for preprocessing the recorded data, means for converting the preprocessed data into multimodal data, means for analyzing the converted multimodal data to generate product proposals, means for notifying the generated product proposals, analysis means for clustering customer behavior, means for generating personalized product proposals based on the clustering results, and means for notifying the customer of the generated product proposals by email. This makes it possible to comprehensively analyze a variety of customer behavior data and provide optimal product proposals for each individual customer.

[1020] "Means for recording customer operations" refers to means for collecting operation logs such as clicks and page transitions that customers make on websites and applications.

[1021] "Means for recording customer usage" means means for monitoring and recording the extent to which customers use specific features or services.

[1022] The "means for recording customer inquiries" refers to a means for collecting questions and problem reports made by customers to customer support as text data.

[1023] "Means for pre-processing recorded data" refers to means for filtering and normalizing the collected raw data to prepare it for easier subsequent analysis.

[1024] A "means for transforming preprocessed data into multimodal data" is a means for integrating data of different formats (e.g., textual data, numerical data, temporal data) into a single analyzable data set.

[1025] "Means for analyzing converted multimodal data and generating product proposals" refers to means for analyzing data using techniques such as machine learning and generating proposals for products and services that are optimal for customers.

[1026] The "means for notifying the generated product proposal" refers to a means for notifying the customer or the sales department of the generated proposal using a method such as email or a dashboard.

[1027] The "analysis means for clustering customer behavior" is a means for analyzing collected customer behavior data and dividing them into groups of customers with common characteristics.

[1028] The "means for generating personalized product proposals based on clustering results" is a means for proposing individually optimized products and services based on the characteristics of each customer group identified by clustering.

[1029] The "means for notifying the customer of the generated product proposal by email" is a means for directly notifying the customer of the content of the proposal by sending the generated proposal to the customer's email address.

[1030] This invention is a system that collects and analyzes customer behavior data, and generates and notifies individually optimized product proposals. A specific embodiment of this system is described below.

[1031] System Overview

[1032] The system consists of the following main components:

[1033] 1. Data Collection Module

[1034] 2. Data Preprocessing Module

[1035] 3. Data Conversion Module

[1036] 4. AI analysis module

[1037] 5. Product proposal generation module

[1038] 6. Notification Module

[1039] Hardware and Software Used

[1040] Hardware: The system operates using a server or cloud environment, while data collection and notifications are performed using the user's smartphone, tablet, or PC.

[1041] Software: Python and Pandas are used for data processing, and machine learning libraries such as Scikit-learn and TensorFlow are used for analysis. A library (smtplib) that sends emails using the SMTP protocol is used for notifications.

[1042] Program processing explanation

[1043] Data Collection Module

[1044] The server collects user operations, usage status, and inquiries in real time, for example, by recording website click information, page browsing history, and product inquiries.

[1045] Data Preprocessing Module

[1046] The collected data is pre-processed on the server, which removes noise and duplicate data and arranges all data into a standard format. For example, if the same click information is recorded multiple times, it will be combined into one.

[1047] Data Conversion Module

[1048] The preprocessed data is converted into multimodal data, a process that combines text data (e.g., inquiry content), numerical data (e.g., product usage), and temporal data (e.g., operation duration) into a single dataset.

[1049] AI Analysis Module

[1050] The server then inputs the converted multimodal data into an AI model to analyze the user's behavioral patterns and needs, using a clustering algorithm to classify user behavior into groups with common characteristics.

[1051] Product proposal generation module

[1052] Based on the clustering results, the server generates product recommendations optimized for each user group. For example, if a user frequently inquires about how to use a particular product, the server can suggest new accessories related to that product.

[1053] Notification Module

[1054] The generated product proposal will be notified by email. The server will send the generated proposal to the user's registered email address. The email is sent using the SMTP protocol.

[1055] Specific examples

[1056] For example, if a user does the following:

[1057] Viewing a specific product page multiple times

[1058] Make an inquiry about "How to use the product"

[1059] Browse additional related products

[1060] The server collects this data, pre-processes and analyzes it, and then sends the following email to the user based on the analysis results:

[1061] Subject: Personalized product recommendations

[1062] Main text:

[1063] Hello,

[1064] Get recommended products related to your recently viewed products.

[1065] 1. Upgrade kit for product X

[1066] 2. Accessories related to product X

[1067] Please check it out.

[1068] An example prompt for a generative AI model would be:

[1069] Generate optimal product recommendations based on user behavior data, including:

[1070] Click information: / home, / product / 3, / product / 5

[1071] Inquiry: How to return product? Warranty information

[1072] In this way, personalized product proposals based on user-specific data can be made, thereby improving the user experience and streamlining sales activities.

[1073] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1074] Step 1:

[1075] The server collects user operations, usage, and inquiries in real time, for example, logging the links users click on websites, the pages they view, the search terms they use, and the information they enter into inquiry forms. The collected data is stored as raw data.

[1076] Input: User operation logs, usage data, and inquiry details

[1077] Output: Raw data collected

[1078] Step 2:

[1079] The server pre-processes the collected raw data, removing unnecessary and duplicate data and arranging the data into a standard format, specifically combining multiple records of the same click and removing noise data (e.g., error logs).

[1080] Input: Raw data collected

[1081] Output: Preprocessed data

[1082] Step 3:

[1083] The server converts the preprocessed data into multimodal data, a process that integrates data of different formats (e.g., textual, numerical, and temporal data) into a single dataset that facilitates subsequent analysis.

[1084] Input: Preprocessed data

[1085] Output: Multimodal data

[1086] Step 4:

[1087] The server inputs the multimodal data into an AI model to analyze user behavior patterns and needs. A clustering algorithm is used for the analysis, classifying user behavior into groups with common characteristics. For example, KMeans clustering is used to classify customers into multiple groups.

[1088] Input: Multimodal data

[1089] Output: Clustered user groups

[1090] Step 5:

[1091] The server generates personalized product recommendations based on the clustering results. This process involves suggesting the best products and services for each user group. For example, if a user is interested in a particular product, it will suggest accessories and upgrades related to that product.

[1092] Input: Clustered user groups

[1093] Output: Personalized product proposals

[1094] Step 6:

[1095] The server notifies the user of the generated product proposal via email. During this process, the product proposal is sent to the user's registered email address. The email containing the proposal is sent using the SMTP protocol.

[1096] Input: Personalized product suggestions

[1097] Output: Product proposal email sent to user

[1098] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1099] This invention combines a system that effectively collects customer operations, usage status, and inquiries on a customer portal site, generates product proposals based on that data, and notifies the sales department of the proposals with an emotion engine that recognizes user emotions.

[1100] System Configuration

[1101] 1. Data Collection

[1102] The server records in real time the series of operations that customers perform on the portal site. For example, it saves click information and page view information when customers use a certain function in a log file. It also records customer usage status (e.g., connection time and data consumption) and inquiry content (e.g., support messages and FAQ search history).

[1103] 2. Data Preprocessing

[1104] The server filters the collected log data to remove unnecessary and duplicate data, such as duplicate entries of the same operation repeated multiple times and meaningless text (noise), and converts the data into a standard format that makes it easier to analyze.

[1105] 3. Data Conversion

[1106] The server converts the preprocessed data into multimodal data. For example, it combines the inquiry content as text data, the usage amount as numerical data, and the operation date and time as time data. This makes it possible to handle different types of data in a unified manner.

[1107] 4. Emotion Recognition by Emotion Engine

[1108] The server analyzes the user's inquiries and operations to recognize their emotions. For example, it determines whether the user is frustrated or excited based on the content of the text and the frequency of operations. This emotional data is also included in the multimodal data.

[1109] 5. AI analysis

[1110] The server then feeds the converted multimodal data into an AI algorithm to analyze customer behavior patterns and needs. Specifically, it uses machine learning models to predict whether a customer wants a specific feature. For example, if a customer frequently inquires about a new feature, it will recommend products or services that include that feature.

[1111] 6. Product proposal generation taking into account emotional data

[1112] The server generates product proposals based on the results of the AI ​​analysis and also takes into account the user's emotional data. For example, if a customer expresses dissatisfaction, it will make proposals to resolve that dissatisfaction (e.g., expanding support services).

[1113] 7. Notification

[1114] The server notifies the sales department of the generated product proposal. Specifically, the proposal details are communicated to the sales representative via the company's internal email system and notification dashboard. The sales representative then follows up with the customer based on this notification.

[1115] Specific examples

[1116] 1. Data Collection

[1117] The server collects operation logs when Customer A starts using the "Subscription Service."

[1118] The server records the details of the inquiry made by Customer A regarding "how to set up this service."

[1119] 2. Data Preprocessing

[1120] The server removes duplicate entries and unnecessary noise from the operations log.

[1121] The server formats the normal log data into a standard format.

[1122] 3. Data Conversion

[1123] The server integrates the inquiry content as text data, the usage amount as numerical data, and the service start date and time as time data.

[1124] 4. Emotion Recognition by Emotion Engine

[1125] The server analyzes the content of the inquiry and the frequency of operations and detects that customer A is dissatisfied.

[1126] 5. AI analysis

[1127] The server inputs the integrated data into an AI model and analyzes customer A's behavioral patterns.

[1128] The server proposes an upgrade plan with a new setup guide because Customer A frequently asks about "how to set it up."

[1129] 6. Product proposal generation taking into account emotional data

[1130] To resolve Customer A's dissatisfaction, the server generates an upgrade plan with a setup guide as well as a proposal to expand the support content.

[1131] 7. Notification

[1132] The server notifies the sales representative of the generated proposal by email, and the sales representative contacts Customer A based on the proposal.

[1133] In this way, by combining this system with an emotion engine, it is possible to make efficient and precise product proposals that take customer emotions into consideration, thereby improving the efficiency of sales activities and customer satisfaction.

[1134] The processing flow will be explained below.

[1135] Step 1: Data collection

[1136] The server records in real time the operations performed by customers on the portal site. For example, it stores click information and page browsing history in a log when a customer starts a "subscription service." It also records connection time and data consumption while the customer is using the service. It also records the content of inquiries made by customers regarding "how to set up this service."

[1137] Step 2: Data Preprocessing

[1138] The server filters the collected log data to remove unnecessary and duplicate data, such as duplicate entries of the same operation repeated multiple times and meaningless text (noise), and converts the data into a standard format to ensure consistency.

[1139] Step 3: Data conversion

[1140] The server converts the preprocessed data into multimodal data. Specifically, it integrates the inquiry content as text data, the usage amount as numerical data, and the operation date and time as time data. This makes it possible to handle different types of data in a unified manner.

[1141] Step 4: Emotion Recognition with the Emotion Engine

[1142] The server analyzes the user's inquiries and operations to recognize the user's emotions. For example, it can determine whether the user is happy or dissatisfied from the text of the inquiry. It can also estimate the user's emotions from the frequency and patterns of operations. This emotional data is included in the multimodal data.

[1143] Step 5: AI analysis

[1144] The server then feeds the converted multimodal data into an AI algorithm to analyze customer behavior patterns and needs. Specifically, it uses machine learning models to predict whether a customer wants a specific feature. For example, if a customer frequently inquires about a new feature, it will recommend products or services that include that feature.

[1145] Step 6: Generate product recommendations taking into account emotional data

[1146] The server generates product proposals based on the results of the AI ​​analysis and also takes into account the user's emotional data. For example, if a customer expresses dissatisfaction, it makes proposals to resolve that dissatisfaction (e.g., an upgrade plan with setup guides and additional support). Based on the emotional data, it adjusts the proposal content to increase customer satisfaction.

[1147] Step 7: Notification

[1148] The server notifies the sales department of the generated product proposal. Specifically, the proposal details are communicated to the sales representative via the company's internal email system or notification dashboard. The sales representative then follows up with the customer based on this notification. For example, they may contact Customer A to propose a new upgrade plan with a setup guide and additional support.

[1149] Example 2

[1150] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1151] Conventional customer portal sites were able to collect customer operations, usage status, and inquiries, but simply recording and analyzing this data made it difficult to make appropriate product proposals based on the customer's true needs and emotions. Furthermore, the collected data sometimes contained noise or duplicate data, which reduced the accuracy of the analysis. Furthermore, even if product proposals were generated, they were sometimes not efficiently communicated to the sales department. An effective solution to these issues was needed.

[1152] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1153] In this invention, the server includes means for recording customer operation details, means for recording customer usage status, means for recording customer inquiry details, means for preprocessing the recorded data, means for converting the preprocessed data into multimodal data, means for analyzing the converted multimodal data to generate product proposals, means for recognizing user emotions, means for generating product proposals taking into account the emotion data, and means for notifying the generated product proposals. This enables precise product proposals that also take emotions into account based on the customer's operation history and inquiry details. Furthermore, preprocessing the collected data removes noise and duplicate data, enabling highly accurate analysis and enabling efficient notification to the sales department.

[1154] "Customer operations" refers to a series of operations performed by customers on the portal site, such as click information and page viewing information.

[1155] "Customer usage" refers to the usage history, such as connection time and data consumption, when a customer uses a portal site.

[1156] "Customer inquiry content" refers to inquiry records such as support messages and FAQ search history made by customers when using the portal site.

[1157] "Recorded data" refers to all data stored on the server regarding customer operations, usage, and inquiries.

[1158] "Preprocessing" refers to the process of removing unnecessary noise and redundant data from recorded data and adjusting the data format.

[1159] "Multimodal data" refers to a format that integrates data of different formats (text data, numerical data, time data, etc.).

[1160] "Means for recognizing user emotions" refers to an emotion engine that analyzes the content of customer inquiries and operations and determines the customer's emotions (dissatisfaction, excitement, etc.).

[1161] "Means for generating product proposals" refers to means for predicting customer needs using AI analysis and proposing optimal products and services.

[1162] The "means for generating a product proposal taking into consideration emotional data" refers to a means for generating a more precise product proposal based on the emotional data of the user.

[1163] The "means for notifying the generated product proposal" refers to a means for effectively communicating the generated product proposal to the sales department.

[1164] This invention combines a system that effectively collects customer operations, usage status, and inquiries on a customer portal site, generates product proposals based on that data, and notifies the sales department of the proposals, with an emotion engine that recognizes user emotions. Specific embodiments of the present invention are described below.

[1165] System Configuration

[1166] The system is primarily composed of a server and terminals, and generates product proposals by recording and analyzing the operations, usage status, and inquiries that occur when customers (users) use the portal site, and notifies these proposals to the sales department.

[1167] Hardware and software used

[1168] Server: The central location for data collection, pre-processing, analysis, proposal generation, and notifications.

[1169] Terminal: The device (PC, smartphone, tablet, etc.) that users use to access the portal site.

[1170] Emotion engine: Software that analyzes user emotions and generates emotion data.

[1171] Data collection

[1172] The server collects the operations performed by customers on the portal site in real time. Specifically, click information, page view information, and inquiry details are instantly saved in a log file. When a user starts a subscription service, the operation log is immediately transferred to the server.

[1173] Examples:

[1174] The operation log generated by the terminal is sent to the server.

[1175] The server saves click information in a log file when Customer A starts using the "Subscription Service."

[1176] Data Preprocessing

[1177] The collected log data is filtered by the server to remove unnecessary noise and duplicate data, and through this process the data is organized into a standard format suitable for subsequent analysis.

[1178] Examples:

[1179] The server removes duplicate entries and meaningless data and formats the normal log data into a standard format.

[1180] Data Conversion

[1181] The preprocessed data is converted into multimodal data by the server, which integrates different types of data (text data, numerical data, time data, etc.) into one and allows for unified handling.

[1182] Examples:

[1183] The server saves the inquiry content as text data, converts the usage amount into numerical data, and integrates the operation date and time as time data.

[1184] Emotion recognition by emotion engine

[1185] The emotion engine analyzes the user's inquiries and operations to recognize their emotions. For example, it determines whether the user is dissatisfied or excited based on the content of the text and the frequency of operations, and generates emotion data.

[1186] Examples:

[1187] The server performs text analysis of the inquiry, detects keywords such as "dissatisfaction" and "irritation," and determines the user's emotions.

[1188] Product proposal generation and notification

[1189] The server uses an AI model based on multimodal data to analyze customer behavior patterns and needs. It then generates optimal product proposals taking into account emotional data and notifies the sales department of these proposals. Notifications are sent via the company's internal email system and notification dashboard.

[1190] Examples:

[1191] The server detects that Customer A frequently inquires about new features and suggests an upgrade plan with a guided setup.

[1192] The server notifies the sales department staff of this proposal by email, and the staff contacts Customer A based on the proposal.

[1193] Prompt Sentence Examples

[1194] Below are examples of prompt sentences to efficiently generate product proposals based on an AI model.

[1195] "Based on this customer's operation history and inquiry details, predict the products and services that the customer may be interested in, and generate product recommendations that take into account usage history and sentiment data."

[1196] This system makes it possible to make precise and effective product proposals based on the customer's operation history and inquiry details, as well as taking into account the user's emotions, dramatically improving the efficiency of sales activities and customer satisfaction.

[1197] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1198] Step 1:

[1199] Data collection

[1200] The server collects the operations performed by customers on the portal site in real time. Specifically, click information and page view information generated by the device are sent to the server and saved in a log file. In addition, connection time, data consumption, and inquiry content are also recorded.

[1201] Input: Customer operation details, usage status, inquiry details

[1202] Output: Operation logs, usage data, and query records saved in log files

[1203] Specific behavior:

[1204] The terminal transmits the user's click information on the portal site to the server.

[1205] The server records click information and page view information in a log file.

[1206] Step 2:

[1207] Data Preprocessing

[1208] The server filters the collected log data, removing unnecessary and duplicate data, and formats the data for subsequent analysis.

[1209] Input: Recorded log data

[1210] Output: filtered and standard formatted data

[1211] Specific behavior:

[1212] The server removes duplicate entries where the same operation has been performed multiple times.

[1213] Remove unnecessary noise data.

[1214] Convert the data into a standard format.

[1215] Step 3:

[1216] Data Conversion

[1217] The server converts the preprocessed data into multimodal data, integrating different types of data (text data, numerical data, time data, etc.) and handling them in a unified manner.

[1218] Input: Preprocessed log data

[1219] Output: Multimodal data

[1220] Specific behavior:

[1221] The server saves the inquiry content as text data.

[1222] Convert usage into numerical data.

[1223] The operation date and time are recorded as time data.

[1224] Integrate all data as multimodal data.

[1225] Step 4:

[1226] Emotion recognition by emotion engine

[1227] The server analyzes the user's inquiries and operations, recognizes the user's emotions, and adds the emotion information to the multimodal data.

[1228] Input: Inquiry details, operation frequency data

[1229] Output: Multimodal data including emotion data

[1230] Specific behavior:

[1231] The server performs text analysis of the inquiry content and extracts keywords that indicate emotions.

[1232] The frequency of operations is analyzed to determine the user's emotions.

[1233] Adding emotion data to multimodal data.

[1234] Step 5:

[1235] AI analysis

[1236] The server inputs multimodal data into an AI algorithm to analyze customer behavior patterns and needs, generating product recommendations that customers may be interested in.

[1237] Input: Multimodal data including emotion data

[1238] Output: Product proposals based on customer needs

[1239] Specific behavior:

[1240] The server inputs multimodal data, including emotional data, into the AI ​​model.

[1241] Analyze behavioral patterns and needs to generate product proposals.

[1242] Step 6:

[1243] Product proposal generation taking into account emotional data

[1244] The server generates more sophisticated product proposals based on the results of AI analysis and emotional data.

[1245] Input: AI analysis results, emotional data

[1246] Output: Product proposals that take emotions into account

[1247] Specific behavior:

[1248] If the customer is dissatisfied, the server generates a proposal to resolve the dissatisfaction.

[1249] Reflect emotional data in proposal content.

[1250] Step 7:

[1251] notification

[1252] The server notifies the sales department of the generated product proposals, using an email system and a notification dashboard to communicate the proposal details in real time.

[1253] Input: Generated product proposal

[1254] Output: Product proposal notified to the sales department

[1255] Specific behavior:

[1256] The server sends the generated product proposal to the sales representative via an email system.

[1257] Sales representatives receive the proposal and follow up with the customer.

[1258] (Application example 2)

[1259] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1260] With conventional portal sites, it was difficult to effectively collect customer operations, usage status, and inquiry details and generate product proposals based on that information. Furthermore, proposals were made without taking into account customer emotions, making it difficult to improve customer satisfaction. Therefore, there is a need for a system that can accurately grasp customer emotions and needs and make optimal product proposals.

[1261] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for recording customer operation details, means for recording customer usage status, means for recording customer inquiry details, means for preprocessing the recorded data, means for converting the preprocessed data into multimodal data, means for analyzing the converted multimodal data to generate product proposals, means for generating user emotion data using an emotion engine that recognizes user emotions and including the emotion data in the multimodal data, and means for notifying the generated product proposals. This makes it possible to comprehensively analyze customer operations, usage status, and inquiry details, and to make precise product proposals that take emotions into consideration.

[1262] 1. "Means for recording customer operations" refers to means for saving information about operations performed by customers on electronic devices or websites as logs.

[1263] 2. "Means for recording customer usage" means means for recording usage data such as the time and amount consumed when a customer uses a service or product.

[1264] 3. "Means for recording customer inquiries" refers to means for saving the details and history of support requests and inquiries made by customers.

[1265] 4. "Means for pre-processing recorded data" means means for removing unnecessary information and redundant data from collected data and converting it into a format that can be analyzed.

[1266] 5. "Means for converting preprocessed data into multimodal data" refers to means for converting data of different formats (text data, numerical data, time data, etc.) so that they can be handled in a unified manner.

[1267] 6. "Means for analyzing converted multimodal data and generating product proposals" refers to means for analyzing customer needs and behavioral patterns based on converted data and creating appropriate product proposals.

[1268] 7. "Means for generating user emotion data using an emotion engine that recognizes user emotions and including the emotion data in multimodal data" refers to means for analyzing user emotions, saving them as numerical data, and integrating them with other data.

[1269] 8. "Means for notifying the generated product proposal" refers to the means for notifying the customer or sales representative of the product proposal by means such as email or push notification.

[1270] This invention combines a system that effectively collects customer operations, usage status, and inquiries on a customer portal site, and generates and notifies product proposals based on that data with an emotion engine that recognizes user emotions.

[1271] First, the server records the operations performed by customers when they use the portal site or smartphone app in real time. The operations include click information and page browsing history. The server also records the customer's usage status (e.g., connection time and data consumption) and inquiries (e.g., support messages and FAQ search history).

[1272] The server then preprocesses the collected log data, using pandas to remove unnecessary and duplicate data and convert it into a standard format for easier analysis. This preprocessing ensures that only meaningful data remains.

[1273] The server then converts the preprocessed data into multimodal data, integrating the query content as text data, the usage amount as numerical data, and the operation date and time as time data, making it possible to handle different types of data in a unified manner.

[1274] Furthermore, the server uses an emotion engine to recognize the user's emotions. Specifically, it uses the SentimentIntensityAnalyzer in the nltk library to determine the user's emotions from the content of the text and the frequency of operations. This emotion data is also included in the multimodal data.

[1275] The server uses the converted multimodal data to analyze customer behavior patterns and needs using AI algorithms. Specifically, it quantifies the text data using TfidfVectorizer, scales it using StandardScaler, and performs cluster analysis using KMeans. This analysis predicts whether the customer is interested in a specific product or service.

[1276] The server then generates product recommendations based on the results of the AI ​​analysis and the emotional data. For example, if the emotion engine indicates that the user is dissatisfied, it can make suggestions to resolve that dissatisfaction (e.g., expanding support services).

[1277] Finally, the server notifies the customer and the sales department of the generated product proposal via email or push notification, allowing the sales representative to follow up on the customer's needs.

[1278] Specific examples

[1279] 1. If Customer A views Product B multiple times and posts a question about how to set it up, the emotion engine determines that they are dissatisfied. In this case, the server will suggest a guided upgrade plan for Product B.

[1280] 2. If Customer B expresses dissatisfaction with the service, a proposal to resolve the dissatisfaction by expanding the support content is generated and notified to Customer B.

[1281] Prompt Sentence Examples

[1282] "Generate a list of recommended products related to product B, which this customer has recently viewed frequently."

[1283] "Please tell us the best product or support you can offer to resolve the issue for Customer B, who is dissatisfied with your service."

[1284] This makes it possible to comprehensively analyze customer operations and inquiries, and to make precise product proposals that take emotions into consideration.

[1285] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1286] Step 1:

[1287] The server records customer operations in real time. Every time a customer views or clicks on a product page, that information is saved in a log. For example, an operation log is generated stating, "Customer clicked on product A." The input is the customer's operation, and the output is the recorded operation log.

[1288] Step 2:

[1289] The server records the customer's usage, which includes information such as the time the customer uses the service and the amount of data consumed. For example, a usage log is generated stating that "the customer used the service for 30 minutes." The input is the customer's service usage, and the output is the recorded usage log.

[1290] Step 3:

[1291] The server records customer inquiries. Every time a customer sends a support message or searches an FAQ, the content is saved in a log. For example, an inquiry log is generated that reads, "A customer inquired about how to set up product B." The input is the customer's inquiry, and the output is the recorded inquiry log.

[1292] Step 4:

[1293] The server preprocesses the collected log data. It uses pandas to remove unnecessary and duplicate data and convert it into a standard format. For example, it removes duplicate click information and meaningless text. The input is the recorded operation log, usage log, and query log, and the output is the preprocessed data.

[1294] Step 5:

[1295] The server converts the preprocessed data into multimodal data, which allows different types of data to be handled in a unified manner. For example, query content can be integrated as text data, usage amount as numerical data, and operation date and time as time data. The input is preprocessed data, and the output is multimodal data.

[1296] Step 6:

[1297] The server uses an emotion engine to recognize the user's emotions. Specifically, it uses the SentimentIntensityAnalyzer from the nltk library. It determines the user's emotions (e.g., dissatisfaction or excitement) from the text content and frequency of operations. The input is preprocessed text data, and the output is emotion data.

[1298] Step 7:

[1299] The server integrates the emotion data into multimodal data, which allows the emotion data to be analyzed together with other data. The input is emotion data and multimodal data, and the output is the integrated multimodal data.

[1300] Step 8:

[1301] The server runs an AI algorithm using the converted multimodal data to analyze customer behavior patterns and needs. The text data is quantified using TfidfVectorizer, scaled using StandardScaler, and subjected to cluster analysis using KMeans. The input is the integrated multimodal data, and the output is customer cluster information and analysis results.

[1302] Step 9:

[1303] The server generates product proposals based on the results of AI analysis and emotional data. For example, if a customer expresses dissatisfaction, it makes proposals to resolve the dissatisfaction (e.g., expanding support services). The input is the customer's cluster information and emotional data, and the output is the generated product proposal.

[1304] Step 10:

[1305] The server notifies the customer and the sales department of the generated product proposal. Notifications are sent by email, push notification, or other methods. The input is the generated product proposal, and the output is the notified product proposal.

[1306] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1307] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1308] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1309] [Fourth embodiment]

[1310] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1311] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1312] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1313] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1314] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1315] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1316] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1317] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1318] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1319] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1320] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1321] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1322] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1323] This invention is a system that effectively collects operation details, usage status, and inquiry details on a customer portal site, generates product proposals based on this data, and notifies the sales department of the proposals.

[1324] System Configuration

[1325] 1. Data Collection

[1326] The server has the function of recording the series of operations that customers perform on the portal site. For example, it collects click information when customers use a certain function and information on the browsing of specific pages. It also records how customers use the product (usage status) and the content of their inquiries.

[1327] 2. Data Preprocessing

[1328] The server filters the collected data to remove unnecessary and duplicated data, such as repeated logs of the same operation and noise data, and normalizes the data into a standard format to facilitate subsequent analysis.

[1329] 3. Data Conversion

[1330] The server converts the preprocessed data into multimodal data, specifically integrating text data (inquiry content), numerical data (usage amount), and time data (operation timing) into a single dataset.

[1331] 4. AI analysis

[1332] The server feeds multimodal data into AI models to analyze customer behavior patterns and needs, for example using machine learning algorithms to predict whether a customer is interested in a particular product feature.

[1333] 5. Generate product proposals

[1334] Based on the results of the AI ​​analysis, the server generates optimal product proposals for customers, such as upgrades and new services related to features that customers frequently inquire about.

[1335] 6. Notification

[1336] The server notifies the sales department of the generated product proposals via an internal email system, a notification dashboard, or by sending a direct message to a specific sales representative.

[1337] Specific examples

[1338] 1. Data Collection

[1339] The server collects operation logs when Customer A starts using the "new subscription service."

[1340] The server also records the inquiries made by Customer A regarding "how to set up this service."

[1341] 2. Data Preprocessing

[1342] The server removes duplicate entries and unnecessary noise from the operation log.

[1343] The server formats the normal log data into a standard format.

[1344] 3. Data Conversion

[1345] The server integrates the inquiry content as text data, the usage amount as numerical data, and the service start date and time as time data.

[1346] 4. AI analysis

[1347] The server inputs this integrated data into an AI model and analyzes Customer A's behavioral patterns.

[1348] The server proposes an upgrade plan with a new setup guide because Customer A frequently asks about "how to set it up."

[1349] 5. Generate product proposals

[1350] The server generates a guided configuration upgrade plan based on AI analysis.

[1351] 6. Notification

[1352] The server notifies the sales representative of the generated proposal by email, and the sales representative contacts Customer A.

[1353] In this way, this system significantly improves the efficiency of sales activities by making efficient and precise product proposals based on customer operations and inquiries.

[1354] The processing flow will be explained below.

[1355] Step 1: Data collection

[1356] The server records the operations performed by customers on the portal site in real time. Specifically, it saves click information and page viewing information when customers use certain functions in a log file. It also records customer usage status (e.g., connection time and data consumption) and inquiry content (e.g., support messages and FAQ search history).

[1357] Step 2: Data Preprocessing

[1358] The server filters the collected log data to remove duplicates and noise, such as duplicate entries of the same operation repeated multiple times and meaningless text (noise). It also converts the data into a standard format that makes it easier to analyze.

[1359] Step 3: Data conversion

[1360] The server converts the preprocessed data into multimodal data. For example, it combines the inquiry content as text data, the usage amount as numerical data, and the operation date and time as time data. This makes it possible to handle different types of data in a unified manner.

[1361] Step 4: AI analysis

[1362] The server then feeds the converted multimodal data into an AI algorithm to analyze customer behavior patterns and needs. Specifically, it uses machine learning models to predict whether a customer wants a specific feature. For example, if a customer frequently inquires about a new feature, it will recommend products or services that include that feature.

[1363] Step 5: Generate product proposals

[1364] The server generates product proposals based on the results of AI analysis. For example, it proposes upgrade plans or new services related to features that customers frequently inquire about. The proposals include the products and services offered, their benefits, and how to use them.

[1365] Step 6: Notification

[1366] The server notifies the sales department of the generated product proposal. Specifically, the proposal details are communicated to the sales representative via the company's internal email system and notification dashboard. The sales representative then follows up with the customer based on this notification.

[1367] These are the specific processing steps of this system, which makes it possible to effectively utilize customer operation and inquiry data and improve the efficiency of sales activities.

[1368] Example 1

[1369] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1370] Conventional customer support systems have difficulty effectively collecting customer operations, usage status, and inquiry details, and quickly and efficiently generating appropriate product proposals based on this data. This has led to problems such as reduced sales efficiency and difficulty in improving customer satisfaction. Furthermore, a lack of technology for precisely analyzing customer behavior patterns has made it difficult to provide accurate proposals. A new system that can solve these issues is needed.

[1371] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1372] In this invention, the server includes means for recording customer operation details, means for recording customer usage status, means for recording customer inquiry details, means for preprocessing the recorded data, means for converting the preprocessed data into multimodal data, means for inputting the converted multimodal data into a generative AI model to analyze customer behavior patterns, means for generating product proposals based on the analysis results, and means for notifying the sales department of the generated product proposals. This makes it possible to effectively collect and preprocess customer data and analyze it using an AI model, thereby enabling accurate and efficient generation of product proposals and notification to the sales department.

[1373] The "means for recording customer operations" is a function for recording operations such as clicks and page views that customers perform on the web portal site.

[1374] "Means for recording customer usage" refers to a function for recording how and to what extent customers use products and services.

[1375] "Means for recording customer inquiries" is a function for recording the inquiries made by customers to support or the help desk in text format.

[1376] "Means for preprocessing recorded data" refers to a function for removing noise and redundant data from collected data and preparing the data in a format suitable for analysis.

[1377] The "means for converting preprocessed data into multimodal data" is a function for integrating preprocessed data into multiple formats, such as text data, numerical data, and time data, to create a single data set.

[1378] "Means for inputting converted multimodal data into a generative artificial intelligence model to analyze customer behavior patterns" refers to a function for inputting multimodal data into an AI model to analyze customer behavior patterns and needs.

[1379] "Means for generating product proposals based on analysis results" refers to a function that automatically creates optimal product and service proposals for customers based on the results of analysis by AI.

[1380] The "means for notifying the sales department of the generated product proposal" is a function for communicating the generated product proposal to the sales department using an appropriate notification means.

[1381] MODE FOR CARRYING OUT THE INVENTION

[1382] This invention is a system that effectively collects operation details, usage status, and inquiry details on a customer portal site, generates product proposals based on this data, and notifies the sales department of the proposals. The configuration and operation of this system will be specifically described.

[1383] 1. Data Collection

[1384] The server has the function of recording a series of operations performed by customers on the portal site. For example, it collects information on customers clicking specific buttons and logs the specific pages they view. It also records how customers use products and services (usage status) and the content of their inquiries. For this purpose, the server uses a log collection module and database.

[1385] Examples:

[1386] The server collects operation logs when Customer A starts using the "new subscription service."

[1387] The server also records the inquiries made by Customer A regarding "how to set up this service."

[1388] 2. Data Preprocessing

[1389] The server filters the collected data to remove unnecessary or redundant data, such as repeated logs of the same operation or noise data, and normalizes the data into a standard format to facilitate subsequent analysis.

[1390] Examples:

[1391] The server removes duplicate entries and unnecessary noise from the operation log.

[1392] The server formats the normal log data into a standard format.

[1393] 3. Data Conversion

[1394] The server converts the preprocessed data into multimodal data, specifically integrating text data (inquiry content), numerical data (usage amount), and time data (operation timing) into a single dataset.

[1395] Examples:

[1396] The server integrates the inquiry content as text data, the usage amount as numerical data, and the service start date and time as time data.

[1397] 4. Analysis using AI models

[1398] The server feeds multimodal data into generative AI models to analyze customer behavior patterns and needs. For example, it uses machine learning algorithms to predict whether a customer is interested in a particular product feature. To do this, the server uses machine learning libraries and AI models.

[1399] Examples:

[1400] The server inputs this integrated data into an AI model and analyzes Customer A's behavioral patterns.

[1401] The server detects that customer A frequently asks about "how to set things up" and proposes a new guided setup upgrade plan.

[1402] 5. Generate product proposals

[1403] The server generates optimal product proposals for customers based on the analysis results of the AI ​​model, such as proposing upgrades or new services related to features that customers frequently inquire about.

[1404] Examples:

[1405] The server generates a guided upgrade plan based on the output of the AI ​​model.

[1406] 6. Notification of Proposal

[1407] The server notifies the sales department of the generated product proposals using the company's internal email system, a notification dashboard, or direct messaging to specific sales representatives.

[1408] Examples:

[1409] The server notifies the sales representative of the generated proposal via email, and the sales representative contacts Customer A and proposes a guided configuration upgrade plan.

[1410] Prompt Sentence Examples

[1411] "Design a system that generates optimal product proposals based on a customer's operation log and inquiry details, and notifies the sales department."

[1412] This system significantly improves the efficiency of sales activities by making efficient and precise product proposals based on customer operations and inquiries.

[1413] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1414] Divide the program's processing flow into processing steps

[1415] Step 1: Data collection

[1416] The server records the series of operations and inquiries that the user makes on the portal site.

[1417] Input: User clicks, page views, inquiries

[1418] Specific operation: When a customer clicks a specific button, the click information is recorded as a log along with a timestamp, and the content submitted in the inquiry form is saved as text data.

[1419] Output: Operation log and inquiry data

[1420] Step 2: Data Preprocessing

[1421] The server filters the collected data, removing unnecessary and redundant data, and normalizes the data into a standard format.

[1422] Input: Operation log and inquiry data

[1423] What it does: It finds and removes duplicate entries and noise data, and converts the remaining data into a standard format such as JSON.

[1424] Output: Preprocessed operation log and query data

[1425] Step 3: Data Integration and Transformation

[1426] The server converts the pre-processed data into multimodal data.

[1427] Input: Preprocessed operation log and query data

[1428] Specific operation: Text data (e.g., inquiry content), numerical data (e.g., usage amount), and time data (e.g., operation timing) are integrated into a single data set.

[1429] Output: Multimodal dataset

[1430] Step 4: Analysis by AI model

[1431] The server feeds the multimodal dataset into a generative AI model to analyze customer behavior patterns.

[1432] Input: Multimodal dataset

[1433] What it does: Multimodal data is fed into an AI model and analyzed using machine learning algorithms.

[1434] Output: Analysis of customer behavior patterns and needs

[1435] Step 5: Generate product proposals

[1436] The server generates optimal product proposals for customers based on the analysis results of the AI ​​model.

[1437] Input: Analysis results of customer behavior patterns and needs

[1438] What it does: Based on the analysis results, it generates recommendations for new services and additional features that customers may be interested in.

[1439] Output: Product proposal data

[1440] Step 6: Proposal Notification

[1441] The server notifies the sales department of the generated product proposal.

[1442] Input: Product proposal data

[1443] What happens next? Communicate proposals to the sales team via email and notification dashboards, sending messages directly to specific salespeople as needed.

[1444] Output: Notification data to the sales department

[1445] (Application example 1)

[1446] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1447] There is a need for a system that can effectively collect and analyze customer behavior data and generate appropriate product proposals to improve the efficiency of sales activities and customer satisfaction. However, current systems do not fully utilize customer behavior data, limiting the accuracy and degree of personalization of product proposals. In particular, there is a lack of a mechanism for integrating and analyzing various customer operations and inquiry data, making it difficult to provide proposals that are optimized for each individual customer.

[1448] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1449] In this invention, the server includes means for recording customer operation details, means for recording customer usage status, means for recording customer inquiry details, means for preprocessing the recorded data, means for converting the preprocessed data into multimodal data, means for analyzing the converted multimodal data to generate product proposals, means for notifying the generated product proposals, analysis means for clustering customer behavior, means for generating personalized product proposals based on the clustering results, and means for notifying the customer of the generated product proposals by email. This makes it possible to comprehensively analyze a variety of customer behavior data and provide optimal product proposals for each individual customer.

[1450] "Means for recording customer operations" refers to means for collecting operation logs such as clicks and page transitions that customers make on websites and applications.

[1451] "Means for recording customer usage" means means for monitoring and recording the extent to which customers use specific features or services.

[1452] The "means for recording customer inquiries" refers to a means for collecting questions and problem reports made by customers to customer support as text data.

[1453] "Means for pre-processing recorded data" refers to means for filtering and normalizing the collected raw data to prepare it for easier subsequent analysis.

[1454] A "means for transforming preprocessed data into multimodal data" is a means for integrating data of different formats (e.g., textual data, numerical data, temporal data) into a single analyzable data set.

[1455] "Means for analyzing converted multimodal data and generating product proposals" refers to means for analyzing data using techniques such as machine learning and generating proposals for products and services that are optimal for customers.

[1456] The "means for notifying the generated product proposal" refers to a means for notifying the customer or the sales department of the generated proposal using a method such as email or a dashboard.

[1457] The "analysis means for clustering customer behavior" is a means for analyzing collected customer behavior data and dividing them into groups of customers with common characteristics.

[1458] The "means for generating personalized product proposals based on clustering results" is a means for proposing individually optimized products and services based on the characteristics of each customer group identified by clustering.

[1459] The "means for notifying the customer of the generated product proposal by email" is a means for directly notifying the customer of the content of the proposal by sending the generated proposal to the customer's email address.

[1460] This invention is a system that collects and analyzes customer behavior data, and generates and notifies individually optimized product proposals. A specific embodiment of this system is described below.

[1461] System Overview

[1462] The system consists of the following main components:

[1463] 1. Data Collection Module

[1464] 2. Data Preprocessing Module

[1465] 3. Data Conversion Module

[1466] 4. AI analysis module

[1467] 5. Product proposal generation module

[1468] 6. Notification Module

[1469] Hardware and Software Used

[1470] Hardware: The system operates using a server or cloud environment, while data collection and notifications are performed using the user's smartphone, tablet, or PC.

[1471] Software: Python and Pandas are used for data processing, and machine learning libraries such as Scikit-learn and TensorFlow are used for analysis. A library (smtplib) that sends emails using the SMTP protocol is used for notifications.

[1472] Program processing explanation

[1473] Data Collection Module

[1474] The server collects user operations, usage status, and inquiries in real time, for example, by recording website click information, page browsing history, and product inquiries.

[1475] Data Preprocessing Module

[1476] The collected data is pre-processed on the server, which removes noise and duplicate data and arranges all data into a standard format. For example, if the same click information is recorded multiple times, it will be combined into one.

[1477] Data Conversion Module

[1478] The preprocessed data is converted into multimodal data, a process that combines text data (e.g., inquiry content), numerical data (e.g., product usage), and temporal data (e.g., operation duration) into a single dataset.

[1479] AI Analysis Module

[1480] The server then inputs the converted multimodal data into an AI model to analyze the user's behavioral patterns and needs, using a clustering algorithm to classify user behavior into groups with common characteristics.

[1481] Product proposal generation module

[1482] Based on the clustering results, the server generates product recommendations optimized for each user group. For example, if a user frequently inquires about how to use a particular product, the server can suggest new accessories related to that product.

[1483] Notification Module

[1484] The generated product proposal will be notified by email. The server will send the generated proposal to the user's registered email address. The email is sent using the SMTP protocol.

[1485] Specific examples

[1486] For example, if a user does the following:

[1487] Viewing a specific product page multiple times

[1488] Make an inquiry about "How to use the product"

[1489] Browse additional related products

[1490] The server collects this data, pre-processes and analyzes it, and then sends the following email to the user based on the analysis results:

[1491] Subject: Personalized product recommendations

[1492] Main text:

[1493] Hello,

[1494] Get recommended products related to your recently viewed products.

[1495] 1. Upgrade kit for product X

[1496] 2. Accessories related to product X

[1497] Please check it out.

[1498] An example prompt for a generative AI model would be:

[1499] Generate optimal product recommendations based on user behavior data, including:

[1500] Click information: / home, / product / 3, / product / 5

[1501] Inquiry: How to return product? Warranty information

[1502] In this way, personalized product proposals based on user-specific data can be made, thereby improving the user experience and streamlining sales activities.

[1503] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1504] Step 1:

[1505] The server collects user operations, usage, and inquiries in real time, for example, logging the links users click on websites, the pages they view, the search terms they use, and the information they enter into inquiry forms. The collected data is stored as raw data.

[1506] Input: User operation logs, usage data, and inquiry details

[1507] Output: Raw data collected

[1508] Step 2:

[1509] The server pre-processes the collected raw data, removing unnecessary and duplicate data and arranging the data into a standard format, specifically combining multiple records of the same click and removing noise data (e.g., error logs).

[1510] Input: Raw data collected

[1511] Output: Preprocessed data

[1512] Step 3:

[1513] The server converts the preprocessed data into multimodal data, a process that integrates data of different formats (e.g., textual, numerical, and temporal data) into a single dataset that facilitates subsequent analysis.

[1514] Input: Preprocessed data

[1515] Output: Multimodal data

[1516] Step 4:

[1517] The server inputs the multimodal data into an AI model to analyze user behavior patterns and needs. A clustering algorithm is used for the analysis, classifying user behavior into groups with common characteristics. For example, KMeans clustering is used to classify customers into multiple groups.

[1518] Input: Multimodal data

[1519] Output: Clustered user groups

[1520] Step 5:

[1521] The server generates personalized product recommendations based on the clustering results. This process involves suggesting the best products and services for each user group. For example, if a user is interested in a particular product, it will suggest accessories and upgrades related to that product.

[1522] Input: Clustered user groups

[1523] Output: Personalized product proposals

[1524] Step 6:

[1525] The server notifies the user of the generated product proposal via email. During this process, the product proposal is sent to the user's registered email address. The email containing the proposal is sent using the SMTP protocol.

[1526] Input: Personalized product suggestions

[1527] Output: Product proposal email sent to user

[1528] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1529] This invention combines a system that effectively collects customer operations, usage status, and inquiries on a customer portal site, generates product proposals based on that data, and notifies the sales department of the proposals with an emotion engine that recognizes user emotions.

[1530] System Configuration

[1531] 1. Data Collection

[1532] The server records in real time the series of operations that customers perform on the portal site. For example, it saves click information and page view information when customers use a certain function in a log file. It also records customer usage status (e.g., connection time and data consumption) and inquiry content (e.g., support messages and FAQ search history).

[1533] 2. Data Preprocessing

[1534] The server filters the collected log data to remove unnecessary and duplicate data, such as duplicate entries of the same operation repeated multiple times and meaningless text (noise), and converts the data into a standard format that makes it easier to analyze.

[1535] 3. Data Conversion

[1536] The server converts the preprocessed data into multimodal data. For example, it combines the inquiry content as text data, the usage amount as numerical data, and the operation date and time as time data. This makes it possible to handle different types of data in a unified manner.

[1537] 4. Emotion Recognition by Emotion Engine

[1538] The server analyzes the user's inquiries and operations to recognize their emotions. For example, it determines whether the user is frustrated or excited based on the content of the text and the frequency of operations. This emotional data is also included in the multimodal data.

[1539] 5. AI analysis

[1540] The server then feeds the converted multimodal data into an AI algorithm to analyze customer behavior patterns and needs. Specifically, it uses machine learning models to predict whether a customer wants a specific feature. For example, if a customer frequently inquires about a new feature, it will recommend products or services that include that feature.

[1541] 6. Product proposal generation taking into account emotional data

[1542] The server generates product proposals based on the results of the AI ​​analysis and also takes into account the user's emotional data. For example, if a customer expresses dissatisfaction, it will make proposals to resolve that dissatisfaction (e.g., expanding support services).

[1543] 7. Notification

[1544] The server notifies the sales department of the generated product proposal. Specifically, the proposal details are communicated to the sales representative via the company's internal email system and notification dashboard. The sales representative then follows up with the customer based on this notification.

[1545] Specific examples

[1546] 1. Data Collection

[1547] The server collects operation logs when Customer A starts using the "Subscription Service."

[1548] The server records the details of the inquiry made by Customer A regarding "how to set up this service."

[1549] 2. Data Preprocessing

[1550] The server removes duplicate entries and unnecessary noise from the operations log.

[1551] The server formats the normal log data into a standard format.

[1552] 3. Data Conversion

[1553] The server integrates the inquiry content as text data, the usage amount as numerical data, and the service start date and time as time data.

[1554] 4. Emotion Recognition by Emotion Engine

[1555] The server analyzes the content of the inquiry and the frequency of operations and detects that customer A is dissatisfied.

[1556] 5. AI analysis

[1557] The server inputs the integrated data into an AI model and analyzes customer A's behavioral patterns.

[1558] The server proposes an upgrade plan with a new setup guide because Customer A frequently asks about "how to set it up."

[1559] 6. Product proposal generation taking into account emotional data

[1560] To resolve Customer A's dissatisfaction, the server generates an upgrade plan with a setup guide as well as a proposal to expand the support content.

[1561] 7. Notification

[1562] The server notifies the sales representative of the generated proposal by email, and the sales representative contacts Customer A based on the proposal.

[1563] In this way, by combining this system with an emotion engine, it is possible to make efficient and precise product proposals that take customer emotions into consideration, thereby improving the efficiency of sales activities and customer satisfaction.

[1564] The processing flow will be explained below.

[1565] Step 1: Data collection

[1566] The server records in real time the operations performed by customers on the portal site. For example, it stores click information and page browsing history in a log when a customer starts a "subscription service." It also records connection time and data consumption while the customer is using the service. It also records the content of inquiries made by customers regarding "how to set up this service."

[1567] Step 2: Data Preprocessing

[1568] The server filters the collected log data to remove unnecessary and duplicate data, such as duplicate entries of the same operation repeated multiple times and meaningless text (noise), and converts the data into a standard format to ensure consistency.

[1569] Step 3: Data conversion

[1570] The server converts the preprocessed data into multimodal data. Specifically, it integrates the inquiry content as text data, the usage amount as numerical data, and the operation date and time as time data. This makes it possible to handle different types of data in a unified manner.

[1571] Step 4: Emotion Recognition with the Emotion Engine

[1572] The server analyzes the user's inquiries and operations to recognize the user's emotions. For example, it can determine whether the user is happy or dissatisfied from the text of the inquiry. It can also estimate the user's emotions from the frequency and patterns of operations. This emotional data is included in the multimodal data.

[1573] Step 5: AI analysis

[1574] The server then feeds the converted multimodal data into an AI algorithm to analyze customer behavior patterns and needs. Specifically, it uses machine learning models to predict whether a customer wants a specific feature. For example, if a customer frequently inquires about a new feature, it will recommend products or services that include that feature.

[1575] Step 6: Generate product recommendations taking into account emotional data

[1576] The server generates product proposals based on the results of the AI ​​analysis and also takes into account the user's emotional data. For example, if a customer expresses dissatisfaction, it makes proposals to resolve that dissatisfaction (e.g., an upgrade plan with setup guides and additional support). Based on the emotional data, it adjusts the proposal content to increase customer satisfaction.

[1577] Step 7: Notification

[1578] The server notifies the sales department of the generated product proposal. Specifically, the proposal details are communicated to the sales representative via the company's internal email system or notification dashboard. The sales representative then follows up with the customer based on this notification. For example, they may contact Customer A to propose a new upgrade plan with a setup guide and additional support.

[1579] Example 2

[1580] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1581] Conventional customer portal sites were able to collect customer operations, usage status, and inquiries, but simply recording and analyzing this data made it difficult to make appropriate product proposals based on the customer's true needs and emotions. Furthermore, the collected data sometimes contained noise or duplicate data, which reduced the accuracy of the analysis. Furthermore, even if product proposals were generated, they were sometimes not efficiently communicated to the sales department. An effective solution to these issues was needed.

[1582] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1583] In this invention, the server includes means for recording customer operation details, means for recording customer usage status, means for recording customer inquiry details, means for preprocessing the recorded data, means for converting the preprocessed data into multimodal data, means for analyzing the converted multimodal data to generate product proposals, means for recognizing user emotions, means for generating product proposals taking into account the emotion data, and means for notifying the generated product proposals. This enables precise product proposals that also take emotions into account based on the customer's operation history and inquiry details. Furthermore, preprocessing the collected data removes noise and duplicate data, enabling highly accurate analysis and enabling efficient notification to the sales department.

[1584] "Customer operations" refers to a series of operations performed by customers on the portal site, such as click information and page viewing information.

[1585] "Customer usage" refers to the usage history, such as connection time and data consumption, when a customer uses a portal site.

[1586] "Customer inquiry content" refers to inquiry records such as support messages and FAQ search history made by customers when using the portal site.

[1587] "Recorded data" refers to all data stored on the server regarding customer operations, usage, and inquiries.

[1588] "Preprocessing" refers to the process of removing unnecessary noise and redundant data from recorded data and adjusting the data format.

[1589] "Multimodal data" refers to a format that integrates data of different formats (text data, numerical data, time data, etc.).

[1590] "Means for recognizing user emotions" refers to an emotion engine that analyzes the content of customer inquiries and operations and determines the customer's emotions (dissatisfaction, excitement, etc.).

[1591] "Means for generating product proposals" refers to means for predicting customer needs using AI analysis and proposing optimal products and services.

[1592] The "means for generating a product proposal taking into consideration emotional data" refers to a means for generating a more precise product proposal based on the emotional data of the user.

[1593] The "means for notifying the generated product proposal" refers to a means for effectively communicating the generated product proposal to the sales department.

[1594] This invention combines a system that effectively collects customer operations, usage status, and inquiries on a customer portal site, generates product proposals based on that data, and notifies the sales department of the proposals, with an emotion engine that recognizes user emotions. Specific embodiments of the present invention are described below.

[1595] System Configuration

[1596] The system is primarily composed of a server and terminals, and generates product proposals by recording and analyzing the operations, usage status, and inquiries that occur when customers (users) use the portal site, and notifies these proposals to the sales department.

[1597] Hardware and software used

[1598] Server: The central location for data collection, pre-processing, analysis, proposal generation, and notifications.

[1599] Terminal: The device (PC, smartphone, tablet, etc.) that users use to access the portal site.

[1600] Emotion engine: Software that analyzes user emotions and generates emotion data.

[1601] Data collection

[1602] The server collects the operations performed by customers on the portal site in real time. Specifically, click information, page view information, and inquiry details are instantly saved in a log file. When a user starts a subscription service, the operation log is immediately transferred to the server.

[1603] Examples:

[1604] The operation log generated by the terminal is sent to the server.

[1605] The server saves click information in a log file when Customer A starts using the "Subscription Service."

[1606] Data Preprocessing

[1607] The collected log data is filtered by the server to remove unnecessary noise and duplicate data, and through this process the data is organized into a standard format suitable for subsequent analysis.

[1608] Examples:

[1609] The server removes duplicate entries and meaningless data and formats the normal log data into a standard format.

[1610] Data Conversion

[1611] The preprocessed data is converted into multimodal data by the server, which integrates different types of data (text data, numerical data, time data, etc.) into one and allows for unified handling.

[1612] Examples:

[1613] The server saves the inquiry content as text data, converts the usage amount into numerical data, and integrates the operation date and time as time data.

[1614] Emotion recognition by emotion engine

[1615] The emotion engine analyzes the user's inquiries and operations to recognize their emotions. For example, it determines whether the user is dissatisfied or excited based on the content of the text and the frequency of operations, and generates emotion data.

[1616] Examples:

[1617] The server performs text analysis of the inquiry, detects keywords such as "dissatisfaction" and "irritation," and determines the user's emotions.

[1618] Product proposal generation and notification

[1619] The server uses an AI model based on multimodal data to analyze customer behavior patterns and needs. It then generates optimal product proposals taking into account emotional data and notifies the sales department of these proposals. Notifications are sent via the company's internal email system and notification dashboard.

[1620] Examples:

[1621] The server detects that Customer A frequently inquires about new features and suggests an upgrade plan with a guided setup.

[1622] The server notifies the sales department staff of this proposal by email, and the staff contacts Customer A based on the proposal.

[1623] Prompt Sentence Examples

[1624] Below are examples of prompt sentences to efficiently generate product proposals based on an AI model.

[1625] "Based on this customer's operation history and inquiry details, predict the products and services that the customer may be interested in, and generate product recommendations that take into account usage history and sentiment data."

[1626] This system makes it possible to make precise and effective product proposals based on the customer's operation history and inquiry details, as well as taking into account the user's emotions, dramatically improving the efficiency of sales activities and customer satisfaction.

[1627] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1628] Step 1:

[1629] Data collection

[1630] The server collects the operations performed by customers on the portal site in real time. Specifically, click information and page view information generated by the device are sent to the server and saved in a log file. In addition, connection time, data consumption, and inquiry content are also recorded.

[1631] Input: Customer operation details, usage status, inquiry details

[1632] Output: Operation logs, usage data, and query records saved in log files

[1633] Specific behavior:

[1634] The terminal transmits the user's click information on the portal site to the server.

[1635] The server records click information and page view information in a log file.

[1636] Step 2:

[1637] Data Preprocessing

[1638] The server filters the collected log data, removing unnecessary and duplicate data, and formats the data for subsequent analysis.

[1639] Input: Recorded log data

[1640] Output: filtered and standard formatted data

[1641] Specific behavior:

[1642] The server removes duplicate entries where the same operation has been performed multiple times.

[1643] Remove unnecessary noise data.

[1644] Convert the data into a standard format.

[1645] Step 3:

[1646] Data Conversion

[1647] The server converts the preprocessed data into multimodal data, integrating different types of data (text data, numerical data, time data, etc.) and handling them in a unified manner.

[1648] Input: Preprocessed log data

[1649] Output: Multimodal data

[1650] Specific behavior:

[1651] The server saves the inquiry content as text data.

[1652] Convert usage into numerical data.

[1653] The operation date and time are recorded as time data.

[1654] Integrate all data as multimodal data.

[1655] Step 4:

[1656] Emotion recognition by emotion engine

[1657] The server analyzes the user's inquiries and operations, recognizes the user's emotions, and adds the emotion information to the multimodal data.

[1658] Input: Inquiry details, operation frequency data

[1659] Output: Multimodal data including emotion data

[1660] Specific behavior:

[1661] The server performs text analysis of the inquiry content and extracts keywords that indicate emotions.

[1662] The frequency of operations is analyzed to determine the user's emotions.

[1663] Adding emotion data to multimodal data.

[1664] Step 5:

[1665] AI analysis

[1666] The server inputs multimodal data into an AI algorithm to analyze customer behavior patterns and needs, generating product recommendations that customers may be interested in.

[1667] Input: Multimodal data including emotion data

[1668] Output: Product proposals based on customer needs

[1669] Specific behavior:

[1670] The server inputs multimodal data, including emotional data, into the AI ​​model.

[1671] Analyze behavioral patterns and needs to generate product proposals.

[1672] Step 6:

[1673] Product proposal generation taking into account emotional data

[1674] The server generates more sophisticated product proposals based on the results of AI analysis and emotional data.

[1675] Input: AI analysis results, emotional data

[1676] Output: Product proposals that take emotions into account

[1677] Specific behavior:

[1678] If the customer is dissatisfied, the server generates a proposal to resolve the dissatisfaction.

[1679] Reflect emotional data in proposal content.

[1680] Step 7:

[1681] notification

[1682] The server notifies the sales department of the generated product proposals, using an email system and a notification dashboard to communicate the proposal details in real time.

[1683] Input: Generated product proposal

[1684] Output: Product proposal notified to the sales department

[1685] Specific behavior:

[1686] The server sends the generated product proposal to the sales representative via an email system.

[1687] Sales representatives receive the proposal and follow up with the customer.

[1688] (Application example 2)

[1689] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1690] With conventional portal sites, it was difficult to effectively collect customer operations, usage status, and inquiry details and generate product proposals based on that information. Furthermore, proposals were made without taking into account customer emotions, making it difficult to improve customer satisfaction. Therefore, there is a need for a system that can accurately grasp customer emotions and needs and make optimal product proposals.

[1691] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for recording customer operation details, means for recording customer usage status, means for recording customer inquiry details, means for preprocessing the recorded data, means for converting the preprocessed data into multimodal data, means for analyzing the converted multimodal data to generate product proposals, means for generating user emotion data using an emotion engine that recognizes user emotions and including the emotion data in the multimodal data, and means for notifying the generated product proposals. This makes it possible to comprehensively analyze customer operations, usage status, and inquiry details, and to make precise product proposals that take emotions into consideration.

[1692] 1. "Means for recording customer operations" refers to means for saving information about operations performed by customers on electronic devices or websites as logs.

[1693] 2. "Means for recording customer usage" means means for recording usage data such as the time and amount consumed when a customer uses a service or product.

[1694] 3. "Means for recording customer inquiries" refers to means for saving the details and history of support requests and inquiries made by customers.

[1695] 4. "Means for pre-processing recorded data" means means for removing unnecessary information and redundant data from collected data and converting it into a format that can be analyzed.

[1696] 5. "Means for converting preprocessed data into multimodal data" refers to means for converting data of different formats (text data, numerical data, time data, etc.) so that they can be handled in a unified manner.

[1697] 6. "Means for analyzing converted multimodal data and generating product proposals" refers to means for analyzing customer needs and behavioral patterns based on converted data and creating appropriate product proposals.

[1698] 7. "Means for generating user emotion data using an emotion engine that recognizes user emotions and including the emotion data in multimodal data" refers to means for analyzing user emotions, saving them as numerical data, and integrating them with other data.

[1699] 8. "Means for notifying the generated product proposal" refers to the means for notifying the customer or sales representative of the product proposal by means such as email or push notification.

[1700] This invention combines a system that effectively collects customer operations, usage status, and inquiries on a customer portal site, and generates and notifies product proposals based on that data with an emotion engine that recognizes user emotions.

[1701] First, the server records the operations performed by customers when they use the portal site or smartphone app in real time. The operations include click information and page browsing history. The server also records the customer's usage status (e.g., connection time and data consumption) and inquiries (e.g., support messages and FAQ search history).

[1702] The server then preprocesses the collected log data, using pandas to remove unnecessary and duplicate data and convert it into a standard format for easier analysis. This preprocessing ensures that only meaningful data remains.

[1703] The server then converts the preprocessed data into multimodal data, integrating the query content as text data, the usage amount as numerical data, and the operation date and time as time data, making it possible to handle different types of data in a unified manner.

[1704] Furthermore, the server uses an emotion engine to recognize the user's emotions. Specifically, it uses the SentimentIntensityAnalyzer in the nltk library to determine the user's emotions from the content of the text and the frequency of operations. This emotion data is also included in the multimodal data.

[1705] The server uses the converted multimodal data to analyze customer behavior patterns and needs using AI algorithms. Specifically, it quantifies the text data using TfidfVectorizer, scales it using StandardScaler, and performs cluster analysis using KMeans. This analysis predicts whether the customer is interested in a specific product or service.

[1706] The server then generates product recommendations based on the results of the AI ​​analysis and the emotional data. For example, if the emotion engine indicates that the user is dissatisfied, it can make suggestions to resolve that dissatisfaction (e.g., expanding support services).

[1707] Finally, the server notifies the customer and the sales department of the generated product proposal via email or push notification, allowing the sales representative to follow up on the customer's needs.

[1708] Specific examples

[1709] 1. If Customer A views Product B multiple times and posts a question about how to set it up, the emotion engine determines that they are dissatisfied. In this case, the server will suggest a guided upgrade plan for Product B.

[1710] 2. If Customer B expresses dissatisfaction with the service, a proposal to resolve the dissatisfaction by expanding the support content is generated and notified to Customer B.

[1711] Prompt Sentence Examples

[1712] "Generate a list of recommended products related to product B, which this customer has recently viewed frequently."

[1713] "Please tell us the best product or support you can offer to resolve the issue for Customer B, who is dissatisfied with your service."

[1714] This makes it possible to comprehensively analyze customer operations and inquiries, and to make precise product proposals that take emotions into consideration.

[1715] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1716] Step 1:

[1717] The server records customer operations in real time. Every time a customer views or clicks on a product page, that information is saved in a log. For example, an operation log is generated stating, "Customer clicked on product A." The input is the customer's operation, and the output is the recorded operation log.

[1718] Step 2:

[1719] The server records the customer's usage, which includes information such as the time the customer uses the service and the amount of data consumed. For example, a usage log is generated stating that "the customer used the service for 30 minutes." The input is the customer's service usage, and the output is the recorded usage log.

[1720] Step 3:

[1721] The server records customer inquiries. Every time a customer sends a support message or searches an FAQ, the content is saved in a log. For example, an inquiry log is generated that reads, "A customer inquired about how to set up product B." The input is the customer's inquiry, and the output is the recorded inquiry log.

[1722] Step 4:

[1723] The server preprocesses the collected log data. It uses pandas to remove unnecessary and duplicate data and convert it into a standard format. For example, it removes duplicate click information and meaningless text. The input is the recorded operation log, usage log, and query log, and the output is the preprocessed data.

[1724] Step 5:

[1725] The server converts the preprocessed data into multimodal data, which allows different types of data to be handled in a unified manner. For example, query content can be integrated as text data, usage amount as numerical data, and operation date and time as time data. The input is preprocessed data, and the output is multimodal data.

[1726] Step 6:

[1727] The server uses an emotion engine to recognize the user's emotions. Specifically, it uses the SentimentIntensityAnalyzer from the nltk library. It determines the user's emotions (e.g., dissatisfaction or excitement) from the text content and frequency of operations. The input is preprocessed text data, and the output is emotion data.

[1728] Step 7:

[1729] The server integrates the emotion data into multimodal data, which allows the emotion data to be analyzed together with other data. The input is emotion data and multimodal data, and the output is the integrated multimodal data.

[1730] Step 8:

[1731] The server runs an AI algorithm using the converted multimodal data to analyze customer behavior patterns and needs. The text data is quantified using TfidfVectorizer, scaled using StandardScaler, and subjected to cluster analysis using KMeans. The input is the integrated multimodal data, and the output is customer cluster information and analysis results.

[1732] Step 9:

[1733] The server generates product proposals based on the results of AI analysis and emotional data. For example, if a customer expresses dissatisfaction, it makes proposals to resolve the dissatisfaction (e.g., expanding support services). The input is the customer's cluster information and emotional data, and the output is the generated product proposal.

[1734] Step 10:

[1735] The server notifies the customer and the sales department of the generated product proposal. Notifications are sent by email, push notification, or other methods. The input is the generated product proposal, and the output is the notified product proposal.

[1736] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1737] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1738] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1739] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1740] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1741] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1742] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1743] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1744] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1745] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1746] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1747] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1748] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1749] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1750] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1751] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1752] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1753] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1754] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1755] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1756] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1757] The following is further disclosed regarding the above embodiment.

[1758] (Claim 1)

[1759] A means for recording customer operations;

[1760] a means of recording customer usage;

[1761] a means for recording customer inquiries;

[1762] means for pre-processing the recorded data;

[1763] means for converting the preprocessed data into multimodal data;

[1764] means for analyzing the converted multimodal data to generate product proposals;

[1765] The system includes a means for notifying the generated product proposal.

[1766] (Claim 2)

[1767] 10. The system of claim 1, further comprising means for filtering unwanted data from the recorded data.

[1768] (Claim 3)

[1769] 10. The system of claim 1, further comprising means for including textual data, numerical data, and temporal data in the multimodal data.

[1770] "Example 1"

[1771] (Claim 1)

[1772] A means for recording customer operations;

[1773] a means of recording customer usage;

[1774] a means for recording customer inquiries;

[1775] means for pre-processing the recorded data;

[1776] means for converting the preprocessed data into multimodal data;

[1777] a means for inputting the converted multimodal data into a generative artificial intelligence model to analyze customer behavior patterns;

[1778] A means for generating a product proposal based on the analysis result;

[1779] The system includes a means for notifying the sales department of the generated product proposal.

[1780] (Claim 2)

[1781] 10. The system of claim 1, further comprising means for filtering unwanted data from the recorded data.

[1782] (Claim 3)

[1783] 10. The system of claim 1, further comprising means for including textual data, numerical data, and temporal data in the multimodal data.

[1784] "Application Example 1"

[1785] (Claim 1)

[1786] A means for recording customer operations;

[1787] a means of recording customer usage;

[1788] a means for recording customer inquiries;

[1789] means for pre-processing the recorded data;

[1790] means for converting the preprocessed data into multimodal data;

[1791] means for analyzing the converted multimodal data to generate product proposals;

[1792] A means for notifying the generated product proposal;

[1793] an analytical means for clustering customer behavior;

[1794] means for generating personalized product suggestions based on the clustering results;

[1795] means for notifying the customer of the generated product proposal by email;

[1796] A system including:

[1797] (Claim 2)

[1798] 10. The system of claim 1, further comprising: means for filtering unnecessary data from the recorded data; and means for optimizing product recommendations using the clustering results.

[1799] (Claim 3)

[1800] 10. The system of claim 1, further comprising: means for including text data, numerical data, and time data in the multimodal data; and means for analyzing customer behavior using a generative AI model.

[1801] "Example 2: Combining Emotion Engines"

[1802] (Claim 1)

[1803] A means for recording customer operations;

[1804] a means of recording customer usage;

[1805] a means for recording customer inquiries;

[1806] means for pre-processing the recorded data;

[1807] means for converting the preprocessed data into multimodal data;

[1808] means for analyzing the converted multimodal data to generate product proposals;

[1809] means for recognizing a user's emotion;

[1810] A means for generating product proposals taking into account emotion data;

[1811] The system includes a means for notifying the generated product proposal.

[1812] (Claim 2)

[1813] 10. The system of claim 1, further comprising means for filtering unwanted data from the recorded data.

[1814] (Claim 3)

[1815] 10. The system of claim 1, further comprising means for including textual data, numerical data, and temporal data in the multimodal data.

[1816] "Application example 2 when combining emotion engines"

[1817] (Claim 1)

[1818] A means for recording customer operations;

[1819] a means of recording customer usage;

[1820] a means for recording customer inquiries;

[1821] means for pre-processing the recorded data;

[1822] means for converting the preprocessed data into multimodal data;

[1823] means for analyzing the converted multimodal data to generate product proposals;

[1824] means for generating emotion data of a user using an emotion engine that recognizes emotions of the user, and including the emotion data in the multimodal data;

[1825] The system includes a means for notifying the generated product proposal.

[1826] (Claim 2)

[1827] 10. The system of claim 1, further comprising means for filtering unwanted data from the recorded data.

[1828] (Claim 3)

[1829] 10. The system of claim 1, further comprising means for including textual data, numerical data, and temporal data in the multimodal data. [Explanation of symbols]

[1830] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. A means for recording customer operations; a means of recording customer usage; a means for recording customer inquiries; means for pre-processing the recorded data; means for converting the preprocessed data into multimodal data; means for analyzing the converted multimodal data to generate product proposals; The system includes a means for notifying the generated product proposal.

2. 10. The system of claim 1, further comprising means for filtering unwanted data from the recorded data.

3. 10. The system of claim 1, further comprising means for including textual data, numerical data, and temporal data in the multimodal data.

Images