System

Abstract

A system is provided.SOLUTION: A system comprising: means for monitoring user behavior in real-time; means for collecting and storing the monitored behavior in a database; means for classifying the collected behavior into specific categories; means for analyzing the classified behavior with a AI to identify the cause and condition of the behavior; means for visualizing and presenting the analysis results to an administrator; and means for generating system improvement suggestions based on the analysis results.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] An increase in transaction cancellations and bid retractions has become a problem on online auction platforms. Such behavior undermines the platform's credibility and degrades the user experience. However, there is currently no effective way to efficiently understand the causes and conditions behind these behaviors and take countermeasures. Current methods mainly involve surveys and interviews with specific users, which consume a great deal of time and resources and make it difficult to collect and analyze sufficient data. Therefore, there is a need for a system that can quickly and accurately analyze the causes of transaction cancellations and bid retractions and propose countermeasures. [Means for solving the problem]

[0005] This invention is a system that monitors user behavior data in real time and stores it in a database. The collected behavior data is classified into specific categories and then analyzed by AI. This analysis identifies the causes and conditions of user behavior and reveals trends. The analysis results are then visualized using graphs and charts and presented to administrators. Based on the analysis results, proposals for system improvement are also generated. This makes it possible to efficiently identify the causes of transaction cancellations and bid revocations and take appropriate measures.

[0006] These measures will help curb the increase in transaction cancellations and bid retractions, and improve the reliability and user experience of the online auction platform.

[0007] "User Behavior Data" means a record of all activities performed by a User on the Online Auction Platform, including bids, cancellation requests, transaction cancellations, and bid retractions.

[0008] "Real-time monitoring methods" are technologies and methods for observing and recording user behavior in real time.

[0009] A "database" is a system for storing, organizing, and making easily accessible collected behavioral data.

[0010] "Means for classifying into specific categories" refers to methods or techniques for classifying collected behavioral data based on attributes or types.

[0011] "AI analytical methods" are methods and techniques that use artificial intelligence algorithms to analyze data and identify patterns and trends.

[0012] "Means for identifying causes and conditions" refers to techniques and methods that clarify the reasons and factors behind behavior from the analysis results.

[0013] "Means for visualizing analysis results and presenting them to administrators" refers to techniques and methods for visually displaying analysis results in the form of graphs, charts, etc., and providing them in a format that is easy for administrators to understand.

[0014] "Means for generating proposals for system improvement" refers to techniques and methods for creating specific proposals for improving the functionality and operability of a system based on the analysis results. [Brief explanation of the drawings]

[0015] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0016] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0017] First, the terms used in the following description will be explained.

[0018] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0019] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0020] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0021] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0022] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0023] [First embodiment]

[0024] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0025] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0026] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0027] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0028] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0029] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0030] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0031] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0032] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0033] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0034] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0035] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0036] User behavior monitoring

[0037] When users access the online auction platform and make transactions, their actions are recorded in real time. The terminal (the user's PC or mobile device) collects and transmits user behavior data to the server. This behavior data includes bids, cancellation requests, transaction cancellations, and bid retractions.

[0038] Collecting and classifying behavioral data

[0039] The server temporarily stores the received behavioral data in a buffer. It then performs batch processing, stores the collected behavioral data in a database, and classifies the data into specific categories. For example, this classification is performed in categories such as "cancellation request from the successful bidder," "cancellation of bids below a certain price," "deletion of specific users," "skipping notes," and "long time required to complete the flow."

[0040] AI analysis of data

[0041] An AI module installed on the server analyzes the stored behavioral data. The AI ​​uses machine learning algorithms to identify behavioral patterns, frequency, conditions, and causes. For example, it can identify product categories with a high number of cancellation requests or specific time periods where cancellations are concentrated.

[0042] Visualizing and presenting results

[0043] The server converts the analyzed results into graphs and charts, and this visual data is displayed on a dashboard on the terminal (PC or mobile device used by the administrator), allowing the administrator to grasp the trends and causes of user behavior at a glance.

[0044] System improvement proposals

[0045] Based on the analysis results, the server generates specific system improvement suggestions, such as providing more detailed descriptions of specific products or simplifying the transaction flow, to improve the user experience. These suggestions are displayed on the terminal (PC or mobile device used by the administrator), allowing the administrator to select and implement appropriate measures.

[0046] Specific examples

[0047] Analysis and improvement of cancellation requests

[0048] When a user makes a cancellation request for a specific product, the terminal sends data such as the "user ID," "product ID," "action type (cancellation request)," "reason for action," and "date and time" to the server.

[0049] The server collects this data and stores it in a database, after which an AI module analyzes it to identify patterns, such as times of day when cancellation requests are most common, product categories, or specific user attributes.

[0050] The analysis results are converted into graphs and charts by the server and displayed on the terminal (administrator's dashboard). For example, information such as "there are many cancellation requests in the evening hours" or "there are frequent cancellations for products in a particular category" can be immediately confirmed.

[0051] Furthermore, the server generates specific suggestions based on the analysis results to reduce transaction cancellations, such as "provide more detailed product descriptions" or "highlight cautions."

[0052] In this way, the present invention enables detailed monitoring of user behavior and analysis of data using AI to efficiently identify the causes of transaction cancellations and bid retractions and take appropriate countermeasures, thereby improving the reliability of the online auction platform and the user experience.

[0053] The processing flow will be explained below.

[0054] Step 1:

[0055] A user logs into the online auction platform and performs trading operations, including placing a bid, requesting a cancellation, canceling a transaction, and withdrawing a bid.

[0056] Step 2:

[0057] The terminal (PC or mobile device used by the user) records each operation performed by the user as an event and generates detailed data related to it (user ID, product ID, action type, reason for action, date and time, etc.).

[0058] Step 3:

[0059] The terminal transmits the generated detailed data to the server in real time.

[0060] Step 4:

[0061] The server temporarily stores the received behavioral data in a buffer, which acts as a temporary storage location for subsequent batch processing of the data into a database.

[0062] Step 5:

[0063] The server periodically runs a batch job to move the data from the buffer to a database, where all user activity data is stored.

[0064] Step 6:

[0065] The server categorizes the behavioral data stored in the database for analysis, including categories such as "cancellation request from the successful bidder," "cancellation of bids below a certain price," "deletion of specific users," "skipping notes," and "long time required to complete the flow."

[0066] Step 7:

[0067] An AI module installed on the server analyzes the categorized behavioral data and uses machine learning algorithms to identify behavioral frequencies, patterns, and conditions for occurrence.

[0068] Step 8:

[0069] The server collects the results of the AI ​​analysis and converts them into visual formats such as graphs and charts, which makes the data easier for administrators to understand.

[0070] Step 9:

[0071] The terminal (PC or mobile device used by the administrator) displays the visualized data generated by the server on a dashboard, allowing the administrator to grasp trends in cancellations and bid withdrawals in real time.

[0072] Step 10:

[0073] The server generates specific suggestions for system improvement based on the results of the AI ​​analysis, such as "providing more detailed product descriptions" or "strengthening transaction management during specific time periods."

[0074] Step 11:

[0075] The terminal (PC or mobile device used by the administrator) displays the suggestions from the server on a dashboard. The administrator can check these suggestions and implement them as necessary.

[0076] Example 1

[0077] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0078] Conventional online auction platforms face challenges in effectively monitoring, collecting, and analyzing user behavior data, making it difficult to quickly and appropriately address issues such as transaction cancellations and bid revocations. Furthermore, there is no well-established method for analyzing this data and presenting it to administrators in an intuitive and easy-to-understand format. As a result, it is difficult to improve the user experience and maintain the reliability of the platform.

[0079] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0080] In this invention, the server includes means for monitoring user behavior data in real time, means for collecting the monitored behavior data and storing it in a storage device, means for classifying the collected behavior data into specific categories, means for analyzing the behavior data using a machine learning algorithm and identifying the causes and conditions of the behavior, means for visualizing the analysis results in graphs and charts and presenting them on an administrator terminal, and means for generating system improvement proposals based on the analysis results and displaying them on the administrator terminal. This enables detailed monitoring of user behavior, effective analysis and visual presentation of data, and enables an improved user experience and maintenance of platform reliability.

[0081] "User" means any person or legal entity that transacts or operates using the online auction platform.

[0082] "Behavioral Data" means information about your transactions and interactions with the online auction platform.

[0083] "Real-time" refers to processing occurring immediately at the moment a user action occurs.

[0084] "Monitoring" refers to the process of continuously watching and recording user activity.

[0085] A "storage device" is hardware or software for storing data.

[0086] A "category" is a specific criterion or grouping for classifying behavioral data.

[0087] A "machine learning algorithm" is an artificial intelligence technology for data analysis that learns from data based on a set of rules and makes predictions and classifications.

[0088] "Analysis results" refer to the information and insights obtained after analyzing behavioral data using machine learning algorithms.

[0089] "Visualization" refers to the presentation of analytical results in a visual format such as a graph or chart.

[0090] An "administrator terminal" is an electronic device used by a system administrator, which has the function of displaying analysis results and suggestions.

[0091] "System improvement proposals" are specific measures or strategies for optimizing or improving the system based on the analysis results.

[0092] The present invention provides a system for real-time monitoring of user behavior data on an online auction platform, and effectively collects, classifies, analyzes, and visualizes the data. Specific embodiments of the system are described below.

[0093] User behavior monitoring

[0094] When a user accesses an online auction platform and conducts transactions such as browsing items, placing bids, or requesting cancellations, their actions are recorded in real time. The PC or mobile device (hereinafter referred to as the "terminal") used by the user collects this behavioral data and sends it to the server. This behavioral data includes the user ID, item ID, action type (e.g., bid, cancellation request), reason for the action, date and time, etc. For example, if a user places a bid on a specific item, the terminal collects data such as "User ID: 12345," "Item ID: 67890," "Action Type: Bid," "Bid Amount: 1,000 yen," and "Date and Time: October 1, 2023, 6:30 PM," and sends it to the server.

[0095] Collecting and classifying behavioral data

[0096] The server temporarily stores the behavioral data sent from the device in a buffer. Then, it performs batch processing and stores the collected behavioral data in an SQL database (e.g., MySQL or PostgreSQL). This storage process is performed every hour. For example, the insertion query "INSERT INTO user_actions (user_id, action_type, item_id, amount, timestamp) VALUES (...)" is used. The server then classifies this data into specific categories (e.g., "cancellation request," "low bid cancellation," etc.).

[0097] AI analysis of data

[0098] The behavioral data is analyzed using machine learning algorithms (using TensorFlow or PyTorch, for example) installed on the server. The AI ​​module identifies behavioral patterns, frequency, and occurrence conditions from the collected data. For example, it extracts information such as "cancellation requests are concentrated at certain times of the day" or "there are many cancellations for products in certain categories."

[0099] Visualizing and presenting results

[0100] The analysis results are converted into graphs and charts by the server. Data visualization tools (such as D3.js and Chart.js) are used to display histograms of the time-of-day distribution of cancellation requests. This visual data is displayed on the administrator's terminal, allowing the administrator to understand user behavior trends and causes at a glance.

[0101] System improvement proposals

[0102] Based on the analysis results, the server generates specific suggestions for system improvement, such as "provide more detailed descriptions of the relevant products" or "highlight warnings." These suggestions are displayed on the administrator's terminal, allowing the administrator to select and implement countermeasures.

[0103] Examples of concrete examples and prompts

[0104] When a user makes a cancellation request for a specific product, the terminal sends data such as the "user ID," "product ID," "action type (cancellation request)," "reason for the action," and "date and time" to the server. The server collects this data and stores it in a database, after which the AI ​​module analyzes it. The analysis results are converted into graphs and charts by the server and displayed on the administrator's terminal. For example, users can immediately see information such as "there are many cancellation requests in the evening hours" or "cancellations are frequent for products in a specific category." The server also generates specific suggestions for reducing transaction cancellations and displays them on the administrator's terminal.

[0105] Prompt Sentence Examples

[0106] Analyze user behavior based on the following behavioral data and generate specific improvement suggestions.

[0107] User ID: 12345

[0108] Product ID: 67890

[0109] Action Type: Cancellation Request

[0110] Reason for action: Insufficient explanation

[0111] Date and Time: 2023-10-01 18:30:00

[0112] In this way, the present invention enables detailed monitoring of user behavior and analysis of data using AI to efficiently identify the causes of transaction cancellations and bid retractions and take appropriate countermeasures, thereby improving the reliability of the online auction platform and the user experience.

[0113] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0114] Step 1:

[0115] Recording user behavior

[0116] When a user accesses an online auction platform and makes a transaction, the data is recorded on the terminal when the user takes an action such as placing a bid or requesting cancellation for a specific item.

[0117] Input: User action (e.g. bidding on an item, requesting cancellation)

[0118] Specific operation: When a user places a bid on "Product A," the terminal records data such as "User ID," "Product ID," "Action Type (bid)," "Bid Amount," and "Date and Time" in its internal memory.

[0119] Output: Collected user behavior data

[0120] Step 2:

[0121] Sending data

[0122] The device transmits the collected behavioral data to a server.

[0123] Input: User behavior data from the device

[0124] Specific operation: The terminal uses an HTTP request to POST the collected data to the server and send it to the endpoint "api / server / collect".

[0125] Output: User behavior data transferred to the server

[0126] Step 3:

[0127] Data buffering

[0128] The behavioral data received by the server is temporarily stored in a buffer.

[0129] Input: User behavior data transferred from the device

[0130] Specific operation: The server temporarily saves the received data in a buffer area in memory, and temporarily stores "User ID: 12345, Action Type: Bid, Product ID: 67890" in memory.

[0131] Output: Buffered user behavior data

[0132] Step 4:

[0133] Batch processing and database storage

[0134] The server performs batch processing at regular intervals (every hour) and stores the behavioral data in an SQL database.

[0135] Input: Buffered user behavior data

[0136] Specific operation: Every hour, the server executes a batch process, executes the query "INSERT INTO user_actions (user_id, action_type, item_id, amount, timestamp) VALUES (...)", and saves it in the database.

[0137] Output: User behavior data stored in a database

[0138] Step 5:

[0139] Behavioral Data Classification

[0140] The server classifies the data in the database into specific categories.

[0141] Input: User behavior data stored in a database

[0142] Specific operation: The server refers to the database, classifies the behavioral data into categories such as "cancellation request" and "low bid cancellation", and adds the information to the "category" column.

[0143] Output: Categorized user behavior data

[0144] Step 6:

[0145] AI analysis of data

[0146] The classified behavioral data is analyzed using machine learning algorithms within the server.

[0147] Input: Categorized user behavior data

[0148] How it works: Using TensorFlow, PyTorch, and other AI modules, the AI ​​module analyzes behavioral patterns, frequency, and occurrence conditions. For example, it performs clustering to identify "time periods and product categories with high rates of cancellation requests."

[0149] Output: Analysis results (e.g., time periods and product categories with the most cancellation requests)

[0150] Step 7:

[0151] Visual Data Generation

[0152] The server converts the analysis results into graphs and charts.

[0153] Input: Analysis results

[0154] What it does: Use data visualization tools such as D3.js and Chart.js to display a histogram of the distribution of cancellation requests by time period.

[0155] Output: Visual data in the form of graphs and charts

[0156] Step 8:

[0157] Display on the dashboard

[0158] Display visual data on the terminal (for administrator).

[0159] Input: Visual data

[0160] Specific operation: The server generates graphs and charts and renders them on an administrator's dashboard, displaying the results in real time when the administrator accesses it via a browser.

[0161] Output: Visual data displayed on the administrator's terminal

[0162] Step 9:

[0163] Generate improvement suggestions

[0164] The server generates suggestions for improving the system based on the analysis results.

[0165] Input: Analysis results

[0166] Specific operation: Based on the analysis results generated by the AI ​​module, improvement suggestions such as "provide more detailed descriptions of the relevant products" and "highlight cautions" are generated in text format.

[0167] Output: Generated improvement suggestions

[0168] Step 10:

[0169] View Suggestions

[0170] Improvement suggestions are displayed on the terminal (for administrators) so that the administrator can select and implement appropriate countermeasures.

[0171] Input: Generated improvement suggestions

[0172] What it does: The proposal will be displayed on the admin dashboard, allowing the admin to select an action, such as "Approve Proposal."

[0173] Output: Improvement proposals displayed on the administrator's terminal

[0174] (Application example 1)

[0175] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0176] There is a need for a system that can monitor user behavior in real time, analyze collected data, and detect suspicious behavior on online platforms early and suggest appropriate preventive measures. Ignoring suspicious behavior not only increases security risks, but ultimately leads to a poor user experience. Therefore, a system is needed that monitors user behavior, categorizes it, performs AI analysis, and visualizes the results to present them to administrators. In addition, it is also a challenge to establish a system that allows administrators to respond quickly and effectively by proposing specific preventive measures based on the analysis results.

[0177] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0178] In this invention, the server includes a means for detecting suspicious activity based on user behavior data and suggesting appropriate preventive measures, a means for monitoring user behavior data in real time, and a means for collecting the monitored behavior data and storing it in a database, thereby making it possible to detect suspicious activity in real time, identify its causes and conditions, and suggest specific preventive measures.

[0179] Definition of Terms

[0180] "User behavior data" refers to information about various actions and events that users perform within the system, including logins, clicks, data accesses, file changes, etc.

[0181] "Real-time monitoring means" means a system or method for collecting and recording user behavior data in real time, thereby enabling tracking of user behavior without delay.

[0182] A "database storage means" is a system or method that efficiently stores collected behavioral data and allows for quick retrieval upon need.

[0183] A "means for classifying into specific categories" is a system or method for dividing collected behavioral data into pre-defined categories according to its nature or type.

[0184] "AI analytics" is the process of using artificial intelligence to analyze behavioral data and identify patterns and anomalies. This includes machine learning algorithms and data mining techniques.

[0185] A "means for identifying causes and conditions" is a system or method that clarifies the causes and conditions that cause behavior based on analyzed data.

[0186] The "means for visualizing and presenting to the administrator" refers to a system or method for displaying the analysis results in a visual format such as a graph or chart, so that the administrator can easily understand them.

[0187] A "means for generating system improvement proposals" is a system or method that, based on the analysis results, presents a specific action plan aimed at improving user experience or security.

[0188] "Means for detecting suspicious activity and suggesting appropriate preventative measures" refers to a system or method that detects anomalies and risks from user behavior data and suggests measures to strengthen security or encourage careful behavior as countermeasures.

[0189] MODE FOR CARRYING OUT THE INVENTION

[0190] This invention is a system that monitors user behavior data in real time, classifies the collected data into specific categories, and analyzes it using AI. The analysis results are visualized and presented to administrators, and if suspicious behavior is detected, appropriate preventive measures are suggested.

[0191] System configuration

[0192] The server requires the following hardware and software:

[0193] Hardware: Servers with powerful CPUs, RAM, and storage, including resources for rapid data processing and analysis.

[0194] Software: This includes programming languages ​​such as Python, machine learning libraries (e.g., Scikit-learn, TensorFlow), database management systems (e.g., MySQL or PostgreSQL), and visualization libraries (e.g., Matplotlib, Plotly).

[0195] Monitoring Data

[0196] When a user uses the system, their behavioral data is monitored in real time. For example, when a user logs in and accesses a specific file, that information is sent from the user's device to the server in real time. This information includes the user's ID, the timestamp of the action, and the type of action.

[0197] Data collection and storage

[0198] The server temporarily stores the behavioral data received in real time in a buffer, then performs batch processing to store the behavioral data in a database designed to support rapid acquisition and analysis of the behavioral data.

[0199] Data classification and analysis

[0200] The collected behavioral data is categorized into specific categories (e.g., logins, data access, file modifications). The server's AI module analyzes this data and identifies suspicious behavioral patterns. For example, a large number of login attempts in a short period of time can be flagged as a possible account takeover.

[0201] Visualizing the results and presenting them to managers

[0202] The server converts the analysis results into graphs and charts, which are then displayed on the administrator's dashboard, allowing the administrator to intuitively understand user behavioral trends and suspicious behavior.

[0203] Preventive measures suggested

[0204] The server generates specific system improvement suggestions based on the analysis results, such as "If there are a large number of login attempts in a short period of time, suggest strengthening authentication for users." This allows administrators to take security measures quickly and effectively.

[0205] Specific examples

[0206] When a user attempts to access a particular file multiple times, the server immediately records the activity and stores it in a database. The AI ​​module analyzes this data and, if the access attempts exceed the normal range, identifies the activity as suspicious. The results are then presented to the administrator in a graph format, suggesting additional security measures (such as strengthening access restrictions).

[0207] Prompt Sentence Examples

[0208] The prompts for instructing a generative AI model on an analysis task are as follows:

[0209] Analyze real-time monitoring data of user behavior, identify suspicious behavior patterns, and generate specific security improvement suggestions, such as detecting abnormal increases in login counts or data access frequency.

[0210] This allows the invention to be effectively implemented.

[0211] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0212] Program processing steps

[0213] Step 1:

[0214] When a user uses the system, their behavioral data is monitored in real time.

[0215] Input: User behavior data (e.g., login attempts, file access)

[0216] How it works: The user's device collects this behavioral data and packages it with metadata such as a timestamp and user ID.

[0217] Output: Real-time user behavior data packets

[0218] Step 2:

[0219] The monitored behavioral data is sent from the user's device to a server.

[0220] Input: Real-time user behavior data packets

[0221] How it works: The user's device uses HTTP requests and WebSockets to send data to the server.

[0222] Output: Raw behavioral data received by the server

[0223] Step 3:

[0224] The server temporarily stores the received behavioral data in a buffer.

[0225] Input: Raw behavioral data received by the server

[0226] How it works: The server temporarily stores the data in memory or in a high-speed database.

[0227] Output: Buffered behavioral data

[0228] Step 4:

[0229] The server uses batch processing to store the behavioral data in the buffer into a database.

[0230] Input: Buffered behavioral data

[0231] What happens: The server executes a query to store the data in a database.

[0232] Output: Behavioral data stored in a database

[0233] Step 5:

[0234] The server classifies the stored behavioral data into specific categories.

[0235] Input: Behavioral data stored in a database

[0236] Actions: The server separates the behavioral data into categories (e.g., login, data access, file modification) based on predefined rules and conditions.

[0237] Output: Categorized behavioral data

[0238] Step 6:

[0239] The classified behavioral data is analyzed using an AI module.

[0240] Input: Categorized behavioral data

[0241] How it works: Generative AI models use machine learning algorithms to analyze behavioral data and identify suspicious behavioral patterns.

[0242] Output: Analysis results (e.g., abnormal login attempts detected)

[0243] Step 7:

[0244] The analysis results are visualized and presented to the administrator.

[0245] Input: Analysis results

[0246] How it works: The server converts the analysis results into graphs and charts and displays them as a dashboard on the administrator's device.

[0247] Output: Visualized analysis results displayed to the administrator

[0248] Step 8:

[0249] The server generates specific proposals for improving the system based on the analysis results.

[0250] Input: Analysis results

[0251] How it works: The server generates recommendations based on the analysis results, such as security enhancements or system configuration changes.

[0252] Output: System improvement proposals presented to administrators

[0253] Prompt Sentence Examples

[0254] Enter prompts like the following into the generative AI model:

[0255] Analyze real-time monitoring data of user behavior, identify suspicious behavior patterns, and generate specific security improvement suggestions, such as detecting abnormal increases in login counts or data access frequency.

[0256] This specific processing flow allows suspicious user behavior to be detected in real time, enabling a prompt response.

[0257] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0258] Monitoring user behavior and sentiment

[0259] A user accesses the online auction platform and performs trading operations. These operations include bidding, cancellation requests, transaction cancellations, and bid cancellations. The terminal (the user's PC or mobile device) is equipped with an emotion engine that recognizes the user's emotions. The emotion engine uses a camera and microphone to analyze emotions from the user's facial expressions and voice.

[0260] Collecting and classifying behavioral and emotional data

[0261] The device collects user behavioral and emotional data in real time, generates detailed data (user ID, product ID, behavior type, reason for behavior, emotional information, date and time, etc.) and sends it to the server.

[0262] The server temporarily stores the received behavioral and emotional data in a buffer. After that, it performs batch processing and stores the collected data in a database. The database stores all behavioral and emotional data.

[0263] Data analysis

[0264] The server categorizes the behavioral and emotional data stored in the database for analysis. These categorizations include categories such as "cancellation request from the successful bidder," "cancellation of bids below a certain price," "deletion of specific users," "skipping notes," and "slow flow completion." Emotional data is similarly categorized, including emotion categories such as "dissatisfaction," "joy," and "surprise."

[0265] An AI module installed on the server analyzes the classified behavioral and emotional data, using machine learning algorithms to identify behavioral frequency, patterns, conditions for occurrence, and emotional changes.

[0266] Visualizing and presenting results

[0267] The server converts the analyzed results into graphs and charts. This visual data is displayed on a dashboard on the terminal (PC or mobile device used by the administrator). Through the dashboard, administrators can grasp user behavioral trends and changes in emotions in real time.

[0268] System improvement proposals

[0269] Based on the analysis results, the server generates specific suggestions for system improvements, such as providing more detailed descriptions of specific products, simplifying the transaction flow, and presenting customer messages tailored to user sentiment. These suggestions are displayed on the terminal (PC or mobile device used by the administrator), allowing the administrator to select and implement appropriate measures.

[0270] Specific examples

[0271] Sentiment analysis and improvement of cancellation requests

[0272] When a user makes a cancellation request for a specific product, the terminal sends data such as the "user ID," "product ID," "action type (cancellation request)," "reason for action," "date and time," and "emotional information (e.g., dissatisfaction)" to the server.

[0273] The server collects this data and stores it in a database. An AI module then analyzes the data to identify times of day when cancellation requests are most common, product categories, specific user attributes, and associated emotional patterns (e.g., high dissatisfaction).

[0274] The analysis results are converted into graphs and charts by the server and displayed on the terminal (administrator's dashboard). For example, information such as "There are many dissatisfied users in the evening hours" or "There are frequent cancellations and dissatisfaction with products in a particular category" can be immediately confirmed.

[0275] Furthermore, the server generates specific suggestions based on the analysis results to reduce transaction cancellations, such as "providing more detailed product descriptions," "highlighting important points," and "sending follow-up messages to dissatisfied users."

[0276] In this way, the present invention enables detailed monitoring of user behavior and sentiment, and uses AI to analyze the data, thereby efficiently identifying the causes of transaction cancellations and bid retractions and taking appropriate countermeasures, thereby improving the reliability of the online auction platform and improving the user experience.

[0277] The processing flow will be explained below.

[0278] Step 1:

[0279] A user logs in to the online auction platform and performs trading operations, including bidding, requesting cancellation, canceling a transaction, and canceling a bid. The terminal (the user's PC or mobile device) then activates an emotion engine to recognize the user's emotions.

[0280] Step 2:

[0281] In parallel with the transaction, the terminal uses an emotion engine to collect real-time emotional data from the user's facial expressions and voice, including categories such as "happiness," "anger," "anxiety," and "surprise."

[0282] Step 3:

[0283] The terminal generates user behavior data (user ID, product ID, behavior type, reason for behavior, date and time, etc.) and emotion data, and transmits them to the server in real time.

[0284] Step 4:

[0285] The server temporarily stores the received behavioral and emotional data in a buffer, which serves as a temporary storage location for the data.

[0286] Step 5:

[0287] The server periodically runs a batch job and stores the buffered data in a database, where all user behavior and emotion data is stored.

[0288] Step 6:

[0289] The server categorizes the behavioral data and emotional data stored in the database for analysis. Behavioral data is categorized into categories such as "cancellation request from the successful bidder," "cancellation of bids below a certain price," "deletion of specific users," "skipping notes," and "long time required to complete the flow." Emotional data is categorized into emotion categories such as "dissatisfaction," "joy," and "surprise."

[0290] Step 7:

[0291] An AI module installed on the server analyzes the classified behavioral and emotional data and uses machine learning algorithms to identify behavioral frequency, patterns, conditions for occurrence, and emotional changes.

[0292] Step 8:

[0293] The server collects the results of the AI ​​analysis and converts them into visual formats such as graphs and charts. This visualized data is displayed on a dashboard on the terminal (PC or mobile device used by the administrator).

[0294] Step 9:

[0295] The terminal (PC or mobile device used by the administrator) displays the visualized data generated by the server on a dashboard, allowing the administrator to grasp trends in cancellations and bid cancellations, as well as changes in user sentiment, in real time.

[0296] Step 10:

[0297] Based on the results of the AI ​​analysis, the server generates specific suggestions for improving the system, such as "providing more detailed product descriptions," "strengthening transaction management during specific time periods," and "sending customer messages to dissatisfied users."

[0298] Step 11:

[0299] The terminal (PC or mobile device used by the administrator) displays the suggestions from the server on a dashboard, and the administrator can review these suggestions and implement them as necessary.

[0300] Specific examples

[0301] Sentiment analysis and improvement of cancellation requests

[0302] Step 1:

[0303] A user requests cancellation of a particular product, for example, if the product is not what they expected.

[0304] Step 2:

[0305] The device uses a camera to capture the user's facial expression when making a cancellation request and records the audio with a microphone.

[0306] Step 3:

[0307] The device uses an emotion engine to recognize the emotion "dissatisfaction" from the user's facial expressions and voice.

[0308] Step 4:

[0309] The terminal generates detailed data such as "user ID," "product ID," "action type (cancellation request)," "reason for action," "date and time," and "emotional information (dissatisfaction)," and sends it to the server.

[0310] Step 5:

[0311] The server stores this data in a buffer and then periodically moves it to a database.

[0312] Step 6:

[0313] The server categorizes the data stored in the database and analyzes it using an AI module, identifying time periods and product categories with high cancellation requests, as well as related emotional patterns (e.g., high levels of dissatisfaction).

[0314] Step 7:

[0315] The analysis results are converted into graphs and charts by the server and displayed on the terminal (administrator's dashboard). For example, it is possible to see information such as "Many users are dissatisfied in the evening hours" or "Cancellations and dissatisfaction occur frequently for products in a particular category."

[0316] Step 8:

[0317] Based on the analysis results, the server generates specific suggestions to reduce transaction cancellations, such as "make product descriptions more detailed," "highlight important points," or "send follow-up messages to dissatisfied users."

[0318] Step 9:

[0319] The terminal (PC or mobile device used by the administrator) displays the suggestions from the server on a dashboard, and the administrator can review these suggestions and implement them as necessary.

[0320] This series of processes enables detailed monitoring of user behavior and sentiment, and by analyzing the data using AI, it is possible to efficiently identify the causes of transaction cancellations and bid retractions and take appropriate countermeasures, thereby improving the reliability and user experience of the online auction platform.

[0321] Example 2

[0322] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0323] Traditional online platforms typically collect and analyze only user behavioral data. However, this makes it difficult to formulate improvement measures that take user emotions into account, resulting in insufficient improvements to the user experience. Furthermore, analyzing behavioral data alone makes it difficult to accurately grasp the user's emotions and psychological state behind specific actions, making it difficult to find appropriate countermeasures.

[0324] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0325] In this invention, the server includes means for monitoring user behavioral data and emotional data in real time, means for collecting the monitored behavioral data and emotional data and storing it in a database, means for classifying the collected behavioral data and emotional data into specific categories, means for analyzing the classified behavioral data and emotional data using AI and identifying the causes and conditions of the behavior, means for visualizing and presenting the analysis results to an administrator, and means for generating system improvement proposals based on the analysis results. This enables advanced data analysis that takes into account not only user behavior but also emotions, making it possible to formulate more appropriate system improvement measures.

[0326] "User" means any individual or legal entity using the online auction platform.

[0327] "Behavioral Data" refers to information about the specific operations and actions that users take on the online auction platform.

[0328] "Emotion data" is information obtained by analyzing the emotions shown by the user when performing operations.

[0329] "Real-time monitoring means" is a general term for hardware and software for instantly acquiring and monitoring user behavioral and emotional data.

[0330] The "collection means" is a system for collecting monitored data and transmitting it to a database.

[0331] "Means for saving in a database" refers to a mechanism for storing and saving collected data in a database in a certain format.

[0332] "Means for categorizing data into specific categories" refers to methods or techniques for dividing collected data into predetermined categories.

[0333] "Means of analysis" refers to the technology that uses AI to analyze classified data and extract useful information and patterns from that data.

[0334] "Cause and condition identification" is a method for identifying the reasons and circumstances behind user behavior based on data analysis.

[0335] "Means for visualization and presentation to administrators" refers to technology for converting analysis results into visual formats such as graphs and charts, and displaying them in a way that administrators can easily understand.

[0336] "Means for generating system improvement proposals" is a process for creating specific improvement measures to improve user experience based on the analysis results.

[0337] The system of the present invention monitors user behavioral data and emotional data in real time, analyzes this data, and generates system improvement proposals. Specific embodiments for implementing this system will be described below.

[0338] Monitoring user behavior and sentiment

[0339] A user accesses the online auction platform and performs operations such as placing a bid, requesting a cancellation, canceling a transaction, or canceling a bid. At this time, the user's device (PC or mobile device) analyzes the user's facial expressions and voice using an emotion engine. The emotion engine uses hardware such as a camera and microphone to obtain the user's emotion data.

[0340] Collecting and classifying behavioral and emotional data

[0341] The device collects user behavioral and emotional data in real time. This includes detailed data such as user ID, product ID, behavior type, reason for behavior, emotional information, and date and time. This detailed data is sent to the server, which temporarily stores the received data in a buffer. The server then performs batch processing and stores the collected data in a database. All behavioral and emotional data is stored in the database.

[0342] Data analysis

[0343] The server classifies the behavioral and emotional data stored in the database for analysis. This classification includes categories such as "cancellation request from the winning bidder," "cancellation of bids below a certain price," "deletion of specific users," "skipping notes," and "long time it takes to process the flow." Emotional data is also classified into emotion categories such as "dissatisfaction," "happiness," and "surprise." An AI module installed on the server analyzes the classified behavioral and emotional data. The AI ​​module uses machine learning algorithms to identify behavioral frequency, patterns, conditions for occurrence, and emotional changes.

[0344] Visualizing and presenting results

[0345] The server converts the analyzed results into graphs and charts. This visual data is displayed on a dashboard on the terminal (PC or mobile device) used by the administrator. Through the dashboard, administrators can grasp changes in user behavioral trends and emotions in real time.

[0346] System improvement proposals

[0347] Based on the analysis results, the server generates specific suggestions for system improvements, such as providing more detailed descriptions of specific products, simplifying transaction flows, and presenting customer messages tailored to user emotions. These suggestions are displayed on a dashboard on the device used by the administrator, who can then take appropriate action as needed.

[0348] Specific examples

[0349] Sentiment analysis and improvement of cancellation requests

[0350] When a user makes a cancellation request for a specific product, the terminal sends data such as the user ID, product ID, action type (cancellation request), reason for the action, date and time, and emotional information (e.g., dissatisfaction) to the server. The server collects this data and stores it in a database. An AI module then analyzes the data to identify time periods with high cancellation requests, product categories, specific user attributes, and related emotional patterns (e.g., high dissatisfaction). The server converts the analysis results into graphs and charts and displays them on the administrator's dashboard. For example, users can immediately see information such as "there are many dissatisfied users in the evening" or "cancellations and dissatisfaction frequently occur for products in a specific category."

[0351] Furthermore, based on the analysis results, the server generates specific suggestions to reduce transaction cancellations. For example, suggestions include "providing more detailed descriptions of the relevant products," "highlighting important points," and "sending follow-up messages to dissatisfied users." This allows for detailed monitoring of user behavior and sentiment, and by analyzing the data using AI, it is possible to efficiently identify the causes of transaction cancellations and bid retractions and take appropriate countermeasures. The introduction of this system will improve the reliability and user experience of online auction platforms.

[0352] Example of input prompt for generative AI model

[0353] "Please explain the specific methods you use to collect user behavioral and sentiment data."

[0354] "Please detail how you will analyze the collected data and generate improvement measures."

[0355] Please provide a sentiment analysis of users when they make cancellation requests and provide specific suggestions for improvement based on that.

[0356] Using such prompts allows the generative AI model to provide specific and useful information.

[0357] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0358] Step 1:

[0359] A user accesses an online auction platform and logs in. The user performs a transaction (e.g., bid, cancellation request, transaction cancellation, bid cancellation). During this process, the device uses the built-in camera and microphone to collect the user's facial expressions and voice, which are then analyzed by the emotion engine. The input is the user's behavior (transaction) and emotional data (facial expressions, voice), and the output is the analyzed emotional data (e.g., dissatisfaction, joy, surprise).

[0360] Step 2:

[0361] The device analyzes the collected behavioral and emotional data in real time and generates detailed data (user ID, product ID, behavior type, reason for behavior, emotional information, date and time). This detailed data is sent to the server. The input is the behavioral and emotional data obtained in step 1, and the output is the detailed data sent to the server.

[0362] Step 3:

[0363] The server temporarily stores the received detailed data in a buffer, then periodically batch processes it and stores it in a database. This allows all behavioral and emotional data to be accumulated centrally. The input is the detailed data sent from the device, and the output is the behavioral and emotional data stored in the database.

[0364] Step 4:

[0365] The server categorizes the behavioral data and emotional data stored in the database into categories for analysis. For example, behavioral data is categorized into categories such as "cancellation request from the winning bidder" and "cancellation of bids below a certain price," while emotional data is categorized into categories such as "dissatisfaction," "joy," and "surprise." The input is the behavioral data and emotional data stored in the database, and the output is the categorized data.

[0366] Step 5:

[0367] An AI module installed on the server analyzes the categorized behavioral and emotional data. The AI ​​module uses machine learning algorithms to identify behavioral frequency, patterns, conditions for occurrence, and emotional changes. The input is the categorized data, and the output is the analysis results, such as identified behavioral patterns and emotional changes.

[0368] Step 6:

[0369] The server converts the data into graphs and charts based on the analysis results. This visual data is displayed in real time on the dashboard of the administrator's device. The input is the result data analyzed by the AI ​​module, and the output is visual data in the form of graphs and charts.

[0370] Step 7:

[0371] The server generates specific suggestions for system improvement based on the analysis results. These suggestions include providing more detailed descriptions of specific products, simplifying the transaction flow, and presenting customer messages based on user sentiment. These suggestions are displayed on a dashboard on the terminal used by the administrator. The input is the analysis results, and the output is specific suggestions for system improvement.

[0372] By introducing this system, it is possible to closely monitor user behavior and emotions and analyze the data using AI, efficiently identifying the causes of transaction cancellations and bid revocations, and taking appropriate countermeasures.

[0373] (Application example 2)

[0374] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0375] Conventional systems mainly analyze data based solely on user behavior data, and do not take emotional data into account. This makes it difficult to quickly implement appropriate countermeasures to improve user experience and optimize systems. While it is particularly important in brick-and-mortar stores to grasp changes in customer emotions in real time and immediately reflect them in service improvements, achieving this has presented many challenges.

[0376] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0377] In this invention, the server includes means for monitoring user behavioral data and emotional data in real time, means for collecting the monitored behavioral data and emotional data and storing it in a database, means for classifying the collected behavioral data and emotional data into specific categories, means for analyzing the classified behavioral data and emotional data using an AI module and identifying the causes and conditions of changes in the behavior and emotions, means for visualizing and presenting the analysis results to an administrator, means for generating system improvement proposals based on the analysis results, means for collecting new user behavioral and emotional data via specific devices used, and means for transmitting the collected data to the server. This enables analysis that takes into account both user behavioral and emotional data, making it possible to take prompt and appropriate service improvement measures even in physical stores.

[0378] "User behavior data" is information recorded when a user performs a specific action, such as picking up a product, looking at a shelf, or canceling a transaction.

[0379] "Emotional data" is information that is analyzed from the user's facial expressions and voice and indicates their emotional state, such as joy, dissatisfaction, or surprise.

[0380] "Real-time monitoring" is a technology that allows data to be collected and analyzed instantly without delay.

[0381] A "database" is a system for systematically storing and managing collected data.

[0382] "Classifying into specific categories" refers to the process of dividing collected data into predetermined categories, such as "cancellation requests" or "expressions of joy."

[0383] The "AI module" is a program that uses artificial intelligence to analyze collected data and identify patterns and causes of behavior and emotions.

[0384] "Visualizing the analysis results" means displaying the analyzed data in a visual format such as a graph or chart so that the administrator can intuitively understand it.

[0385] "System improvement proposals" refers to generating specific proposals for better services and system optimization based on the analysis results.

[0386] "Specific Equipment Used" refers to hardware such as cameras, microphones, and smart glasses used to collect data.

[0387] A system embodying this invention is designed to monitor user behavioral and emotional data in real time. The system mainly utilizes the following hardware and software:

[0388] Hardware Configuration

[0389] 1. Smart glasses: These are devices that use a built-in camera and microphone to capture the user's facial expressions and voice. This device is used to collect data in real time.

[0390] 2. Server: It is a central control unit for storing collected data, analyzing them and visualizing the results.

[0391] 3. Administrator terminal: A PC or tablet device used to receive analysis results and system improvement proposals and perform management.

[0392] Software Configuration

[0393] 1. EmotionRecognition module: Software that analyzes emotions from the user's facial expressions.

[0394] 2. Voice Analysis Module: Software that analyzes emotions from the user's tone of voice.

[0395] 3. Data collection module: This is software for receiving data sent from the smart glasses and transferring it to the server.

[0396] 4. AI module: An AI program that analyzes collected behavioral and emotional data to identify behavioral patterns and emotional changes.

[0397] 5. Database: A system for systematically storing and managing data.

[0398] 6. Dashboard: An interface for visualizing analysis results in graphs and charts and presenting them to administrators.

[0399] Data processing and calculation

[0400] Data collection: The smart glasses capture the user's facial expressions with a camera and voice with a microphone. These data are analyzed in real time by the EmotionRecognition module and VoiceAnalysis module. The analysis results are collected as behavioral and emotional data.

[0401] Data Transfer: The collected data is sent to the server through the data collection module.

[0402] Data storage: The server stores the received data in a database, which manages both behavioral and emotional data.

[0403] Data analysis: The server's AI module uses the stored data to apply machine learning algorithms to analyze behavioral patterns and emotional changes. Specifically, it analyzes specific user actions (e.g., canceling a specific product) and the emotions they experience (e.g., dissatisfaction) to identify their causes.

[0404] Visualization of results: The analysis results are visualized and displayed on the administrator's dashboard as graphs and charts, allowing administrators to check user behavioral trends and changes in sentiment in real time.

[0405] System improvement proposals: Based on the analysis results, improvement proposals such as detailed product descriptions and simplified transaction flows are automatically generated and notified to the administrator's terminal.

[0406] Specific examples

[0407] For example, if the analysis results show a trend of "many users feeling dissatisfied in the evening," the system will notify the administrator with suggestions for improvement, such as "make the product descriptions more detailed" or "highlight warnings." The following prompt sentences could be considered as specific examples:

[0408] Prompt Sentence Examples

[0409] "Recent data shows that many customers are frustrated when reading the descriptions of new products. What can you do to improve this?"

[0410] As a result, the present invention makes it possible to comprehensively analyze user behavioral data and emotional data, and to propose prompt and appropriate service improvement measures even in physical stores.

[0411] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0412] Step 1:

[0413] The user puts on the smart glasses and walks around the store.

[0414] Input: User's facial expression and audio.

[0415] Output: Image data captured by the camera and audio data recorded by the microphone.

[0416] Specific operation: The camera in the smart glasses continuously captures the user's facial expressions, and the microphone records audio.

[0417] Step 2:

[0418] The smart glasses device uses the EmotionRecognition and VoiceAnalysis modules to analyze the collected data in real time.

[0419] Input: Image data captured by the camera and recorded audio data.

[0420] Output: Parsed emotion and speech data (e.g., happy, frustrated, surprised).

[0421] Specific operation: The EmotionRecognition module analyzes facial expressions, and the VoiceAnalysis module analyzes voice tones to generate emotion data.

[0422] Step 3:

[0423] The analyzed data is sent to a server via a data collection module.

[0424] Input: Emotion data and speech data.

[0425] Output: Data stored on the server.

[0426] Specific operation: The data collection module packages the analyzed data and sends it to the server over the network.

[0427] Step 4:

[0428] The server stores the received data in a database.

[0429] Input: Emotion data and voice data sent to the server.

[0430] Output: Structured data stored in a database.

[0431] Specific operation: The server analyzes the received data and stores it appropriately in the database.

[0432] Step 5:

[0433] The server's AI module performs analysis using behavioral and emotional data stored in a database.

[0434] Input: User behavioral and emotional data stored in a database.

[0435] Output: Analysis results (e.g., many users feel dissatisfied at certain times of the day).

[0436] How it works: The AI ​​module applies machine learning algorithms to identify behavioral patterns and causes of emotional changes.

[0437] Step 6:

[0438] The analysis results are visualized and displayed on the administrator's terminal (PC or tablet).

[0439] Input: Analysis result data.

[0440] Output: Analysis results displayed as graphs and charts.

[0441] Specific operation: The analysis results data is converted into graphs and charts and displayed on a dashboard.

[0442] Step 7:

[0443] Based on the analysis results, proposals for system improvement are generated and notified to the administrator's terminal.

[0444] Input: Analysis result data.

[0445] Output: Suggestions for improving the system (e.g., making product descriptions more detailed, highlighting cautions).

[0446] Specific operation: The AI ​​module generates improvement suggestions based on the analysis results and notifies the administrator terminal.

[0447] Step 8:

[0448] Administrators can review the suggested improvements through the dashboard and take appropriate action.

[0449] Input: System improvement suggestions.

[0450] Output: Corrective action taken (e.g. product description update, repositioning).

[0451] Specific Action: Administrator interacts with the dashboard and takes appropriate remedial action.

[0452] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0453] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0454] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0455] [Second embodiment]

[0456] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0457] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0458] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0459] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0460] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0461] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0462] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0463] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0464] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0465] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0466] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0467] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0468] User behavior monitoring

[0469] When users access the online auction platform and make transactions, their actions are recorded in real time. The terminal (the user's PC or mobile device) collects and transmits user behavior data to the server. This behavior data includes bids, cancellation requests, transaction cancellations, and bid retractions.

[0470] Collecting and classifying behavioral data

[0471] The server temporarily stores the received behavioral data in a buffer. It then performs batch processing, stores the collected behavioral data in a database, and classifies the data into specific categories. For example, this classification is performed in categories such as "cancellation request from the successful bidder," "cancellation of bids below a certain price," "deletion of specific users," "skipping notes," and "long time required to complete the flow."

[0472] AI analysis of data

[0473] An AI module installed on the server analyzes the stored behavioral data. The AI ​​uses machine learning algorithms to identify behavioral patterns, frequency, conditions, and causes. For example, it can identify product categories with a high number of cancellation requests or specific time periods where cancellations are concentrated.

[0474] Visualizing and presenting results

[0475] The server converts the analyzed results into graphs and charts, and this visual data is displayed on a dashboard on the terminal (PC or mobile device used by the administrator), allowing the administrator to grasp the trends and causes of user behavior at a glance.

[0476] System improvement proposals

[0477] Based on the analysis results, the server generates specific system improvement suggestions, such as providing more detailed descriptions of specific products or simplifying the transaction flow, to improve the user experience. These suggestions are displayed on the terminal (PC or mobile device used by the administrator), allowing the administrator to select and implement appropriate measures.

[0478] Specific examples

[0479] Analysis and improvement of cancellation requests

[0480] When a user makes a cancellation request for a specific product, the terminal sends data such as the "user ID," "product ID," "action type (cancellation request)," "reason for action," and "date and time" to the server.

[0481] The server collects this data and stores it in a database, after which an AI module analyzes it to identify patterns, such as times of day when cancellation requests are most common, product categories, or specific user attributes.

[0482] The analysis results are converted into graphs and charts by the server and displayed on the terminal (administrator's dashboard). For example, information such as "there are many cancellation requests in the evening hours" or "there are frequent cancellations for products in a particular category" can be immediately confirmed.

[0483] Furthermore, the server generates specific suggestions based on the analysis results to reduce transaction cancellations, such as "provide more detailed product descriptions" or "highlight cautions."

[0484] In this way, the present invention enables detailed monitoring of user behavior and analysis of data using AI to efficiently identify the causes of transaction cancellations and bid retractions and take appropriate countermeasures, thereby improving the reliability of the online auction platform and the user experience.

[0485] The processing flow will be explained below.

[0486] Step 1:

[0487] A user logs into the online auction platform and performs trading operations, including placing a bid, requesting a cancellation, canceling a transaction, and withdrawing a bid.

[0488] Step 2:

[0489] The terminal (PC or mobile device used by the user) records each operation performed by the user as an event and generates detailed data related to it (user ID, product ID, action type, reason for action, date and time, etc.).

[0490] Step 3:

[0491] The terminal transmits the generated detailed data to the server in real time.

[0492] Step 4:

[0493] The server temporarily stores the received behavioral data in a buffer, which acts as a temporary storage location for subsequent batch processing of the data into a database.

[0494] Step 5:

[0495] The server periodically runs a batch job to move the data from the buffer to a database, where all user activity data is stored.

[0496] Step 6:

[0497] The server categorizes the behavioral data stored in the database for analysis, including categories such as "cancellation request from the successful bidder," "cancellation of bids below a certain price," "deletion of specific users," "skipping notes," and "long time required to complete the flow."

[0498] Step 7:

[0499] An AI module installed on the server analyzes the categorized behavioral data and uses machine learning algorithms to identify behavioral frequencies, patterns, and conditions for occurrence.

[0500] Step 8:

[0501] The server collects the results of the AI ​​analysis and converts them into visual formats such as graphs and charts, which makes the data easier for administrators to understand.

[0502] Step 9:

[0503] The terminal (PC or mobile device used by the administrator) displays the visualized data generated by the server on a dashboard, allowing the administrator to grasp trends in cancellations and bid withdrawals in real time.

[0504] Step 10:

[0505] The server generates specific suggestions for system improvement based on the results of the AI ​​analysis, such as "providing more detailed product descriptions" or "strengthening transaction management during specific time periods."

[0506] Step 11:

[0507] The terminal (PC or mobile device used by the administrator) displays the suggestions from the server on a dashboard. The administrator can check these suggestions and implement them as necessary.

[0508] Example 1

[0509] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0510] Conventional online auction platforms face challenges in effectively monitoring, collecting, and analyzing user behavior data, making it difficult to quickly and appropriately address issues such as transaction cancellations and bid revocations. Furthermore, there is no well-established method for analyzing this data and presenting it to administrators in an intuitive and easy-to-understand format. As a result, it is difficult to improve the user experience and maintain the reliability of the platform.

[0511] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0512] In this invention, the server includes means for monitoring user behavior data in real time, means for collecting the monitored behavior data and storing it in a storage device, means for classifying the collected behavior data into specific categories, means for analyzing the behavior data using a machine learning algorithm and identifying the causes and conditions of the behavior, means for visualizing the analysis results in graphs and charts and presenting them on an administrator terminal, and means for generating system improvement proposals based on the analysis results and displaying them on the administrator terminal. This enables detailed monitoring of user behavior, effective analysis and visual presentation of data, and enables an improved user experience and maintenance of platform reliability.

[0513] "User" means any person or legal entity that transacts or operates using the online auction platform.

[0514] "Behavioral Data" means information about your transactions and interactions with the online auction platform.

[0515] "Real-time" refers to processing occurring immediately at the moment a user action occurs.

[0516] "Monitoring" refers to the process of continuously watching and recording user activity.

[0517] A "storage device" is hardware or software for storing data.

[0518] A "category" is a specific criterion or grouping for classifying behavioral data.

[0519] A "machine learning algorithm" is an artificial intelligence technology for data analysis that learns from data based on a set of rules and makes predictions and classifications.

[0520] "Analysis results" refer to the information and insights obtained after analyzing behavioral data using machine learning algorithms.

[0521] "Visualization" refers to the presentation of analytical results in a visual format such as a graph or chart.

[0522] An "administrator terminal" is an electronic device used by a system administrator, which has the function of displaying analysis results and suggestions.

[0523] "System improvement proposals" are specific measures or strategies for optimizing or improving the system based on the analysis results.

[0524] The present invention provides a system for real-time monitoring of user behavior data on an online auction platform, and effectively collects, classifies, analyzes, and visualizes the data. Specific embodiments of the system are described below.

[0525] User behavior monitoring

[0526] When a user accesses an online auction platform and conducts transactions such as browsing items, placing bids, or requesting cancellations, their actions are recorded in real time. The PC or mobile device (hereinafter referred to as the "terminal") used by the user collects this behavioral data and sends it to the server. This behavioral data includes the user ID, item ID, action type (e.g., bid, cancellation request), reason for the action, date and time, etc. For example, if a user places a bid on a specific item, the terminal collects data such as "User ID: 12345," "Item ID: 67890," "Action Type: Bid," "Bid Amount: 1,000 yen," and "Date and Time: October 1, 2023, 6:30 PM," and sends it to the server.

[0527] Collecting and classifying behavioral data

[0528] The server temporarily stores the behavioral data sent from the device in a buffer. Then, it performs batch processing and stores the collected behavioral data in an SQL database (e.g., MySQL or PostgreSQL). This storage process is performed every hour. For example, the insertion query "INSERT INTO user_actions (user_id, action_type, item_id, amount, timestamp) VALUES (...)" is used. The server then classifies this data into specific categories (e.g., "cancellation request," "low bid cancellation," etc.).

[0529] AI analysis of data

[0530] The behavioral data is analyzed using machine learning algorithms (using TensorFlow or PyTorch, for example) installed on the server. The AI ​​module identifies behavioral patterns, frequency, and occurrence conditions from the collected data. For example, it extracts information such as "cancellation requests are concentrated at certain times of the day" or "there are many cancellations for products in certain categories."

[0531] Visualizing and presenting results

[0532] The analysis results are converted into graphs and charts by the server. Data visualization tools (such as D3.js and Chart.js) are used to display histograms of the time-of-day distribution of cancellation requests. This visual data is displayed on the administrator's terminal, allowing the administrator to understand user behavior trends and causes at a glance.

[0533] System improvement proposals

[0534] Based on the analysis results, the server generates specific suggestions for system improvement, such as "provide more detailed descriptions of the relevant products" or "highlight warnings." These suggestions are displayed on the administrator's terminal, allowing the administrator to select and implement countermeasures.

[0535] Examples of concrete examples and prompts

[0536] When a user makes a cancellation request for a specific product, the terminal sends data such as the "user ID," "product ID," "action type (cancellation request)," "reason for the action," and "date and time" to the server. The server collects this data and stores it in a database, after which the AI ​​module analyzes it. The analysis results are converted into graphs and charts by the server and displayed on the administrator's terminal. For example, users can immediately see information such as "there are many cancellation requests in the evening hours" or "cancellations are frequent for products in a specific category." The server also generates specific suggestions for reducing transaction cancellations and displays them on the administrator's terminal.

[0537] Prompt Sentence Examples

[0538] Analyze user behavior based on the following behavioral data and generate specific improvement suggestions.

[0539] User ID: 12345

[0540] Product ID: 67890

[0541] Action Type: Cancellation Request

[0542] Reason for action: Insufficient explanation

[0543] Date and Time: 2023-10-01 18:30:00

[0544] In this way, the present invention enables detailed monitoring of user behavior and analysis of data using AI to efficiently identify the causes of transaction cancellations and bid retractions and take appropriate countermeasures, thereby improving the reliability of the online auction platform and the user experience.

[0545] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0546] Step 1:

[0547] Recording user behavior

[0548] When a user accesses an online auction platform and makes a transaction, the data is recorded on the terminal when the user takes an action such as placing a bid or requesting cancellation for a specific item.

[0549] Input: User action (e.g. bidding on an item, requesting cancellation)

[0550] Specific operation: When a user places a bid on "Product A," the terminal records data such as "User ID," "Product ID," "Action Type (bid)," "Bid Amount," and "Date and Time" in its internal memory.

[0551] Output: Collected user behavior data

[0552] Step 2:

[0553] Sending data

[0554] The device transmits the collected behavioral data to a server.

[0555] Input: User behavior data from the device

[0556] Specific operation: The terminal uses an HTTP request to POST the collected data to the server and send it to the endpoint "api / server / collect".

[0557] Output: User behavior data transferred to the server

[0558] Step 3:

[0559] Data buffering

[0560] The behavioral data received by the server is temporarily stored in a buffer.

[0561] Input: User behavior data transferred from the device

[0562] Specific operation: The server temporarily saves the received data in a buffer area in memory, and temporarily stores "User ID: 12345, Action Type: Bid, Product ID: 67890" in memory.

[0563] Output: Buffered user behavior data

[0564] Step 4:

[0565] Batch processing and database storage

[0566] The server performs batch processing at regular intervals (every hour) and stores the behavioral data in an SQL database.

[0567] Input: Buffered user behavior data

[0568] Specific operation: Every hour, the server executes a batch process, executes the query "INSERT INTO user_actions (user_id, action_type, item_id, amount, timestamp) VALUES (...)", and saves it in the database.

[0569] Output: User behavior data stored in a database

[0570] Step 5:

[0571] Behavioral Data Classification

[0572] The server classifies the data in the database into specific categories.

[0573] Input: User behavior data stored in a database

[0574] Specific operation: The server refers to the database, classifies the behavioral data into categories such as "cancellation request" and "low bid cancellation", and adds the information to the "category" column.

[0575] Output: Categorized user behavior data

[0576] Step 6:

[0577] AI analysis of data

[0578] The classified behavioral data is analyzed using machine learning algorithms within the server.

[0579] Input: Categorized user behavior data

[0580] How it works: Using TensorFlow, PyTorch, and other AI modules, the AI ​​module analyzes behavioral patterns, frequency, and occurrence conditions. For example, it performs clustering to identify "time periods and product categories with high rates of cancellation requests."

[0581] Output: Analysis results (e.g., time periods and product categories with the most cancellation requests)

[0582] Step 7:

[0583] Visual Data Generation

[0584] The server converts the analysis results into graphs and charts.

[0585] Input: Analysis results

[0586] What it does: Use data visualization tools such as D3.js and Chart.js to display a histogram of the distribution of cancellation requests by time period.

[0587] Output: Visual data in the form of graphs and charts

[0588] Step 8:

[0589] Display on the dashboard

[0590] Display visual data on the terminal (for administrator).

[0591] Input: Visual data

[0592] Specific operation: The server generates graphs and charts and renders them on an administrator's dashboard, displaying the results in real time when the administrator accesses it via a browser.

[0593] Output: Visual data displayed on the administrator's terminal

[0594] Step 9:

[0595] Generate improvement suggestions

[0596] The server generates suggestions for improving the system based on the analysis results.

[0597] Input: Analysis results

[0598] Specific operation: Based on the analysis results generated by the AI ​​module, improvement suggestions such as "provide more detailed descriptions of the relevant products" and "highlight cautions" are generated in text format.

[0599] Output: Generated improvement suggestions

[0600] Step 10:

[0601] View Suggestions

[0602] Improvement suggestions are displayed on the terminal (for administrators) so that the administrator can select and implement appropriate countermeasures.

[0603] Input: Generated improvement suggestions

[0604] What it does: The proposal will be displayed on the admin dashboard, allowing the admin to select an action, such as "Approve Proposal."

[0605] Output: Improvement proposals displayed on the administrator's terminal

[0606] (Application example 1)

[0607] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0608] There is a need for a system that can monitor user behavior in real time, analyze collected data, and detect suspicious behavior on online platforms early and suggest appropriate preventive measures. Ignoring suspicious behavior not only increases security risks, but ultimately leads to a poor user experience. Therefore, a system is needed that monitors user behavior, categorizes it, performs AI analysis, and visualizes the results to present them to administrators. In addition, it is also a challenge to establish a system that allows administrators to respond quickly and effectively by proposing specific preventive measures based on the analysis results.

[0609] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0610] In this invention, the server includes a means for detecting suspicious activity based on user behavior data and suggesting appropriate preventive measures, a means for monitoring user behavior data in real time, and a means for collecting the monitored behavior data and storing it in a database, thereby making it possible to detect suspicious activity in real time, identify its causes and conditions, and suggest specific preventive measures.

[0611] Definition of Terms

[0612] "User behavior data" refers to information about various actions and events that users perform within the system, including logins, clicks, data accesses, file changes, etc.

[0613] "Real-time monitoring means" means a system or method for collecting and recording user behavior data in real time, thereby enabling tracking of user behavior without delay.

[0614] A "database storage means" is a system or method that efficiently stores collected behavioral data and allows for quick retrieval upon need.

[0615] A "means for classifying into specific categories" is a system or method for dividing collected behavioral data into pre-defined categories according to its nature or type.

[0616] "AI analytics" is the process of using artificial intelligence to analyze behavioral data and identify patterns and anomalies. This includes machine learning algorithms and data mining techniques.

[0617] A "means for identifying causes and conditions" is a system or method that clarifies the causes and conditions that cause behavior based on analyzed data.

[0618] The "means for visualizing and presenting to the administrator" refers to a system or method for displaying the analysis results in a visual format such as a graph or chart, so that the administrator can easily understand them.

[0619] A "means for generating system improvement proposals" is a system or method that, based on the analysis results, presents a specific action plan aimed at improving user experience or security.

[0620] "Means for detecting suspicious activity and suggesting appropriate preventative measures" refers to a system or method that detects anomalies and risks from user behavior data and suggests measures to strengthen security or encourage careful behavior as countermeasures.

[0621] MODE FOR CARRYING OUT THE INVENTION

[0622] This invention is a system that monitors user behavior data in real time, classifies the collected data into specific categories, and analyzes it using AI. The analysis results are visualized and presented to administrators, and if suspicious behavior is detected, appropriate preventive measures are suggested.

[0623] System configuration

[0624] The server requires the following hardware and software:

[0625] Hardware: Servers with powerful CPUs, RAM, and storage, including resources for rapid data processing and analysis.

[0626] Software: This includes programming languages ​​such as Python, machine learning libraries (e.g., Scikit-learn, TensorFlow), database management systems (e.g., MySQL or PostgreSQL), and visualization libraries (e.g., Matplotlib, Plotly).

[0627] Monitoring Data

[0628] When a user uses the system, their behavioral data is monitored in real time. For example, when a user logs in and accesses a specific file, that information is sent from the user's device to the server in real time. This information includes the user's ID, the timestamp of the action, and the type of action.

[0629] Data collection and storage

[0630] The server temporarily stores the behavioral data received in real time in a buffer, then performs batch processing to store the behavioral data in a database designed to support rapid acquisition and analysis of the behavioral data.

[0631] Data classification and analysis

[0632] The collected behavioral data is categorized into specific categories (e.g., logins, data access, file modifications). The server's AI module analyzes this data and identifies suspicious behavioral patterns. For example, a large number of login attempts in a short period of time can be flagged as a possible account takeover.

[0633] Visualizing the results and presenting them to managers

[0634] The server converts the analysis results into graphs and charts, which are then displayed on the administrator's dashboard, allowing the administrator to intuitively understand user behavioral trends and suspicious behavior.

[0635] Preventive measures suggested

[0636] The server generates specific system improvement suggestions based on the analysis results, such as "If there are a large number of login attempts in a short period of time, suggest strengthening authentication for users." This allows administrators to take security measures quickly and effectively.

[0637] Specific examples

[0638] When a user attempts to access a particular file multiple times, the server immediately records the activity and stores it in a database. The AI ​​module analyzes this data and, if the access attempts exceed the normal range, identifies the activity as suspicious. The results are then presented to the administrator in a graph format, suggesting additional security measures (such as strengthening access restrictions).

[0639] Prompt Sentence Examples

[0640] The prompts for instructing a generative AI model on an analysis task are as follows:

[0641] Analyze real-time monitoring data of user behavior, identify suspicious behavior patterns, and generate specific security improvement suggestions, such as detecting abnormal increases in login counts or data access frequency.

[0642] This allows the invention to be effectively implemented.

[0643] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0644] Program processing steps

[0645] Step 1:

[0646] When a user uses the system, their behavioral data is monitored in real time.

[0647] Input: User behavior data (e.g., login attempts, file access)

[0648] How it works: The user's device collects this behavioral data and packages it with metadata such as a timestamp and user ID.

[0649] Output: Real-time user behavior data packets

[0650] Step 2:

[0651] The monitored behavioral data is sent from the user's device to a server.

[0652] Input: Real-time user behavior data packets

[0653] How it works: The user's device uses HTTP requests and WebSockets to send data to the server.

[0654] Output: Raw behavioral data received by the server

[0655] Step 3:

[0656] The server temporarily stores the received behavioral data in a buffer.

[0657] Input: Raw behavioral data received by the server

[0658] How it works: The server temporarily stores the data in memory or in a high-speed database.

[0659] Output: Buffered behavioral data

[0660] Step 4:

[0661] The server uses batch processing to store the behavioral data in the buffer into a database.

[0662] Input: Buffered behavioral data

[0663] What happens: The server executes a query to store the data in a database.

[0664] Output: Behavioral data stored in a database

[0665] Step 5:

[0666] The server classifies the stored behavioral data into specific categories.

[0667] Input: Behavioral data stored in a database

[0668] Actions: The server separates the behavioral data into categories (e.g., login, data access, file modification) based on predefined rules and conditions.

[0669] Output: Categorized behavioral data

[0670] Step 6:

[0671] The classified behavioral data is analyzed using an AI module.

[0672] Input: Categorized behavioral data

[0673] How it works: Generative AI models use machine learning algorithms to analyze behavioral data and identify suspicious behavioral patterns.

[0674] Output: Analysis results (e.g., abnormal login attempts detected)

[0675] Step 7:

[0676] The analysis results are visualized and presented to the administrator.

[0677] Input: Analysis results

[0678] How it works: The server converts the analysis results into graphs and charts and displays them as a dashboard on the administrator's device.

[0679] Output: Visualized analysis results displayed to the administrator

[0680] Step 8:

[0681] The server generates specific proposals for improving the system based on the analysis results.

[0682] Input: Analysis results

[0683] How it works: The server generates recommendations based on the analysis results, such as security enhancements or system configuration changes.

[0684] Output: System improvement proposals presented to administrators

[0685] Prompt Sentence Examples

[0686] Enter prompts like the following into the generative AI model:

[0687] Analyze real-time monitoring data of user behavior, identify suspicious behavior patterns, and generate specific security improvement suggestions, such as detecting abnormal increases in login counts or data access frequency.

[0688] This specific processing flow allows suspicious user behavior to be detected in real time, enabling a prompt response.

[0689] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0690] Monitoring user behavior and sentiment

[0691] A user accesses the online auction platform and performs trading operations. These operations include bidding, cancellation requests, transaction cancellations, and bid cancellations. The terminal (the user's PC or mobile device) is equipped with an emotion engine that recognizes the user's emotions. The emotion engine uses a camera and microphone to analyze emotions from the user's facial expressions and voice.

[0692] Collecting and classifying behavioral and emotional data

[0693] The device collects user behavioral and emotional data in real time, generates detailed data (user ID, product ID, behavior type, reason for behavior, emotional information, date and time, etc.) and sends it to the server.

[0694] The server temporarily stores the received behavioral and emotional data in a buffer. After that, it performs batch processing and stores the collected data in a database. The database stores all behavioral and emotional data.

[0695] Data analysis

[0696] The server categorizes the behavioral and emotional data stored in the database for analysis. These categorizations include categories such as "cancellation request from the successful bidder," "cancellation of bids below a certain price," "deletion of specific users," "skipping notes," and "slow flow completion." Emotional data is similarly categorized, including emotion categories such as "dissatisfaction," "joy," and "surprise."

[0697] An AI module installed on the server analyzes the classified behavioral and emotional data, using machine learning algorithms to identify behavioral frequency, patterns, conditions for occurrence, and emotional changes.

[0698] Visualizing and presenting results

[0699] The server converts the analyzed results into graphs and charts. This visual data is displayed on a dashboard on the terminal (PC or mobile device used by the administrator). Through the dashboard, administrators can grasp user behavioral trends and changes in emotions in real time.

[0700] System improvement proposals

[0701] Based on the analysis results, the server generates specific suggestions for system improvements, such as providing more detailed descriptions of specific products, simplifying the transaction flow, and presenting customer messages tailored to user sentiment. These suggestions are displayed on the terminal (PC or mobile device used by the administrator), allowing the administrator to select and implement appropriate measures.

[0702] Specific examples

[0703] Sentiment analysis and improvement of cancellation requests

[0704] When a user makes a cancellation request for a specific product, the terminal sends data such as the "user ID," "product ID," "action type (cancellation request)," "reason for action," "date and time," and "emotional information (e.g., dissatisfaction)" to the server.

[0705] The server collects this data and stores it in a database. An AI module then analyzes the data to identify times of day when cancellation requests are most common, product categories, specific user attributes, and associated emotional patterns (e.g., high dissatisfaction).

[0706] The analysis results are converted into graphs and charts by the server and displayed on the terminal (administrator's dashboard). For example, information such as "There are many dissatisfied users in the evening hours" or "There are frequent cancellations and dissatisfaction with products in a particular category" can be immediately confirmed.

[0707] Furthermore, the server generates specific suggestions based on the analysis results to reduce transaction cancellations, such as "providing more detailed product descriptions," "highlighting important points," and "sending follow-up messages to dissatisfied users."

[0708] In this way, the present invention enables detailed monitoring of user behavior and sentiment, and uses AI to analyze the data, thereby efficiently identifying the causes of transaction cancellations and bid retractions and taking appropriate countermeasures, thereby improving the reliability of the online auction platform and improving the user experience.

[0709] The processing flow will be explained below.

[0710] Step 1:

[0711] A user logs in to the online auction platform and performs trading operations, including bidding, requesting cancellation, canceling a transaction, and canceling a bid. The terminal (the user's PC or mobile device) then activates an emotion engine to recognize the user's emotions.

[0712] Step 2:

[0713] In parallel with the transaction, the terminal uses an emotion engine to collect real-time emotional data from the user's facial expressions and voice, including categories such as "happiness," "anger," "anxiety," and "surprise."

[0714] Step 3:

[0715] The terminal generates user behavior data (user ID, product ID, behavior type, reason for behavior, date and time, etc.) and emotion data, and transmits them to the server in real time.

[0716] Step 4:

[0717] The server temporarily stores the received behavioral and emotional data in a buffer, which serves as a temporary storage location for the data.

[0718] Step 5:

[0719] The server periodically runs a batch job and stores the buffered data in a database, where all user behavior and emotion data is stored.

[0720] Step 6:

[0721] The server categorizes the behavioral data and emotional data stored in the database for analysis. Behavioral data is categorized into categories such as "cancellation request from the successful bidder," "cancellation of bids below a certain price," "deletion of specific users," "skipping notes," and "long time required to complete the flow." Emotional data is categorized into emotion categories such as "dissatisfaction," "joy," and "surprise."

[0722] Step 7:

[0723] An AI module installed on the server analyzes the classified behavioral and emotional data and uses machine learning algorithms to identify behavioral frequency, patterns, conditions for occurrence, and emotional changes.

[0724] Step 8:

[0725] The server collects the results of the AI ​​analysis and converts them into visual formats such as graphs and charts. This visualized data is displayed on a dashboard on the terminal (PC or mobile device used by the administrator).

[0726] Step 9:

[0727] The terminal (PC or mobile device used by the administrator) displays the visualized data generated by the server on a dashboard, allowing the administrator to grasp trends in cancellations and bid cancellations, as well as changes in user sentiment, in real time.

[0728] Step 10:

[0729] Based on the results of the AI ​​analysis, the server generates specific suggestions for improving the system, such as "providing more detailed product descriptions," "strengthening transaction management during specific time periods," and "sending customer messages to dissatisfied users."

[0730] Step 11:

[0731] The terminal (PC or mobile device used by the administrator) displays the suggestions from the server on a dashboard, and the administrator can review these suggestions and implement them as necessary.

[0732] Specific examples

[0733] Sentiment analysis and improvement of cancellation requests

[0734] Step 1:

[0735] A user requests cancellation of a particular product, for example, if the product is not what they expected.

[0736] Step 2:

[0737] The device uses a camera to capture the user's facial expression when making a cancellation request and records the audio with a microphone.

[0738] Step 3:

[0739] The device uses an emotion engine to recognize the emotion "dissatisfaction" from the user's facial expressions and voice.

[0740] Step 4:

[0741] The terminal generates detailed data such as "user ID," "product ID," "action type (cancellation request)," "reason for action," "date and time," and "emotional information (dissatisfaction)," and sends it to the server.

[0742] Step 5:

[0743] The server stores this data in a buffer and then periodically moves it to a database.

[0744] Step 6:

[0745] The server categorizes the data stored in the database and analyzes it using an AI module, identifying time periods and product categories with high cancellation requests, as well as related emotional patterns (e.g., high levels of dissatisfaction).

[0746] Step 7:

[0747] The analysis results are converted into graphs and charts by the server and displayed on the terminal (administrator's dashboard). For example, it is possible to see information such as "Many users are dissatisfied in the evening hours" or "Cancellations and dissatisfaction occur frequently for products in a particular category."

[0748] Step 8:

[0749] Based on the analysis results, the server generates specific suggestions to reduce transaction cancellations, such as "make product descriptions more detailed," "highlight important points," or "send follow-up messages to dissatisfied users."

[0750] Step 9:

[0751] The terminal (PC or mobile device used by the administrator) displays the suggestions from the server on a dashboard, and the administrator can review these suggestions and implement them as necessary.

[0752] This series of processes enables detailed monitoring of user behavior and sentiment, and by analyzing the data using AI, it is possible to efficiently identify the causes of transaction cancellations and bid retractions and take appropriate countermeasures, thereby improving the reliability and user experience of the online auction platform.

[0753] Example 2

[0754] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0755] Traditional online platforms typically collect and analyze only user behavioral data. However, this makes it difficult to formulate improvement measures that take user emotions into account, resulting in insufficient improvements to the user experience. Furthermore, analyzing behavioral data alone makes it difficult to accurately grasp the user's emotions and psychological state behind specific actions, making it difficult to find appropriate countermeasures.

[0756] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0757] In this invention, the server includes means for monitoring user behavioral data and emotional data in real time, means for collecting the monitored behavioral data and emotional data and storing it in a database, means for classifying the collected behavioral data and emotional data into specific categories, means for analyzing the classified behavioral data and emotional data using AI and identifying the causes and conditions of the behavior, means for visualizing and presenting the analysis results to an administrator, and means for generating system improvement proposals based on the analysis results. This enables advanced data analysis that takes into account not only user behavior but also emotions, making it possible to formulate more appropriate system improvement measures.

[0758] "User" means any individual or legal entity using the online auction platform.

[0759] "Behavioral Data" refers to information about the specific operations and actions that users take on the online auction platform.

[0760] "Emotion data" is information obtained by analyzing the emotions shown by the user when performing operations.

[0761] "Real-time monitoring means" is a general term for hardware and software for instantly acquiring and monitoring user behavioral and emotional data.

[0762] The "collection means" is a system for collecting monitored data and transmitting it to a database.

[0763] "Means for saving in a database" refers to a mechanism for storing and saving collected data in a database in a certain format.

[0764] "Means for categorizing data into specific categories" refers to methods or techniques for dividing collected data into predetermined categories.

[0765] "Means of analysis" refers to the technology that uses AI to analyze classified data and extract useful information and patterns from that data.

[0766] "Cause and condition identification" is a method for identifying the reasons and circumstances behind user behavior based on data analysis.

[0767] "Means for visualization and presentation to administrators" refers to technology for converting analysis results into visual formats such as graphs and charts, and displaying them in a way that administrators can easily understand.

[0768] "Means for generating system improvement proposals" is a process for creating specific improvement measures to improve user experience based on the analysis results.

[0769] The system of the present invention monitors user behavioral data and emotional data in real time, analyzes this data, and generates system improvement proposals. Specific embodiments for implementing this system will be described below.

[0770] Monitoring user behavior and sentiment

[0771] A user accesses the online auction platform and performs operations such as placing a bid, requesting a cancellation, canceling a transaction, or canceling a bid. At this time, the user's device (PC or mobile device) analyzes the user's facial expressions and voice using an emotion engine. The emotion engine uses hardware such as a camera and microphone to obtain the user's emotion data.

[0772] Collecting and classifying behavioral and emotional data

[0773] The device collects user behavioral and emotional data in real time. This includes detailed data such as user ID, product ID, behavior type, reason for behavior, emotional information, and date and time. This detailed data is sent to the server, which temporarily stores the received data in a buffer. The server then performs batch processing and stores the collected data in a database. All behavioral and emotional data is stored in the database.

[0774] Data analysis

[0775] The server classifies the behavioral and emotional data stored in the database for analysis. This classification includes categories such as "cancellation request from the winning bidder," "cancellation of bids below a certain price," "deletion of specific users," "skipping notes," and "long time it takes to process the flow." Emotional data is also classified into emotion categories such as "dissatisfaction," "happiness," and "surprise." An AI module installed on the server analyzes the classified behavioral and emotional data. The AI ​​module uses machine learning algorithms to identify behavioral frequency, patterns, conditions for occurrence, and emotional changes.

[0776] Visualizing and presenting results

[0777] The server converts the analyzed results into graphs and charts. This visual data is displayed on a dashboard on the terminal (PC or mobile device) used by the administrator. Through the dashboard, administrators can grasp changes in user behavioral trends and emotions in real time.

[0778] System improvement proposals

[0779] Based on the analysis results, the server generates specific suggestions for system improvements, such as providing more detailed descriptions of specific products, simplifying transaction flows, and presenting customer messages tailored to user emotions. These suggestions are displayed on a dashboard on the device used by the administrator, who can then take appropriate action as needed.

[0780] Specific examples

[0781] Sentiment analysis and improvement of cancellation requests

[0782] When a user makes a cancellation request for a specific product, the terminal sends data such as the user ID, product ID, action type (cancellation request), reason for the action, date and time, and emotional information (e.g., dissatisfaction) to the server. The server collects this data and stores it in a database. An AI module then analyzes the data to identify time periods with high cancellation requests, product categories, specific user attributes, and related emotional patterns (e.g., high dissatisfaction). The server converts the analysis results into graphs and charts and displays them on the administrator's dashboard. For example, users can immediately see information such as "there are many dissatisfied users in the evening" or "cancellations and dissatisfaction frequently occur for products in a specific category."

[0783] Furthermore, based on the analysis results, the server generates specific suggestions to reduce transaction cancellations. For example, suggestions include "providing more detailed descriptions of the relevant products," "highlighting important points," and "sending follow-up messages to dissatisfied users." This allows for detailed monitoring of user behavior and sentiment, and by analyzing the data using AI, it is possible to efficiently identify the causes of transaction cancellations and bid retractions and take appropriate countermeasures. The introduction of this system will improve the reliability and user experience of online auction platforms.

[0784] Example of input prompt for generative AI model

[0785] "Please explain the specific methods you use to collect user behavioral and sentiment data."

[0786] "Please detail how you will analyze the collected data and generate improvement measures."

[0787] Please provide a sentiment analysis of users when they make cancellation requests and provide specific suggestions for improvement based on that.

[0788] Using such prompts allows the generative AI model to provide specific and useful information.

[0789] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0790] Step 1:

[0791] A user accesses an online auction platform and logs in. The user performs a transaction (e.g., bid, cancellation request, transaction cancellation, bid cancellation). During this process, the device uses the built-in camera and microphone to collect the user's facial expressions and voice, which are then analyzed by the emotion engine. The input is the user's behavior (transaction) and emotional data (facial expressions, voice), and the output is the analyzed emotional data (e.g., dissatisfaction, joy, surprise).

[0792] Step 2:

[0793] The device analyzes the collected behavioral and emotional data in real time and generates detailed data (user ID, product ID, behavior type, reason for behavior, emotional information, date and time). This detailed data is sent to the server. The input is the behavioral and emotional data obtained in step 1, and the output is the detailed data sent to the server.

[0794] Step 3:

[0795] The server temporarily stores the received detailed data in a buffer, then periodically batch processes it and stores it in a database. This allows all behavioral and emotional data to be accumulated centrally. The input is the detailed data sent from the device, and the output is the behavioral and emotional data stored in the database.

[0796] Step 4:

[0797] The server categorizes the behavioral data and emotional data stored in the database into categories for analysis. For example, behavioral data is categorized into categories such as "cancellation request from the winning bidder" and "cancellation of bids below a certain price," while emotional data is categorized into categories such as "dissatisfaction," "joy," and "surprise." The input is the behavioral data and emotional data stored in the database, and the output is the categorized data.

[0798] Step 5:

[0799] An AI module installed on the server analyzes the categorized behavioral and emotional data. The AI ​​module uses machine learning algorithms to identify behavioral frequency, patterns, conditions for occurrence, and emotional changes. The input is the categorized data, and the output is the analysis results, such as identified behavioral patterns and emotional changes.

[0800] Step 6:

[0801] The server converts the data into graphs and charts based on the analysis results. This visual data is displayed in real time on the dashboard of the administrator's device. The input is the result data analyzed by the AI ​​module, and the output is visual data in the form of graphs and charts.

[0802] Step 7:

[0803] The server generates specific suggestions for system improvement based on the analysis results. These suggestions include providing more detailed descriptions of specific products, simplifying the transaction flow, and presenting customer messages based on user sentiment. These suggestions are displayed on a dashboard on the terminal used by the administrator. The input is the analysis results, and the output is specific suggestions for system improvement.

[0804] By introducing this system, it is possible to closely monitor user behavior and emotions and analyze the data using AI, efficiently identifying the causes of transaction cancellations and bid revocations, and taking appropriate countermeasures.

[0805] (Application example 2)

[0806] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0807] Conventional systems mainly analyze data based solely on user behavior data, and do not take emotional data into account. This makes it difficult to quickly implement appropriate countermeasures to improve user experience and optimize systems. While it is particularly important in brick-and-mortar stores to grasp changes in customer emotions in real time and immediately reflect them in service improvements, achieving this has presented many challenges.

[0808] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0809] In this invention, the server includes means for monitoring user behavioral data and emotional data in real time, means for collecting the monitored behavioral data and emotional data and storing it in a database, means for classifying the collected behavioral data and emotional data into specific categories, means for analyzing the classified behavioral data and emotional data using an AI module and identifying the causes and conditions of changes in the behavior and emotions, means for visualizing and presenting the analysis results to an administrator, means for generating system improvement proposals based on the analysis results, means for collecting new user behavioral and emotional data via specific devices used, and means for transmitting the collected data to the server. This enables analysis that takes into account both user behavioral and emotional data, making it possible to take prompt and appropriate service improvement measures even in physical stores.

[0810] "User behavior data" is information recorded when a user performs a specific action, such as picking up a product, looking at a shelf, or canceling a transaction.

[0811] "Emotional data" is information that is analyzed from the user's facial expressions and voice and indicates their emotional state, such as joy, dissatisfaction, or surprise.

[0812] "Real-time monitoring" is a technology that allows data to be collected and analyzed instantly without delay.

[0813] A "database" is a system for systematically storing and managing collected data.

[0814] "Classifying into specific categories" refers to the process of dividing collected data into predetermined categories, such as "cancellation requests" or "expressions of joy."

[0815] The "AI module" is a program that uses artificial intelligence to analyze collected data and identify patterns and causes of behavior and emotions.

[0816] "Visualizing the analysis results" means displaying the analyzed data in a visual format such as a graph or chart so that the administrator can intuitively understand it.

[0817] "System improvement proposals" refers to generating specific proposals for better services and system optimization based on the analysis results.

[0818] "Specific Equipment Used" refers to hardware such as cameras, microphones, and smart glasses used to collect data.

[0819] A system embodying this invention is designed to monitor user behavioral and emotional data in real time. The system mainly utilizes the following hardware and software:

[0820] Hardware Configuration

[0821] 1. Smart glasses: These are devices that use a built-in camera and microphone to capture the user's facial expressions and voice. This device is used to collect data in real time.

[0822] 2. Server: It is a central control unit for storing collected data, analyzing them and visualizing the results.

[0823] 3. Administrator terminal: A PC or tablet device used to receive analysis results and system improvement proposals and perform management.

[0824] Software Configuration

[0825] 1. EmotionRecognition module: Software that analyzes emotions from the user's facial expressions.

[0826] 2. Voice Analysis Module: Software that analyzes emotions from the user's tone of voice.

[0827] 3. Data collection module: This is software for receiving data sent from the smart glasses and transferring it to the server.

[0828] 4. AI module: An AI program that analyzes collected behavioral and emotional data to identify behavioral patterns and emotional changes.

[0829] 5. Database: A system for systematically storing and managing data.

[0830] 6. Dashboard: An interface for visualizing analysis results in graphs and charts and presenting them to administrators.

[0831] Data processing and calculation

[0832] Data collection: The smart glasses capture the user's facial expressions with a camera and voice with a microphone. These data are analyzed in real time by the EmotionRecognition module and VoiceAnalysis module. The analysis results are collected as behavioral and emotional data.

[0833] Data Transfer: The collected data is sent to the server through the data collection module.

[0834] Data storage: The server stores the received data in a database, which manages both behavioral and emotional data.

[0835] Data analysis: The server's AI module uses the stored data to apply machine learning algorithms to analyze behavioral patterns and emotional changes. Specifically, it analyzes specific user actions (e.g., canceling a specific product) and the emotions they experience (e.g., dissatisfaction) to identify their causes.

[0836] Visualization of results: The analysis results are visualized and displayed on the administrator's dashboard as graphs and charts, allowing administrators to check user behavioral trends and changes in sentiment in real time.

[0837] System improvement proposals: Based on the analysis results, improvement proposals such as detailed product descriptions and simplified transaction flows are automatically generated and notified to the administrator's terminal.

[0838] Specific examples

[0839] For example, if the analysis results show a trend of "many users feeling dissatisfied in the evening," the system will notify the administrator with suggestions for improvement, such as "make the product descriptions more detailed" or "highlight warnings." The following prompt sentences could be considered as specific examples:

[0840] Prompt Sentence Examples

[0841] "Recent data shows that many customers are frustrated when reading the descriptions of new products. What can you do to improve this?"

[0842] As a result, the present invention makes it possible to comprehensively analyze user behavioral data and emotional data, and to propose prompt and appropriate service improvement measures even in physical stores.

[0843] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0844] Step 1:

[0845] The user puts on the smart glasses and walks around the store.

[0846] Input: User's facial expression and audio.

[0847] Output: Image data captured by the camera and audio data recorded by the microphone.

[0848] Specific operation: The camera in the smart glasses continuously captures the user's facial expressions, and the microphone records audio.

[0849] Step 2:

[0850] The smart glasses device uses the EmotionRecognition and VoiceAnalysis modules to analyze the collected data in real time.

[0851] Input: Image data captured by the camera and recorded audio data.

[0852] Output: Parsed emotion and speech data (e.g., happy, frustrated, surprised).

[0853] Specific operation: The EmotionRecognition module analyzes facial expressions, and the VoiceAnalysis module analyzes voice tones to generate emotion data.

[0854] Step 3:

[0855] The analyzed data is sent to a server via a data collection module.

[0856] Input: Emotion data and speech data.

[0857] Output: Data stored on the server.

[0858] Specific operation: The data collection module packages the analyzed data and sends it to the server over the network.

[0859] Step 4:

[0860] The server stores the received data in a database.

[0861] Input: Emotion data and voice data sent to the server.

[0862] Output: Structured data stored in a database.

[0863] Specific operation: The server analyzes the received data and stores it appropriately in the database.

[0864] Step 5:

[0865] The server's AI module performs analysis using behavioral and emotional data stored in a database.

[0866] Input: User behavioral and emotional data stored in a database.

[0867] Output: Analysis results (e.g., many users feel dissatisfied at certain times of the day).

[0868] How it works: The AI ​​module applies machine learning algorithms to identify behavioral patterns and causes of emotional changes.

[0869] Step 6:

[0870] The analysis results are visualized and displayed on the administrator's terminal (PC or tablet).

[0871] Input: Analysis result data.

[0872] Output: Analysis results displayed as graphs and charts.

[0873] Specific operation: The analysis results data is converted into graphs and charts and displayed on a dashboard.

[0874] Step 7:

[0875] Based on the analysis results, proposals for system improvement are generated and notified to the administrator's terminal.

[0876] Input: Analysis result data.

[0877] Output: Suggestions for improving the system (e.g., making product descriptions more detailed, highlighting cautions).

[0878] Specific operation: The AI ​​module generates improvement suggestions based on the analysis results and notifies the administrator terminal.

[0879] Step 8:

[0880] Administrators can review the suggested improvements through the dashboard and take appropriate action.

[0881] Input: System improvement suggestions.

[0882] Output: Corrective action taken (e.g. product description update, repositioning).

[0883] Specific Action: Administrator interacts with the dashboard and takes appropriate remedial action.

[0884] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0885] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0886] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0887] [Third embodiment]

[0888] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0889] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0890] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0891] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0892] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0893] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0894] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0895] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0896] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0897] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0898] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0899] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0900] User behavior monitoring

[0901] When users access the online auction platform and make transactions, their actions are recorded in real time. The terminal (the user's PC or mobile device) collects and transmits user behavior data to the server. This behavior data includes bids, cancellation requests, transaction cancellations, and bid retractions.

[0902] Collecting and classifying behavioral data

[0903] The server temporarily stores the received behavioral data in a buffer. It then performs batch processing, stores the collected behavioral data in a database, and classifies the data into specific categories. For example, this classification is performed in categories such as "cancellation request from the successful bidder," "cancellation of bids below a certain price," "deletion of specific users," "skipping notes," and "long time required to complete the flow."

[0904] AI analysis of data

[0905] An AI module installed on the server analyzes the stored behavioral data. The AI ​​uses machine learning algorithms to identify behavioral patterns, frequency, conditions, and causes. For example, it can identify product categories with a high number of cancellation requests or specific time periods where cancellations are concentrated.

[0906] Visualizing and presenting results

[0907] The server converts the analyzed results into graphs and charts, and this visual data is displayed on a dashboard on the terminal (PC or mobile device used by the administrator), allowing the administrator to grasp the trends and causes of user behavior at a glance.

[0908] System improvement proposals

[0909] Based on the analysis results, the server generates specific system improvement suggestions, such as providing more detailed descriptions of specific products or simplifying the transaction flow, to improve the user experience. These suggestions are displayed on the terminal (PC or mobile device used by the administrator), allowing the administrator to select and implement appropriate measures.

[0910] Specific examples

[0911] Analysis and improvement of cancellation requests

[0912] When a user makes a cancellation request for a specific product, the terminal sends data such as the "user ID," "product ID," "action type (cancellation request)," "reason for action," and "date and time" to the server.

[0913] The server collects this data and stores it in a database, after which an AI module analyzes it to identify patterns, such as times of day when cancellation requests are most common, product categories, or specific user attributes.

[0914] The analysis results are converted into graphs and charts by the server and displayed on the terminal (administrator's dashboard). For example, information such as "there are many cancellation requests in the evening hours" or "there are frequent cancellations for products in a particular category" can be immediately confirmed.

[0915] Furthermore, the server generates specific suggestions based on the analysis results to reduce transaction cancellations, such as "provide more detailed product descriptions" or "highlight cautions."

[0916] In this way, the present invention enables detailed monitoring of user behavior and analysis of data using AI to efficiently identify the causes of transaction cancellations and bid retractions and take appropriate countermeasures, thereby improving the reliability of the online auction platform and the user experience.

[0917] The processing flow will be explained below.

[0918] Step 1:

[0919] A user logs into the online auction platform and performs trading operations, including placing a bid, requesting a cancellation, canceling a transaction, and withdrawing a bid.

[0920] Step 2:

[0921] The terminal (PC or mobile device used by the user) records each operation performed by the user as an event and generates detailed data related to it (user ID, product ID, action type, reason for action, date and time, etc.).

[0922] Step 3:

[0923] The terminal transmits the generated detailed data to the server in real time.

[0924] Step 4:

[0925] The server temporarily stores the received behavioral data in a buffer, which acts as a temporary storage location for subsequent batch processing of the data into a database.

[0926] Step 5:

[0927] The server periodically runs a batch job to move the data from the buffer to a database, where all user activity data is stored.

[0928] Step 6:

[0929] The server categorizes the behavioral data stored in the database for analysis, including categories such as "cancellation request from the successful bidder," "cancellation of bids below a certain price," "deletion of specific users," "skipping notes," and "long time required to complete the flow."

[0930] Step 7:

[0931] An AI module installed on the server analyzes the categorized behavioral data and uses machine learning algorithms to identify behavioral frequencies, patterns, and conditions for occurrence.

[0932] Step 8:

[0933] The server collects the results of the AI ​​analysis and converts them into visual formats such as graphs and charts, which makes the data easier for administrators to understand.

[0934] Step 9:

[0935] The terminal (PC or mobile device used by the administrator) displays the visualized data generated by the server on a dashboard, allowing the administrator to grasp trends in cancellations and bid withdrawals in real time.

[0936] Step 10:

[0937] The server generates specific suggestions for system improvement based on the results of the AI ​​analysis, such as "providing more detailed product descriptions" or "strengthening transaction management during specific time periods."

[0938] Step 11:

[0939] The terminal (PC or mobile device used by the administrator) displays the suggestions from the server on a dashboard. The administrator can check these suggestions and implement them as necessary.

[0940] Example 1

[0941] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0942] Conventional online auction platforms face challenges in effectively monitoring, collecting, and analyzing user behavior data, making it difficult to quickly and appropriately address issues such as transaction cancellations and bid revocations. Furthermore, there is no well-established method for analyzing this data and presenting it to administrators in an intuitive and easy-to-understand format. As a result, it is difficult to improve the user experience and maintain the reliability of the platform.

[0943] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0944] In this invention, the server includes means for monitoring user behavior data in real time, means for collecting the monitored behavior data and storing it in a storage device, means for classifying the collected behavior data into specific categories, means for analyzing the behavior data using a machine learning algorithm and identifying the causes and conditions of the behavior, means for visualizing the analysis results in graphs and charts and presenting them on an administrator terminal, and means for generating system improvement proposals based on the analysis results and displaying them on the administrator terminal. This enables detailed monitoring of user behavior, effective analysis and visual presentation of data, and enables an improved user experience and maintenance of platform reliability.

[0945] "User" means any person or legal entity that transacts or operates using the online auction platform.

[0946] "Behavioral Data" means information about your transactions and interactions with the online auction platform.

[0947] "Real-time" refers to processing occurring immediately at the moment a user action occurs.

[0948] "Monitoring" refers to the process of continuously watching and recording user activity.

[0949] A "storage device" is hardware or software for storing data.

[0950] A "category" is a specific criterion or grouping for classifying behavioral data.

[0951] A "machine learning algorithm" is an artificial intelligence technology for data analysis that learns from data based on a set of rules and makes predictions and classifications.

[0952] "Analysis results" refer to the information and insights obtained after analyzing behavioral data using machine learning algorithms.

[0953] "Visualization" refers to the presentation of analytical results in a visual format such as a graph or chart.

[0954] An "administrator terminal" is an electronic device used by a system administrator, which has the function of displaying analysis results and suggestions.

[0955] "System improvement proposals" are specific measures or strategies for optimizing or improving the system based on the analysis results.

[0956] The present invention provides a system for real-time monitoring of user behavior data on an online auction platform, and effectively collects, classifies, analyzes, and visualizes the data. Specific embodiments of the system are described below.

[0957] User behavior monitoring

[0958] When a user accesses an online auction platform and conducts transactions such as browsing items, placing bids, or requesting cancellations, their actions are recorded in real time. The PC or mobile device (hereinafter referred to as the "terminal") used by the user collects this behavioral data and sends it to the server. This behavioral data includes the user ID, item ID, action type (e.g., bid, cancellation request), reason for the action, date and time, etc. For example, if a user places a bid on a specific item, the terminal collects data such as "User ID: 12345," "Item ID: 67890," "Action Type: Bid," "Bid Amount: 1,000 yen," and "Date and Time: October 1, 2023, 6:30 PM," and sends it to the server.

[0959] Collecting and classifying behavioral data

[0960] The server temporarily stores the behavioral data sent from the device in a buffer. Then, it performs batch processing and stores the collected behavioral data in an SQL database (e.g., MySQL or PostgreSQL). This storage process is performed every hour. For example, the insertion query "INSERT INTO user_actions (user_id, action_type, item_id, amount, timestamp) VALUES (...)" is used. The server then classifies this data into specific categories (e.g., "cancellation request," "low bid cancellation," etc.).

[0961] AI analysis of data

[0962] The behavioral data is analyzed using machine learning algorithms (using TensorFlow or PyTorch, for example) installed on the server. The AI ​​module identifies behavioral patterns, frequency, and occurrence conditions from the collected data. For example, it extracts information such as "cancellation requests are concentrated at certain times of the day" or "there are many cancellations for products in certain categories."

[0963] Visualizing and presenting results

[0964] The analysis results are converted into graphs and charts by the server. Data visualization tools (such as D3.js and Chart.js) are used to display histograms of the time-of-day distribution of cancellation requests. This visual data is displayed on the administrator's terminal, allowing the administrator to understand user behavior trends and causes at a glance.

[0965] System improvement proposals

[0966] Based on the analysis results, the server generates specific suggestions for system improvement, such as "provide more detailed descriptions of the relevant products" or "highlight warnings." These suggestions are displayed on the administrator's terminal, allowing the administrator to select and implement countermeasures.

[0967] Examples of concrete examples and prompts

[0968] When a user makes a cancellation request for a specific product, the terminal sends data such as the "user ID," "product ID," "action type (cancellation request)," "reason for the action," and "date and time" to the server. The server collects this data and stores it in a database, after which the AI ​​module analyzes it. The analysis results are converted into graphs and charts by the server and displayed on the administrator's terminal. For example, users can immediately see information such as "there are many cancellation requests in the evening hours" or "cancellations are frequent for products in a specific category." The server also generates specific suggestions for reducing transaction cancellations and displays them on the administrator's terminal.

[0969] Prompt Sentence Examples

[0970] Analyze user behavior based on the following behavioral data and generate specific improvement suggestions.

[0971] User ID: 12345

[0972] Product ID: 67890

[0973] Action Type: Cancellation Request

[0974] Reason for action: Insufficient explanation

[0975] Date and Time: 2023-10-01 18:30:00

[0976] In this way, the present invention enables detailed monitoring of user behavior and analysis of data using AI to efficiently identify the causes of transaction cancellations and bid retractions and take appropriate countermeasures, thereby improving the reliability of the online auction platform and the user experience.

[0977] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0978] Step 1:

[0979] Recording user behavior

[0980] When a user accesses an online auction platform and makes a transaction, the data is recorded on the terminal when the user takes an action such as placing a bid or requesting cancellation for a specific item.

[0981] Input: User action (e.g. bidding on an item, requesting cancellation)

[0982] Specific operation: When a user places a bid on "Product A," the terminal records data such as "User ID," "Product ID," "Action Type (bid)," "Bid Amount," and "Date and Time" in its internal memory.

[0983] Output: Collected user behavior data

[0984] Step 2:

[0985] Sending data

[0986] The device transmits the collected behavioral data to a server.

[0987] Input: User behavior data from the device

[0988] Specific operation: The terminal uses an HTTP request to POST the collected data to the server and send it to the endpoint "api / server / collect".

[0989] Output: User behavior data transferred to the server

[0990] Step 3:

[0991] Data buffering

[0992] The behavioral data received by the server is temporarily stored in a buffer.

[0993] Input: User behavior data transferred from the device

[0994] Specific operation: The server temporarily saves the received data in a buffer area in memory, and temporarily stores "User ID: 12345, Action Type: Bid, Product ID: 67890" in memory.

[0995] Output: Buffered user behavior data

[0996] Step 4:

[0997] Batch processing and database storage

[0998] The server performs batch processing at regular intervals (every hour) and stores the behavioral data in an SQL database.

[0999] Input: Buffered user behavior data

[1000] Specific operation: Every hour, the server executes a batch process, executes the query "INSERT INTO user_actions (user_id, action_type, item_id, amount, timestamp) VALUES (...)", and saves it in the database.

[1001] Output: User behavior data stored in a database

[1002] Step 5:

[1003] Behavioral Data Classification

[1004] The server classifies the data in the database into specific categories.

[1005] Input: User behavior data stored in a database

[1006] Specific operation: The server refers to the database, classifies the behavioral data into categories such as "cancellation request" and "low bid cancellation", and adds the information to the "category" column.

[1007] Output: Categorized user behavior data

[1008] Step 6:

[1009] AI analysis of data

[1010] The classified behavioral data is analyzed using machine learning algorithms within the server.

[1011] Input: Categorized user behavior data

[1012] How it works: Using TensorFlow, PyTorch, and other AI modules, the AI ​​module analyzes behavioral patterns, frequency, and occurrence conditions. For example, it performs clustering to identify "time periods and product categories with high rates of cancellation requests."

[1013] Output: Analysis results (e.g., time periods and product categories with the most cancellation requests)

[1014] Step 7:

[1015] Visual Data Generation

[1016] The server converts the analysis results into graphs and charts.

[1017] Input: Analysis results

[1018] What it does: Use data visualization tools such as D3.js and Chart.js to display a histogram of the distribution of cancellation requests by time period.

[1019] Output: Visual data in the form of graphs and charts

[1020] Step 8:

[1021] Display on the dashboard

[1022] Display visual data on the terminal (for administrator).

[1023] Input: Visual data

[1024] Specific operation: The server generates graphs and charts and renders them on an administrator's dashboard, displaying the results in real time when the administrator accesses it via a browser.

[1025] Output: Visual data displayed on the administrator's terminal

[1026] Step 9:

[1027] Generate improvement suggestions

[1028] The server generates suggestions for improving the system based on the analysis results.

[1029] Input: Analysis results

[1030] Specific operation: Based on the analysis results generated by the AI ​​module, improvement suggestions such as "provide more detailed descriptions of the relevant products" and "highlight cautions" are generated in text format.

[1031] Output: Generated improvement suggestions

[1032] Step 10:

[1033] View Suggestions

[1034] Improvement suggestions are displayed on the terminal (for administrators) so that the administrator can select and implement appropriate countermeasures.

[1035] Input: Generated improvement suggestions

[1036] What it does: The proposal will be displayed on the admin dashboard, allowing the admin to select an action, such as "Approve Proposal."

[1037] Output: Improvement proposals displayed on the administrator's terminal

[1038] (Application example 1)

[1039] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1040] There is a need for a system that can monitor user behavior in real time, analyze collected data, and detect suspicious behavior on online platforms early and suggest appropriate preventive measures. Ignoring suspicious behavior not only increases security risks, but ultimately leads to a poor user experience. Therefore, a system is needed that monitors user behavior, categorizes it, performs AI analysis, and visualizes the results to present them to administrators. In addition, it is also a challenge to establish a system that allows administrators to respond quickly and effectively by proposing specific preventive measures based on the analysis results.

[1041] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1042] In this invention, the server includes a means for detecting suspicious activity based on user behavior data and suggesting appropriate preventive measures, a means for monitoring user behavior data in real time, and a means for collecting the monitored behavior data and storing it in a database, thereby making it possible to detect suspicious activity in real time, identify its causes and conditions, and suggest specific preventive measures.

[1043] Definition of Terms

[1044] "User behavior data" refers to information about various actions and events that users perform within the system, including logins, clicks, data accesses, file changes, etc.

[1045] "Real-time monitoring means" means a system or method for collecting and recording user behavior data in real time, thereby enabling tracking of user behavior without delay.

[1046] A "database storage means" is a system or method that efficiently stores collected behavioral data and allows for quick retrieval upon need.

[1047] A "means for classifying into specific categories" is a system or method for dividing collected behavioral data into pre-defined categories according to its nature or type.

[1048] "AI analytics" is the process of using artificial intelligence to analyze behavioral data and identify patterns and anomalies. This includes machine learning algorithms and data mining techniques.

[1049] A "means for identifying causes and conditions" is a system or method that clarifies the causes and conditions that cause behavior based on analyzed data.

[1050] The "means for visualizing and presenting to the administrator" refers to a system or method for displaying the analysis results in a visual format such as a graph or chart, so that the administrator can easily understand them.

[1051] A "means for generating system improvement proposals" is a system or method that, based on the analysis results, presents a specific action plan aimed at improving user experience or security.

[1052] "Means for detecting suspicious activity and suggesting appropriate preventative measures" refers to a system or method that detects anomalies and risks from user behavior data and suggests measures to strengthen security or encourage careful behavior as countermeasures.

[1053] MODE FOR CARRYING OUT THE INVENTION

[1054] This invention is a system that monitors user behavior data in real time, classifies the collected data into specific categories, and analyzes it using AI. The analysis results are visualized and presented to administrators, and if suspicious behavior is detected, appropriate preventive measures are suggested.

[1055] System configuration

[1056] The server requires the following hardware and software:

[1057] Hardware: Servers with powerful CPUs, RAM, and storage, including resources for rapid data processing and analysis.

[1058] Software: This includes programming languages ​​such as Python, machine learning libraries (e.g., Scikit-learn, TensorFlow), database management systems (e.g., MySQL or PostgreSQL), and visualization libraries (e.g., Matplotlib, Plotly).

[1059] Monitoring Data

[1060] When a user uses the system, their behavioral data is monitored in real time. For example, when a user logs in and accesses a specific file, that information is sent from the user's device to the server in real time. This information includes the user's ID, the timestamp of the action, and the type of action.

[1061] Data collection and storage

[1062] The server temporarily stores the behavioral data received in real time in a buffer, then performs batch processing to store the behavioral data in a database designed to support rapid acquisition and analysis of the behavioral data.

[1063] Data classification and analysis

[1064] The collected behavioral data is categorized into specific categories (e.g., logins, data access, file modifications). The server's AI module analyzes this data and identifies suspicious behavioral patterns. For example, a large number of login attempts in a short period of time can be flagged as a possible account takeover.

[1065] Visualizing the results and presenting them to managers

[1066] The server converts the analysis results into graphs and charts, which are then displayed on the administrator's dashboard, allowing the administrator to intuitively understand user behavioral trends and suspicious behavior.

[1067] Preventive measures suggested

[1068] The server generates specific system improvement suggestions based on the analysis results, such as "If there are a large number of login attempts in a short period of time, suggest strengthening authentication for users." This allows administrators to take security measures quickly and effectively.

[1069] Specific examples

[1070] When a user attempts to access a particular file multiple times, the server immediately records the activity and stores it in a database. The AI ​​module analyzes this data and, if the access attempts exceed the normal range, identifies the activity as suspicious. The results are then presented to the administrator in a graph format, suggesting additional security measures (such as strengthening access restrictions).

[1071] Prompt Sentence Examples

[1072] The prompts for instructing a generative AI model on an analysis task are as follows:

[1073] Analyze real-time monitoring data of user behavior, identify suspicious behavior patterns, and generate specific security improvement suggestions, such as detecting abnormal increases in login counts or data access frequency.

[1074] This allows the invention to be effectively implemented.

[1075] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1076] Program processing steps

[1077] Step 1:

[1078] When a user uses the system, their behavioral data is monitored in real time.

[1079] Input: User behavior data (e.g., login attempts, file access)

[1080] How it works: The user's device collects this behavioral data and packages it with metadata such as a timestamp and user ID.

[1081] Output: Real-time user behavior data packets

[1082] Step 2:

[1083] The monitored behavioral data is sent from the user's device to a server.

[1084] Input: Real-time user behavior data packets

[1085] How it works: The user's device uses HTTP requests and WebSockets to send data to the server.

[1086] Output: Raw behavioral data received by the server

[1087] Step 3:

[1088] The server temporarily stores the received behavioral data in a buffer.

[1089] Input: Raw behavioral data received by the server

[1090] How it works: The server temporarily stores the data in memory or in a high-speed database.

[1091] Output: Buffered behavioral data

[1092] Step 4:

[1093] The server uses batch processing to store the behavioral data in the buffer into a database.

[1094] Input: Buffered behavioral data

[1095] What happens: The server executes a query to store the data in a database.

[1096] Output: Behavioral data stored in a database

[1097] Step 5:

[1098] The server classifies the stored behavioral data into specific categories.

[1099] Input: Behavioral data stored in a database

[1100] Actions: The server separates the behavioral data into categories (e.g., login, data access, file modification) based on predefined rules and conditions.

[1101] Output: Categorized behavioral data

[1102] Step 6:

[1103] The classified behavioral data is analyzed using an AI module.

[1104] Input: Categorized behavioral data

[1105] How it works: Generative AI models use machine learning algorithms to analyze behavioral data and identify suspicious behavioral patterns.

[1106] Output: Analysis results (e.g., abnormal login attempts detected)

[1107] Step 7:

[1108] The analysis results are visualized and presented to the administrator.

[1109] Input: Analysis results

[1110] How it works: The server converts the analysis results into graphs and charts and displays them as a dashboard on the administrator's device.

[1111] Output: Visualized analysis results displayed to the administrator

[1112] Step 8:

[1113] The server generates specific proposals for improving the system based on the analysis results.

[1114] Input: Analysis results

[1115] How it works: The server generates recommendations based on the analysis results, such as security enhancements or system configuration changes.

[1116] Output: System improvement proposals presented to administrators

[1117] Prompt Sentence Examples

[1118] Enter prompts like the following into the generative AI model:

[1119] Analyze real-time monitoring data of user behavior, identify suspicious behavior patterns, and generate specific security improvement suggestions, such as detecting abnormal increases in login counts or data access frequency.

[1120] This specific processing flow allows suspicious user behavior to be detected in real time, enabling a prompt response.

[1121] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1122] Monitoring user behavior and sentiment

[1123] A user accesses the online auction platform and performs trading operations. These operations include bidding, cancellation requests, transaction cancellations, and bid cancellations. The terminal (the user's PC or mobile device) is equipped with an emotion engine that recognizes the user's emotions. The emotion engine uses a camera and microphone to analyze emotions from the user's facial expressions and voice.

[1124] Collecting and classifying behavioral and emotional data

[1125] The device collects user behavioral and emotional data in real time, generates detailed data (user ID, product ID, behavior type, reason for behavior, emotional information, date and time, etc.) and sends it to the server.

[1126] The server temporarily stores the received behavioral and emotional data in a buffer. After that, it performs batch processing and stores the collected data in a database. The database stores all behavioral and emotional data.

[1127] Data analysis

[1128] The server categorizes the behavioral and emotional data stored in the database for analysis. These categorizations include categories such as "cancellation request from the successful bidder," "cancellation of bids below a certain price," "deletion of specific users," "skipping notes," and "slow flow completion." Emotional data is similarly categorized, including emotion categories such as "dissatisfaction," "joy," and "surprise."

[1129] An AI module installed on the server analyzes the classified behavioral and emotional data, using machine learning algorithms to identify behavioral frequency, patterns, conditions for occurrence, and emotional changes.

[1130] Visualizing and presenting results

[1131] The server converts the analyzed results into graphs and charts. This visual data is displayed on a dashboard on the terminal (PC or mobile device used by the administrator). Through the dashboard, administrators can grasp user behavioral trends and changes in emotions in real time.

[1132] System improvement proposals

[1133] Based on the analysis results, the server generates specific suggestions for system improvements, such as providing more detailed descriptions of specific products, simplifying the transaction flow, and presenting customer messages tailored to user sentiment. These suggestions are displayed on the terminal (PC or mobile device used by the administrator), allowing the administrator to select and implement appropriate measures.

[1134] Specific examples

[1135] Sentiment analysis and improvement of cancellation requests

[1136] When a user makes a cancellation request for a specific product, the terminal sends data such as the "user ID," "product ID," "action type (cancellation request)," "reason for action," "date and time," and "emotional information (e.g., dissatisfaction)" to the server.

[1137] The server collects this data and stores it in a database. An AI module then analyzes the data to identify times of day when cancellation requests are most common, product categories, specific user attributes, and associated emotional patterns (e.g., high dissatisfaction).

[1138] The analysis results are converted into graphs and charts by the server and displayed on the terminal (administrator's dashboard). For example, information such as "There are many dissatisfied users in the evening hours" or "There are frequent cancellations and dissatisfaction with products in a particular category" can be immediately confirmed.

[1139] Furthermore, the server generates specific suggestions based on the analysis results to reduce transaction cancellations, such as "providing more detailed product descriptions," "highlighting important points," and "sending follow-up messages to dissatisfied users."

[1140] In this way, the present invention enables detailed monitoring of user behavior and sentiment, and uses AI to analyze the data, thereby efficiently identifying the causes of transaction cancellations and bid retractions and taking appropriate countermeasures, thereby improving the reliability of the online auction platform and improving the user experience.

[1141] The processing flow will be explained below.

[1142] Step 1:

[1143] A user logs in to the online auction platform and performs trading operations, including bidding, requesting cancellation, canceling a transaction, and canceling a bid. The terminal (the user's PC or mobile device) then activates an emotion engine to recognize the user's emotions.

[1144] Step 2:

[1145] In parallel with the transaction, the terminal uses an emotion engine to collect real-time emotional data from the user's facial expressions and voice, including categories such as "happiness," "anger," "anxiety," and "surprise."

[1146] Step 3:

[1147] The terminal generates user behavior data (user ID, product ID, behavior type, reason for behavior, date and time, etc.) and emotion data, and transmits them to the server in real time.

[1148] Step 4:

[1149] The server temporarily stores the received behavioral and emotional data in a buffer, which serves as a temporary storage location for the data.

[1150] Step 5:

[1151] The server periodically runs a batch job and stores the buffered data in a database, where all user behavior and emotion data is stored.

[1152] Step 6:

[1153] The server categorizes the behavioral data and emotional data stored in the database for analysis. Behavioral data is categorized into categories such as "cancellation request from the successful bidder," "cancellation of bids below a certain price," "deletion of specific users," "skipping notes," and "long time required to complete the flow." Emotional data is categorized into emotion categories such as "dissatisfaction," "joy," and "surprise."

[1154] Step 7:

[1155] An AI module installed on the server analyzes the classified behavioral and emotional data and uses machine learning algorithms to identify behavioral frequency, patterns, conditions for occurrence, and emotional changes.

[1156] Step 8:

[1157] The server collects the results of the AI ​​analysis and converts them into visual formats such as graphs and charts. This visualized data is displayed on a dashboard on the terminal (PC or mobile device used by the administrator).

[1158] Step 9:

[1159] The terminal (PC or mobile device used by the administrator) displays the visualized data generated by the server on a dashboard, allowing the administrator to grasp trends in cancellations and bid cancellations, as well as changes in user sentiment, in real time.

[1160] Step 10:

[1161] Based on the results of the AI ​​analysis, the server generates specific suggestions for improving the system, such as "providing more detailed product descriptions," "strengthening transaction management during specific time periods," and "sending customer messages to dissatisfied users."

[1162] Step 11:

[1163] The terminal (PC or mobile device used by the administrator) displays the suggestions from the server on a dashboard, and the administrator can review these suggestions and implement them as necessary.

[1164] Specific examples

[1165] Sentiment analysis and improvement of cancellation requests

[1166] Step 1:

[1167] A user requests cancellation of a particular product, for example, if the product is not what they expected.

[1168] Step 2:

[1169] The device uses a camera to capture the user's facial expression when making a cancellation request and records the audio with a microphone.

[1170] Step 3:

[1171] The device uses an emotion engine to recognize the emotion "dissatisfaction" from the user's facial expressions and voice.

[1172] Step 4:

[1173] The terminal generates detailed data such as "user ID," "product ID," "action type (cancellation request)," "reason for action," "date and time," and "emotional information (dissatisfaction)," and sends it to the server.

[1174] Step 5:

[1175] The server stores this data in a buffer and then periodically moves it to a database.

[1176] Step 6:

[1177] The server categorizes the data stored in the database and analyzes it using an AI module, identifying time periods and product categories with high cancellation requests, as well as related emotional patterns (e.g., high levels of dissatisfaction).

[1178] Step 7:

[1179] The analysis results are converted into graphs and charts by the server and displayed on the terminal (administrator's dashboard). For example, it is possible to see information such as "Many users are dissatisfied in the evening hours" or "Cancellations and dissatisfaction occur frequently for products in a particular category."

[1180] Step 8:

[1181] Based on the analysis results, the server generates specific suggestions to reduce transaction cancellations, such as "make product descriptions more detailed," "highlight important points," or "send follow-up messages to dissatisfied users."

[1182] Step 9:

[1183] The terminal (PC or mobile device used by the administrator) displays the suggestions from the server on a dashboard, and the administrator can review these suggestions and implement them as necessary.

[1184] This series of processes enables detailed monitoring of user behavior and sentiment, and by analyzing the data using AI, it is possible to efficiently identify the causes of transaction cancellations and bid retractions and take appropriate countermeasures, thereby improving the reliability and user experience of the online auction platform.

[1185] Example 2

[1186] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1187] Traditional online platforms typically collect and analyze only user behavioral data. However, this makes it difficult to formulate improvement measures that take user emotions into account, resulting in insufficient improvements to the user experience. Furthermore, analyzing behavioral data alone makes it difficult to accurately grasp the user's emotions and psychological state behind specific actions, making it difficult to find appropriate countermeasures.

[1188] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1189] In this invention, the server includes means for monitoring user behavioral data and emotional data in real time, means for collecting the monitored behavioral data and emotional data and storing it in a database, means for classifying the collected behavioral data and emotional data into specific categories, means for analyzing the classified behavioral data and emotional data using AI and identifying the causes and conditions of the behavior, means for visualizing and presenting the analysis results to an administrator, and means for generating system improvement proposals based on the analysis results. This enables advanced data analysis that takes into account not only user behavior but also emotions, making it possible to formulate more appropriate system improvement measures.

[1190] "User" means any individual or legal entity using the online auction platform.

[1191] "Behavioral Data" refers to information about the specific operations and actions that users take on the online auction platform.

[1192] "Emotion data" is information obtained by analyzing the emotions shown by the user when performing operations.

[1193] "Real-time monitoring means" is a general term for hardware and software for instantly acquiring and monitoring user behavioral and emotional data.

[1194] The "collection means" is a system for collecting monitored data and transmitting it to a database.

[1195] "Means for saving in a database" refers to a mechanism for storing and saving collected data in a database in a certain format.

[1196] "Means for categorizing data into specific categories" refers to methods or techniques for dividing collected data into predetermined categories.

[1197] "Means of analysis" refers to the technology that uses AI to analyze classified data and extract useful information and patterns from that data.

[1198] "Cause and condition identification" is a method for identifying the reasons and circumstances behind user behavior based on data analysis.

[1199] "Means for visualization and presentation to administrators" refers to technology for converting analysis results into visual formats such as graphs and charts, and displaying them in a way that administrators can easily understand.

[1200] "Means for generating system improvement proposals" is a process for creating specific improvement measures to improve user experience based on the analysis results.

[1201] The system of the present invention monitors user behavioral data and emotional data in real time, analyzes this data, and generates system improvement proposals. Specific embodiments for implementing this system will be described below.

[1202] Monitoring user behavior and sentiment

[1203] A user accesses the online auction platform and performs operations such as placing a bid, requesting a cancellation, canceling a transaction, or canceling a bid. At this time, the user's device (PC or mobile device) analyzes the user's facial expressions and voice using an emotion engine. The emotion engine uses hardware such as a camera and microphone to obtain the user's emotion data.

[1204] Collecting and classifying behavioral and emotional data

[1205] The device collects user behavioral and emotional data in real time. This includes detailed data such as user ID, product ID, behavior type, reason for behavior, emotional information, and date and time. This detailed data is sent to the server, which temporarily stores the received data in a buffer. The server then performs batch processing and stores the collected data in a database. All behavioral and emotional data is stored in the database.

[1206] Data analysis

[1207] The server classifies the behavioral and emotional data stored in the database for analysis. This classification includes categories such as "cancellation request from the winning bidder," "cancellation of bids below a certain price," "deletion of specific users," "skipping notes," and "long time it takes to process the flow." Emotional data is also classified into emotion categories such as "dissatisfaction," "happiness," and "surprise." An AI module installed on the server analyzes the classified behavioral and emotional data. The AI ​​module uses machine learning algorithms to identify behavioral frequency, patterns, conditions for occurrence, and emotional changes.

[1208] Visualizing and presenting results

[1209] The server converts the analyzed results into graphs and charts. This visual data is displayed on a dashboard on the terminal (PC or mobile device) used by the administrator. Through the dashboard, administrators can grasp changes in user behavioral trends and emotions in real time.

[1210] System improvement proposals

[1211] Based on the analysis results, the server generates specific suggestions for system improvements, such as providing more detailed descriptions of specific products, simplifying transaction flows, and presenting customer messages tailored to user emotions. These suggestions are displayed on a dashboard on the device used by the administrator, who can then take appropriate action as needed.

[1212] Specific examples

[1213] Sentiment analysis and improvement of cancellation requests

[1214] When a user makes a cancellation request for a specific product, the terminal sends data such as the user ID, product ID, action type (cancellation request), reason for the action, date and time, and emotional information (e.g., dissatisfaction) to the server. The server collects this data and stores it in a database. An AI module then analyzes the data to identify time periods with high cancellation requests, product categories, specific user attributes, and related emotional patterns (e.g., high dissatisfaction). The server converts the analysis results into graphs and charts and displays them on the administrator's dashboard. For example, users can immediately see information such as "there are many dissatisfied users in the evening" or "cancellations and dissatisfaction frequently occur for products in a specific category."

[1215] Furthermore, based on the analysis results, the server generates specific suggestions to reduce transaction cancellations. For example, suggestions include "providing more detailed descriptions of the relevant products," "highlighting important points," and "sending follow-up messages to dissatisfied users." This allows for detailed monitoring of user behavior and sentiment, and by analyzing the data using AI, it is possible to efficiently identify the causes of transaction cancellations and bid retractions and take appropriate countermeasures. The introduction of this system will improve the reliability and user experience of online auction platforms.

[1216] Example of input prompt for generative AI model

[1217] "Please explain the specific methods you use to collect user behavioral and sentiment data."

[1218] "Please detail how you will analyze the collected data and generate improvement measures."

[1219] Please provide a sentiment analysis of users when they make cancellation requests and provide specific suggestions for improvement based on that.

[1220] Using such prompts allows the generative AI model to provide specific and useful information.

[1221] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1222] Step 1:

[1223] A user accesses an online auction platform and logs in. The user performs a transaction (e.g., bid, cancellation request, transaction cancellation, bid cancellation). During this process, the device uses the built-in camera and microphone to collect the user's facial expressions and voice, which are then analyzed by the emotion engine. The input is the user's behavior (transaction) and emotional data (facial expressions, voice), and the output is the analyzed emotional data (e.g., dissatisfaction, joy, surprise).

[1224] Step 2:

[1225] The device analyzes the collected behavioral and emotional data in real time and generates detailed data (user ID, product ID, behavior type, reason for behavior, emotional information, date and time). This detailed data is sent to the server. The input is the behavioral and emotional data obtained in step 1, and the output is the detailed data sent to the server.

[1226] Step 3:

[1227] The server temporarily stores the received detailed data in a buffer, then periodically batch processes it and stores it in a database. This allows all behavioral and emotional data to be accumulated centrally. The input is the detailed data sent from the device, and the output is the behavioral and emotional data stored in the database.

[1228] Step 4:

[1229] The server categorizes the behavioral data and emotional data stored in the database into categories for analysis. For example, behavioral data is categorized into categories such as "cancellation request from the winning bidder" and "cancellation of bids below a certain price," while emotional data is categorized into categories such as "dissatisfaction," "joy," and "surprise." The input is the behavioral data and emotional data stored in the database, and the output is the categorized data.

[1230] Step 5:

[1231] An AI module installed on the server analyzes the categorized behavioral and emotional data. The AI ​​module uses machine learning algorithms to identify behavioral frequency, patterns, conditions for occurrence, and emotional changes. The input is the categorized data, and the output is the analysis results, such as identified behavioral patterns and emotional changes.

[1232] Step 6:

[1233] The server converts the data into graphs and charts based on the analysis results. This visual data is displayed in real time on the dashboard of the administrator's device. The input is the result data analyzed by the AI ​​module, and the output is visual data in the form of graphs and charts.

[1234] Step 7:

[1235] The server generates specific suggestions for system improvement based on the analysis results. These suggestions include providing more detailed descriptions of specific products, simplifying the transaction flow, and presenting customer messages based on user sentiment. These suggestions are displayed on a dashboard on the terminal used by the administrator. The input is the analysis results, and the output is specific suggestions for system improvement.

[1236] By introducing this system, it is possible to closely monitor user behavior and emotions and analyze the data using AI, efficiently identifying the causes of transaction cancellations and bid revocations, and taking appropriate countermeasures.

[1237] (Application example 2)

[1238] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1239] Conventional systems mainly analyze data based solely on user behavior data, and do not take emotional data into account. This makes it difficult to quickly implement appropriate countermeasures to improve user experience and optimize systems. While it is particularly important in brick-and-mortar stores to grasp changes in customer emotions in real time and immediately reflect them in service improvements, achieving this has presented many challenges.

[1240] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1241] In this invention, the server includes means for monitoring user behavioral data and emotional data in real time, means for collecting the monitored behavioral data and emotional data and storing it in a database, means for classifying the collected behavioral data and emotional data into specific categories, means for analyzing the classified behavioral data and emotional data using an AI module and identifying the causes and conditions of changes in the behavior and emotions, means for visualizing and presenting the analysis results to an administrator, means for generating system improvement proposals based on the analysis results, means for collecting new user behavioral and emotional data via specific devices used, and means for transmitting the collected data to the server. This enables analysis that takes into account both user behavioral and emotional data, making it possible to take prompt and appropriate service improvement measures even in physical stores.

[1242] "User behavior data" is information recorded when a user performs a specific action, such as picking up a product, looking at a shelf, or canceling a transaction.

[1243] "Emotional data" is information that is analyzed from the user's facial expressions and voice and indicates their emotional state, such as joy, dissatisfaction, or surprise.

[1244] "Real-time monitoring" is a technology that allows data to be collected and analyzed instantly without delay.

[1245] A "database" is a system for systematically storing and managing collected data.

[1246] "Classifying into specific categories" refers to the process of dividing collected data into predetermined categories, such as "cancellation requests" or "expressions of joy."

[1247] The "AI module" is a program that uses artificial intelligence to analyze collected data and identify patterns and causes of behavior and emotions.

[1248] "Visualizing the analysis results" means displaying the analyzed data in a visual format such as a graph or chart so that the administrator can intuitively understand it.

[1249] "System improvement proposals" refers to generating specific proposals for better services and system optimization based on the analysis results.

[1250] "Specific Equipment Used" refers to hardware such as cameras, microphones, and smart glasses used to collect data.

[1251] A system embodying this invention is designed to monitor user behavioral and emotional data in real time. The system mainly utilizes the following hardware and software:

[1252] Hardware Configuration

[1253] 1. Smart glasses: These are devices that use a built-in camera and microphone to capture the user's facial expressions and voice. This device is used to collect data in real time.

[1254] 2. Server: It is a central control unit for storing collected data, analyzing them and visualizing the results.

[1255] 3. Administrator terminal: A PC or tablet device used to receive analysis results and system improvement proposals and perform management.

[1256] Software Configuration

[1257] 1. EmotionRecognition module: Software that analyzes emotions from the user's facial expressions.

[1258] 2. Voice Analysis Module: Software that analyzes emotions from the user's tone of voice.

[1259] 3. Data collection module: This is software for receiving data sent from the smart glasses and transferring it to the server.

[1260] 4. AI module: An AI program that analyzes collected behavioral and emotional data to identify behavioral patterns and emotional changes.

[1261] 5. Database: A system for systematically storing and managing data.

[1262] 6. Dashboard: An interface for visualizing analysis results in graphs and charts and presenting them to administrators.

[1263] Data processing and calculation

[1264] Data collection: The smart glasses capture the user's facial expressions with a camera and voice with a microphone. These data are analyzed in real time by the EmotionRecognition module and VoiceAnalysis module. The analysis results are collected as behavioral and emotional data.

[1265] Data Transfer: The collected data is sent to the server through the data collection module.

[1266] Data storage: The server stores the received data in a database, which manages both behavioral and emotional data.

[1267] Data analysis: The server's AI module uses the stored data to apply machine learning algorithms to analyze behavioral patterns and emotional changes. Specifically, it analyzes specific user actions (e.g., canceling a specific product) and the emotions they experience (e.g., dissatisfaction) to identify their causes.

[1268] Visualization of results: The analysis results are visualized and displayed on the administrator's dashboard as graphs and charts, allowing administrators to check user behavioral trends and changes in sentiment in real time.

[1269] System improvement proposals: Based on the analysis results, improvement proposals such as detailed product descriptions and simplified transaction flows are automatically generated and notified to the administrator's terminal.

[1270] Specific examples

[1271] For example, if the analysis results show a trend of "many users feeling dissatisfied in the evening," the system will notify the administrator with suggestions for improvement, such as "make the product descriptions more detailed" or "highlight warnings." The following prompt sentences could be considered as specific examples:

[1272] Prompt Sentence Examples

[1273] "Recent data shows that many customers are frustrated when reading the descriptions of new products. What can you do to improve this?"

[1274] As a result, the present invention makes it possible to comprehensively analyze user behavioral data and emotional data, and to propose prompt and appropriate service improvement measures even in physical stores.

[1275] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1276] Step 1:

[1277] The user puts on the smart glasses and walks around the store.

[1278] Input: User's facial expression and audio.

[1279] Output: Image data captured by the camera and audio data recorded by the microphone.

[1280] Specific operation: The camera in the smart glasses continuously captures the user's facial expressions, and the microphone records audio.

[1281] Step 2:

[1282] The smart glasses device uses the EmotionRecognition and VoiceAnalysis modules to analyze the collected data in real time.

[1283] Input: Image data captured by the camera and recorded audio data.

[1284] Output: Parsed emotion and speech data (e.g., happy, frustrated, surprised).

[1285] Specific operation: The EmotionRecognition module analyzes facial expressions, and the VoiceAnalysis module analyzes voice tones to generate emotion data.

[1286] Step 3:

[1287] The analyzed data is sent to a server via a data collection module.

[1288] Input: Emotion data and speech data.

[1289] Output: Data stored on the server.

[1290] Specific operation: The data collection module packages the analyzed data and sends it to the server over the network.

[1291] Step 4:

[1292] The server stores the received data in a database.

[1293] Input: Emotion data and voice data sent to the server.

[1294] Output: Structured data stored in a database.

[1295] Specific operation: The server analyzes the received data and stores it appropriately in the database.

[1296] Step 5:

[1297] The server's AI module performs analysis using behavioral and emotional data stored in a database.

[1298] Input: User behavioral and emotional data stored in a database.

[1299] Output: Analysis results (e.g., many users feel dissatisfied at certain times of the day).

[1300] How it works: The AI ​​module applies machine learning algorithms to identify behavioral patterns and causes of emotional changes.

[1301] Step 6:

[1302] The analysis results are visualized and displayed on the administrator's terminal (PC or tablet).

[1303] Input: Analysis result data.

[1304] Output: Analysis results displayed as graphs and charts.

[1305] Specific operation: The analysis results data is converted into graphs and charts and displayed on a dashboard.

[1306] Step 7:

[1307] Based on the analysis results, proposals for system improvement are generated and notified to the administrator's terminal.

[1308] Input: Analysis result data.

[1309] Output: Suggestions for improving the system (e.g., making product descriptions more detailed, highlighting cautions).

[1310] Specific operation: The AI ​​module generates improvement suggestions based on the analysis results and notifies the administrator terminal.

[1311] Step 8:

[1312] Administrators can review the suggested improvements through the dashboard and take appropriate action.

[1313] Input: System improvement suggestions.

[1314] Output: Corrective action taken (e.g. product description update, repositioning).

[1315] Specific Action: Administrator interacts with the dashboard and takes appropriate remedial action.

[1316] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1317] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1318] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1319] [Fourth embodiment]

[1320] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1321] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1322] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1323] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1324] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1325] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1326] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1327] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1328] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1329] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1330] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1331] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1332] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1333] User behavior monitoring

[1334] When users access the online auction platform and make transactions, their actions are recorded in real time. The terminal (the user's PC or mobile device) collects and transmits user behavior data to the server. This behavior data includes bids, cancellation requests, transaction cancellations, and bid retractions.

[1335] Collecting and classifying behavioral data

[1336] The server temporarily stores the received behavioral data in a buffer. It then performs batch processing, stores the collected behavioral data in a database, and classifies the data into specific categories. For example, this classification is performed in categories such as "cancellation request from the successful bidder," "cancellation of bids below a certain price," "deletion of specific users," "skipping notes," and "long time required to complete the flow."

[1337] AI analysis of data

[1338] An AI module installed on the server analyzes the stored behavioral data. The AI ​​uses machine learning algorithms to identify behavioral patterns, frequency, conditions, and causes. For example, it can identify product categories with a high number of cancellation requests or specific time periods where cancellations are concentrated.

[1339] Visualizing and presenting results

[1340] The server converts the analyzed results into graphs and charts, and this visual data is displayed on a dashboard on the terminal (PC or mobile device used by the administrator), allowing the administrator to grasp the trends and causes of user behavior at a glance.

[1341] System improvement proposals

[1342] Based on the analysis results, the server generates specific system improvement suggestions, such as providing more detailed descriptions of specific products or simplifying the transaction flow, to improve the user experience. These suggestions are displayed on the terminal (PC or mobile device used by the administrator), allowing the administrator to select and implement appropriate measures.

[1343] Specific examples

[1344] Analysis and improvement of cancellation requests

[1345] When a user makes a cancellation request for a specific product, the terminal sends data such as the "user ID," "product ID," "action type (cancellation request)," "reason for action," and "date and time" to the server.

[1346] The server collects this data and stores it in a database, after which an AI module analyzes it to identify patterns, such as times of day when cancellation requests are most common, product categories, or specific user attributes.

[1347] The analysis results are converted into graphs and charts by the server and displayed on the terminal (administrator's dashboard). For example, information such as "there are many cancellation requests in the evening hours" or "there are frequent cancellations for products in a particular category" can be immediately confirmed.

[1348] Furthermore, the server generates specific suggestions based on the analysis results to reduce transaction cancellations, such as "provide more detailed product descriptions" or "highlight cautions."

[1349] In this way, the present invention enables detailed monitoring of user behavior and analysis of data using AI to efficiently identify the causes of transaction cancellations and bid retractions and take appropriate countermeasures, thereby improving the reliability of the online auction platform and the user experience.

[1350] The processing flow will be explained below.

[1351] Step 1:

[1352] A user logs into the online auction platform and performs trading operations, including placing a bid, requesting a cancellation, canceling a transaction, and withdrawing a bid.

[1353] Step 2:

[1354] The terminal (PC or mobile device used by the user) records each operation performed by the user as an event and generates detailed data related to it (user ID, product ID, action type, reason for action, date and time, etc.).

[1355] Step 3:

[1356] The terminal transmits the generated detailed data to the server in real time.

[1357] Step 4:

[1358] The server temporarily stores the received behavioral data in a buffer, which acts as a temporary storage location for subsequent batch processing of the data into a database.

[1359] Step 5:

[1360] The server periodically runs a batch job to move the data from the buffer to a database, where all user activity data is stored.

[1361] Step 6:

[1362] The server categorizes the behavioral data stored in the database for analysis, including categories such as "cancellation request from the successful bidder," "cancellation of bids below a certain price," "deletion of specific users," "skipping notes," and "long time required to complete the flow."

[1363] Step 7:

[1364] An AI module installed on the server analyzes the categorized behavioral data and uses machine learning algorithms to identify behavioral frequencies, patterns, and conditions for occurrence.

[1365] Step 8:

[1366] The server collects the results of the AI ​​analysis and converts them into visual formats such as graphs and charts, which makes the data easier for administrators to understand.

[1367] Step 9:

[1368] The terminal (PC or mobile device used by the administrator) displays the visualized data generated by the server on a dashboard, allowing the administrator to grasp trends in cancellations and bid withdrawals in real time.

[1369] Step 10:

[1370] The server generates specific suggestions for system improvement based on the results of the AI ​​analysis, such as "providing more detailed product descriptions" or "strengthening transaction management during specific time periods."

[1371] Step 11:

[1372] The terminal (PC or mobile device used by the administrator) displays the suggestions from the server on a dashboard. The administrator can check these suggestions and implement them as necessary.

[1373] Example 1

[1374] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1375] Conventional online auction platforms face challenges in effectively monitoring, collecting, and analyzing user behavior data, making it difficult to quickly and appropriately address issues such as transaction cancellations and bid revocations. Furthermore, there is no well-established method for analyzing this data and presenting it to administrators in an intuitive and easy-to-understand format. As a result, it is difficult to improve the user experience and maintain the reliability of the platform.

[1376] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1377] In this invention, the server includes means for monitoring user behavior data in real time, means for collecting the monitored behavior data and storing it in a storage device, means for classifying the collected behavior data into specific categories, means for analyzing the behavior data using a machine learning algorithm and identifying the causes and conditions of the behavior, means for visualizing the analysis results in graphs and charts and presenting them on an administrator terminal, and means for generating system improvement proposals based on the analysis results and displaying them on the administrator terminal. This enables detailed monitoring of user behavior, effective analysis and visual presentation of data, and enables an improved user experience and maintenance of platform reliability.

[1378] "User" means any person or legal entity that transacts or operates using the online auction platform.

[1379] "Behavioral Data" means information about your transactions and interactions with the online auction platform.

[1380] "Real-time" refers to processing occurring immediately at the moment a user action occurs.

[1381] "Monitoring" refers to the process of continuously watching and recording user activity.

[1382] A "storage device" is hardware or software for storing data.

[1383] A "category" is a specific criterion or grouping for classifying behavioral data.

[1384] A "machine learning algorithm" is an artificial intelligence technology for data analysis that learns from data based on a set of rules and makes predictions and classifications.

[1385] "Analysis results" refer to the information and insights obtained after analyzing behavioral data using machine learning algorithms.

[1386] "Visualization" refers to the presentation of analytical results in a visual format such as a graph or chart.

[1387] An "administrator terminal" is an electronic device used by a system administrator, which has the function of displaying analysis results and suggestions.

[1388] "System improvement proposals" are specific measures or strategies for optimizing or improving the system based on the analysis results.

[1389] The present invention provides a system for real-time monitoring of user behavior data on an online auction platform, and effectively collects, classifies, analyzes, and visualizes the data. Specific embodiments of the system are described below.

[1390] User behavior monitoring

[1391] When a user accesses an online auction platform and conducts transactions such as browsing items, placing bids, or requesting cancellations, their actions are recorded in real time. The PC or mobile device (hereinafter referred to as the "terminal") used by the user collects this behavioral data and sends it to the server. This behavioral data includes the user ID, item ID, action type (e.g., bid, cancellation request), reason for the action, date and time, etc. For example, if a user places a bid on a specific item, the terminal collects data such as "User ID: 12345," "Item ID: 67890," "Action Type: Bid," "Bid Amount: 1,000 yen," and "Date and Time: October 1, 2023, 6:30 PM," and sends it to the server.

[1392] Collecting and classifying behavioral data

[1393] The server temporarily stores the behavioral data sent from the device in a buffer. Then, it performs batch processing and stores the collected behavioral data in an SQL database (e.g., MySQL or PostgreSQL). This storage process is performed every hour. For example, the insertion query "INSERT INTO user_actions (user_id, action_type, item_id, amount, timestamp) VALUES (...)" is used. The server then classifies this data into specific categories (e.g., "cancellation request," "low bid cancellation," etc.).

[1394] AI analysis of data

[1395] The behavioral data is analyzed using machine learning algorithms (using TensorFlow or PyTorch, for example) installed on the server. The AI ​​module identifies behavioral patterns, frequency, and occurrence conditions from the collected data. For example, it extracts information such as "cancellation requests are concentrated at certain times of the day" or "there are many cancellations for products in certain categories."

[1396] Visualizing and presenting results

[1397] The analysis results are converted into graphs and charts by the server. Data visualization tools (such as D3.js and Chart.js) are used to display histograms of the time-of-day distribution of cancellation requests. This visual data is displayed on the administrator's terminal, allowing the administrator to understand user behavior trends and causes at a glance.

[1398] System improvement proposals

[1399] Based on the analysis results, the server generates specific suggestions for system improvement, such as "provide more detailed descriptions of the relevant products" or "highlight warnings." These suggestions are displayed on the administrator's terminal, allowing the administrator to select and implement countermeasures.

[1400] Examples of concrete examples and prompts

[1401] When a user makes a cancellation request for a specific product, the terminal sends data such as the "user ID," "product ID," "action type (cancellation request)," "reason for the action," and "date and time" to the server. The server collects this data and stores it in a database, after which the AI ​​module analyzes it. The analysis results are converted into graphs and charts by the server and displayed on the administrator's terminal. For example, users can immediately see information such as "there are many cancellation requests in the evening hours" or "cancellations are frequent for products in a specific category." The server also generates specific suggestions for reducing transaction cancellations and displays them on the administrator's terminal.

[1402] Prompt Sentence Examples

[1403] Analyze user behavior based on the following behavioral data and generate specific improvement suggestions.

[1404] User ID: 12345

[1405] Product ID: 67890

[1406] Action Type: Cancellation Request

[1407] Reason for action: Insufficient explanation

[1408] Date and Time: 2023-10-01 18:30:00

[1409] In this way, the present invention enables detailed monitoring of user behavior and analysis of data using AI to efficiently identify the causes of transaction cancellations and bid retractions and take appropriate countermeasures, thereby improving the reliability of the online auction platform and the user experience.

[1410] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1411] Step 1:

[1412] Recording user behavior

[1413] When a user accesses an online auction platform and makes a transaction, the data is recorded on the terminal when the user takes an action such as placing a bid or requesting cancellation for a specific item.

[1414] Input: User action (e.g. bidding on an item, requesting cancellation)

[1415] Specific operation: When a user places a bid on "Product A," the terminal records data such as "User ID," "Product ID," "Action Type (bid)," "Bid Amount," and "Date and Time" in its internal memory.

[1416] Output: Collected user behavior data

[1417] Step 2:

[1418] Sending data

[1419] The device transmits the collected behavioral data to a server.

[1420] Input: User behavior data from the device

[1421] Specific operation: The terminal uses an HTTP request to POST the collected data to the server and send it to the endpoint "api / server / collect".

[1422] Output: User behavior data transferred to the server

[1423] Step 3:

[1424] Data buffering

[1425] The behavioral data received by the server is temporarily stored in a buffer.

[1426] Input: User behavior data transferred from the device

[1427] Specific operation: The server temporarily saves the received data in a buffer area in memory, and temporarily stores "User ID: 12345, Action Type: Bid, Product ID: 67890" in memory.

[1428] Output: Buffered user behavior data

[1429] Step 4:

[1430] Batch processing and database storage

[1431] The server performs batch processing at regular intervals (every hour) and stores the behavioral data in an SQL database.

[1432] Input: Buffered user behavior data

[1433] Specific operation: Every hour, the server executes a batch process, executes the query "INSERT INTO user_actions (user_id, action_type, item_id, amount, timestamp) VALUES (...)", and saves it in the database.

[1434] Output: User behavior data stored in a database

[1435] Step 5:

[1436] Behavioral Data Classification

[1437] The server classifies the data in the database into specific categories.

[1438] Input: User behavior data stored in a database

[1439] Specific operation: The server refers to the database, classifies the behavioral data into categories such as "cancellation request" and "low bid cancellation", and adds the information to the "category" column.

[1440] Output: Categorized user behavior data

[1441] Step 6:

[1442] AI analysis of data

[1443] The classified behavioral data is analyzed using machine learning algorithms within the server.

[1444] Input: Categorized user behavior data

[1445] How it works: Using TensorFlow, PyTorch, and other AI modules, the AI ​​module analyzes behavioral patterns, frequency, and occurrence conditions. For example, it performs clustering to identify "time periods and product categories with high rates of cancellation requests."

[1446] Output: Analysis results (e.g., time periods and product categories with the most cancellation requests)

[1447] Step 7:

[1448] Visual Data Generation

[1449] The server converts the analysis results into graphs and charts.

[1450] Input: Analysis results

[1451] What it does: Use data visualization tools such as D3.js and Chart.js to display a histogram of the distribution of cancellation requests by time period.

[1452] Output: Visual data in the form of graphs and charts

[1453] Step 8:

[1454] Display on the dashboard

[1455] Display visual data on the terminal (for administrator).

[1456] Input: Visual data

[1457] Specific operation: The server generates graphs and charts and renders them on an administrator's dashboard, displaying the results in real time when the administrator accesses it via a browser.

[1458] Output: Visual data displayed on the administrator's terminal

[1459] Step 9:

[1460] Generate improvement suggestions

[1461] The server generates suggestions for improving the system based on the analysis results.

[1462] Input: Analysis results

[1463] Specific operation: Based on the analysis results generated by the AI ​​module, improvement suggestions such as "provide more detailed descriptions of the relevant products" and "highlight cautions" are generated in text format.

[1464] Output: Generated improvement suggestions

[1465] Step 10:

[1466] View Suggestions

[1467] Improvement suggestions are displayed on the terminal (for administrators) so that the administrator can select and implement appropriate countermeasures.

[1468] Input: Generated improvement suggestions

[1469] What it does: The proposal will be displayed on the admin dashboard, allowing the admin to select an action, such as "Approve Proposal."

[1470] Output: Improvement proposals displayed on the administrator's terminal

[1471] (Application example 1)

[1472] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1473] There is a need for a system that can monitor user behavior in real time, analyze collected data, and detect suspicious behavior on online platforms early and suggest appropriate preventive measures. Ignoring suspicious behavior not only increases security risks, but ultimately leads to a poor user experience. Therefore, a system is needed that monitors user behavior, categorizes it, performs AI analysis, and visualizes the results to present them to administrators. In addition, it is also a challenge to establish a system that allows administrators to respond quickly and effectively by proposing specific preventive measures based on the analysis results.

[1474] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1475] In this invention, the server includes a means for detecting suspicious activity based on user behavior data and suggesting appropriate preventive measures, a means for monitoring user behavior data in real time, and a means for collecting the monitored behavior data and storing it in a database, thereby making it possible to detect suspicious activity in real time, identify its causes and conditions, and suggest specific preventive measures.

[1476] Definition of Terms

[1477] "User behavior data" refers to information about various actions and events that users perform within the system, including logins, clicks, data accesses, file changes, etc.

[1478] "Real-time monitoring means" means a system or method for collecting and recording user behavior data in real time, thereby enabling tracking of user behavior without delay.

[1479] A "database storage means" is a system or method that efficiently stores collected behavioral data and allows for quick retrieval upon need.

[1480] A "means for classifying into specific categories" is a system or method for dividing collected behavioral data into pre-defined categories according to its nature or type.

[1481] "AI analytics" is the process of using artificial intelligence to analyze behavioral data and identify patterns and anomalies. This includes machine learning algorithms and data mining techniques.

[1482] A "means for identifying causes and conditions" is a system or method that clarifies the causes and conditions that cause behavior based on analyzed data.

[1483] The "means for visualizing and presenting to the administrator" refers to a system or method for displaying the analysis results in a visual format such as a graph or chart, so that the administrator can easily understand them.

[1484] A "means for generating system improvement proposals" is a system or method that, based on the analysis results, presents a specific action plan aimed at improving user experience or security.

[1485] "Means for detecting suspicious activity and suggesting appropriate preventative measures" refers to a system or method that detects anomalies and risks from user behavior data and suggests measures to strengthen security or encourage careful behavior as countermeasures.

[1486] MODE FOR CARRYING OUT THE INVENTION

[1487] This invention is a system that monitors user behavior data in real time, classifies the collected data into specific categories, and analyzes it using AI. The analysis results are visualized and presented to administrators, and if suspicious behavior is detected, appropriate preventive measures are suggested.

[1488] System configuration

[1489] The server requires the following hardware and software:

[1490] Hardware: Servers with powerful CPUs, RAM, and storage, including resources for rapid data processing and analysis.

[1491] Software: This includes programming languages ​​such as Python, machine learning libraries (e.g., Scikit-learn, TensorFlow), database management systems (e.g., MySQL or PostgreSQL), and visualization libraries (e.g., Matplotlib, Plotly).

[1492] Monitoring Data

[1493] When a user uses the system, their behavioral data is monitored in real time. For example, when a user logs in and accesses a specific file, that information is sent from the user's device to the server in real time. This information includes the user's ID, the timestamp of the action, and the type of action.

[1494] Data collection and storage

[1495] The server temporarily stores the behavioral data received in real time in a buffer, then performs batch processing to store the behavioral data in a database designed to support rapid acquisition and analysis of the behavioral data.

[1496] Data classification and analysis

[1497] The collected behavioral data is categorized into specific categories (e.g., logins, data access, file modifications). The server's AI module analyzes this data and identifies suspicious behavioral patterns. For example, a large number of login attempts in a short period of time can be flagged as a possible account takeover.

[1498] Visualizing the results and presenting them to managers

[1499] The server converts the analysis results into graphs and charts, which are then displayed on the administrator's dashboard, allowing the administrator to intuitively understand user behavioral trends and suspicious behavior.

[1500] Preventive measures suggested

[1501] The server generates specific system improvement suggestions based on the analysis results, such as "If there are a large number of login attempts in a short period of time, suggest strengthening authentication for users." This allows administrators to take security measures quickly and effectively.

[1502] Specific examples

[1503] When a user attempts to access a particular file multiple times, the server immediately records the activity and stores it in a database. The AI ​​module analyzes this data and, if the access attempts exceed the normal range, identifies the activity as suspicious. The results are then presented to the administrator in a graph format, suggesting additional security measures (such as strengthening access restrictions).

[1504] Prompt Sentence Examples

[1505] The prompts for instructing a generative AI model on an analysis task are as follows:

[1506] Analyze real-time monitoring data of user behavior, identify suspicious behavior patterns, and generate specific security improvement suggestions, such as detecting abnormal increases in login counts or data access frequency.

[1507] This allows the invention to be effectively implemented.

[1508] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1509] Program processing steps

[1510] Step 1:

[1511] When a user uses the system, their behavioral data is monitored in real time.

[1512] Input: User behavior data (e.g., login attempts, file access)

[1513] How it works: The user's device collects this behavioral data and packages it with metadata such as a timestamp and user ID.

[1514] Output: Real-time user behavior data packets

[1515] Step 2:

[1516] The monitored behavioral data is sent from the user's device to a server.

[1517] Input: Real-time user behavior data packets

[1518] How it works: The user's device uses HTTP requests and WebSockets to send data to the server.

[1519] Output: Raw behavioral data received by the server

[1520] Step 3:

[1521] The server temporarily stores the received behavioral data in a buffer.

[1522] Input: Raw behavioral data received by the server

[1523] How it works: The server temporarily stores the data in memory or in a high-speed database.

[1524] Output: Buffered behavioral data

[1525] Step 4:

[1526] The server uses batch processing to store the behavioral data in the buffer into a database.

[1527] Input: Buffered behavioral data

[1528] What happens: The server executes a query to store the data in a database.

[1529] Output: Behavioral data stored in a database

[1530] Step 5:

[1531] The server classifies the stored behavioral data into specific categories.

[1532] Input: Behavioral data stored in a database

[1533] Actions: The server separates the behavioral data into categories (e.g., login, data access, file modification) based on predefined rules and conditions.

[1534] Output: Categorized behavioral data

[1535] Step 6:

[1536] The classified behavioral data is analyzed using an AI module.

[1537] Input: Categorized behavioral data

[1538] How it works: Generative AI models use machine learning algorithms to analyze behavioral data and identify suspicious behavioral patterns.

[1539] Output: Analysis results (e.g., abnormal login attempts detected)

[1540] Step 7:

[1541] The analysis results are visualized and presented to the administrator.

[1542] Input: Analysis results

[1543] How it works: The server converts the analysis results into graphs and charts and displays them as a dashboard on the administrator's device.

[1544] Output: Visualized analysis results displayed to the administrator

[1545] Step 8:

[1546] The server generates specific proposals for improving the system based on the analysis results.

[1547] Input: Analysis results

[1548] How it works: The server generates recommendations based on the analysis results, such as security enhancements or system configuration changes.

[1549] Output: System improvement proposals presented to administrators

[1550] Prompt Sentence Examples

[1551] Enter prompts like the following into the generative AI model:

[1552] Analyze real-time monitoring data of user behavior, identify suspicious behavior patterns, and generate specific security improvement suggestions, such as detecting abnormal increases in login counts or data access frequency.

[1553] This specific processing flow allows suspicious user behavior to be detected in real time, enabling a prompt response.

[1554] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1555] Monitoring user behavior and sentiment

[1556] A user accesses the online auction platform and performs trading operations. These operations include bidding, cancellation requests, transaction cancellations, and bid cancellations. The terminal (the user's PC or mobile device) is equipped with an emotion engine that recognizes the user's emotions. The emotion engine uses a camera and microphone to analyze emotions from the user's facial expressions and voice.

[1557] Collecting and classifying behavioral and emotional data

[1558] The device collects user behavioral and emotional data in real time, generates detailed data (user ID, product ID, behavior type, reason for behavior, emotional information, date and time, etc.) and sends it to the server.

[1559] The server temporarily stores the received behavioral and emotional data in a buffer. After that, it performs batch processing and stores the collected data in a database. The database stores all behavioral and emotional data.

[1560] Data analysis

[1561] The server categorizes the behavioral and emotional data stored in the database for analysis. These categorizations include categories such as "cancellation request from the successful bidder," "cancellation of bids below a certain price," "deletion of specific users," "skipping notes," and "slow flow completion." Emotional data is similarly categorized, including emotion categories such as "dissatisfaction," "joy," and "surprise."

[1562] An AI module installed on the server analyzes the classified behavioral and emotional data, using machine learning algorithms to identify behavioral frequency, patterns, conditions for occurrence, and emotional changes.

[1563] Visualizing and presenting results

[1564] The server converts the analyzed results into graphs and charts. This visual data is displayed on a dashboard on the terminal (PC or mobile device used by the administrator). Through the dashboard, administrators can grasp user behavioral trends and changes in emotions in real time.

[1565] System improvement proposals

[1566] Based on the analysis results, the server generates specific suggestions for system improvements, such as providing more detailed descriptions of specific products, simplifying the transaction flow, and presenting customer messages tailored to user sentiment. These suggestions are displayed on the terminal (PC or mobile device used by the administrator), allowing the administrator to select and implement appropriate measures.

[1567] Specific examples

[1568] Sentiment analysis and improvement of cancellation requests

[1569] When a user makes a cancellation request for a specific product, the terminal sends data such as the "user ID," "product ID," "action type (cancellation request)," "reason for action," "date and time," and "emotional information (e.g., dissatisfaction)" to the server.

[1570] The server collects this data and stores it in a database. An AI module then analyzes the data to identify times of day when cancellation requests are most common, product categories, specific user attributes, and associated emotional patterns (e.g., high dissatisfaction).

[1571] The analysis results are converted into graphs and charts by the server and displayed on the terminal (administrator's dashboard). For example, information such as "There are many dissatisfied users in the evening hours" or "There are frequent cancellations and dissatisfaction with products in a particular category" can be immediately confirmed.

[1572] Furthermore, the server generates specific suggestions based on the analysis results to reduce transaction cancellations, such as "providing more detailed product descriptions," "highlighting important points," and "sending follow-up messages to dissatisfied users."

[1573] In this way, the present invention enables detailed monitoring of user behavior and sentiment, and uses AI to analyze the data, thereby efficiently identifying the causes of transaction cancellations and bid retractions and taking appropriate countermeasures, thereby improving the reliability of the online auction platform and improving the user experience.

[1574] The processing flow will be explained below.

[1575] Step 1:

[1576] A user logs in to the online auction platform and performs trading operations, including bidding, requesting cancellation, canceling a transaction, and canceling a bid. The terminal (the user's PC or mobile device) then activates an emotion engine to recognize the user's emotions.

[1577] Step 2:

[1578] In parallel with the transaction, the terminal uses an emotion engine to collect real-time emotional data from the user's facial expressions and voice, including categories such as "happiness," "anger," "anxiety," and "surprise."

[1579] Step 3:

[1580] The terminal generates user behavior data (user ID, product ID, behavior type, reason for behavior, date and time, etc.) and emotion data, and transmits them to the server in real time.

[1581] Step 4:

[1582] The server temporarily stores the received behavioral and emotional data in a buffer, which serves as a temporary storage location for the data.

[1583] Step 5:

[1584] The server periodically runs a batch job and stores the buffered data in a database, where all user behavior and emotion data is stored.

[1585] Step 6:

[1586] The server categorizes the behavioral data and emotional data stored in the database for analysis. Behavioral data is categorized into categories such as "cancellation request from the successful bidder," "cancellation of bids below a certain price," "deletion of specific users," "skipping notes," and "long time required to complete the flow." Emotional data is categorized into emotion categories such as "dissatisfaction," "joy," and "surprise."

[1587] Step 7:

[1588] An AI module installed on the server analyzes the classified behavioral and emotional data and uses machine learning algorithms to identify behavioral frequency, patterns, conditions for occurrence, and emotional changes.

[1589] Step 8:

[1590] The server collects the results of the AI ​​analysis and converts them into visual formats such as graphs and charts. This visualized data is displayed on a dashboard on the terminal (PC or mobile device used by the administrator).

[1591] Step 9:

[1592] The terminal (PC or mobile device used by the administrator) displays the visualized data generated by the server on a dashboard, allowing the administrator to grasp trends in cancellations and bid cancellations, as well as changes in user sentiment, in real time.

[1593] Step 10:

[1594] Based on the results of the AI ​​analysis, the server generates specific suggestions for improving the system, such as "providing more detailed product descriptions," "strengthening transaction management during specific time periods," and "sending customer messages to dissatisfied users."

[1595] Step 11:

[1596] The terminal (PC or mobile device used by the administrator) displays the suggestions from the server on a dashboard, and the administrator can review these suggestions and implement them as necessary.

[1597] Specific examples

[1598] Sentiment analysis and improvement of cancellation requests

[1599] Step 1:

[1600] A user requests cancellation of a particular product, for example, if the product is not what they expected.

[1601] Step 2:

[1602] The device uses a camera to capture the user's facial expression when making a cancellation request and records the audio with a microphone.

[1603] Step 3:

[1604] The device uses an emotion engine to recognize the emotion "dissatisfaction" from the user's facial expressions and voice.

[1605] Step 4:

[1606] The terminal generates detailed data such as "user ID," "product ID," "action type (cancellation request)," "reason for action," "date and time," and "emotional information (dissatisfaction)," and sends it to the server.

[1607] Step 5:

[1608] The server stores this data in a buffer and then periodically moves it to a database.

[1609] Step 6:

[1610] The server categorizes the data stored in the database and analyzes it using an AI module, identifying time periods and product categories with high cancellation requests, as well as related emotional patterns (e.g., high levels of dissatisfaction).

[1611] Step 7:

[1612] The analysis results are converted into graphs and charts by the server and displayed on the terminal (administrator's dashboard). For example, it is possible to see information such as "Many users are dissatisfied in the evening hours" or "Cancellations and dissatisfaction occur frequently for products in a particular category."

[1613] Step 8:

[1614] Based on the analysis results, the server generates specific suggestions to reduce transaction cancellations, such as "make product descriptions more detailed," "highlight important points," or "send follow-up messages to dissatisfied users."

[1615] Step 9:

[1616] The terminal (PC or mobile device used by the administrator) displays the suggestions from the server on a dashboard, and the administrator can review these suggestions and implement them as necessary.

[1617] This series of processes enables detailed monitoring of user behavior and sentiment, and by analyzing the data using AI, it is possible to efficiently identify the causes of transaction cancellations and bid retractions and take appropriate countermeasures, thereby improving the reliability and user experience of the online auction platform.

[1618] Example 2

[1619] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1620] Traditional online platforms typically collect and analyze only user behavioral data. However, this makes it difficult to formulate improvement measures that take user emotions into account, resulting in insufficient improvements to the user experience. Furthermore, analyzing behavioral data alone makes it difficult to accurately grasp the user's emotions and psychological state behind specific actions, making it difficult to find appropriate countermeasures.

[1621] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1622] In this invention, the server includes means for monitoring user behavioral data and emotional data in real time, means for collecting the monitored behavioral data and emotional data and storing it in a database, means for classifying the collected behavioral data and emotional data into specific categories, means for analyzing the classified behavioral data and emotional data using AI and identifying the causes and conditions of the behavior, means for visualizing and presenting the analysis results to an administrator, and means for generating system improvement proposals based on the analysis results. This enables advanced data analysis that takes into account not only user behavior but also emotions, making it possible to formulate more appropriate system improvement measures.

[1623] "User" means any individual or legal entity using the online auction platform.

[1624] "Behavioral Data" refers to information about the specific operations and actions that users take on the online auction platform.

[1625] "Emotion data" is information obtained by analyzing the emotions shown by the user when performing operations.

[1626] "Real-time monitoring means" is a general term for hardware and software for instantly acquiring and monitoring user behavioral and emotional data.

[1627] The "collection means" is a system for collecting monitored data and transmitting it to a database.

[1628] "Means for saving in a database" refers to a mechanism for storing and saving collected data in a database in a certain format.

[1629] "Means for categorizing data into specific categories" refers to methods or techniques for dividing collected data into predetermined categories.

[1630] "Means of analysis" refers to the technology that uses AI to analyze classified data and extract useful information and patterns from that data.

[1631] "Cause and condition identification" is a method for identifying the reasons and circumstances behind user behavior based on data analysis.

[1632] "Means for visualization and presentation to administrators" refers to technology for converting analysis results into visual formats such as graphs and charts, and displaying them in a way that administrators can easily understand.

[1633] "Means for generating system improvement proposals" is a process for creating specific improvement measures to improve user experience based on the analysis results.

[1634] The system of the present invention monitors user behavioral data and emotional data in real time, analyzes this data, and generates system improvement proposals. Specific embodiments for implementing this system will be described below.

[1635] Monitoring user behavior and sentiment

[1636] A user accesses the online auction platform and performs operations such as placing a bid, requesting a cancellation, canceling a transaction, or canceling a bid. At this time, the user's device (PC or mobile device) analyzes the user's facial expressions and voice using an emotion engine. The emotion engine uses hardware such as a camera and microphone to obtain the user's emotion data.

[1637] Collecting and classifying behavioral and emotional data

[1638] The device collects user behavioral and emotional data in real time. This includes detailed data such as user ID, product ID, behavior type, reason for behavior, emotional information, and date and time. This detailed data is sent to the server, which temporarily stores the received data in a buffer. The server then performs batch processing and stores the collected data in a database. All behavioral and emotional data is stored in the database.

[1639] Data analysis

[1640] The server classifies the behavioral and emotional data stored in the database for analysis. This classification includes categories such as "cancellation request from the winning bidder," "cancellation of bids below a certain price," "deletion of specific users," "skipping notes," and "long time it takes to process the flow." Emotional data is also classified into emotion categories such as "dissatisfaction," "happiness," and "surprise." An AI module installed on the server analyzes the classified behavioral and emotional data. The AI ​​module uses machine learning algorithms to identify behavioral frequency, patterns, conditions for occurrence, and emotional changes.

[1641] Visualizing and presenting results

[1642] The server converts the analyzed results into graphs and charts. This visual data is displayed on a dashboard on the terminal (PC or mobile device) used by the administrator. Through the dashboard, administrators can grasp changes in user behavioral trends and emotions in real time.

[1643] System improvement proposals

[1644] Based on the analysis results, the server generates specific suggestions for system improvements, such as providing more detailed descriptions of specific products, simplifying transaction flows, and presenting customer messages tailored to user emotions. These suggestions are displayed on a dashboard on the device used by the administrator, who can then take appropriate action as needed.

[1645] Specific examples

[1646] Sentiment analysis and improvement of cancellation requests

[1647] When a user makes a cancellation request for a specific product, the terminal sends data such as the user ID, product ID, action type (cancellation request), reason for the action, date and time, and emotional information (e.g., dissatisfaction) to the server. The server collects this data and stores it in a database. An AI module then analyzes the data to identify time periods with high cancellation requests, product categories, specific user attributes, and related emotional patterns (e.g., high dissatisfaction). The server converts the analysis results into graphs and charts and displays them on the administrator's dashboard. For example, users can immediately see information such as "there are many dissatisfied users in the evening" or "cancellations and dissatisfaction frequently occur for products in a specific category."

[1648] Furthermore, based on the analysis results, the server generates specific suggestions to reduce transaction cancellations. For example, suggestions include "providing more detailed descriptions of the relevant products," "highlighting important points," and "sending follow-up messages to dissatisfied users." This allows for detailed monitoring of user behavior and sentiment, and by analyzing the data using AI, it is possible to efficiently identify the causes of transaction cancellations and bid retractions and take appropriate countermeasures. The introduction of this system will improve the reliability and user experience of online auction platforms.

[1649] Example of input prompt for generative AI model

[1650] "Please explain the specific methods you use to collect user behavioral and sentiment data."

[1651] "Please detail how you will analyze the collected data and generate improvement measures."

[1652] Please provide a sentiment analysis of users when they make cancellation requests and provide specific suggestions for improvement based on that.

[1653] Using such prompts allows the generative AI model to provide specific and useful information.

[1654] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1655] Step 1:

[1656] A user accesses an online auction platform and logs in. The user performs a transaction (e.g., bid, cancellation request, transaction cancellation, bid cancellation). During this process, the device uses the built-in camera and microphone to collect the user's facial expressions and voice, which are then analyzed by the emotion engine. The input is the user's behavior (transaction) and emotional data (facial expressions, voice), and the output is the analyzed emotional data (e.g., dissatisfaction, joy, surprise).

[1657] Step 2:

[1658] The device analyzes the collected behavioral and emotional data in real time and generates detailed data (user ID, product ID, behavior type, reason for behavior, emotional information, date and time). This detailed data is sent to the server. The input is the behavioral and emotional data obtained in step 1, and the output is the detailed data sent to the server.

[1659] Step 3:

[1660] The server temporarily stores the received detailed data in a buffer, then periodically batch processes it and stores it in a database. This allows all behavioral and emotional data to be accumulated centrally. The input is the detailed data sent from the device, and the output is the behavioral and emotional data stored in the database.

[1661] Step 4:

[1662] The server categorizes the behavioral data and emotional data stored in the database into categories for analysis. For example, behavioral data is categorized into categories such as "cancellation request from the winning bidder" and "cancellation of bids below a certain price," while emotional data is categorized into categories such as "dissatisfaction," "joy," and "surprise." The input is the behavioral data and emotional data stored in the database, and the output is the categorized data.

[1663] Step 5:

[1664] An AI module installed on the server analyzes the categorized behavioral and emotional data. The AI ​​module uses machine learning algorithms to identify behavioral frequency, patterns, conditions for occurrence, and emotional changes. The input is the categorized data, and the output is the analysis results, such as identified behavioral patterns and emotional changes.

[1665] Step 6:

[1666] The server converts the data into graphs and charts based on the analysis results. This visual data is displayed in real time on the dashboard of the administrator's device. The input is the result data analyzed by the AI ​​module, and the output is visual data in the form of graphs and charts.

[1667] Step 7:

[1668] The server generates specific suggestions for system improvement based on the analysis results. These suggestions include providing more detailed descriptions of specific products, simplifying the transaction flow, and presenting customer messages based on user sentiment. These suggestions are displayed on a dashboard on the terminal used by the administrator. The input is the analysis results, and the output is specific suggestions for system improvement.

[1669] By introducing this system, it is possible to closely monitor user behavior and emotions and analyze the data using AI, efficiently identifying the causes of transaction cancellations and bid revocations, and taking appropriate countermeasures.

[1670] (Application example 2)

[1671] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1672] Conventional systems mainly analyze data based solely on user behavior data, and do not take emotional data into account. This makes it difficult to quickly implement appropriate countermeasures to improve user experience and optimize systems. While it is particularly important in brick-and-mortar stores to grasp changes in customer emotions in real time and immediately reflect them in service improvements, achieving this has presented many challenges.

[1673] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1674] In this invention, the server includes means for monitoring user behavioral data and emotional data in real time, means for collecting the monitored behavioral data and emotional data and storing it in a database, means for classifying the collected behavioral data and emotional data into specific categories, means for analyzing the classified behavioral data and emotional data using an AI module and identifying the causes and conditions of changes in the behavior and emotions, means for visualizing and presenting the analysis results to an administrator, means for generating system improvement proposals based on the analysis results, means for collecting new user behavioral and emotional data via specific devices used, and means for transmitting the collected data to the server. This enables analysis that takes into account both user behavioral and emotional data, making it possible to take prompt and appropriate service improvement measures even in physical stores.

[1675] "User behavior data" is information recorded when a user performs a specific action, such as picking up a product, looking at a shelf, or canceling a transaction.

[1676] "Emotional data" is information that is analyzed from the user's facial expressions and voice and indicates their emotional state, such as joy, dissatisfaction, or surprise.

[1677] "Real-time monitoring" is a technology that allows data to be collected and analyzed instantly without delay.

[1678] A "database" is a system for systematically storing and managing collected data.

[1679] "Classifying into specific categories" refers to the process of dividing collected data into predetermined categories, such as "cancellation requests" or "expressions of joy."

[1680] The "AI module" is a program that uses artificial intelligence to analyze collected data and identify patterns and causes of behavior and emotions.

[1681] "Visualizing the analysis results" means displaying the analyzed data in a visual format such as a graph or chart so that the administrator can intuitively understand it.

[1682] "System improvement proposals" refers to generating specific proposals for better services and system optimization based on the analysis results.

[1683] "Specific Equipment Used" refers to hardware such as cameras, microphones, and smart glasses used to collect data.

[1684] A system embodying this invention is designed to monitor user behavioral and emotional data in real time. The system mainly utilizes the following hardware and software:

[1685] Hardware Configuration

[1686] 1. Smart glasses: These are devices that use a built-in camera and microphone to capture the user's facial expressions and voice. This device is used to collect data in real time.

[1687] 2. Server: It is a central control unit for storing collected data, analyzing them and visualizing the results.

[1688] 3. Administrator terminal: A PC or tablet device used to receive analysis results and system improvement proposals and perform management.

[1689] Software Configuration

[1690] 1. EmotionRecognition module: Software that analyzes emotions from the user's facial expressions.

[1691] 2. Voice Analysis Module: Software that analyzes emotions from the user's tone of voice.

[1692] 3. Data collection module: This is software for receiving data sent from the smart glasses and transferring it to the server.

[1693] 4. AI module: An AI program that analyzes collected behavioral and emotional data to identify behavioral patterns and emotional changes.

[1694] 5. Database: A system for systematically storing and managing data.

[1695] 6. Dashboard: An interface for visualizing analysis results in graphs and charts and presenting them to administrators.

[1696] Data processing and calculation

[1697] Data collection: The smart glasses capture the user's facial expressions with a camera and voice with a microphone. These data are analyzed in real time by the EmotionRecognition module and VoiceAnalysis module. The analysis results are collected as behavioral and emotional data.

[1698] Data Transfer: The collected data is sent to the server through the data collection module.

[1699] Data storage: The server stores the received data in a database, which manages both behavioral and emotional data.

[1700] Data analysis: The server's AI module uses the stored data to apply machine learning algorithms to analyze behavioral patterns and emotional changes. Specifically, it analyzes specific user actions (e.g., canceling a specific product) and the emotions they experience (e.g., dissatisfaction) to identify their causes.

[1701] Visualization of results: The analysis results are visualized and displayed on the administrator's dashboard as graphs and charts, allowing administrators to check user behavioral trends and changes in sentiment in real time.

[1702] System improvement proposals: Based on the analysis results, improvement proposals such as detailed product descriptions and simplified transaction flows are automatically generated and notified to the administrator's terminal.

[1703] Specific examples

[1704] For example, if the analysis results show a trend of "many users feeling dissatisfied in the evening," the system will notify the administrator with suggestions for improvement, such as "make the product descriptions more detailed" or "highlight warnings." The following prompt sentences could be considered as specific examples:

[1705] Prompt Sentence Examples

[1706] "Recent data shows that many customers are frustrated when reading the descriptions of new products. What can you do to improve this?"

[1707] As a result, the present invention makes it possible to comprehensively analyze user behavioral data and emotional data, and to propose prompt and appropriate service improvement measures even in physical stores.

[1708] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1709] Step 1:

[1710] The user puts on the smart glasses and walks around the store.

[1711] Input: User's facial expression and audio.

[1712] Output: Image data captured by the camera and audio data recorded by the microphone.

[1713] Specific operation: The camera in the smart glasses continuously captures the user's facial expressions, and the microphone records audio.

[1714] Step 2:

[1715] The smart glasses device uses the EmotionRecognition and VoiceAnalysis modules to analyze the collected data in real time.

[1716] Input: Image data captured by the camera and recorded audio data.

[1717] Output: Parsed emotion and speech data (e.g., happy, frustrated, surprised).

[1718] Specific operation: The EmotionRecognition module analyzes facial expressions, and the VoiceAnalysis module analyzes voice tones to generate emotion data.

[1719] Step 3:

[1720] The analyzed data is sent to a server via a data collection module.

[1721] Input: Emotion data and speech data.

[1722] Output: Data stored on the server.

[1723] Specific operation: The data collection module packages the analyzed data and sends it to the server over the network.

[1724] Step 4:

[1725] The server stores the received data in a database.

[1726] Input: Emotion data and voice data sent to the server.

[1727] Output: Structured data stored in a database.

[1728] Specific operation: The server analyzes the received data and stores it appropriately in the database.

[1729] Step 5:

[1730] The server's AI module performs analysis using behavioral and emotional data stored in a database.

[1731] Input: User behavioral and emotional data stored in a database.

[1732] Output: Analysis results (e.g., many users feel dissatisfied at certain times of the day).

[1733] How it works: The AI ​​module applies machine learning algorithms to identify behavioral patterns and causes of emotional changes.

[1734] Step 6:

[1735] The analysis results are visualized and displayed on the administrator's terminal (PC or tablet).

[1736] Input: Analysis result data.

[1737] Output: Analysis results displayed as graphs and charts.

[1738] Specific operation: The analysis results data is converted into graphs and charts and displayed on a dashboard.

[1739] Step 7:

[1740] Based on the analysis results, proposals for system improvement are generated and notified to the administrator's terminal.

[1741] Input: Analysis result data.

[1742] Output: Suggestions for improving the system (e.g., making product descriptions more detailed, highlighting cautions).

[1743] Specific operation: The AI ​​module generates improvement suggestions based on the analysis results and notifies the administrator terminal.

[1744] Step 8:

[1745] Administrators can review the suggested improvements through the dashboard and take appropriate action.

[1746] Input: System improvement suggestions.

[1747] Output: Corrective action taken (e.g. product description update, repositioning).

[1748] Specific Action: Administrator interacts with the dashboard and takes appropriate remedial action.

[1749] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1750] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1751] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1752] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1753] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1754] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1755] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1756] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1757] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1758] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1759] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1760] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1761] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1762] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1763] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1764] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1765] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1766] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1767] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1768] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1769] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1770] The following is further disclosed regarding the above embodiment.

[1771] (Claim 1)

[1772] A means of monitoring user behavior data in real time,

[1773] a means for collecting and storing the monitored behavioral data in a database;

[1774] A means of classifying collected behavioral data into specific categories;

[1775] A method for analyzing classified behavioral data using AI to identify the causes and conditions of the behavior, and

[1776] A means to visualize the analysis results and present them to the administrator,

[1777] means for generating system improvement suggestions based on the analysis results;

[1778] A system including:

[1779] (Claim 2)

[1780] 2. The system according to claim 1, wherein the system records behavioral data when a user cancels a transaction or bid and analyzes the data.

[1781] (Claim 3)

[1782] 2. The system according to claim 1, further comprising means for visualizing user behavioral trends in graphs or charts based on the analysis results.

[1783] "Example 1"

[1784] (Claim 1)

[1785] A means of monitoring user behavior data in real time,

[1786] means for collecting and storing the monitored behavioral data in a storage device;

[1787] A means of classifying collected behavioral data into specific categories;

[1788] A means for analyzing the classified behavioral data using a machine learning algorithm to identify the causes and conditions of the behavior;

[1789] A means to visualize the analysis results in graphs and charts and present them on an administrator's terminal,

[1790] A means for generating a proposal for system improvement based on the analysis result and displaying the proposal on an administrator terminal;

[1791] A system including:

[1792] (Claim 2)

[1793] 2. The system according to claim 1, wherein the system records behavioral data when a user cancels a transaction or bid and analyzes the data.

[1794] (Claim 3)

[1795] 2. The system according to claim 1, further comprising means for visualizing user behavioral trends in graphs or charts based on the analysis results.

[1796] "Application Example 1"

[1797] Rewriting of claims

[1798] (Claim 1)

[1799] A means of monitoring user behavior data in real time,

[1800] a means for collecting and storing the monitored behavioral data in a database;

[1801] A means of classifying collected behavioral data into specific categories;

[1802] A method for analyzing classified behavioral data using AI to identify the causes and conditions of the behavior, and

[1803] A means to visualize the analysis results and present them to the administrator,

[1804] means for generating system improvement suggestions based on the analysis results;

[1805] A means of detecting suspicious activity based on user behavior data and suggesting appropriate preventative measures;

[1806] A system including:

[1807] (Claim 2)

[1808] 2. The system according to claim 1, wherein the system records behavioral data when a user cancels a transaction or bid and analyzes the data.

[1809] (Claim 3)

[1810] 2. The system according to claim 1, further comprising means for visualizing user behavioral trends in graphs or charts based on the analysis results.

[1811] "Example 2: Combining Emotion Engines"

[1812] (Claim 1)

[1813] a means for monitoring user behavioral and emotional data in real time;

[1814] a means for collecting and storing the monitored behavioral and emotional data in a database;

[1815] a means for classifying the collected behavioral and emotional data into specific categories;

[1816] A method for analyzing classified behavioral and emotional data using AI to identify the causes and conditions of the behavior;

[1817] A means to visualize the analysis results and present them to the administrator,

[1818] means for generating system improvement suggestions based on the analysis results;

[1819] A system including:

[1820] (Claim 2)

[1821] 2. The system of claim 1, wherein behavioral data and emotional data are recorded when a user cancels a transaction or bid, and the data is analyzed.

[1822] (Claim 3)

[1823] 2. The system according to claim 1, further comprising means for visualizing the user's behavioral trends and emotional changes in graphs and charts based on the analysis results.

[1824] "Application example 2 when combining emotion engines"

[1825] (Claim 1)

[1826] a means for monitoring user behavioral and emotional data in real time;

[1827] a means for collecting and storing the monitored behavioral and emotional data in a database;

[1828] a means for classifying the collected behavioral and emotional data into specific categories;

[1829] A means for analyzing the classified behavioral data and emotional data using an AI module to identify the causes and conditions of changes in behavior and emotion;

[1830] A means to visualize the analysis results and present them to the administrator,

[1831] means for generating system improvement suggestions based on the analysis results;

[1832] a means for collecting new user behavioral and emotional data via specific devices used;

[1833] means for transmitting the collected data to a server;

[1834] A system including:

[1835] (Claim 2)

[1836] 2. The system of claim 1, wherein behavioral data and emotional data are recorded when a user cancels a transaction or bid, and the data is analyzed.

[1837] (Claim 3)

[1838] 2. The system according to claim 1, further comprising means for visualizing the user's behavioral tendencies and emotional changes in graphs and charts based on the analysis results. [Explanation of symbols]

[1839] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. A means of monitoring user behavior data in real time, a means for collecting and storing the monitored behavioral data in a database; A means of classifying collected behavioral data into specific categories; A method for analyzing classified behavioral data using AI to identify the causes and conditions of the behavior, and A means to visualize the analysis results and present them to the administrator, means for generating system improvement suggestions based on the analysis results; A system including:

2. 2. The system according to claim 1, wherein behavioral data when a user cancels a transaction or bid is recorded and the data is analyzed.

3. 2. The system according to claim 1, further comprising means for visualizing the user's behavioral trends in graphs or charts based on the analysis results.

Images