system

Abstract

We provide the system. [Solution] The present invention relates to a system that integrates diverse customer behavior data such as demographic information, preferences, purchase history, and survey responses held by companies and businesses, automatically generates virtual customers that closely resemble real customers using generative AI, automatically conducts, aggregates, and analyzes surveys on new products and existing services for these virtual customers, extracts satisfaction levels, improvement requests, and future needs in real time and continuously, and further automatically proposes various marketing measures such as quality improvement, price revisions, advertising media selection, and campaign timing optimization based on the analysis results. This enables efficient and sustainable improvement of customer satisfaction, sales expansion, and strengthening of competitiveness by responding immediately to changing market environments and consumer needs.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method executed by at least one processor, the method comprising the steps of: receiving a user utterance; adding the user utterance to a prompt including an instruction sentence related to a description of a chatbot character; encoding the prompt; and inputting the encoded prompt into a language model to generate a chatbot utterance that responds to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] The problem this invention aims to solve is that companies and businesses face the challenge of constantly changing public opinion and market trends, as well as the diversification of consumer values ​​and preferences. Conventional methods make it difficult to quickly and accurately grasp user needs. Furthermore, traditional surveys and market analyses are costly and time-consuming, and the data obtained is fragmented and static, making it impossible to conduct real-time customer evaluations, grasp latent needs, or plan and execute timely marketing strategies. As a result, improvements in customer satisfaction and sales, and strengthening competitiveness, cannot be fully achieved. Therefore, this invention aims to integrate diverse customer trend data held by companies and businesses, automatically generate virtual customers that closely resemble real customers using generative AI, automatically conduct, aggregate, and analyze surveys on new products and existing services for these virtual customers, extract satisfaction levels, improvement requests, and future needs in real time and continuously, and then automatically propose optimal marketing strategies based on the analysis results. This will enable companies and businesses to respond immediately to the changing market environment and efficiently and sustainably achieve improved customer satisfaction, increased sales, and strengthened competitiveness. [Means for solving the problem]

[0005] To address the challenges, the first step is to establish a customer data integration unit that collects multiple types of customer trend data held by companies and businesses, such as demographic information, preferences, past purchase history, and survey responses, and then manages them comprehensively after preprocessing such as data duplication and data imputation. Next, a virtual customer generation unit is established that uses generative AI to analyze customer attributes and behavioral patterns based on the integrated customer trend data, employing machine learning and clustering techniques to automatically generate multiple virtual customers that closely resemble real customers, each with specific attribute information such as age, gender, place of residence, preferences, purchase trends, and survey response trends. Furthermore, new products and existing products are introduced to these virtual customers. By establishing a survey analysis and strategy proposal department that automatically conducts service-related surveys using generative AI, automatically generates responses such as satisfaction levels and improvement requests tailored to the attributes and past trends of each virtual customer, aggregates and analyzes these responses to quantify and visualize trends in satisfaction scores and improvement requests, and then automatically proposes various marketing measures such as quality improvement, price revisions, strengthened delivery systems, enhanced support systems, and optimized advertising channels using generative AI based on the analysis results, and continuously executes this entire process 24 hours a day, companies and businesses can respond promptly to changing market environments and customer needs, efficiently and sustainably achieving improved customer satisfaction, increased sales, and strengthened competitiveness. [Brief explanation of the drawing]

[0006] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6]This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Modes for carrying out the invention]

[0007] Hereinafter, an example of an embodiment of the system relating to the technology of this disclosure will be described with reference to the attached drawings.

[0008] First, let's explain the terminology used in the following explanation.

[0009] In the following embodiments, the signed processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Furthermore, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include CPU (Central Processing Unit), GPU (Graphics Processing Unit), GPGPU (General-Purpose computing on Graphics Processing Units), and APU (Accelerated Processing Unit).

[0010] In the following embodiments, signed RAM (Random Access Memory) is a memory that temporarily stores information and is used as work memory by the processor.

[0011] In the following embodiments, the storage with a reference sign is one or a plurality of non-volatile storage devices that store various programs, various parameters, and the like. Examples of the non-volatile storage device include a flash memory (SSD (Solid State Drive)), a magnetic disk (for example, a hard disk), a magnetic tape, and the like.

[0012] In the following embodiments, the communication I / F (Interface) with a reference sign is an interface including a communication processor, an antenna, and the like. The communication I / F controls communication between a plurality of computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), and the like.

[0013] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B". That is, "A and / or B" means that it may be only A, may be only B, or may be a combination of A and B. Further, in the present specification, even when three or more matters are expressed by being connected with "and / or", the same concept as that for "A and / or B" is applied.

[0014] [First Embodiment]

[0015] FIG. 1 illustrates an example of a configuration of a data processing system 10 according to the first embodiment.

[0016] As shown in FIG. 1, the data processing system 10 includes a data processing apparatus 12 and a smart device 14. An example of the data processing apparatus 12 is a server.

[0017] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of the "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0018] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, the camera 42, and the communication I / F 44 are also connected to the bus 52.

[0019] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A receives user input from contact of an indicator by detecting contact of the indicator (for example, a pen or a finger). The microphone 38B receives voice-based user input by detecting the user's voice. A control unit 46A in the processor 46 transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, a specific processing unit 290 acquires the data indicating the user input.

[0020] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0021] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0022] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0023] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0024] Storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotions using the emotion identification model 59 and perform identification processing using the user's emotions. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotions, including but not limited to these examples. Furthermore, emotion estimation and prediction also include, for example, emotion analysis.

[0025] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output processing program 60 executed on the RAM 48. The smart device 14 also has a data generation model 58 and an emotion identification model 59, similar to the data generation model and emotion identification model 59, and can perform processing similar to that of the specific processing unit 290 using these models. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0026] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device (e.g., a generation server) may have the data generation model 58. In this case, the data processing device 12 obtains processing results (such as prediction results) using the data generation model 58 by communicating with the server device having the data generation model 58. The data processing device 12 may also be a server device or a terminal device owned by a user (e.g., a mobile phone, robot, home appliance, etc.). Next, an example of processing by the data processing system 10 according to the first embodiment will be described.

[0027] (Example 1)

[0028] The flow of a specific process in Example 1 will now be described. Each part of the system described below is implemented by a data processing device 12 and a smart device 14. The data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0029] (Embodiments for carrying out the present invention)

[0030] Embodiments of the present invention will be described in great detail and comprehensively, based on a configuration using a server and a terminal, including the specific operation of each component, methods for utilizing generative AI, the overall system flow, and various application examples.

[0031] First, the customer data integration unit is implemented on a server. This unit automatically collects customer demographic information (e.g., age, gender, place of residence, occupation, family structure, annual income, education level, etc.), preferences (e.g., purchasing trends, preferences in surveys, content on social media and blogs, web browsing history, app usage history, loyalty card usage history, event participation history, etc.), past purchase history (e.g., category of purchased products or services, brand, price range, purchase frequency, purchase channel, purchase timing, return / exchange history, coupon usage history, etc.), and survey responses (e.g., satisfaction level, improvement requests, intention to repurchase, intention to recommend, opinions in free-response fields, comparative evaluations with competitors, etc.) from multiple databases and external data sources owned by companies and businesses, using various means such as API integration, batch processing, and file import. For example, it obtains purchase history in real time from POS systems, customer attributes and inquiry history from CRM systems, the latest survey responses from web survey systems, and public posts and follower attributes from social media. Because this data differs in format, granularity, and descriptive language, generative AI is used on the server to automatically summarize and classify the content of free-text fields written in natural language, automatically map inconsistencies in attribute names and categories between different data sources, automatically complete and correct missing and outlier values, and automatically integrate and eliminate duplicate data. For example, it automatically standardizes inconsistencies in spelling such as "Tokyo Metropolitan Area," "Tokyo," and "TOKYO," and automatically classifies age information such as "early 30s," "35 years old," and "34 years old" into appropriate categories. Furthermore, the generative AI can also automatically extract positive and negative emotions from customers' free-text comments and classify them by keyword or topic.

[0032] Next, the virtual customer generation unit is also implemented on the server. In this unit, virtual customers are generated using generative AI based on the integrated customer trend data. Specifically, a generative AI model (such as a large-scale language model or generative adversarial network) that has learned the attribute distribution and behavioral patterns of real customers is used to automatically generate a large number of virtual customers with multidimensional attributes such as age, gender, place of residence, occupation, family structure, preferences, purchasing tendencies, survey response tendencies, web browsing tendencies, and SNS posting tendencies. For example, a specific virtual customer such as "a man in his 40s, living in a regional city, health-conscious, likes to eat out with family on weekends, has purchased health foods three times in the past six months, and frequently shares health trends on SNS" is generated. Furthermore, generative AI can incorporate time-series data and external market trend data (e.g., economic indicators, weather data, social trends, new product information from competitors, buzzwords on social media, etc.) to generate virtual customers that reflect real-world market changes and social conditions in real time. For example, "virtual customers who have become more frugal due to recent price increases," "virtual customers who are interested in new diet methods trending on social media," or "virtual customers who tend to stay home more during the pandemic." In addition, generative AI can use techniques such as clustering and latent space exploration to create potential customer profiles that do not exist in existing customer segments but may emerge in the future (e.g., "Generation Z consumers with new values," "emerging consumers who prioritize sustainability," etc.). The number of virtual customers generated and their attribute distribution can be flexibly set according to the company's strategy and analytical objectives.

[0033] The survey analysis and policy proposal department is also implemented on the server. In this department, surveys regarding new products and existing services are automatically conducted for virtual customers using generative AI. For example, the generative AI automatically generates questions such as, "Please rate your satisfaction with the design of new product Y on a 5-point scale," "Please give us your opinion on the pricing," "What additional features would you like to see?", and "In what ways is it superior to similar products from competitors?" Based on the attributes and past trends of the virtual customers, the generative AI automatically creates responses such as satisfaction levels, improvement requests, and free-response comments. For example, a price-sensitive virtual customer might answer, "I would like the price to be lowered a little more," a feature-oriented virtual customer might answer, "I would like smartphone connectivity added," and a brand-conscious virtual customer might answer, "I would like the design to be more luxurious." The generative AI automatically summarizes and categorizes the content of the free-response comments and extracts trends and characteristics of the answers. For example, it automatically extracts specific improvement requests such as "slow delivery," "poor customer support," and "difficult to open the package," and aggregates them by frequently occurring keywords and topics. Once the survey results are collected, they are aggregated and analyzed on the server to quantify and visualize things like the average satisfaction score, items with the most improvement requests, trends by attribute group, changes over time, evaluations by product category, and comparative evaluations with competitors. Based on these analysis results, the generative AI automatically extracts specific insights such as "women in their 30s highly rate the design," "men in their 50s are often dissatisfied with the price," and "new product Y is particularly popular among young people living in urban areas."

[0034] Furthermore, the Survey Analysis and Strategy Proposal Department automatically proposes optimal marketing strategies based on the analysis results using generative AI. For example, if satisfaction is low, it might suggest "implementing a product improvement campaign"; if there are many complaints about price, it might suggest "implementing a limited-time discount"; if there are many requests regarding delivery and support, it might suggest "strengthening the delivery system and customer support"; and if there are many requests for new features, it might suggest "launching a new feature addition project." The generative AI automatically selects the most suitable advertising media, messaging, and campaign timing for each customer segment, making practical suggestions such as "strengthening SNS advertising for younger demographics," "utilizing direct mail for senior demographics," "the best time to launch a new product is before a major holiday," and "proposing a transportation discount campaign for those living in rural areas." The generative AI also references past strategy results and external market trend data, automatically optimizing the strategy content, for example, "setting the discount rate from 10% to 15% based on the effectiveness of similar strategies implemented in the past," "reusing advertising expressions that received a strong response on SNS," and "bringing forward our own campaign to coincide with the launch of new products by competitors."

[0035] These processes run continuously on the server 24 hours a day. Every time new customer data or market trend data is imported into the server, virtual customer generation, surveys, analysis, and policy proposals are updated in real time, ensuring that the latest customer evaluations and needs are always understood. For example, it can immediately respond to new trends that spread rapidly on social media or sudden social events (disasters, epidemics, legal changes, etc.), instantly updating the attributes and needs of virtual customers and reflecting them in policy proposals.

[0036] On the terminal side, the user interface is primarily implemented, allowing marketing personnel at companies and businesses to view and confirm analysis results and proposed strategies performed on the server in real time through dashboards and notification functions. For example, using the terminal's browser or a dedicated app, survey aggregate results by attribute, time series, and product category, as well as proposed strategies from the generative AI, can be intuitively displayed in the form of graphs, heatmaps, recommendation lists, and alert notifications. Furthermore, it is possible to perform operations on the server from the terminal, such as specifying products and services to be analyzed, customizing survey questions, sending feedback on proposed strategies, and inputting data on the effectiveness of strategies after implementation. For example, a person in charge can input instructions from the terminal such as "add survey questions about new product Z," "conduct analysis focused on a specific customer segment," or "change the priority of proposed strategies," and the generative AI on the server side can process them immediately.

[0037] Furthermore, as examples of applications, it can be used in a variety of ways depending on the industry, business type, and purpose. For example, a retailer with multiple stores could generate virtual customers with different customer segments for each store and automatically propose the most suitable promotional measures for each store; a restaurant could generate virtual customers that reflect seasonal consumption trends and weather data and automatically propose seasonal menus and event measures; or a financial institution could generate virtual customers that reflect economic indicators and legal change information and automatically perform risk assessments and predict the effectiveness of new services and campaigns.

[0038] Thus, in the embodiments of the present invention, a customer data integration unit, a virtual customer generation unit, and a survey analysis / policy proposal unit are implemented on the server side, and a user interface unit is implemented on the terminal side. This makes it possible to maximize the use of diverse customer trend data held by companies and businesses, fully utilize the capabilities of generative AI, and provide a system that can grasp customer needs in real time and continuously, and automatically plan and execute optimal marketing measures. For example, it becomes possible to respond immediately to all changes in the market environment, such as seasonal changes in consumption trends, changes in social trends, changes in the actions of competitors, and sudden changes in social conditions, thereby continuously strengthening the competitiveness of companies and improving customer satisfaction.

[0039] (System Configuration)

[0040] The system according to this embodiment comprises a customer data integration unit, a virtual customer generation unit, a questionnaire generation and implementation unit, a questionnaire analysis and policy proposal unit, a user interface unit, a database management unit, an external data linkage unit, and a security management unit. The customer data integration unit has the function of collecting and integrating diverse customer information held by companies and businesses, and automatically collects data of different formats and granularities, such as purchase history data from POS systems, customer attribute data from CRM systems, questionnaire response data from web questionnaire systems, public statement data from SNS and blogs, app usage history, point card usage history, and event participation history, through means such as API linkage, batch processing, and file import. The customer data integration unit uses generative AI to automatically summarize and classify the contents of free-text fields written in natural language, automatically unify inconsistencies in attribute names and categories between different data sources, automatically complete and correct missing and abnormal values, and automatically integrate and eliminate duplicate data. For example, it can standardize variations in spelling such as "Tokyo Metropolitan Area," "Tokyo," and "TOKYO," classify age information such as "early 30s," "35 years old," and "34 years old" into appropriate categories, extract positive and negative emotions from free-response comments, and classify them by keyword or topic.

[0041] The virtual customer generation unit has the function of generating virtual customers using generative AI based on the integrated customer trend data. The virtual customer generation unit uses a generative AI model that has learned the attribute distribution and behavioral patterns of real customers to automatically generate a large number of virtual customers with multidimensional attributes such as age, gender, place of residence, occupation, family structure, preferences, purchasing tendencies, survey response tendencies, web browsing tendencies, and SNS posting tendencies. For example, it can generate a specific virtual customer such as "a man in his 40s, living in a regional city, health-conscious, likes to eat out with family on weekends, has purchased health foods three times in the past six months, and frequently shares health trends on SNS." Furthermore, the virtual customer generation unit can incorporate time-series data and external market trend data to generate virtual customers that reflect real-world market changes and social conditions in real time, such as "a virtual customer whose frugality has increased due to recent price increases," "a virtual customer who is interested in new diet methods that are trending on SNS," or "a virtual customer who tends to refrain from going out during the new infectious disease outbreak." Furthermore, by using techniques such as clustering and latent space exploration, it is possible to create potential customer profiles that do not currently exist in existing customer segments but may emerge in the future. Examples include "Generation Z consumers with new values" and "emerging segments that prioritize sustainability."

[0042] The survey generation and implementation unit has the function of automatically generating and implementing surveys about new products and existing services for virtual customers using a generation-based AI. For example, the survey generation and implementation unit automatically generates questions such as "Please rate your satisfaction with the design of new product Y on a 5-point scale," "Please give us your opinion on the pricing," "What additional features would you like to see?", and "In what ways is it superior to similar products from competitors?". Based on the attributes and past trends of the virtual customer, the generation-based AI automatically creates responses such as satisfaction levels, improvement requests, and free-response comments. For example, a price-sensitive virtual customer might answer "I would like the price to be lowered a little more," a feature-oriented virtual customer might answer "I would like smartphone connectivity added," and a brand-oriented virtual customer might answer "I would like the design to be more luxurious." Furthermore, the survey generation and implementation unit automatically summarizes and categorizes the content of the free-response section and extracts trends and characteristics of the responses. For example, it extracts specific improvement requests such as "Delivery is slow," "Support is poor," and "The package is difficult to open," and aggregates them by frequently occurring keywords and topics.

[0043] The Survey Analysis and Strategy Proposal Department has the function of aggregating and analyzing survey results and automatically proposing optimal marketing strategies using a generative AI. The Survey Analysis and Strategy Proposal Department quantifies and visualizes, for example, the average satisfaction score, items with many improvement requests, trends by attribute group, time-series changes, evaluations by product category, and comparative evaluations with competitors. For example, it automatically extracts specific insights such as "Women in their 30s highly rate the design," "Men in their 50s are often dissatisfied with the price," and "New product Y is particularly popular among young people living in urban areas." Furthermore, the Survey Analysis and Strategy Proposal Department generates specific strategy proposals, such as "implement a product improvement campaign" if satisfaction is low, "implement a limited-time discount" if there are many complaints about price, "strengthen the delivery system and customer support" if there are many requests regarding delivery and support, and "launch a project to add new features" if there are many requests for new features. Furthermore, the system automatically selects the most suitable advertising media, messaging, and campaign timing for each customer segment, making practical suggestions such as "strengthen social media advertising for younger demographics," "utilize direct mail for senior demographics," "the best time to launch a new product is before a major holiday," and "propose a transportation discount campaign for those living in rural areas." In addition, it automatically optimizes campaign content by referencing past campaign results and external market trend data, such as "setting the discount rate from 10% to 15% based on the effectiveness of similar campaigns implemented in the past," "reusing advertising expressions that received a strong response on social media," and "bringing forward our own campaign to coincide with the launch of new products by competitors."

[0044] The user interface unit operates on the terminal, allowing marketing personnel at companies and businesses to view and confirm analysis results and proposed strategies performed on the server in real time through dashboards and notification functions. The user interface unit can intuitively display survey aggregate results by attribute, time series, and product category, as well as strategy proposals suggested by generative AI, in formats such as graphs, heatmaps, recommendation lists, and alert notifications. Furthermore, the user interface unit allows operations from the terminal to the server, such as specifying products and services to be analyzed, customizing survey questions, sending feedback on strategy proposals, and inputting data on the effectiveness of strategy implementation. For example, a person in charge can input instructions from the terminal such as "add survey questions about new product Z," "conduct analysis focused on a specific customer segment," or "change the priority of strategy proposals," and the generative AI on the server side can process them immediately.

[0045] The Database Management Department centrally manages customer data, virtual customer data, survey results, and policy proposal history used throughout the entire system. The Database Management Department ensures system reliability and security by performing tasks such as data backup, restore, access control, data version control, and encrypted data storage. The External Data Integration Department automatically acquires external market trend data, economic indicators, weather data, competitor information, and SNS trend data via APIs and web crawling, and utilizes this data for analysis and virtual customer generation within the system. For example, weather data can be incorporated to analyze "changes in purchasing trends during rainy weather," or SNS trend data can be incorporated to generate "virtual customers interested in trending products." The Security Management Department is responsible for system-wide access control, unauthorized access detection, data encryption, audit log management, and personal information protection, enabling companies and businesses to operate the system with confidence.

[0046] Specific examples of prompts to be fed into the generation AI necessary for implementing the present invention include, for example, "Generate 10 patterns of virtual customers that closely resemble real customers based on the following customer attribute data. Each virtual customer should include age, gender, place of residence, occupation, preferences, purchasing tendencies, and survey response tendencies," or "Automatically generate survey questions regarding the next new product X. The questions should include satisfaction, price, function, design, competitor comparison, and free-response comments," or "Automatically generate survey responses for each virtual customer based on the following virtual customer attributes and survey questions. The responses should include satisfaction scores and specific improvement requests," or "Analyze the following survey results and automatically propose three optimal marketing strategy proposals for each customer segment. The strategy proposals should include advertising media, appeal content, and campaign timing." Furthermore, a variety of prompts can be used in each part of the system to maximize the use of the generation AI, such as "Summarize the following free-response survey responses and extract the main improvement requests," or "Based on past strategy performance data and current market trend data, propose the optimal content and implementation timing for the next promotional strategy."

[0047] (Implementation Steps)

[0048] Step 1: Collect and integrate customer data This step involves automatically collecting and integrating diverse customer data held by companies and businesses on a server. Specifically, it aggregates data of different formats and granularities, such as purchase history from POS systems, customer attributes from CRM systems, survey responses from web survey systems, public posts from social media and blogs, app usage history, loyalty card usage history, and event participation history, using methods such as API integration, batch processing, and file import. Furthermore, it performs preprocessing such as data deduplication, missing value imputation, normalization, and anonymization, and manages the data as an integrated customer profile.

[0049] Step 2: Data preprocessing, summarization, and classification using generative AI Using generative AI, the collected customer data is automatically summarized and categorized using natural language in free-response fields, unified inconsistencies in attribute names and categories across different data sources, automatically imputes and corrects missing and outlier values, and automatically merges and eliminates duplicate data. For example, inconsistencies in spelling such as "Tokyo," "Tokyo," and "TOKYO" are unified, and age information such as "early 30s," "35 years old," and "34 years old" is classified into the appropriate category. Specific examples of prompts to be fed into the generative AI include: "Unify the inconsistencies in spelling of the following customer attribute data and classify them by category," and "Summarize the following free-response questionnaire answers and extract the main opinions and sentiments."

[0050] Step 3: Generating virtual customers using AI This step involves automatically generating a large number of virtual customers using generative AI based on integrated customer behavior data. Using a generative AI model that has learned the attribute distribution and behavioral patterns of real customers, it generates virtual customers with multidimensional attributes such as age, gender, place of residence, occupation, family structure, preferences, purchasing tendencies, survey response tendencies, web browsing tendencies, and social media posting tendencies. For example, it generates a specific virtual customer such as "a man in his 40s, living in a regional city, health-conscious, enjoys eating out with family on weekends, has purchased health foods 3 times in the past 6 months, and frequently shares health trends on social media." Specific examples of prompts to be fed into the generative AI include: "Based on the following customer attribute data, please generate 10 patterns of virtual customers that closely resemble real customers. Each virtual customer should include age, gender, place of residence, occupation, preferences, purchasing tendencies, and survey response tendencies."

[0051] Step 4: Automatic generation of survey questions using AI. This step involves automatically generating survey questions for virtual customers using a generative AI. For example, it automatically generates questions such as, "Please rate your satisfaction with the design of new product Y on a 5-point scale," "Please give us your opinion on the pricing," "What additional features would you like to see?", and "How is it superior to similar products from competitors?". A specific example of a prompt to be fed into the generative AI would be, "Please automatically generate survey questions for the following new product X. The questions should include satisfaction, price, features, design, competitor comparison, and open-ended comments."

[0052] Step 5: Automatic generation of virtual customer survey responses using AI. This step involves using a generative AI to automatically generate responses to the generated survey questions, based on each virtual customer's attributes and past trends, including satisfaction levels, improvement requests, and free-response comments. For example, a price-sensitive virtual customer might respond, "I'd like the price to be lowered a bit," while a feature-oriented virtual customer might respond, "I'd like smartphone connectivity added." A concrete example of a prompt to be fed into the generative AI would be, "Please automatically generate survey responses for each virtual customer based on the following virtual customer attributes and survey questions. Please include satisfaction scores and specific improvement requests in the responses."

[0053] Step 6: Compiling and analyzing survey results This step involves aggregating and analyzing survey responses obtained from virtual customers on a server. It quantifies and visualizes average satisfaction scores, frequently requested improvements, trends by attribute group, time-series changes, product category evaluations, and comparative evaluations with competitors. For example, it extracts specific insights such as "women in their 30s highly rate the design," "men in their 50s are often dissatisfied with the price," and "new product Y is particularly popular among young people living in urban areas."

[0054] Step 7: Automatic generation of policy proposals using AI. This step involves using a generative AI to automatically generate optimal marketing strategies based on the results of a survey. For example, if satisfaction is low, it might suggest "implementing a product improvement campaign"; if there are many complaints about price, it might suggest "implementing a limited-time discount"; if there are many requests regarding delivery and support, it might suggest "strengthening the delivery system and customer support"; and if there are many requests for new features, it might suggest "launching a project to add new features." The AI ​​will generate specific strategies like these. Furthermore, it will automatically select the most suitable advertising media, messaging, and campaign timing for each customer segment. A concrete example of a prompt to feed into the generative AI would be: "Analyze the following survey results and automatically propose three optimal marketing strategies for each customer segment. The strategies should include advertising media, messaging, and campaign timing."

[0055] Step 8: Visualization of proposed policies and analysis results, and presentation to those in charge. This step involves presenting the survey results and proposed measures suggested by the generative AI to the person in charge through the user interface in formats such as graphs, heatmaps, recommendation lists, and alert notifications. The person in charge can also perform operations from their terminal, such as specifying the products or services to be analyzed, customizing survey questions, sending feedback on proposed measures, and inputting data on the effectiveness of the measures after their implementation.

[0056] (Specific use cases)

[0057] For example, let's consider a case where a nationwide retailer launches a new health food product and uses the system of the present invention to conduct pre-market customer evaluation and plan optimal marketing strategies. First, the customer data integration unit automatically collects diverse data such as purchase history data from the POS system for the past three years, customer attribute data from the CRM system, past survey responses regarding health foods from the web survey system, public posts on health consciousness from social media, point card usage history, app usage history, and event participation history, using API integration and batch processing, and stores it in the database management unit. Next, using the generation AI, it unifies inconsistencies in spelling such as "Tokyo," "Tokyo," and "TOKYO," classifies age information such as "early 30s," "35 years old," and "34 years old" into appropriate categories, and extracts and summarizes key opinions and feelings from free-response comments such as "the price is high," "I like the taste," and "the packaging is confusing." Specific examples of prompts to be fed into the generation AI include, "Unify the inconsistencies in the following customer attribute data and classify them by category," and "Summarize the following free-response survey responses and extract key opinions and feelings."

[0058] The virtual customer generation unit uses integrated customer trend data and generation AI to automatically generate a large number of virtual customers with multidimensional attributes such as age, gender, place of residence, preferences, purchasing tendencies, and survey response tendencies. For example, "female in her 30s, lives in an urban area, health-conscious, has purchased health foods multiple times in the past six months, and frequently shares health trends on social media" or "male in his 50s, lives in a regional city, is price-conscious, does not purchase health foods frequently but was recently told he was at risk of lifestyle-related diseases in a health checkup." A specific example of a prompt message to be fed into the generation AI is: "Based on the following customer attribute data, please generate 10 patterns of virtual customers that closely resemble real customers. Each virtual customer should include age, gender, place of residence, occupation, preferences, purchasing tendencies, and survey response tendencies."

[0059] The survey generation and implementation department uses a generation AI to automatically generate questions such as, "Please rate your satisfaction with the taste of the new health food on a 5-point scale," "Please give us your opinion on the pricing," "What kind of package design would you like?", and "In what ways is it superior to health foods from competitors?". Based on the attributes and past trends of virtual customers, the generation AI automatically creates responses such as satisfaction levels, suggestions for improvement, and free-response comments. Specific examples of prompts to be fed into the generation AI include, "Please automatically generate the following survey questions regarding the new health food. The questions should include taste, price, packaging, competitor comparison, and free-response comments," and "Based on the following virtual customer attributes and survey questions, please automatically generate survey responses for each virtual customer. The responses should include a satisfaction score and specific suggestions for improvement."

[0060] The Survey Analysis and Strategy Proposal Department aggregates and analyzes survey responses obtained from virtual customers on a server, extracting specific insights such as, "Women in their 30s living in urban areas give high marks to the taste, but tend to be somewhat dissatisfied with the price," and "Men in their 50s living in regional cities value easy-to-understand packaging and are more responsive to advertising that strongly emphasizes health." Furthermore, using a generation AI, it automatically generates optimal marketing strategy proposals for each customer segment, such as, "For young people in urban areas, strengthen SNS advertising and implement a first-time purchase discount campaign," "For middle-aged and elderly people living in regional areas, develop sampling measures linked to health checkup events," and "As a suggestion for improving the package design, enlarge the ingredient list and clearly communicate the health benefits." An example of a prompt to be fed into the generation AI is, "Analyze the following survey results and automatically propose three optimal marketing strategy proposals for each customer segment. The strategy proposals should include advertising media, appeal content, and campaign timing."

[0061] The user interface section presents these analysis results and proposed measures to the person in charge on a dashboard in the form of graphs, heatmaps, recommendation lists, and alert notifications. The person in charge can then perform operations from their terminal, such as specifying the products or services to be analyzed, customizing survey questions, sending feedback on proposed measures, and inputting data to measure the effectiveness of the measures after their implementation.

[0062] In such use cases, the system of the present invention can automatically provide real-time, multidimensional customer evaluations that cannot be obtained through conventional market research or survey data collection, as well as specific marketing strategy proposals optimized for each customer segment. This enables companies and businesses to respond promptly to changing market environments and consumer needs, efficiently and sustainably achieving improved customer satisfaction, increased sales, and enhanced competitiveness.

[0063] (Application Example 1) The flow of a specific process in Application Example 1 will be explained below. Each part of the system described below is implemented by a data processing device 12 and a smart device 14. The data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0064] (Embodiments for carrying out the present invention)

[0065] This section will provide a more detailed, comprehensive, and thorough explanation of the system configuration and operation of each component in the field of e-commerce sites, as well as embodiments for implementing the present invention. The system of this embodiment is configured such that the server side implements a customer data integration unit, a virtual customer generation unit, a site optimization measure proposal unit, a database management unit, an external data linkage unit, and a security management unit, while the terminal side implements a user interface unit.

[0066] First, the customer data integration unit has the function of automatically collecting and integrating diverse customer behavior data from the e-commerce site on the server. For example, member registration information includes age, gender, place of residence, occupation, family structure, email address, phone number, member rank, registration date, birthday, hobbies, interest categories, and past campaign participation history. Purchase history includes purchased product name, product category, brand, price, purchase date and time, purchase frequency, purchase channel (PC, smartphone, tablet, etc.), coupon usage history, return / exchange history, gift usage history, bulk purchase history, subscription history, and whether or not a review was posted at the time of purchase. Browsing history includes the number of times a product page was viewed, time spent on the page, viewing order, search keywords, ranking page views, review viewing history, genre and price range of viewed products, cart addition rate after viewing, and abandonment rate after viewing. Cart addition history includes products added to the cart, date and time of addition, cart abandonment history, price range and category of products in the cart, time spent in the cart, cart deletion history, and purchase conversion rate of products in the cart. Survey responses include satisfaction scores, suggestions for improvement, intention to repurchase, intention to recommend, comments in the free-response section, comparative evaluations with competitor sites, product-specific evaluations, evaluations regarding delivery and support, and evaluations regarding UI / UX. Inquiry history includes inquiry content, inquiry date and time, response history, FAQ viewing history, chatbot usage history, satisfaction with inquiry response, and purchasing behavior after inquiry. Point usage history includes points earned, points used, point expiration date, point campaign participation history, point expiration history, and purchasing trends when using points. Social media integration information includes social media account type, number of followers, post content, share history, like history, traffic history via social media, social media campaign participation history, and social media word-of-mouth content. This data is automatically collected through various means such as API integration, batch processing, file import, web crawling, and real-time streaming, and stored in the database management unit.

[0067] The Customer Data Integration Department utilizes generative AI to automatically summarize and classify the content of free-response sections of surveys written in natural language, automatically map inconsistencies in attribute names and categories between different data sources, automatically complete and correct missing and outlier values, and automatically integrate and eliminate duplicate data. For example, it can standardize variations in spelling such as "Tokyo," "Tokyo," and "TOKYO," classify age information such as "early 30s," "35 years old," and "34 years old" into appropriate categories, extract positive and negative emotions and major requests from free-response opinions, and classify them by keyword and topic. Furthermore, it can consolidate data of the same person from multiple data sources, anonymize it, and manage it as a unified customer profile. For example, it can integrate customer information from different databases using the same email address, phone number, or SNS account as a key, and automatically perform anonymization in accordance with personal information protection regulations.

[0068] The virtual customer generation unit has the function of generating virtual customers using generative AI based on the integrated customer trend data. Specifically, it uses a generative AI model that has learned the attribute distribution and behavioral patterns of real customers to automatically generate a large number of virtual customers with multidimensional attributes such as age, gender, place of residence, occupation, family structure, preferences, purchasing tendencies, survey response tendencies, browsing tendencies, cart addition tendencies, payment method preferences, return history, and SNS posting tendencies. For example, they create virtual customer profiles that reflect real customer demographics, such as: "A woman in her 30s, living in an urban area, fashion-conscious, has purchased shoes and bags multiple times in the past six months, frequently shares trendy items on social media, is sensitive to discounts and free shipping campaigns, and uses her smartphone frequently"; "A man in his 50s, living in a rural area, likes electronics, doesn't purchase frequently but chooses expensive items, prefers electronic money for payment, and values ​​fast delivery"; "A man in his 20s, a student, interested in games and gadgets, frequently reads reviews, values ​​word-of-mouth on social media, and often uses the site late at night"; and "A woman in her 40s, raising children, frequently makes regular purchases of baby products and daily necessities, values ​​point campaigns, and frequently uses specified delivery dates and times." Furthermore, the virtual customer generation unit incorporates market trend data, seasonal fluctuation data, competitor site strategy information, and SNS trend data via the external data linkage unit, enabling the generation of virtual customers that reflect real-world market changes and social conditions in real time. For example, "virtual customers reflecting seasonal demand fluctuations," "virtual customers interested in new products trending on SNS," "virtual customers influenced by new campaigns implemented on competitor sites," "virtual customers reflecting changes in purchasing trends due to weather changes," and "virtual customers influenced by social events (long holidays, disasters, epidemics, etc.)." In addition, by using methods such as clustering and latent space exploration, it is possible to create potential customer profiles that do not exist in existing customer segments but may emerge in the future. Examples include "emerging segments that prioritize sustainability," "Gen Z consumers with new values," and "new purchasing patterns among seniors in an aging society."

[0069] The Site Optimization Strategy Proposal Department uses generative AI to automatically generate survey questions for each element of an e-commerce site for each virtual customer, and automatically generates responses such as satisfaction levels, improvement requests, and free-response comments based on the virtual customer's attributes and past trends. For example, it creates questions that anticipate actual user experience, such as "Are the images and descriptions on the product detail page easy to understand?", "How is the usability of the shopping cart screen?", "Can the desired payment method be selected on the payment screen?", "Are the shipping options sufficient?", "Is the return process easy?", "Are the campaign banners attractive?", "Is the recommendation function useful?", "Is the search function easy to use?", "Are the review displays helpful?", "Is the overall site loading speed satisfactory?", and "Is the smartphone display optimized?". For each virtual customer, the generative AI automatically generates satisfaction scores and improvement requests for these questions. For example, a fashion-oriented virtual customer might answer "I want more product images" and "I want to see outfit examples," a delivery-focused virtual customer might answer "I want a same-day shipping option added" and "I want the real-time tracking function for shipping status strengthened," and a payment method-conscious virtual customer might answer "I want more electronic money and deferred payment options" and "I want an installment payment option added." Virtual customers who prioritize reviews responded with requests such as, "I'd like reviews to be displayed only to purchasers to increase their credibility," and "I'd like a feature to upload photos of reviews." Specific comments such as "The return process is confusing," "Campaign information is hard to find," "The site loads slowly," and "It's difficult to use on smartphones" are also automatically generated in the free-response section.

[0070] The Site Optimization Strategy Proposal Department compiles and analyzes these survey responses to extract trends for each customer segment and overall challenges, such as "Women in their 20s strongly desire more product images and styling examples," "Men in their 40s prioritize diversification of payment methods and improved delivery speed," "Overall, there is a demand for simplified return procedures," "Improving the reliability of the review function contributes to overall satisfaction," and "Optimizing smartphone display is effective in reducing the bounce rate among younger users." Furthermore, using generative AI, it automatically proposes specific site optimization measures, such as "Increase the number of images on product detail pages," "Automatically generate and post styling examples," "Refresh the cart screen UI," "Add electronic money and deferred payment as payment methods," "Introduce same-day delivery as a shipping option," "Make the return process one-click," "Change the display position of campaign banners to the center of the top page," "Add photo posting and purchaser-only display to the review function," "Expand filtering options in the search function," "Increase the overall display speed of the site," and "Optimize the UI / UX for smartphone display." In addition, it automatically indicates the priority of each measure and which measure is effective for which customer segment. For example, it automatically selects the most suitable advertising media, messaging, and campaign timing for each customer segment, such as "strengthen social media advertising for younger demographics," "utilize direct mail for senior demographics," "the best time to launch a new product is before a major holiday," and "enhancing the review function is effective for all generations."

[0071] The Database Management Department centrally manages customer data, virtual customer data, survey results, policy proposal history, external data linkage history, and system logs used throughout the entire system. It also performs data backup, restore, access rights management, data version control, encrypted data storage, and data audit log management. The External Data Linkage Department automatically acquires external market trend data, economic indicators, weather data, competitor site information, SNS trend data, and social event information via APIs and web crawling, and utilizes them for analysis within the system and virtual customer generation. For example, it might incorporate weather data to analyze "changes in purchasing trends during rainy weather," incorporate SNS trend data to generate "virtual customers interested in trending products," or incorporate new campaign information from competitor sites to generate "virtual customers who respond to competitor initiatives." The Security Management Department is responsible for system-wide access control, unauthorized access detection, data encryption, audit log management, personal information protection, and compliance with legal regulations such as GDPR, ensuring that companies and businesses can operate the system with peace of mind.

[0072] The user interface operates on the terminal, allowing site administrators to view and confirm analysis results and proposed strategies performed on the server in real time through dashboards and notification functions. For example, survey results categorized by attribute, time series, product category, and site element, as well as strategy proposals suggested by generative AI, can be intuitively displayed in the form of graphs, heatmaps, recommendation lists, alert notifications, and post-implementation effectiveness measurement results. Furthermore, it is possible to perform operations on the server from the terminal, such as specifying products and services to be analyzed, customizing survey questions, sending feedback on strategy proposals, inputting post-implementation effectiveness measurement data, customizing dashboards, and setting up A / B tests for strategies. For example, an administrator can input instructions from the terminal such as "add survey questions about new product A," "conduct analysis focused on a specific customer segment," "change the priority of strategy proposals," "specify the strategies to be tested in A / B tests," and "input changes in sales, churn rate, and satisfaction after implementation of strategies," and the generative AI on the server side can process them immediately.

[0073] As described above, in the embodiments of the present invention, a customer data integration unit, a virtual customer generation unit, a site optimization measure proposal unit, a database management unit, an external data linkage unit, and a security management unit are implemented on the server side, and a user interface unit is implemented on the terminal side. This makes it possible to maximize the use of diverse customer trend data held by e-commerce site operators, fully utilize the capabilities of generative AI, grasp customer needs in real time and continuously, and provide a system that can automatically plan and execute optimal site operation measures. For example, it is possible to respond immediately to all changes in the market environment, such as seasonal demand fluctuations, changes in social trends, changes in competitor sites' measures, sudden changes in social conditions, changes in weather, changes in laws and regulations, and the emergence of buzzwords on social media, and to continuously achieve maximizing sales, reducing bounce rates, improving customer satisfaction, and strengthening competitiveness of e-commerce sites.

[0074] (System Configuration)

[0075] The system according to this embodiment includes a customer data integration unit, a virtual customer generation unit, a site optimization measure proposal unit, a database management unit, an external data linkage unit, a security management unit, and a user interface unit.The Customer Data Integration Department has the function of automatically collecting and integrating diverse customer behavior data on the e-commerce site on the server. For example, it includes member registration information such as age, gender, place of residence, occupation, family structure, email address, member rank, birthday, hobbies, interest categories, and past campaign participation history; purchase history such as purchased product name, product category, brand, price, purchase date and time, purchase frequency, purchase channel, coupon usage history, return / exchange history, gift usage history, subscription history, and whether or not reviews have been posted; browsing history such as the number of times product pages have been viewed, time spent on the page, viewing order, search keywords, ranking page views, review viewing history, genre and price range of viewed products, cart addition rate after viewing, and abandonment rate after viewing; cart addition history such as products added to the cart, date and time added, cart abandonment history, price range and category of products in the cart, time spent on products in the cart, cart deletion history, and purchase conversion rate of products in the cart; and survey responses such as satisfaction score, improvement requests, repurchase intention, recommendation intention, opinions in the free-response field, comparative evaluation with competitor sites, product evaluation, evaluation regarding delivery and support, and UI / UX. This system automatically collects data such as ratings, inquiry history (including inquiry content, inquiry date and time, response history, FAQ viewing history, chatbot usage history, satisfaction level with inquiry response, and post-inquiry purchasing behavior), point usage history (including points earned, points used, point expiration date, point campaign participation history, point expiration history, and purchasing trends when using points), and SNS integration information (including SNS account type, number of followers, post content, share history, like history, traffic history via SNS, SNS campaign participation history, and SNS word-of-mouth content). This data is automatically collected through various means such as API integration, batch processing, file import, web crawling, and real-time streaming. Generative AI is used to automatically summarize and classify the content of free-response fields in surveys written in natural language, automatically map inconsistencies in attribute names and categories between different data sources, automatically complete and correct missing and outlier values, automatically integrate and eliminate duplicate data, consolidate data for the same person from multiple data sources, anonymize the data, and manage it as a unified customer profile.

[0076] The virtual customer generation unit has the function of generating virtual customers using generative AI based on the integrated customer trend data. Using a generative AI model that has learned the attribute distribution and behavior patterns of real customers, it automatically generates a large number of virtual customers with multidimensional attributes such as age, gender, place of residence, occupation, family structure, preferences, purchasing tendencies, survey response tendencies, browsing tendencies, cart addition tendencies, payment method preferences, return history, and SNS posting tendencies. For example, "female in her 30s, lives in an urban area, fashion-oriented, has purchased shoes and bags multiple times in the past six months, frequently shares trendy products on SNS, is sensitive to discounts and free shipping campaigns, and uses smartphones frequently," "male in his 50s, lives in a rural area, likes home electronics, purchases infrequently but chooses high-priced items, prefers electronic money for payment, values ​​fast delivery," "male in his 20s, student, interested in games and gadgets, frequently reads reviews, values ​​word-of-mouth on SNS, late at night." The system creates virtual customer profiles that reflect real-world customer demographics, such as "frequent site usage," "women in their 40s, raising children, frequently making regular purchases of baby products and daily necessities, prioritizing point campaigns, and frequently using specified delivery dates and times." Through the external data integration department, it also incorporates market trend data, seasonal fluctuation data, competitor site strategy information, SNS trend data, weather data, and social event information to generate virtual customers that reflect real-world market changes and social conditions in real time, such as "virtual customers reflecting seasonal demand fluctuations," "virtual customers interested in new products trending on SNS," "virtual customers influenced by new campaigns implemented on competitor sites," "virtual customers reflecting changes in purchasing trends due to weather changes," and "virtual customers influenced by social events." Using methods such as clustering and latent space exploration, it also creates potential customer profiles that do not exist in existing customer segments but may emerge in the future.

[0077] The Site Optimization Strategy Proposal Department uses generative AI to automatically generate survey questions for each element of an e-commerce site for each virtual customer. Based on the virtual customer's attributes and past trends, it automatically generates responses such as satisfaction levels, improvement requests, and free-response comments. Questions such as "Are the images and descriptions on the product detail page easy to understand?", "How is the usability of the shopping cart screen?", "Can I select my preferred payment method on the checkout screen?", "Are the shipping options sufficient?", "Is the return process easy?", "Are the campaign banners attractive?", "Is the recommendation function useful?", "Is the search function easy to use?", "Are the review displays helpful?", "Are you satisfied with the overall site loading speed?", and "Is the smartphone display optimized?" are created. For each virtual customer, the generative AI automatically generates satisfaction scores and improvement requests for these questions. For example, a fashion-oriented virtual customer might answer "I want more product images" and "I want to see outfit examples," while a delivery-focused virtual customer might answer "I want a same-day delivery option added" and "I want a delivery option added." Some virtual customers responded, "We want enhanced real-time tracking of shipping status," while others, who are particular about payment methods, responded, "We want more electronic money and deferred payment options," and "We want installment payment options added." Virtual customers who prioritize reviews responded, "We want reviews to be displayed only to purchasers to increase their reliability," and "We want a photo posting function for reviews." In the free-response section, specific opinions such as "The return process is confusing," "Campaign information is hard to find," "The site loads slowly," and "It's difficult to use on smartphones" were automatically generated. By aggregating and analyzing these survey responses, trends for each customer segment and overall challenges were extracted, such as "Women in their 20s strongly desire more product images and styling examples," "Men in their 40s prioritize diversification of payment methods and improved delivery speed," "Overall, there is a demand for simplified return procedures," "Improving the reliability of the review function contributes to overall satisfaction," and "Optimizing smartphone display is effective in reducing the dropout rate among younger users."Using generative AI, the system automatically proposes specific site optimization measures, such as "increasing the number of images on product detail pages," "automatically generating and displaying outfit examples," "revamping the shopping cart UI," "adding electronic money and deferred payment options," "introducing same-day delivery as a shipping option," "making the return process a one-click process," "changing the display position of campaign banners to the center of the top page," "adding photo posting and buyer-only display to the review function," "expanding filtering options in the search function," "speeding up the overall site display speed," and "optimizing the UI / UX for smartphone display." It also automatically indicates the priority of each measure and which measure is most effective for which customer segment.

[0078] The Database Management Department centrally manages customer data, virtual customer data, survey results, policy proposal history, external data linkage history, and system logs used throughout the entire system. It also performs data backup, restore, access rights management, data version control, encrypted data storage, and data audit log management. The External Data Linkage Department automatically acquires external market trend data, economic indicators, weather data, competitor site information, SNS trend data, and social event information via APIs and web crawling, and utilizes this data for analysis within the system and virtual customer generation. For example, it might incorporate weather data to analyze "changes in purchasing trends during rainy weather," incorporate SNS trend data to generate "virtual customers interested in trending products," or incorporate new campaign information from competitor sites to generate "virtual customers who respond to competitor initiatives." The Security Management Department is responsible for system-wide access control, unauthorized access detection, data encryption, audit log management, personal information protection, and legal compliance, ensuring that companies and businesses can operate the system with peace of mind.

[0079] The user interface operates on the terminal, allowing site administrators to view and confirm analysis results and proposed measures performed on the server in real time through dashboards and notification functions. It intuitively displays survey aggregate results by attribute, time series, product category, and site element, as well as proposed measures from generative AI, in formats such as graphs, heatmaps, recommendation lists, alert notifications, and post-implementation effect measurement results. From the terminal, it is also possible to perform operations on the server, such as specifying products and services to be analyzed, customizing survey questions, sending feedback on proposed measures, inputting post-implementation effect measurement data, customizing dashboards, and setting up A / B tests for measures.

[0080] Specific examples of prompts to be fed into the generation AI necessary for implementing the present invention include: "Generate 10 patterns of virtual customers that closely resemble real customers based on the following customer attribute data. Each virtual customer should include age, gender, place of residence, occupation, preferences, purchasing tendencies, and survey response tendencies." "Automatically generate survey questions regarding the following new product A. The questions should include satisfaction level, price, functionality, design, competitor comparison, and free-response comments." "Automatically generate survey responses for each virtual customer based on the following virtual customer attributes and survey questions. The responses should include satisfaction scores and specific improvement requests." "Analyze the following survey results and automatically propose three optimal site optimization measures for each customer segment. The measures should include UI improvements, advertising media, appeals, and campaign timing."

[0081] (Implementation Steps)

[0082] Step 1: Collect and integrate customer data The system automatically collects diverse customer behavior data from e-commerce sites, including member registration information, purchase history, browsing history, cart addition history, survey responses, inquiry history, point usage history, and social media integration information, on the server using methods such as API integration, batch processing, file import, web crawling, and real-time streaming, and stores it in the database management unit. For example, it comprehensively captures data of all levels and formats, such as purchased product names and categories, purchase dates and times, cart abandonment history, review posting status, social media traffic history, and customer satisfaction with inquiry response.

[0083] Step 2: Data preprocessing, matching, summarization, and classification using generative AI. Using generative AI, the collected customer data is automatically summarized and categorized from the free-response sections of questionnaires written in natural language, automatically maps inconsistencies in attribute names and categories between different data sources, automatically imputes and corrects missing and outlier values, automatically merges and eliminates duplicate data, consolidates data for the same person from multiple data sources, anonymizes the data, and manages it as a unified customer profile. Specific examples of prompts to be fed into the generative AI include: "Please unify the inconsistencies in the notation of the following customer attribute data and classify them by category," and "Please summarize the following free-response questionnaire answers and extract the main opinions and sentiments."

[0084] Step 3: Generating virtual customers using AI Based on integrated customer behavior data, generative AI is used to automatically generate a large number of virtual customers with multidimensional attributes such as age, gender, place of residence, occupation, family structure, preferences, purchasing tendencies, survey response tendencies, browsing tendencies, cart addition tendencies, payment method preferences, return history, and social media posting tendencies. For example, it can create virtual customer profiles that reflect real customer demographics, such as "female in her 30s, lives in an urban area, fashion-conscious, has purchased shoes and bags multiple times in the past six months, frequently shares trendy products on social media, is sensitive to discounts and free shipping campaigns, and frequently uses a smartphone." A specific example of a prompt to be fed into the generative AI is: "Based on the following customer attribute data, please generate 10 patterns of virtual customers that closely resemble real customer profiles. Each virtual customer should include age, gender, place of residence, occupation, preferences, purchasing tendencies, and survey response tendencies."

[0085] Step 4: Automatic generation of survey questions using AI. The system automatically generates survey questions for virtual customers using a generative AI. For example, it can create questions such as: "Are the images and descriptions on the product details page easy to understand?", "How user-friendly is the shopping cart screen?", "Can I select my preferred payment method on the checkout screen?", "Are the shipping options sufficient?", "Is the return process easy?", "Are the campaign banners attractive?", "Is the recommendation function useful?", "Is the search function easy to use?", "Are the reviews helpful?", "Is the overall site loading speed satisfactory?", and "Is the smartphone display optimized?". A specific example of a prompt to be fed into the generative AI is: "Please automatically generate survey questions for the following new product A. The questions should include satisfaction, price, features, design, competitor comparison, and free-response comments."

[0086] Step 5: Automatic generation of virtual customer survey responses using AI. For the generated survey questions, a generative AI is used to automatically generate responses such as satisfaction levels, improvement requests, and free-response comments based on each virtual customer's attributes and past trends. For example, a fashion-oriented virtual customer might respond with "I want more product images" and "I want outfit examples to be posted," a delivery-focused virtual customer might respond with "I want a same-day delivery option," and a payment-conscious virtual customer might respond with "I want more electronic money and deferred payment options." A specific example of a prompt to be fed into the generative AI is: "Based on the following virtual customer attributes and survey questions, please automatically generate survey responses for each virtual customer. The responses should include a satisfaction score and specific improvement requests."

[0087] Step 6: Compiling and analyzing survey results The system collects and analyzes survey responses from virtual customers on a server to quantify and visualize average satisfaction scores, frequently requested improvements, trends by attribute group, time-series changes, product category evaluations, site element evaluations, and comparative evaluations with competitor sites. For example, it extracts trends for each customer segment and overall issues, such as "women in their 20s strongly desire more product images and styling examples," "men in their 40s prioritize diversification of payment methods and improved delivery speed," and "overall, there is a demand for simplified return procedures."

[0088] Step 7: Automatic generation of site optimization strategy proposals using AI. Based on the results of the survey, a generative AI is used to automatically generate optimal site optimization strategies. For example, it proposes specific site optimization measures such as "increase the number of images on product detail pages," "automatically generate and post outfit examples," "revamp the UI of the shopping cart screen," "add electronic money and deferred payment as payment methods," "introduce same-day delivery as a shipping option," "make the return process one-click," "change the display position of the campaign banner to the center of the top page," "add photo posting and purchaser-only display to the review function," "expand filtering options to the search function," "speed up the overall display speed of the site," and "optimize the UI / UX for smartphone display." It also automatically indicates the priority of each measure and which measure is effective for which customer segment. An example of a prompt to be fed into the generative AI is: "Analyze the following survey results and automatically propose three optimal site optimization strategies for each customer segment. The strategies should include UI improvements, advertising media, appeal content, and campaign timing."

[0089] Step 8: Visualization of proposed policies and analysis results, and presentation to those in charge. Through the user interface, survey results and proposed measures suggested by the generative AI are presented to the person in charge in the form of graphs, heatmaps, recommendation lists, alert notifications, and post-implementation effectiveness measurement results. The person in charge can then perform operations from their terminal, such as specifying the products or services to be analyzed, customizing survey questions, sending feedback on proposed measures, inputting post-implementation effectiveness measurement data, customizing dashboards, and setting up A / B tests for measures.

[0090] (Specific use cases)

[0091] For example, let's consider a case where a nationwide e-commerce site operator plans to launch a new sportswear category. First, the customer data integration department collects member registration information such as the age, gender, place of residence, occupation, family structure, hobbies, and past campaign participation history of existing members; past purchase history including the name, category, brand, price, purchase date and time, purchase frequency, purchase channel, coupon usage history, return / exchange history, review posting status, and subscription history of sports goods and related products; browsing history including the number of views of sportswear-related pages, time spent on the page, search keywords, ranking page views, review viewing history, genre and price range of viewed products, cart addition rate, and abandonment rate; cart addition history including the date and time sportswear was added to the cart, cart abandonment history, price range and category of items in the cart, time spent in the cart, deletion history, and purchase conversion rate; and survey responses including satisfaction scores regarding sportswear, requests for improvement, intention to repurchase, intention to recommend, and comments in the free-response section. The system automatically collects diverse data such as comparative evaluations with competing sites, evaluations regarding delivery and support, and evaluations regarding UI / UX; inquiry history including the content of inquiries related to sportswear, inquiry date and time, response history, FAQ viewing history, chatbot usage history, satisfaction level with inquiry response, and purchasing behavior after inquiry; point usage history including whether points were used when purchasing sportswear, points earned, points used, point expiration date, point campaign participation history, point expiration history, and purchasing trends when using points; and SNS integration information including sports-related SNS account type, number of followers, post content, share history, like history, traffic history via SNS, SNS campaign participation history, and SNS word-of-mouth content. This diverse data is automatically collected via API integration, batch processing, web crawling, etc., and stored in the database management department.

[0092] Next, using a generation AI, variations in spelling such as "Tokyo Metropolitan Area," "Tokyo," and "TOKYO" are standardized, age information such as "early 20s," "25 years old," and "24 years old" is classified into appropriate categories, key opinions and feelings such as "I like the design," "The price is high," "There are few size options," and "I'm concerned about the material" are extracted and summarized from free-response comments, data for the same person is consolidated from multiple data sources, anonymized, and then managed as a unified customer profile. Specific examples of prompts to be fed into the generation AI include: "Standardize the variations in spelling of the following customer attribute data and classify them by category," and "Summarize the following free-response questionnaire answers and extract the key opinions and feelings."

[0093] The virtual customer generation unit uses integrated customer trend data and generation AI to automatically generate a large number of virtual customer profiles that reflect real customer demographics. For example, "Female in her 20s, lives in an urban area, sports-oriented, has purchased running shoes and yoga wear multiple times in the past six months, frequently shares training videos on social media, is sensitive to discounts and free shipping campaigns, and uses smartphones frequently," "Male in his 40s, lives in a rural area, purchases sportswear for health maintenance, purchases infrequently but chooses high-performance products, prefers electronic money for payment, and values ​​fast delivery," and "Female in her 30s, is raising children, buys sportswear for the whole family in bulk, values ​​point campaigns, and frequently uses specified delivery dates and times." A specific example of a prompt message to be fed into the generation AI is, "Based on the following customer attribute data, please generate 10 patterns of virtual customers that closely resemble real customer profiles. Each virtual customer should include age, gender, place of residence, occupation, preferences, purchasing tendencies, and survey response tendencies."

[0094] The Site Optimization Measures Proposal Department uses a generation AI to automatically generate questions for each virtual customer, such as "Are the images and descriptions on the sportswear product details page easy to understand?", "How is the usability of the shopping cart screen?", "Can the desired payment method be selected on the checkout screen?", "Are the shipping options sufficient?", "Is the return process easy?", "Are the campaign banners attractive?", "Is the recommendation function useful?", "Is the search function easy to use?", "Are the reviews helpful?", "Is the overall site loading speed satisfactory?", and "Is the smartphone display optimized?". Based on the attributes and past trends of the virtual customer, the department automatically generates responses such as satisfaction levels, requests for improvement, and free-response comments. For example, a fashion-conscious virtual customer might say, "I want more product images" or "I want to see outfit examples," while a delivery-focused virtual customer might say, "I want a same-day delivery option," a payment method-conscious virtual customer might say, "I want more electronic money and deferred payment options," and a review-focused virtual customer might say, "I want reviews to be displayed only to purchasers to increase their credibility" or "I want a feature to post photos of reviews." In the free-response section, specific opinions such as "The return process is confusing," "Campaign information is hard to find," "The site loads slowly," and "It's difficult to use on smartphones" are also automatically generated. Specific prompts to be fed into the AI ​​include: "Automatically generate the following survey questions regarding the new sportswear category. The questions should include satisfaction, price, functionality, design, competitor comparison, and free-response opinions." and "Automatically generate survey responses for each virtual customer based on the following virtual customer attributes and survey questions. The responses should include a satisfaction score and specific improvement requests."

[0095] After compiling and analyzing the survey results, the generating AI extracts trends for each customer segment and overall challenges, such as "Women in their 20s strongly desire more product images and styling examples," "Men in their 40s prioritize diversification of payment methods and improved delivery speed," "Overall, there is a demand for simplified return procedures," "Improving the reliability of the review function contributes to overall satisfaction," and "Optimizing smartphone display is effective in reducing the dropout rate among younger users." Using the generating AI, it automatically proposes specific site optimization measures, such as "Increase the number of images on product detail pages," "Automatically generate and post styling examples," "Refresh the cart screen UI," "Add electronic money and deferred payment as payment methods," "Introduce same-day delivery as a shipping option," "Make the return process one-click," "Change the display position of campaign banners to the center of the top page," "Add photo posting and buyer-only display to the review function," "Expand filtering options in the search function," "Increase the overall site display speed," and "Optimize the UI / UX for smartphone display." A concrete example of a prompt message to be fed into the generating AI is: "Analyze the following survey results and automatically propose three optimal website optimization strategies for each customer segment. The strategies should include UI improvements, advertising media, messaging, and campaign timing."

[0096] The user interface section presents these analysis results and proposed measures to the person in charge on a dashboard in the form of graphs, heatmaps, recommendation lists, alert notifications, and post-implementation effectiveness measurement results. The person in charge can then perform operations from their terminal, such as specifying the products or services to be analyzed, customizing survey questions, sending feedback on proposed measures, inputting post-implementation effectiveness measurement data, customizing the dashboard, and setting up A / B tests for the measures.

[0097] In such use cases, the system of the present invention automatically provides real-time, multidimensional customer evaluations that cannot be obtained through conventional market research or survey data collection, as well as specific site optimization measures optimized for each customer segment. This allows e-commerce site operators to quickly and accurately determine site management policies when launching new categories or improving existing categories, without having to wait for actual user behavior.

[0098] The specific processing unit 290 transmits the result of the specific processing obtained in the embodiment 1 or application example 1 described above to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0099] Data generation model 58 is a form of so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> Examples of generative AI include text generation AI, image generation AI, and multimodal generation AI. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images (e.g., still image data or video data). The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference result in one or more data formats from audio data, text data, and image data. The data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specific processing unit 290 performs the specific processing described above using the data generation model 58. The data generation model 58 may be a fine-tuned model that outputs inference results from prompts that do not contain instructions, in which case the data generation model 58 can output inference results from prompts that do not contain instructions. In the data processing device 12, etc., there are multiple types of data generation models 58, and the data generation model 58 includes AI other than generative AI. AI other than generative AI includes, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVMs), k-means clustering, convolutional neural networks (CNNs), recurrent neural networks (RNNs), generative adversarial networks (GANs), or naive Bayes, and can perform various processes, but is not limited to these examples. Also, the AI ​​may be an AI agent. Furthermore, when the processing of each of the above parts is performed by the AI, the processing may be performed by the AI ​​in part or in whole, but is not limited to this example.Furthermore, processing performed by AI, including generative AI, may be replaced with rule-based processing, and rule-based processing may be replaced with processing performed by AI, including generative AI.

[0100] Furthermore, the processing performed by the data processing system 10 described above is carried out by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart device 14, but it may also be carried out by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart device 14. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the smart device 14 or an external device, and the smart device 14 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0101] For example, the data collection unit is implemented by the control unit 46A of the smart device 14 or the specific processing unit 290 of the data processing device 12. For example, the acquisition unit acquires step count data using the camera 42 and communication I / F 44 of the smart device 14 and processes it with the specific processing unit 290 of the data processing device 12. For example, the analysis unit is implemented by the specific processing unit 290 of the data processing device 12 and analyzes the data from the data collection unit and the acquisition unit. For example, the generation unit is implemented by the specific processing unit 290 of the data processing device 12 and generates a cooking menu using generation AI. For example, the serving unit is implemented by the output device 40 of the smart device 14 or the specific processing unit 290 of the data processing device 12 and provides the generated cooking menu to the user. The correspondence between each unit and the devices and control units is not limited to the examples described above and can be modified in various ways.

[0102] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0103] [Second Embodiment]

[0104] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0105] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0106] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0107] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and communication interface 44 are also connected to the bus 52.

[0108] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0109] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0110] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0111] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0112] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0113] Storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotions using the emotion identification model 59 and perform identification processing using the user's emotions. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotions, including but not limited to these examples. Furthermore, emotion estimation and prediction also include, for example, emotion analysis.

[0114] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48. The smart glasses 214 also have a data generation model 58 and an emotion identification model 59, similar to the data generation model and emotion identification model 59, and can perform processing similar to that of the identification processing unit 290 using these models.

[0115] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. Each part of the system described below is realized by the data processing device 12 and the smart glasses 214. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0116] (Example 1)

[0117] Since the process flow is the same as in Example 1 described in the First Embodiment above, the explanation will be omitted.

[0118] (Application Example 1) Since the process flow is the same as in Application Example 1 described in the first embodiment above, the explanation will be omitted.

[0119] The specific processing unit 290 transmits the result of the specific processing obtained in the embodiment 1 or application example 1 described above to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0120] Furthermore, the processing by the data processing system 10 described above is performed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart glasses 214, but it may also be performed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart glasses 214. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the smart glasses 214 or an external device, and the smart glasses 214 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0121] For example, the data collection unit is implemented by the control unit 46A of the smart glasses 214 or the specific processing unit 290 of the data processing device 12. For example, the acquisition unit acquires step count data using the camera 42 and communication I / F 44 of the smart glasses 214 and processes it using the specific processing unit 290 of the data processing device 12. For example, the analysis unit is implemented by the specific processing unit 290 of the data processing device 12 and analyzes the data from the data collection unit and the acquisition unit. For example, the generation unit is implemented by the specific processing unit 290 of the data processing device 12 and generates a cooking menu using generation AI. For example, the serving unit is implemented by the speaker 240 of the smart glasses 214 or the specific processing unit 290 of the data processing device 12 and provides the generated cooking menu to the user. The correspondence between each unit and the device or control unit is not limited to the examples described above and can be modified in various ways.

[0122] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0123] [Third Embodiment]

[0124] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0125] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0126] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0127] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, display 343, and communication interface 44 are also connected to the bus 52.

[0128] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0129] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0130] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0131] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0132] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0133] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0134] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0135] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. Each part of the system described below is implemented by the data processing device 12 and the headset terminal 314. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0136] (Example 1) Since the process flow is the same as in Example 1 described in the First Embodiment above, the explanation will be omitted.

[0137] (Application Example 1) Since the process flow is the same as in Application Example 1 described in the first embodiment above, the explanation will be omitted.

[0138] The specific processing unit 290 transmits the result of the specific processing obtained in the embodiment 1 or application example 1 described above to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0139] Furthermore, the processing performed by the data processing system 10 described above is carried out by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the headset terminal 314, but it may also be carried out by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the headset terminal 314. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the headset terminal 314 or an external device, and the headset terminal 314 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0140] For example, the data collection unit is implemented by the control unit 46A of the headset terminal 314 or the specific processing unit 290 of the data processing device 12. For example, the acquisition unit acquires step count data using the camera 42 and communication I / F 44 of the headset terminal 314 and processes it using the specific processing unit 290 of the data processing device 12. For example, the analysis unit is implemented by the specific processing unit 290 of the data processing device 12 and analyzes the data from the data collection unit and the acquisition unit. For example, the generation unit is implemented by the specific processing unit 290 of the data processing device 12 and generates a cooking menu using generation AI. For example, the serving unit is implemented by the speaker 240 and display 343 of the headset terminal 314 or the specific processing unit 290 of the data processing device 12 and provides the generated cooking menu to the user. The correspondence between each unit and the device or control unit is not limited to the examples described above and can be modified in various ways.

[0141] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0142] [Fourth Embodiment]

[0143] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0144] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0145] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0146] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, controlled object 443, and communication interface 44 are also connected to the bus 52.

[0147] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0148] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0149] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0150] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0151] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0152] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0153] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0154] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0155] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. Each part of the system described below is implemented by the data processing device 12 and the robot 414. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0156] (Example 1) Since the process flow is the same as in Example 1 described in the First Embodiment above, the explanation will be omitted.

[0157] (Application Example 1) Since the process flow is the same as in Application Example 1 described in the first embodiment above, the explanation will be omitted.

[0158] The specific processing unit 290 transmits the result of the specific processing obtained in the embodiment 1 or application example 1 described above to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0159] Furthermore, the processing performed by the data processing system 10 described above is carried out by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the robot 414, but it may also be carried out by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the robot 414. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the robot 414 or external devices, and the robot 414 acquires or collects information necessary for processing from the data processing device 12 or external devices.

[0160] For example, the data collection unit is implemented by the control unit 46A of the robot 414 or the specific processing unit 290 of the data processing device 12. For example, the acquisition unit acquires step count data using the camera 42 and communication I / F 44 of the robot 414 and processes it using the specific processing unit 290 of the data processing device 12. For example, the analysis unit is implemented by the specific processing unit 290 of the data processing device 12 and analyzes the data from the data collection unit and the acquisition unit. For example, the generation unit is implemented by the specific processing unit 290 of the data processing device 12 and generates a cooking menu using a generation AI. For example, the serving unit is implemented by the speaker 240 and controlled object 443 of the robot 414 and the specific processing unit 290 of the data processing device 12 and provides the generated cooking menu to the user. The correspondence between each unit and the device or control unit is not limited to the examples described above and can be modified in various ways.

[0161] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0162] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0163] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0164] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0165] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0166] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0167] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0168] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0169] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0170] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0171] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0172] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0173] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0174] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0175] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0176] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0177] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0178] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0179] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0180] Furthermore, the following additional information is disclosed regarding the above explanation.

[0181] (Claim 1)

[0182] The system comprises a customer data integration unit, a virtual customer generation unit, and a site optimization strategy proposal unit. The customer data integration unit automatically collects diverse customer behavior data from e-commerce sites, such as member registration information, purchase history, browsing history, cart entry history, survey responses, inquiry history, point usage history, and SNS linking information, through API integration and batch processing, integrates it into a database, and performs preprocessing using generative AI, including data duplication, missing value imputation, normalization, anonymization, summarization and classification of free-text fields, and standardization of attribute names and categories. The virtual customer generation unit uses generative AI based on the integrated customer behavior data to generate age, gender, place of residence, preferences, purchasing trends, survey trends, return history, and payment method. This system automatically generates a large number of virtual customers with multidimensional attributes such as preferences. The site optimization strategy proposal department uses generative AI to automatically generate satisfaction levels and improvement requests for each element of the site, such as product detail pages, shopping cart screens, payment screens, top pages, search functions, recommendation functions, campaign banners, shipping options, and return procedure screens, for each virtual customer. The system then aggregates and analyzes these survey responses to extract trends for each customer segment and overall issues. Furthermore, it uses generative AI to automatically propose site optimization measures such as enhancing product images, refreshing the UI, adding payment methods, expanding shipping options, simplifying return procedures, and changing the display location of campaigns. This entire process is repeated 24 hours a day. (Claim 2)

[0183] The system according to claim 1, characterized in that the virtual customer generation unit also incorporates external market trend data, seasonal fluctuation data, competitor site strategy information, and SNS trend data, and uses a generation AI to dynamically update the attributes, preferences, and purchasing tendencies of virtual customers in real time, thereby maintaining a virtual customer group that constantly reflects the latest market conditions and social trends. (Claim 3)

[0184] The system according to claim 1, characterized in that the aforementioned site optimization measures proposal department visualizes the aggregated and analyzed survey results in multiple dimensions, such as by attribute, time series, product category, and site element, and further uses generative AI to automatically select the most suitable site improvement measures, promotion measures, advertising media, appeal content, and campaign timing for each customer segment, and proposes them to site operators in real time through a dashboard and notification function. [Explanation of symbols]

[0185] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots

Claims

1. The system comprises a customer data integration unit, a virtual customer generation unit, and a survey analysis and policy proposal unit. The customer data integration unit collects multiple types of customer trend data held by companies and businesses, such as demographic information, preferences, past purchase history, and survey responses, and manages them integrally after preprocessing such as data duplication and missing data completion. The virtual customer generation unit analyzes customer attributes and behavioral patterns using generational AI based on the integrated customer trend data, employing machine learning and clustering techniques, and generates multiple virtual customers that closely resemble real customers, based on age, gender, place of residence, preferences, purchase tendencies, survey response tendencies, etc. The system is characterized by automatically generating specific attribute information, having the survey analysis and policy proposal department automatically conduct surveys on new products and existing services to the virtual customers using a generation-based AI, automatically generating responses such as satisfaction levels and improvement requests according to each virtual customer's attributes and past trends, aggregating and analyzing these responses to quantify and visualize satisfaction scores and improvement request trends, and then automatically proposing various marketing measures such as quality improvement, price revisions, strengthening of delivery systems, strengthening of support systems, and optimization of advertising channels using the generation-based AI based on the analysis results, and continuously executing this series of processes 24 hours a day.

2. The system according to claim 1, characterized in that the virtual customer generation unit incorporates time-series changes in customer trend data and external public opinion and market trend data, dynamically updates the attributes and preferences of virtual customers in real time using a generation AI, and maintains a virtual customer group that always reflects the latest market conditions.

3. The system according to claim 1, characterized in that the aforementioned survey analysis and policy proposal department visualizes the aggregated and analyzed survey results in multiple dimensions, such as by attribute, time series, and product category, and further uses generative AI to automatically select the most suitable advertising media, appeal content, and campaign timing for each customer segment, and proposes them in real time to marketing personnel of companies and businesses through dashboards and notification functions.

Images